Terraform - import aws_s3_bucket does not store important attributes like acl

Recently, I had to import some AWS resources to terraform, and most things went smoothly, but some did not. More specifically, I have encountered this problem. And here is my reply how to deal with it now. In this post, I am going to be more elaborate about this issue.

So, what exactly I have run into? Here is the code:

Such bucket existed and I wanted to import this guy to terraform (the bucket was public). So, I typed terraform import 'aws_s3_bucket.my-bucket' 'my-bucket' and pressed enter:

Wait, what? I understand the force_destroy argument (it is false by default), because I had not specified it, but acl? I have two grant blocks... and according to the documentation, acl conflicts with grant. So, how is it even possible? 🤔

It was tempting to run terraform apply command... so let's do that!

And what happened? Terraform (or should I say aws provider?) ignored these grant blocks and removed some ACL (Access control list) records from my bucket (the bucket was converted to the private one after this action). Why the grant blocks were ignored? They were there the whole time, they are still there.

Hmm, what if I run terraform apply again? 😅

ACL records (grant blocks) had been removed (from terrafom state (and from my bucket ☹️), not from my code) in the first apply and now in the second one terraform wants to add them. I cannot accept such behaviour on production; I don't want to make my bucket private for some minutes/seconds on prod env. It was funny on dev environment, but not on production.

So, it is time for lifecycle Meta-Argument! ⛑ And specifically ignore_changes argument. This way we can define which arguments should be ignored during an update procedure. After changes, my code looked like this:

I had to add both acl and force_destroy arguments to the ignore_changes list. Adding only acl did not resolve the problem. Terraform showed that only this argument would be added, but ACL records were removed also 🤷.

Using this meta-argument I can import production buckets fearlessly. Terraform does not want to change (remove if I want to be more specific) ACL records this way. It works, but it is quite hacky and rather should be resolved on aws provider side, methinks. I have not fixed any issue in terraform repo yet, so maybe it is time to learn something new, help others and fix this bug.

Comments

Post a Comment

Popular posts from this blog

GitLab - extends keyword

Image
In the previous post I had written about the include keyword. Now, let's dive in the extends  keyword. What can you do with this? Here is the definition from GitLab documentation: extends defines entry names that a job that uses extends inherits from. It’s an alternative to using YAML anchors and is a little more flexible and readable. So, when is it useful? It is useful when you want to be DRY and keep your setup cleanly. Let's assume that you want to build a docker image and give developers possibility to deploy the image on dev , staging and prod environments. We are going to do that without the extends keyword, and after that we will think how we can do it better. Example without the extends  keyword: As you can see, there is a lot of code which repeats itself: stage, image, when  and script . We can remove a lot of code! So, let's do that. We are going to create a hidden job .deploy  (you will not see this job in a pipeline) and the rest of the jobs will in
Read more

Managing Secrets in GitLab / Git

Image
Let's say that you have to log in via ssh into an instance, and you work with GitLab, so you want to keep the private key in GitLab somewhere. Is it secure? Let's see! Custom environment variables You can use custom environment variables. Here you can read more about them (Developers cannot change them, only Maintainers and Owners can). There are two types of variables: Variable (the runner creates an environment variable that uses the key for the name and the value for the value) File (the runner creates an environment variable that uses the key for the name. For the value, the runner writes the variable value to a temporary file and uses this path) It seems that we can use File type for our purpose. We can set up it via API or UI . So, let's do that! Go to project's Settings > CI/CD . There will be Variables section (btw, you can specify variables also per group and even for all projects (in admin panel)). Click Add Variable button and add a variable: Key:
Read more

GitLab - trigger keyword

Image
 We are going to talk about the trigger keyword today. With this feature you can define a downstream pipeline trigger. So, you can trigger a pipeline in any project (you must have access to this project of course). There are two types of downstream pipelines: multi-project pipelines child pipelines  Let's omit child pipelines and have a look at multi-project pipelines. It is very easy to use. You just need to provide a path to the project and that's it. Remember that you only can trigger a pipeline, not a job ! You can also provide more information like environment variables. As always, I am going to provide some examples, so you will understand it better. The first example: you want to trigger a pipeline in another project after the deployment of your service/s. How to do that? Here we want to trigger this pipeline (project my/run-tests ). Let's say that we want to run some tests after each deployment: We need to add the job which will trigger this pipeline in my/run-
Read more