GitLab - terraform plan and apply

How do you apply changes in terraform? In most cases you run terraform plan and then terraform apply and type yes. This approach works great on your local machine, but how to apply changes (and only the changes you want!) in GitLab job where you do not have access to shell? How to do that, when you cannot approve the output of apply command?

You can use terraform apply -auto-approve, but it might be risky... No one likes to destroy something on production without a priori knowledge. So, can we run terraform plan, check the output and then run terraform apply in another step? We can, but still it might be risky operation. Why? Because plan and apply are separated operations! They know nothing about each other. So, apply can change something which was not showed in plan.

But... according to Terraform Documentation:

The optional -out argument can be used to save the generated plan to a file for later execution with terraform apply, which can be useful when running Terraform in automation.

This way we can run terraform plan with -out parameter and then use this file in terraform apply command. Therefore, we know that only changes from plan command will be applied (if there were any, of course).

So, let's say that we have two environments: dev and prod. We want to run terraform plan command, and if everything is alright, we want to run terraform apply and it should apply changes from the plan only. We need to save a file from plan command and use it later in apply. For this we can use artifacts keyword. We will also use some other keywords. Here is the first part of the .gitlab-ci.yml file (link to the repository you will find at the end of the post):

We keep our terraform logic in dev and prod catalogs. We have two stages: plan and apply.

    .common job: This is a hidden job which will be used later by extends keyword. We provide terraform image (obviously) and entrypoint for this image (by default, entrypoint is set to terraform command, so without changing entrypoint we couldn't run some commands like cd for example). The job will be manual, and we also have before_script section. We want to go to dev or prod ($ENV_NAME environment variable) catalogue and run terraform init command.

    .plan job: Another hidden job which will be used later. Here we inherit from .common job using extends keyword, we set stage to plan, and run terraform plan command with param -out. This way we tell terraform to save the plan to a specific file: $ENV_NAME.plan. This environment variable $ENV_NAME we will provide in another jobs. Additionally, we set artifacts section. With this we can easily save whatever we want and use it later in other jobs. So, here we want to keep the generated plan.

    .apply job: Here we also inherit from .common job, but stage is set to apply, and the command in script section is terraform apply -input=false $ENV_NAME.plan. We will use this job and .plan soon...

So we have a backbone; now, we are going to add the rest of stuff. We have two catalogs: dev and prod. I am going to provide a code only for dev jobs, because the logic for prod is the same. The only difference is the name of jobs and the value of $ENV_NAME environment variable.

We have three jobs here, but one of them is hidden so in a pipeline you will see only two jobs: dev plan and dev apply.

    .dev job: we only set a variable here.

    dev plan job: in this job we only inherit from the previous ones: .plan and .dev. We get the whole logic from .plan job, and from .dev we get info about $ENV_NAME environment variable.

    dev apply job: it is similar to dev plan job, but we inherit from .apply job. Additionally, we use dependencies keyword, telling GitLab that we want artifacts only from dev plan job.

The same thing has been done for prod environment. So, the pipeline looks that:

The pipeline

In dev plan job you will find such output:

dev plan

As you can see, terraform saved the plan to dev.plan file (in dev catalog) and the artifact has been uploaded. In the next job dev apply you will find this guy:

dev apply

And here, we used dev.plan file which has been uploaded by dev plan job! That's it!

Here is the link to the repository terraform-plan-apply; you will find the rest of code there.

And here is a video:


 

Comments

Post a Comment

Popular posts from this blog

GitLab - extends keyword

Image
In the previous post I had written about the include keyword. Now, let's dive in the extends  keyword. What can you do with this? Here is the definition from GitLab documentation: extends defines entry names that a job that uses extends inherits from. It’s an alternative to using YAML anchors and is a little more flexible and readable. So, when is it useful? It is useful when you want to be DRY and keep your setup cleanly. Let's assume that you want to build a docker image and give developers possibility to deploy the image on dev , staging and prod environments. We are going to do that without the extends keyword, and after that we will think how we can do it better. Example without the extends  keyword: As you can see, there is a lot of code which repeats itself: stage, image, when  and script . We can remove a lot of code! So, let's do that. We are going to create a hidden job .deploy  (you will not see this job in a pipeline) and the rest of the jobs ...
Read more

Managing Secrets in GitLab / Git

Image
Let's say that you have to log in via ssh into an instance, and you work with GitLab, so you want to keep the private key in GitLab somewhere. Is it secure? Let's see! Custom environment variables You can use custom environment variables. Here you can read more about them (Developers cannot change them, only Maintainers and Owners can). There are two types of variables: Variable (the runner creates an environment variable that uses the key for the name and the value for the value) File (the runner creates an environment variable that uses the key for the name. For the value, the runner writes the variable value to a temporary file and uses this path) It seems that we can use File type for our purpose. We can set up it via API or UI . So, let's do that! Go to project's Settings > CI/CD . There will be Variables section (btw, you can specify variables also per group and even for all projects (in admin panel)). Click Add Variable button and add a variable: Key: ...
Read more

GitLab - trigger keyword

Image
 We are going to talk about the trigger keyword today. With this feature you can define a downstream pipeline trigger. So, you can trigger a pipeline in any project (you must have access to this project of course). There are two types of downstream pipelines: multi-project pipelines child pipelines  Let's omit child pipelines and have a look at multi-project pipelines. It is very easy to use. You just need to provide a path to the project and that's it. Remember that you only can trigger a pipeline, not a job ! You can also provide more information like environment variables. As always, I am going to provide some examples, so you will understand it better. The first example: you want to trigger a pipeline in another project after the deployment of your service/s. How to do that? Here we want to trigger this pipeline (project my/run-tests ). Let's say that we want to run some tests after each deployment: We need to add the job which will trigger this pipeline in my/run-...
Read more