Pipeline Editor in GitLab - test your CI configs

Pipeline Editor has been introduced in 13.8 version. What can you do with this tool? Normally, when you edit your .gitlab-ci.yml file, you will find that something is wrong with the code only after you push your changes. It would be great to know that there is a typo or something extremely wrong with this config in advance, right?

And for this, you can use Pipeline Editor. There are three things you can do with that tool:

  • Validate pipeline configuration
  • Visualise the configuration
  • Lint the configuration

Validation is done automatically when you're editing the file. Okay, so how to start? 🤔 Go to CI / CD section and click the Editor link:

Validation

After that, you should see the Pipeline Editor:

Let's break something!

Hmm, what's wrong with this code? 🤔 I've received a message that my CI configuration is invalid because chosen stage does not exist. This make sense! I have a build job and I inherit from hidden job .default, and I had not set a stage in any of these jobs. But, I should! Thank you Pipeline Editor.

Let's fix CI configuration and add a build stage to a build job:

Linter

CI configuration is valid now, I can go to the Lint section and check if syntax is correct:

Everything looks fine here! Syntax is correct and I can even see the preview of my jobs! Nice 👍. Linter checks for syntax and logical errors and it's more powerful than the validation in the editor. I see that the build job will run on branches and tags, and because it's the first job in a pipeline with when set to on_success it will be always spawned.

Let's add another job, but this time for deploy:

Now, when I navigate to Lint section I will see two jobs:

Great 👍. In this way, you can lint your changes before committing them. Linter is updated in real-time. And you can see also some additional info like deploy job is manual.

Visualization 

Okay, and now the next section: Visualize. If you go there now you will see these two jobs:

You don't need this feature if you have two or three jobs. But, what if you have more jobs? It would be useful to visualize them. Let's see another example:

It's not a complicated one; but even in this example, a visualization will be very useful. And here is the viz:

Much better than plain yaml, right? Instantly, you know what's going on here. You have four stages, the first job is lint. Then, an image is built. And now, because I used needs keyword, I see relationships between jobs. Let's hover a cursor on deploy dev job:

Deploy dev job needs only build job. And you can easily see the relationship between these two jobs. deploy dev job runs as soon as build job finishes without waiting for jobs in test stage to finish. So, I want to deploy my changes on dev environment as soon as possible 😉.

Now, let's hover a cursor on deploy prod job:

Here, the situation is different. I don't want to deploy something on production without any tests! So, deploy prod job needs these three guys from test stage. Deploy prod job runs as soon as these three test jobs finish. Do you agree that such visualization is useful?

include keyword

What if you have a more complicated pipeline? Pipeline Editor supports include keyword, so you can use it. Let's say that you split the stages and put each stage into another yaml file. Then you can just include all these files:

And your Pipeline Editor still is able to function normally, because as I said earlier it supports this keyword.

Of course in this way you cannot test everything. For example, you cannot test if your ifs in rules keyword are correct, etc. But, it's still useful. If you just need to add small fix, it might be a good alternative to Merge Request.

Here is the video if you prefer this way of learning:

Comments

Post a Comment

Popular posts from this blog

GitLab - extends keyword

Image
In the previous post I had written about the include keyword. Now, let's dive in the extends  keyword. What can you do with this? Here is the definition from GitLab documentation: extends defines entry names that a job that uses extends inherits from. It’s an alternative to using YAML anchors and is a little more flexible and readable. So, when is it useful? It is useful when you want to be DRY and keep your setup cleanly. Let's assume that you want to build a docker image and give developers possibility to deploy the image on dev , staging and prod environments. We are going to do that without the extends keyword, and after that we will think how we can do it better. Example without the extends  keyword: As you can see, there is a lot of code which repeats itself: stage, image, when  and script . We can remove a lot of code! So, let's do that. We are going to create a hidden job .deploy  (you will not see this job in a pipeline) and the rest of the jobs ...
Read more

Managing Secrets in GitLab / Git

Image
Let's say that you have to log in via ssh into an instance, and you work with GitLab, so you want to keep the private key in GitLab somewhere. Is it secure? Let's see! Custom environment variables You can use custom environment variables. Here you can read more about them (Developers cannot change them, only Maintainers and Owners can). There are two types of variables: Variable (the runner creates an environment variable that uses the key for the name and the value for the value) File (the runner creates an environment variable that uses the key for the name. For the value, the runner writes the variable value to a temporary file and uses this path) It seems that we can use File type for our purpose. We can set up it via API or UI . So, let's do that! Go to project's Settings > CI/CD . There will be Variables section (btw, you can specify variables also per group and even for all projects (in admin panel)). Click Add Variable button and add a variable: Key: ...
Read more

GitLab - trigger keyword

Image
 We are going to talk about the trigger keyword today. With this feature you can define a downstream pipeline trigger. So, you can trigger a pipeline in any project (you must have access to this project of course). There are two types of downstream pipelines: multi-project pipelines child pipelines  Let's omit child pipelines and have a look at multi-project pipelines. It is very easy to use. You just need to provide a path to the project and that's it. Remember that you only can trigger a pipeline, not a job ! You can also provide more information like environment variables. As always, I am going to provide some examples, so you will understand it better. The first example: you want to trigger a pipeline in another project after the deployment of your service/s. How to do that? Here we want to trigger this pipeline (project my/run-tests ). Let's say that we want to run some tests after each deployment: We need to add the job which will trigger this pipeline in my/run-...
Read more