Python/DevOps

The problem : you have many scripts (let's say that they are written in python and you just want to run them typing  python your-script.py ) and sometimes you want to run some of them, sometimes only one, etc. There is no pattern. Additionally you want to trigger these scripts via GitLab API. How can you do this? The first idea: let's create a job for each of them! But... then what? You want to run only a small subset of them and each time this subset might be different ☹️. You might add variables, check them and run only these jobs when IF evaluates to true, like: But, I think that you see the problem of this approach. What about creating a common job without any command? It will be your job to provide a command for script section (for example python run-something-and-upload-to-s3.py ). This way we will have only one job in GitLab and during triggering you must provide a command. The code: 8 lines. Woah! We used rules keyword, because we want to spawn only this job w

Sometimes you want to test your code on different python versions. Using Travis CI it is very easy, but can we do that or something similar in GitLab? Let's see! The problem : there are some tests written in python and we want to test it on different python versions. We can easily use extends keyword, define some common logic in .test  job and then inherit it in other jobs: But... for each new version we have to define a new job. Maybe, we can do better? GitLab has introduced a  parallel keyword. Additionally, there is a  matrix keyword; using this guy you can run a job multiple times in parallel in a single pipeline, but with different variable values for each instance of the job. But, you cannot change image keyword for example 😞. Maybe there is another way? Have you ever heard about dind (Docker-in-Docker)? If not, here is some info about it: Another way to configure GitLab Runner for docker support is to register a runner with the Docker executor and use the Docker im

 We are going to talk about the trigger keyword today. With this feature you can define a downstream pipeline trigger. So, you can trigger a pipeline in any project (you must have access to this project of course). There are two types of downstream pipelines: multi-project pipelines child pipelines  Let's omit child pipelines and have a look at multi-project pipelines. It is very easy to use. You just need to provide a path to the project and that's it. Remember that you only can trigger a pipeline, not a job ! You can also provide more information like environment variables. As always, I am going to provide some examples, so you will understand it better. The first example: you want to trigger a pipeline in another project after the deployment of your service/s. How to do that? Here we want to trigger this pipeline (project my/run-tests ). Let's say that we want to run some tests after each deployment: We need to add the job which will trigger this pipeline in my/run-