Python/DevOps

Sometimes you want to test your code on different python versions. Using Travis CI it is very easy, but can we do that or something similar in GitLab? Let's see! The problem : there are some tests written in python and we want to test it on different python versions. We can easily use extends keyword, define some common logic in .test  job and then inherit it in other jobs: But... for each new version we have to define a new job. Maybe, we can do better? GitLab has introduced a  parallel keyword. Additionally, there is a  matrix keyword; using this guy you can run a job multiple times in parallel in a single pipeline, but with different variable values for each instance of the job. But, you cannot change image keyword for example 😞. Maybe there is another way? Have you ever heard about dind (Docker-in-Docker)? If not, here is some info about it: Another way to configure GitLab Runner for docker support is to register a runner with the Docker executor and use the...

How do you apply changes in terraform ? In most cases you run terraform plan and then terraform apply  and type yes . This approach works great on your local machine, but how to apply changes (and only the changes you want!) in GitLab job where you do not have access to shell? How to do that, when you cannot approve the output of apply command? You can use terraform apply -auto-approve , but it might be risky... No one likes to destroy something on production without a priori knowledge. So, can we run terraform plan , check the output and then run terraform apply  in another step? We can, but still it might be risky operation. Why? Because plan and apply  are separated operations! They know nothing about each other. So, apply  can change something which was not showed in plan . But... according to Terraform Documentation : The optional -out argument can be used to save the generated plan to a file for later execution with terraform apply, which can be useful...

Ahh, yes!  You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit I bet you had encountered this problem if you read now this post. According to this document : Beginning November 2, 2020, progressive enforcement of rate limits for anonymous and authenticated Docker Hub usage goes into effect. This means that anonymous and free Docker Hub users will have usage restrictions gradually placed on container image pull requests. Sadly, this happened on our clusters (AWS EKS). How to fix it? I wanted to spawn DaemonSet object where I could run docker login command and this way change config.json  on every node. But, after that you need to restart docker  process on every node, and I still do not know how to do that on AWS EKS. So, a temporary fix was to create a Secret object and then link it to every ServiceAccount  object. It is "a hack", but we needed very fast working solution. We ...

In the previous post I had written about the include keyword. Now, let's dive in the extends  keyword. What can you do with this? Here is the definition from GitLab documentation: extends defines entry names that a job that uses extends inherits from. It’s an alternative to using YAML anchors and is a little more flexible and readable. So, when is it useful? It is useful when you want to be DRY and keep your setup cleanly. Let's assume that you want to build a docker image and give developers possibility to deploy the image on dev , staging and prod environments. We are going to do that without the extends keyword, and after that we will think how we can do it better. Example without the extends  keyword: As you can see, there is a lot of code which repeats itself: stage, image, when  and script . We can remove a lot of code! So, let's do that. We are going to create a hidden job .deploy  (you will not see this job in a pipeline) and the rest of the jobs ...