Python/DevOps

Today we are going to learn more about the rules  keyword in GitLab. Normally, when you want to spawn your job on a specific branch or only in merge requests you can just use the only/except keywords. But, sometimes it is not enough. For example, what if you want to spawn a job always on develop  branch and run it automatically, but in merge requests you want to run it manually? You cannot (easily) do that with the only/except  keywords. Excerpt from GitLab documentation about the rules  keyword: The rules keyword can be used to include or exclude jobs in pipelines.  Rules are evaluated in order until the first match. When matched, the job is either included or excluded from the pipeline, depending on the configuration. If included, the job also has certain attributes added to it.  rules replaces only/except and can’t be used in conjunction with it. If you attempt to use both keywords in the same job, the linter returns a key may not be used with rules error. There are two very imp

In the previous post I had written about the include keyword. Now, let's dive in the extends  keyword. What can you do with this? Here is the definition from GitLab documentation: extends defines entry names that a job that uses extends inherits from. It’s an alternative to using YAML anchors and is a little more flexible and readable. So, when is it useful? It is useful when you want to be DRY and keep your setup cleanly. Let's assume that you want to build a docker image and give developers possibility to deploy the image on dev , staging and prod environments. We are going to do that without the extends keyword, and after that we will think how we can do it better. Example without the extends  keyword: As you can see, there is a lot of code which repeats itself: stage, image, when  and script . We can remove a lot of code! So, let's do that. We are going to create a hidden job .deploy  (you will not see this job in a pipeline) and the rest of the jobs will in

Some time ago GitLab has introduced include keyword that: allows the inclusion of external YAML files. Everyone knows that microservices like to multiply. Thus, this feature is very useful when you have microservices, because you do not have to copy & paste the code. Let's see what can you do with this. Let's assume that we use Docker and we need to build an image. If you want to build an image and deploy your microservice you can just make a new .gitlab-ci.yml file in your project and add two jobs: build and  deploy : When you have three/four microservices you can tempt to copy & paste this code to the rest of the projects. But, if you have more projects and you need to change the logic of deployments, then you have a problem, because you need to edit many .gitlab-ci.yml files. You do not want to do that, so how to deal with it? Use include  keyword! It is very simple. For me in most cases creating one gitlab template file is enough. Let's call it gitlab-tem

 Issue 7980 This bug occurs only when you use threads and only once. Of course, this method is thread safe, but there is a severe warning. The first use of strptime is not thread secure (underneath _strptime is imported, and the import may throw AttributeError ). If you want to avoid this problem, either you have to call strptime or import _strptime before starting a thread. How to fix it? Just import _strptime or call strptime before starting a thread. It seems that it happens since 2010.

If you have to maintain a massive project, you probably have many fixtures in conftest file. And there is a problem; this file grows and grows. So, at some point, you decide to split this huge file into smaller files. But, py.test has to know the fixtures are keeping in these files. So, what you can do? I see three patterns here for this: import these guys inside conftest.py file create more conftest.py files use pytest_plugins Import fixtures inside conftest file You can for example do this: 1 2 3 4 5 6 7 8 9 from tests.my_fixtures.fixs1 import ( fix11, fix12, fix13, fix14, fix15, ... ) from tests.my_fixtures.fixs2 import ( fix21, fix22, fix23, ) from tests.my_fixtures.fixs3 import ( fix31, fix32, fix33 ) But, there are some problems with this approach. You import them in an explicit way, so if you create a new fixture, you have to remember to add this guy in the import clause. So, this strategy is not the best solution. If you have many fi

pytest-dbfixtures If you use pytest maybe you have heard about pytest-dbfixtures : Pytest dbfixtures is a pytest plugin that makes it a lot easier to set up proper database or storage engine for testing. Simply use one of provided fixtures that start predefined clean database server for your tests or creates server more tailored for your application by using one of provided factories. This plugin is very useful if you have integration tests in your project, and you want to perform tests on a database for example. You will find information how to use it in the documentation . Currently, the plugin supports: Postgresql MySQL Redis Mongo Elasticsearch RabbitMQ And recently, we have added support for DynamoDB . Here , you will find how to run DynamoDB on your computer. And, here we are. If you want to use it in production, you want to test it locally. dynamodb fixture If you still do not use pytest, go to the pytest page and read how to use it, now. If you want to

Introduction I have been using Vim for a long time, but recently I have started using PyCharm too. This tool has many proper features and plugins. I will show you some of them. You will find here many gifs instead of text. In this way, you can easily see how it works, how it looks, etc. Plugins   .ignore This plugin was written by my friend. Here is more info about it. It will generate new ignore files for you. BashSupport Supports syntax highlighting, rename refactoring, documentation lookup, inspections, quickfixes, etc. Dash A smart and simple plugin that provides keyboard shortcut access for Dash . KeyPromoter Shows you how easy you can make the same action using only a keyboard. Lua Support for LUA. Markdown Support for markdown. YAML/Ansible support I don't want to write again "Support for YAML/Ansible". Tips & Tricks You do not know what to do? Do not be scared! Just use TODO. You have access to Termin