Python/DevOps

Some time ago, I wanted to debug messages on a specific topic in Kafka , but I did not want to do that on production (we set up Kafka on Kubernetes using bitnami chart ). So, the question was, how can I do that? How can I debug something on production? It almost always a bad idea, so I wanted to "dump" a topic on my local machine. Something like "dump of topic" does not exist of course, because what does it mean in Kafka world? 🤔 But, you can create a consumer which will consume the topic from the beginning and pipe messages into a file; then using producer you can put these messages onto a specific topic on your local machine. Remember that my Kafka Cluster is deployed on Kubernetes Cluster. I can "ssh" onto Kafka pod, run command and then copy the file on my local machine. Or, I can just exec a command on a Kafka pod and pipe the output (messages) to my local machine and save the messages locally. Here is the comamnd: You tell kafka-0 pod that you ...

Hi, today I am going to introduce Pluto to you. So, what is it? 🤔 Pluto is a utility to help users find deprecated Kubernetes apiVersions in their code repositories and their helm releases. Sounds great! You use Kubernetes right? So, you know that Kubernetes has Deprecation Policy . It is a large system, so many features evolve over time and many of them might be removed. Thus, to avoid problems, Kubernetes provides a deprecation policy. So, you know in advance that something is going to be deprecated and you should act! There are 3 main " API Groups " in Kubernetes: Alpha (v1alpha1) Beta (v1beta1) Stable (v1) From time to time, some objects (or their versions) in Kubernetes can be removed or deprecated and it would be nice to know about it in advance. And for that job you can use: Let's see this guy in an action! You need to install it firstly of course. You can do that by this command: I used minikube ( minikube start --kubernetes-version=v1.16.1 ) to set up a...

Ahh, yes!  You have reached your pull rate limit. You may increase the limit by authenticating and upgrading: https://www.docker.com/increase-rate-limit I bet you had encountered this problem if you read now this post. According to this document : Beginning November 2, 2020, progressive enforcement of rate limits for anonymous and authenticated Docker Hub usage goes into effect. This means that anonymous and free Docker Hub users will have usage restrictions gradually placed on container image pull requests. Sadly, this happened on our clusters (AWS EKS). How to fix it? I wanted to spawn DaemonSet object where I could run docker login command and this way change config.json  on every node. But, after that you need to restart docker  process on every node, and I still do not know how to do that on AWS EKS. So, a temporary fix was to create a Secret object and then link it to every ServiceAccount  object. It is "a hack", but we needed very fast working solution. We ...