Kafka and Kubernetes - How to "dump" topic?
Some time ago, I wanted to debug messages on a specific topic in Kafka , but I did not want to do that on production (we set up Kafka on Kubernetes using bitnami chart ). So, the question was, how can I do that? How can I debug something on production? It almost always a bad idea, so I wanted to "dump" a topic on my local machine. Something like "dump of topic" does not exist of course, because what does it mean in Kafka world? 🤔 But, you can create a consumer which will consume the topic from the beginning and pipe messages into a file; then using producer you can put these messages onto a specific topic on your local machine. Remember that my Kafka Cluster is deployed on Kubernetes Cluster. I can "ssh" onto Kafka pod, run command and then copy the file on my local machine. Or, I can just exec a command on a Kafka pod and pipe the output (messages) to my local machine and save the messages locally. Here is the comamnd: You tell kafka-0 pod that you ...