Kafka is a powerful framework of message queue and more, so let’s install it on our local environment to test!
Our ingredients:
1 – Helm;
2 – Kubernetes;
3 – Helm Apache Kafka.
Let’s init our Helm support:
helm init
Let’s download a copy of the Helm Kafka
git clone https://github.com/confluentinc/cp-helm-charts.git
cp cp-helm-charts/values.yaml .
If you desire, you can expose kafka externally editing the values.yaml file and adding there to the cp-kafka part.
nodeport:
enabled: true
servicePort: 19092
firstListenerPort: 31090
configurationOverrides:
"offsets.topic.replication.factor": "3"
"advertised.listeners": |-
EXTERNAL://localhost:$((31090 + ${KAFKA_BROKER_ID}))
"listener.security.protocol.map": |-
PLAINTEXT:PLAINTEXT,EXTERNAL:PLAINTEXT
Let’s install kafka inside a namespace called kafka
helm install my-confluent -n kafka cp-helm-charts -f ./values.yaml
If everything is ok you can check:
kubectl get all -n kafka
We can use the kafka-client to manipulate the kafka, to install it:
cd cp-helm-chart/examples
kubectl apply -f kafka-client.yaml
To iniciate the client:
kubectl exec -it kafka-client -- /bin/bash
cd /usr/bin
To create a topic example on Kafka:
kafka-topics --zookeeper my-confluent-cp-zookeeper-headless:2181 --topic my-confluent-topic --create --partitions 1 --replication-factor 1 --if-not-exists
To put a message on kafka topic:
echo "hello world" | kafka-console-producer --broker-list my-confluent-cp-kafka-headless:9092 --topic my-confluent-topic
To consume a message of our topic:
kafka-console-consumer --bootstrap-server my-confluent-cp-kafka-headless:9092 --topic my-confluent-topic --from-beginning
Voila! our kafka is running perfectly!
Reference: https://tsuyoshiushio.medium.com/local-kafka-cluster-on-kubernetes-on-your-pc-in-5-minutes-651a2ff4dcde
English