mongodb / docs-kafka-connector Goto Github PK

The docs reference Apache Zookeeper which manages ... at https://www.mongodb.com/docs/kafka-connector/current/quick-start/

However, it is now possible to run Kafka without Zookeeper, which means:

• 1 container test instance
• 3 container production cluster, as opposed to 3 physical machines or 9 VMs as was the case before

Starter docs are at https://www.confluent.io/blog/kafka-without-zookeeper-a-sneak-peek/

Hi all,

My current MongoDB and Kafka environments are hosted within Kubernetes pods. To interact with these applications, we first need to authenticate with the Kubernetes cluster and then establish a port-forwarding session.

To achieve this, I'm using two separate commands in our local terminal environment (note: the following values have been altered for security purposes):

In Terminal 1, we set up the connection to the database by executing kubeConnection.sh.

I have defined environment variables for the database port, endpoint, and the name of the application service. I then use kubectl to run a temporary pod using a specific Docker image, which listens on the designated port and forwards TCP connections to the MongoDB endpoint:

export DEV_DB_PORT=27017
export DEV_DB_ENDPOINT=db.endpoint
export DEV_APP_SERVICE_NAME=docdb-access
kubectl run ${DEV_APP_SERVICE_NAME} --image=dockerurl/docker/data-platforms/common/socat:latest -it --tty --rm --expose=true --port=${DEV_DB_PORT} tcp-listen:${DEV_DB_PORT},fork,reuseaddr tcp connect:${DEV_DB_ENDPOINT}:${DEV_DB_PORT}

In Terminal 2, I establish the port-forwarding session with kubePortForward.sh. Utilizing the same environment variables for consistency, I've forwarded the local port to the service port in the Kubernetes cluster, allowing direct access to the database through the local machine:

export DEV_DB_PORT=27017
export DEV_DB_ENDPOINT=db.endpoint
export DEV_APP_SERVICE_NAME=docdb-access
kubectl port-forward service/${DEV_APP_SERVICE_NAME} ${DEV_DB_PORT}:${DEV_DB_PORT}

Thank you all,
Michael Mendy