A Data Caching Service in Kubernetes

In this article, I will try to explain building a data caching service with MongoDB and Redis deploying under a minikube environment. The web application is written using the Go programming language.

Image for post
Image for post

Github: https://github.com/Deeptiman/go-cache-kubernetes

Installation

The development environment should have Go as a programming language, MongoDB as persistence storage, and Redis as data caching service.

Note: The following installation of MongoDB, Redis Cache is only supported in the local development environment and the Kubernetes deployment environment for each module follows a different setup.

Go

Setup the GOPATH

MongoDB

The dependency module of MongoDB is required to install a mongo-go-driver that has all the Go APIs to query a MongoDB database.

Redis Cache

The go-redis Go client library will work as a dependency module to integrate the data caching service.

redis-server: The development environment also required to install a Redis server that will provide various data structures, caching mechanisms and a message broker.

Build the Docker images

The web application has been built under a docker container environment that will deploy into the Kubernetes.

Installation

Build the image

Tag the image

Login to Docker Hub

Push the image to Docker Hub

Kubernetes Tools

The minikube and kubectl two major Kubernetes tools that required to be installed in the local environment.

minikube will be running as a single-node Kubernetes cluster under a Virtual machine. The computer must support Virtualization and HyperVisor needs to be enabled. Hyperkit is a recommended virtualization kit to start the minikube.

kubectl is a command-line tool that will provide several useful commands to deploy, manage, and troubleshoot a Kubernetes Cluster.

Kubernetes Deployment

The web application will require different layers of deployment strategies to create a Kubernetes cluster running under a minikube environment.

Deployments of Web App

  1. PersistentVolumeClaim

The web application will require 1GB of the storage volume to integrate the docker container inside the Pod.

YAML: go-cache-poc-pvc.yaml

Image for post
Image for post
Persistent Volume Claims metadata of go-cache-kubernetes-pvc-v1-poc

2. Deployment

The web application will run as a Deployment inside a Kubernetes cluster.

YAML: go-cache-poc-app.yaml

Check the Deployment status

Image for post
Image for post
Deployment metadata of go-cache-kubernetes-app-poc

3. Service (LoadBalancer)

There will be LoadBalancer running as a Service to create an endpoint to connect with external traffic.

YAML: go-cache-poc-svc.yaml

Image for post
Image for post
Service runs as LoadBalancer for go-cache-kubernetes-svc-poc

After the Service created successfully, in another terminal execute the following command.

The command requires root privileges and it will create a network route to allow the host to connect to external traffic via a Cluster IP gateway.

MongoDB Replica Set as a Kubernetes StatefulSet

The Kubernetes provides an architecture to create N-replicas with a unique identifier for all kinds of stateful applications. A brief explanation about StatefulSet and the configuration setup of MongoDB as StatefulSet in Kubernetes that you can find in my other medium article.

Image for post
Image for post

Secret Management in Kubernetes

This web application also uses the Secret management technique to store MongoDB credentials. I have explained the Secret management setup for both Kubernetes and HashiCorp Vault in my other medium article.

Image for post
Image for post

Deploy Redis in Kubernetes

The Redis docker image is required to be run in the local environment.

Redis Deployment

YAML: redis-deployment.yaml

Image for post
Image for post
Deployment metadata of redismod

Redis Service

YAML: redis-service.yaml

Image for post
Image for post
Service metadata of redis-service

Deploy the redismod image into the Kubernetes

Expose the deployment

Now verify the Redis connection

You must be getting an ip address with a port that can be used as a connection string for Redis.

Image for post
Image for post
minikube dashboard with all kinds in running status

Troubleshoot with kubectl

kubectl tool can be used debug, analyze Pods, Services in the Kubernetes cluster.

Few useful commands

  1. kubectl get pods
  2. kubectl describe pods
  3. kubectl logs
  4. kubectl exec -ti <pod-name> — bash

So, this is the overview of building a Data Caching Service in Kubernetes using Microservice architecture. If you are a beginner then, this project will help you get started with Kubernetes and understanding deployment in a minikube environment.

I hope you find this article useful :)

Thanks

Written by

Working on various software development projects, [Android, Go, Blockchain, Node.js, MongoDB, JavaScript, Kubernetes, Docker]

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store