About Me

My photo
I am MCSE in Data Management and Analytics with specialization in MS SQL Server and MCP in Azure. I have over 13+ years of experience in IT industry with expertise in data management, Azure Cloud, Data-Canter Migration, Infrastructure Architecture planning and Virtualization and automation. Contact me if you are looking for any sort of guidance in getting your Infrastructure provisioning automated through Terraform. I sometime write for a place to store my own experiences for future search and read by own blog but can hopefully help others along the way. Thanks.

Stateful sets in K8S

 A StatefulSet is a Kubernetes resource used to manage stateful applications, such as databases. It provides a mechanism for deploying, scaling, and managing stateful applications in a Kubernetes cluster. When it comes to scaling a database, there are several reasons why a StatefulSet might be the preferred choice.


Firstly, a StatefulSet provides stable network identities for each replica of a database. Each replica can be given a unique hostname and IP address that can be used to reference it within the Kubernetes cluster. This is important for applications that require stable and predictable network identities, such as databases that need to maintain persistent connections with clients.


Secondly, a StatefulSet ensures that replicas are scaled up or down in a specific order. This is important for stateful applications, such as databases, that require a specific initialization and shutdown sequence. By scaling replicas in a specific order, a StatefulSet can ensure that the database is correctly initialized and shut down without causing data loss or other issues.


Thirdly, a StatefulSet allows for the use of persistent volumes. Persistent volumes are used to store data outside of the container that runs the database. This ensures that data is not lost when containers are restarted or scaled up or down. Persistent volumes can be used to store data on local disks, network-attached storage, or cloud storage, depending on the needs of the application.


Finally, a StatefulSet allows for the use of headless services. Headless services are used to expose the network identities of each replica of a StatefulSet. This allows other applications within the Kubernetes cluster to connect to specific replicas of the database directly, without going through a load balancer or other intermediary. This is important for applications that require direct access to specific replicas of the database, such as read replicas or sharded databases.


In summary, a StatefulSet is a Kubernetes resource that is used to manage stateful applications, such as databases. It provides stable network identities, ensures correct scaling order, allows for the use of persistent volumes, and enables the use of headless services. These features make StatefulSets a good choice for scaling and managing databases in a Kubernetes cluster.


Example of installing mongodb with statefulset using static storage account

# Set variables
namespace=mongodb
statefulsetname=mongodb-statefulset
storagename=mongodb-storage
storageclass=azure-disk
replicas=1
mongodbusername=admina
mongodbpassword=Password@123


cat <<EOF | kubectl apply -f -
# Create a Namespace mongodb-namespace
apiVersion: v1
kind: Namespace
metadata:
  name: $namespace
EOF

cat <<EOF | kubectl apply -f -
kind: StorageClass
apiVersion: storage.k8s.io/v1
metadata:
  name: $storageclass
  namespace: $namespace 
provisioner: kubernetes.io/azure-disk
parameters:
  storageaccounttype: Standard_LRS
  kind: Managed
EOF


  

# Create MongoDB StatefulSet
cat <<EOF | kubectl apply -f -
apiVersion: apps/v1
kind: StatefulSet
metadata:
  name: $statefulsetname
  namespace: $namespace
spec:
  serviceName: $statefulsetname
  replicas: $replicas
  selector:
    matchLabels:
      app: mongodb
  template:
    metadata:
      labels:
        app: mongodb
    spec:
      containers:
      - name: mongodb
        image: mongo:latest
        env:
        - name: MONGO_INITDB_ROOT_USERNAME
          value: $mongodbusername
        - name: MONGO_INITDB_ROOT_PASSWORD
          value: $mongodbpassword
        ports:
        - containerPort: 27017
        volumeMounts:
        - name: $storagename
          mountPath: /data/db
  volumeClaimTemplates:
  - metadata:
      name: $storagename
    spec:
      storageClassName: $storageclass
      accessModes:
      - ReadWriteOnce
      resources:
        requests:
          storage: 5Gi
EOF

# Create MongoDB StatefulSet explanation

This is a Kubernetes YAML file that creates a StatefulSet to run MongoDB in a Kubernetes cluster. A StatefulSet is a Kubernetes object that provides guarantees about the ordering and uniqueness of pods in a Kubernetes cluster. MongoDB is a popular NoSQL database management system.

The YAML file specifies the following attributes for the StatefulSet:

Name: The name of the StatefulSet, which is provided as a parameter to the script.

Namespace: The namespace where the StatefulSet will be created, which is also provided as a parameter.

Service name: The name of the service associated with the StatefulSet, which is the same as the StatefulSet name.

Replicas: The number of replicas of MongoDB that will be created in the StatefulSet, which is also provided as a parameter.

Selector: The labels used to identify the MongoDB pods managed by the StatefulSet.

Template: The pod template used to create MongoDB pods in the StatefulSet. It contains the following:

 Labels: Used to identify the pods created by the StatefulSet.

 Containers: The container specification for MongoDB, which includes the following:

   Name: The name of the container.

   Image: The MongoDB Docker image to use.

  Env: The environment variables to set in the container, which includes the MongoDB root username and password.

  Ports: The port used by MongoDB.

  VolumeMounts: The storage volume to mount to store MongoDB data.

VolumeClaimTemplates: The storage volume template used to create persistent volumes for MongoDB data. It contains the following:

 Metadata: The name of the volume to be created.

 Spec: The specification of the storage volume, which includes the storage class, access modes, and storage requests.

Overall, this YAML file creates a scalable and persistent MongoDB deployment in a Kubernetes cluster, with authentication enabled using a root username and password, and 5GB of storage allocated for each MongoDB pod.

# Create MongoDB Headless service
cat <<EOF | kubectl apply -f -
apiVersion: v1
kind: Service
metadata:
  name: $statefulsetname
  namespace: $namespace
  labels:
    name: mongo
spec:
  ports:
  - port: 27017
    targetPort: 27017
  clusterIP: None
  selector:
    app: mongodb
EOF

Q:- why we create a headless service 

Ans:-why we create a headless service in statefulsets

In a StatefulSet, each pod has a unique and stable hostname that is based on the StatefulSet name and the ordinal index of the pod. For example, a StatefulSet with a name of "mongodb" and replicas set to 3 would create pods with hostnames "mongodb-0", "mongodb-1", and "mongodb-2". This allows applications running inside the pod to use the hostname to connect to other pods in the StatefulSet, rather than relying on IP addresses that may change if pods are added or removed.

By default, Kubernetes creates a cluster IP service for a StatefulSet, which forwards traffic to the pods using their stable hostnames. However, in some cases, it may be useful to have direct access to each pod's hostname, without going through the service. This is where headless services come in.

A headless service is a Kubernetes service that is created with the "clusterIP: None" option. This means that the service does not have a virtual IP address, and instead returns the DNS name of each pod that matches the service selector. In the case of a StatefulSet, a headless service returns the stable hostnames of each pod in the StatefulSet, allowing applications to connect directly to each pod.

There are several reasons why a headless service may be useful in a StatefulSet:

Load balancing: A headless service allows applications to connect directly to each pod, which can be useful for load balancing or distributing workloads across multiple pods.

Stateful application requirements: Some stateful applications require direct access to each pod's hostname in order to function properly.

Database sharding: In databases that support sharding, a headless service can be used to shard data across multiple pods based on their stable hostnames.

Overall, a headless service in a StatefulSet allows for more direct and fine-grained control over how applications connect to pods, and can be useful in a variety of scenarios.

Q:- what if i set replicaset to 3, will  this static storageClass support  replicaset to 3. if not how can we change replicaset to 3. 

ANS:--- ???  



What is the name of the StatefulSet?

How many replicas of the StatefulSet do you want to create?

What is the name of the pod template that will be used to create each replica?

What is the name of the container that will be used to run the database software?

What is the name of the Docker image that will be used to run the container?

What is the command that should be run inside the container when it starts?

What are the arguments that should be passed to the command?

How much CPU and memory resources should be allocated to each replica?

What are the persistent volume claims that should be used by each replica?

What are the names of the services that will be created to provide access to the StatefulSet?

What is the name of the headless service that will be created to provide network identities for each replica?

What is the name of the service that will be created to provide a load-balanced endpoint for the StatefulSet?

What is the port number that the database software listens on?

What is the name of the secret that should be used to store any sensitive configuration data or credentials required by the database software?

What is the name of the ConfigMap that should be used to store any configuration data required by the database software?





No comments: