Setup ETCD CLUSTER

How To Setup etcd Clusters

To begin with - Etcd is a distributed, reliable key-value store designed to securely store configuration data and state information for distributed systems like Kubernetes.

It acts as the backbone of the Kubernetes control plane, ensuring that all components within the cluster are in sync and that the desired state of the cluster is maintained.

How ETCD works:

Etcd operates using a consensus algorithm called RAFT to maintain consistency across the distributed nodes.

In an etcd cluster, one node is elected as the leader, while the others are followers. The leader handles all write requests and propagates these changes to the followers to ensure data consistency across the cluster.

If the leader fails, a new leader is elected from the remaining nodes to maintain cluster operations without downtime.

Deployment Types:

Stacked etcd cluster - etcd instances run on the same nodes as the Kubernetes control plane components. This setup is simple but offers less resilience in the event of node failures.

This is generally suitable for smaller environments or development clusters where ease of setup and management is prioritized over high availability.

External etcd cluster - etcd runs on dedicated nodes separate from the control plane, offering enhanced resilience and fault tolerance.

This setup enhances resilience and fault tolerance, as failures in the control plane do not directly impact etcd, and vice versa.

It provides a higher level of availability, making it the preferred choice for production environments where maintaining cluster stability is crucial.

Practical Step-by-Step Guide:

Step 1. Download and Install etcd

wget -q --show-progress --https-only --timestamping \

"https://github.com/etcd-io/etcd/releases/download/v3.5.15/etcd-v3.5.15-linux-amd64.tar.gz"

tar -xvf etcd-v3.5.15-linux-amd64.tar.gz

sudo mv etcd-v3.5.15-linux-amd64/etcd* /usr/local/bin/

Step 2. Configure etcd

sudo mkdir -p /etc/etcd /var/lib/etcd

sudo cp ca.pem kubernetes-key.pem kubernetes.pem /etc/etcd/

Step 3. Setup Environment Variables

# Example for AWS

INTERNAL_IP=$(curl -s http://169.254.169.254/latest/meta-data/local-ipv4)

ETCD_NAME=$(hostname)

# Example for Azure

INTERNAL_IP=$(curl -H Metadata:true -s "http://169.254.169.254/metadata/instance/network/interface/0/ipv4/ipAddress/0/privateIpAddress?api-version=2021-02-01&format=text")

ETCD_NAME=$(hostname)

# Example for on-prem with static IPs

INTERNAL_IP=<Your_Static_IP>

ETCD_NAME=$(hostname -s)

Step 4. Create etcd Systemd Unit File

cat <<EOF | sudo tee /etc/systemd/system/etcd.service

[Unit]

[Service]

Type=notify

ExecStart=/usr/local/bin/etcd \\

--name ${ETCD_NAME} \\

--cert-file=/etc/etcd/kubernetes.pem \\

--key-file=/etc/etcd/kubernetes-key.pem \\

--peer-cert-file=/etc/etcd/kubernetes.pem \\

--peer-key-file=/etc/etcd/kubernetes-key.pem \\

--trusted-ca-file=/etc/etcd/ca.pem \\

--peer-trusted-ca-file=/etc/etcd/ca.pem \\

--peer-client-cert-auth \\

--client-cert-auth \\

--initial-advertise-peer-urls https://${INTERNAL_IP}:2380 \\

--listen-peer-urls https://${INTERNAL_IP}:2380 \\

--listen-client-urls https://${INTERNAL_IP}:2379,https://127.0.0.1:2379 \\

--advertise-client-urls https://${INTERNAL_IP}:2379 \\

--initial-cluster-token etcd-cluster-0 \\

--initial-cluster controller-0=https://10.240.0.10:2380,controller-1=https://10.240.0.11:2380,controller-2=https://10.240.0.12:2380 \\

--initial-cluster-state new \\

--data-dir=/var/lib/etcd

Restart=on-failure

RestartSec=5

[Install]

WantedBy=multi-user.target

EOF

Step 5. Start the etcd Service

sudo systemctl daemon-reload

sudo systemctl enable etcd

sudo systemctl start etcd

Step 6. Verify etcd Cluster Members

sudo ETCDCTL_API=3 etcdctl member list \

--endpoints=https://127.0.0.1:2379 \

--cacert=/etc/etcd/ca.pem \

--cert=/etc/etcd/kubernetes.pem \

--key=/etc/etcd/kubernetes-key.pem

Tips and Tricks:

Backup Regularly: Etcd stores critical Kubernetes data; regular backups are essential. Use the etcdctl snapshot save command to create backups.

Monitor Leader Election: Keep an eye on the leader election process using etcd logs. Frequent elections might indicate an unstable cluster.

Use TLS Encryption: Always configure TLS encryption for communication between etcd nodes and clients to secure the cluster.

Final reminder,

etcd is the backbone of Kubernetes and keeps everything in sync — get it right, and your Kubernetes stays solid.