How to Setup Kubernetes on Linux Server – Complete Guide

To set up Kubernetes on a Linux server, prepare supported hosts, install a container runtime (containerd), disable swap, configure kernel networking, install kubelet/kubeadm/kubectl, initialize the control plane with kubeadm, deploy a CNI (e.g., Calico), and join worker nodes. Validate with kubectl get nodes and secure the cluster with sensible firewall and RBAC settings.

In this guide, you’ll learn How to Setup Kubernetes on Linux Server the right way—using kubeadm, containerd, and a production-ready networking stack. We’ll cover prerequisites, step-by-step installation on Ubuntu/Debian and RHEL derivatives, common variations, best practices, troubleshooting, and FAQs based on real-world hosting experience.

Why This Guide and Who It’s For

This tutorial targets beginners and sysadmins who want a stable, repeatable Kubernetes cluster on bare-metal or VPS. It reflects current Kubernetes packaging, CRI containerd defaults, and networking settings that pass production smell tests at hosting scale.

Prerequisites and System Requirements

Supported Linux distributions

Use maintained, recent LTS versions. Common choices:

• Ubuntu 22.04/24.04 LTS
• Debian 12 (Bookworm)
• Rocky Linux 9 / AlmaLinux 9 / RHEL 9

Minimum resources per node

• CPU: 2 vCPU (4+ recommended for control-plane)
• RAM: 4 GB minimum (8+ GB recommended for control-plane)
• Disk: 30+ GB free SSD
• Network: Stable, low-latency, outbound internet for images/manifests

Networking and ports

• Control-plane (TCP): 6443 (API server), 2379-2380 (etcd), 10250, 10257, 10259
• Worker nodes (TCP): 10250, NodePort range 30000–32767
• Allow intra-cluster traffic between nodes; disable conflicting firewalls or open the ports above.

Step-by-Step: Kubernetes Setup on Linux with kubeadm

We’ll use containerd (recommended CRI), kubeadm for bootstrapping, and Calico for CNI. Repeat the pre-steps on all nodes. Run init on the control-plane node only.

1) Set hostname, update packages, and time sync

# Set unique hostnames on each node
sudo hostnamectl set-hostname cp-1   # control-plane example
# sudo hostnamectl set-hostname worker-1  # worker example

# Update system packages
sudo apt update && sudo apt -y upgrade    # Ubuntu/Debian
# sudo dnf -y update                      # RHEL/Rocky/Alma

# Install NTP/time sync (Ubuntu/Debian)
sudo apt -y install chrony
sudo systemctl enable --now chronyd
# RHEL/Rocky/Alma typically have chrony enabled by default

2) Disable swap and configure kernel networking

Kubernetes requires swap off. Enable required kernel modules and sysctl for bridged traffic and forwarding.

# Disable swap now and persist across reboots
sudo swapoff -a
sudo sed -i.bak '/\sswap\s/s/^/#/' /etc/fstab

# Load kernel modules
cat <<EOF | sudo tee /etc/modules-load.d/k8s.conf
overlay
br_netfilter
EOF
sudo modprobe overlay
sudo modprobe br_netfilter

# Sysctl settings for Kubernetes networking
cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables  = 1
net.bridge.bridge-nf-call-ip6tables = 1
net.ipv4.ip_forward                 = 1
EOF
sudo sysctl --system

3) Install containerd (CRI) and set systemd cgroups

Containerd is the recommended runtime. Configure it to use systemd cgroups to match kubelet defaults.

# Ubuntu/Debian
sudo apt -y install curl gnupg2 software-properties-common ca-certificates apt-transport-https
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/$(. /etc/os-release; echo $ID)/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo \
  "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/$(. /etc/os-release; echo $ID) \
  $(. /etc/os-release; echo $VERSION_CODENAME) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null

sudo apt update
sudo apt -y install containerd.io

# Generate default config and switch to systemd cgroups
sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml > /dev/null
sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml

sudo systemctl enable --now containerd

# RHEL/Rocky/Alma
sudo dnf -y install yum-utils device-mapper-persistent-data lvm2
sudo dnf config-manager --add-repo https://download.docker.com/linux/centos/docker-ce.repo
sudo dnf -y install containerd.io

sudo mkdir -p /etc/containerd
containerd config default | sudo tee /etc/containerd/config.toml > /dev/null
sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml

sudo systemctl enable --now containerd

# Ensure SELinux has container-selinux installed (preferred) or set permissive if you must:
# sudo dnf -y install container-selinux
# sudo setenforce 0 && sudo sed -i 's/^SELINUX=enforcing/SELINUX=permissive/' /etc/selinux/config

4) Install kubelet, kubeadm, and kubectl

Install the Kubernetes packages from the official repositories (pkgs.k8s.io). Repeat on all nodes.

# Ubuntu/Debian (example uses v1.30 stable channel)
sudo mkdir -p /etc/apt/keyrings
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.30/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.30/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list

sudo apt update
sudo apt -y install kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl
sudo systemctl enable --now kubelet

# RHEL/Rocky/Alma
cat <<EOF | sudo tee /etc/yum.repos.d/kubernetes.repo
[kubernetes]
name=Kubernetes
baseurl=https://pkgs.k8s.io/core:/stable:/v1.30/rpm/
enabled=1
gpgcheck=1
gpgkey=https://pkgs.k8s.io/core:/stable:/v1.30/rpm/repodata/repomd.xml.key
EOF

sudo dnf -y install kubelet kubeadm kubectl
sudo systemctl enable --now kubelet

5) Initialize the control-plane

Run this on your control-plane node only. Choose a Pod CIDR that your CNI supports. Calico defaults to 192.168.0.0/16.

sudo kubeadm init --pod-network-cidr=192.168.0.0/16

# Set kubeconfig for your non-root user
mkdir -p $HOME/.kube
sudo cp -i /etc/kubernetes/admin.conf $HOME/.kube/config
sudo chown $(id -u):$(id -g) $HOME/.kube/config

Note the kubeadm join command printed at the end. You’ll use it on worker nodes.

6) Install a CNI plugin (Calico example)

Apply a networking add-on so Pods can communicate. Calico is a popular choice for policy and performance.

# From the control-plane with kubectl configured
kubectl apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.27.3/manifests/calico.yaml

# Wait for core components to become Ready
kubectl get pods -n kube-system -w

7) Join worker nodes

On each worker node, paste the exact join command with token and discovery-hash from kubeadm init output.

sudo kubeadm join <CONTROL_PLANE_IP>:6443 --token <TOKEN> \
  --discovery-token-ca-cert-hash sha256:<HASH>

8) Validate the cluster

# On the control-plane
kubectl get nodes
kubectl get pods -A

# Test deployment
kubectl create deployment nginx --image=nginx:stable
kubectl expose deployment nginx --port=80 --type=NodePort
kubectl get svc nginx -o wide

Common Variations and Choices

Ubuntu/Debian vs RHEL-like

• Package managers differ (apt vs dnf), but kubeadm flow is identical.
• SELinux: Keep enforcing with container-selinux; permissive only if necessary.
• Firewalld/UFW: Open Kubernetes ports or temporarily disable during testing.

CNI plugins (Calico, Flannel, Cilium)

• Calico: rich NetworkPolicy, BGP options, widely used.
• Flannel: simple overlay, easy for labs.
• Cilium: eBPF-based performance and security visibility.

Best Practices and Security Hardening

• Use distinct control-plane and worker nodes; taint control-plane to avoid app workloads.
• Pin minor Kubernetes version and plan quarterly upgrades; test in staging first.
• Restrict API access to trusted IPs; enforce RBAC and least privilege.
• Rotate join tokens and TLS certificates before expiry.
• Enable audit logs and monitor with a SIEM.
• Use encrypted etcd and secure backups; protect /etc/kubernetes and kubeconfig files.
• Scan container images, sign with SBOM/attestations, and use admission policies.

Troubleshooting Checklist

• kubelet not Ready: Check swap is off, containerd running, and SystemdCgroup=true.
• Pods stuck Pending: CNI not installed or wrong Pod CIDR. Verify calico or chosen plugin.
• Join failures: Ensure ports open (6443), correct token/hash, time sync, DNS working.
• CoreDNS CrashLoop: Verify cluster DNS, node resolv.conf, and CNI status.
• Network policies block traffic: Temporarily remove policies to isolate the issue.

# Quick diagnostics
sudo systemctl status kubelet containerd
journalctl -u kubelet -f
kubectl get nodes -o wide
kubectl get pods -A -o wide
kubectl describe node <name>
kubectl describe pod <name> -n <ns>

Scaling and High Availability (Optional)

For HA, deploy 3 control-plane nodes behind a TCP load balancer on port 6443 and externalize etcd or use stacked etcd with odd quorum. Automate provisioning with configuration management (Ansible/Terraform) and store cluster state backups securely.

When to Choose Managed or Provider-Assisted Kubernetes

If you prefer to focus on workloads instead of bootstrap and lifecycle tasks, consider managed control planes or provider-assisted clusters. At YouStable, our Kubernetes-ready VPS and Bare Metal give you fast NVMe, premium bandwidth, and 24×7 support to help you initialize kubeadm, tune networking, and scale with confidence.

Full Command Recap (Ubuntu/Debian)

# 1) Swap, modules, sysctl
sudo swapoff -a && sudo sed -i.bak '/\sswap\s/s/^/#/' /etc/fstab
echo -e "overlay\nbr_netfilter" | sudo tee /etc/modules-load.d/k8s.conf
sudo modprobe overlay && sudo modprobe br_netfilter
cat <<EOF | sudo tee /etc/sysctl.d/99-kubernetes-cri.conf
net.bridge.bridge-nf-call-iptables=1
net.bridge.bridge-nf-call-ip6tables=1
net.ipv4.ip_forward=1
EOF
sudo sysctl --system

# 2) containerd
sudo apt update && sudo apt -y install ca-certificates curl gnupg apt-transport-https
sudo install -m 0755 -d /etc/apt/keyrings
curl -fsSL https://download.docker.com/linux/$(. /etc/os-release; echo $ID)/gpg | sudo gpg --dearmor -o /etc/apt/keyrings/docker.gpg
echo "deb [arch=$(dpkg --print-architecture) signed-by=/etc/apt/keyrings/docker.gpg] https://download.docker.com/linux/$(. /etc/os-release; echo $ID) $(. /etc/os-release; echo $VERSION_CODENAME) stable" | sudo tee /etc/apt/sources.list.d/docker.list > /dev/null
sudo apt update && sudo apt -y install containerd.io
sudo mkdir -p /etc/containerd && containerd config default | sudo tee /etc/containerd/config.toml > /dev/null
sudo sed -i 's/SystemdCgroup = false/SystemdCgroup = true/' /etc/containerd/config.toml
sudo systemctl enable --now containerd

# 3) kubeadm, kubelet, kubectl
curl -fsSL https://pkgs.k8s.io/core:/stable:/v1.30/deb/Release.key | sudo gpg --dearmor -o /etc/apt/keyrings/kubernetes-apt-keyring.gpg
echo 'deb [signed-by=/etc/apt/keyrings/kubernetes-apt-keyring.gpg] https://pkgs.k8s.io/core:/stable:/v1.30/deb/ /' | sudo tee /etc/apt/sources.list.d/kubernetes.list
sudo apt update && sudo apt -y install kubelet kubeadm kubectl
sudo apt-mark hold kubelet kubeadm kubectl
sudo systemctl enable --now kubelet

# 4) Control-plane init
sudo kubeadm init --pod-network-cidr=192.168.0.0/16
mkdir -p $HOME/.kube && sudo cp /etc/kubernetes/admin.conf $HOME/.kube/config && sudo chown $(id -u):$(id -g) $HOME/.kube/config
kubectl apply -f https://raw.githubusercontent.com/projectcalico/calico/v3.27.3/manifests/calico.yaml
kubectl get nodes

Conclusion

Now you know How to Setup Kubernetes on Linux Server using kubeadm and containerd, from kernel prerequisites to a functioning CNI. Keep versions pinned, automate your steps, and harden security. If you need performance-tuned nodes and expert help, YouStable’s Kubernetes-ready servers can accelerate your rollout.

FAQs: How to Setup Kubernetes on Linux Server