Dung (Donny) Nguyen

Senior Software Engineer

Kubernetes Overview

Kubernetes (often shortened to K8s) is an open-source platform for running containerized applications across a cluster of machines. Docker answers the question “how do I package and run one container?” Kubernetes answers the next one: “how do I run hundreds of containers across many servers, keep them healthy, scale them, and update them without downtime?”

Kubernetes was originally designed at Google, drawing on its internal Borg system, and was donated to the Cloud Native Computing Foundation (CNCF) in 2015. Today it is the de facto standard for container orchestration and is offered as a managed service by every major cloud: Amazon EKS, Google GKE, and Azure AKS.


Why Kubernetes?

Running a few containers on one host with docker run or Docker Compose works well for development. In production, new problems appear:

Kubernetes solves these with a declarative model: you describe the desired state (“run 3 replicas of this image, exposed on port 80”), and Kubernetes continuously works to make the actual state match it.

Capability What Kubernetes Does
Self-healing Restarts failed containers, replaces Pods on failed nodes, kills containers that fail health checks
Scheduling Places containers on nodes based on CPU/memory requests, affinity rules, and constraints
Horizontal scaling Scales replicas manually or automatically based on CPU, memory, or custom metrics
Service discovery & load balancing Gives a stable DNS name and virtual IP to a group of Pods
Rolling updates & rollbacks Replaces Pods incrementally and reverts if the new version is unhealthy
Configuration & secrets Injects config and credentials without rebuilding images
Storage orchestration Attaches cloud disks, network storage, or local volumes to Pods

High-Level Architecture

A Kubernetes cluster consists of a control plane (the brain) and a set of worker nodes (the muscle) that run your workloads.

                    kubectl / CI pipelines / API clients
                                  |
                                  v
  +------------------------- Control Plane --------------------------+
  |                                                                  |
  |   [ kube-apiserver ] <------> [ etcd ]                           |
  |         ^      ^                                                 |
  |         |      |                                                 |
  |   [ scheduler ]  [ controller-manager ]  [ cloud-controller-mgr ]|
  +------------------------------------------------------------------+
                 |                        |
                 v                        v
  +---------- Worker Node 1 ----------+  +---------- Worker Node 2 ----------+
  |  [ kubelet ]   [ kube-proxy ]     |  |  [ kubelet ]   [ kube-proxy ]     |
  |  [ container runtime (containerd)]|  |  [ container runtime (containerd)]|
  |   Pod   Pod   Pod                 |  |   Pod   Pod                       |
  +-----------------------------------+  +-----------------------------------+

Control Plane Components

1. kube-apiserver

The API server is the front door of the cluster. Every interaction, whether from kubectl, a CI pipeline, or an internal component, goes through its REST API. It:

It is the only component that talks to etcd directly. All other components watch the API server for changes.

2. etcd

etcd is a consistent, highly available key-value store that holds the entire cluster state: every Pod, Service, ConfigMap, and Secret. It uses the Raft consensus algorithm, so production clusters typically run 3 or 5 etcd members to tolerate failures. If etcd is lost without a backup, the cluster’s state is lost.

3. kube-scheduler

The scheduler watches for newly created Pods that have no node assigned and picks the best node for each one. It filters out nodes that can’t run the Pod (insufficient resources, taints, node selectors) and then scores the remaining ones (spreading, affinity, resource balance).

4. kube-controller-manager

The controller manager runs the built-in controllers: control loops that watch the cluster state and push it toward the desired state. Examples include:

5. cloud-controller-manager

On cloud providers, the cloud controller manager integrates with the provider’s APIs, for example creating a cloud load balancer for a LoadBalancer Service or attaching block storage volumes.


Worker Node Components

1. kubelet

The kubelet is an agent running on every node. It receives Pod specs from the API server, tells the container runtime to start the containers, runs liveness and readiness probes, and reports node and Pod status back to the control plane.

2. Container Runtime

The container runtime actually runs containers. Kubernetes talks to it via the Container Runtime Interface (CRI). The most common runtimes are containerd and CRI-O. Kubernetes removed its built-in Docker integration (dockershim) in v1.24, but images built with Docker still run unchanged, because they are standard OCI images. (See Docker Architecture for how containerd and runc fit together.)

3. kube-proxy

kube-proxy maintains network rules (iptables, IPVS, or nftables) on each node so that traffic sent to a Service’s virtual IP is forwarded to one of its healthy Pods. Some CNI plugins, such as Cilium, can replace kube-proxy entirely using eBPF.


Core Kubernetes Objects

Everything in Kubernetes is an object (also called a resource) described in YAML or JSON and stored through the API server.

Pod

A Pod is the smallest deployable unit. It wraps one or more containers that share the same network namespace (one IP address) and can share volumes. Most Pods contain a single application container, sometimes with sidecar containers for logging, proxies, or service mesh.

Pods are ephemeral: when one dies, it is replaced by a new Pod with a new IP. You rarely create Pods directly; you use higher-level controllers instead.

Deployment and ReplicaSet

A Deployment manages stateless applications. It creates a ReplicaSet, which keeps a specified number of identical Pods running. Changing the Deployment (for example, a new image tag) triggers a rolling update: a new ReplicaSet scales up while the old one scales down.

Service

A Service gives a group of Pods (selected by labels) a stable virtual IP and DNS name, and load balances across them.

Type Description
ClusterIP Default. Reachable only inside the cluster.
NodePort Exposes the Service on a static port on every node.
LoadBalancer Provisions an external cloud load balancer.
ExternalName Maps the Service to an external DNS name.

Ingress and Gateway API

An Ingress routes external HTTP(S) traffic to Services based on host names and paths, handled by an ingress controller (such as NGINX or Traefik). The newer Gateway API is the successor to Ingress, with a richer, role-oriented model (GatewayClass, Gateway, HTTPRoute).

ConfigMap and Secret

A ConfigMap stores non-sensitive configuration (environment variables, config files). A Secret stores sensitive data such as passwords and tokens. Secrets are only base64-encoded by default, so enable encryption at rest in etcd and restrict access with RBAC, or use an external secrets manager.

Namespace

A Namespace divides one cluster into virtual sub-clusters, for example per team or per environment. Resource quotas, network policies, and RBAC rules can be scoped to a namespace.

Other Workload Types

Object Use Case
StatefulSet Stateful apps (databases, Kafka) needing stable network IDs and persistent storage per replica
DaemonSet Runs one Pod on every (or selected) node, e.g., log collectors, monitoring agents
Job Runs Pods to completion, e.g., a batch task or a database migration
CronJob Runs Jobs on a schedule

Storage Objects


A Simple Example

The manifest below runs three replicas of an NGINX web server and exposes them inside the cluster:

apiVersion: apps/v1
kind: Deployment
metadata:
  name: web
  labels:
    app: web
spec:
  replicas: 3
  selector:
    matchLabels:
      app: web
  template:
    metadata:
      labels:
        app: web
    spec:
      containers:
        - name: nginx
          image: nginx:1.27
          ports:
            - containerPort: 80
          resources:
            requests:
              cpu: "100m"
              memory: "128Mi"
            limits:
              memory: "256Mi"
          readinessProbe:
            httpGet:
              path: /
              port: 80
          livenessProbe:
            httpGet:
              path: /
              port: 80
---
apiVersion: v1
kind: Service
metadata:
  name: web
spec:
  type: ClusterIP
  selector:
    app: web
  ports:
    - port: 80
      targetPort: 80

Apply it and inspect the result:

kubectl apply -f web.yaml
kubectl get deployments
kubectl get pods -l app=web -o wide
kubectl get service web

Notice that the Service finds its Pods through the label selector app: web, not through Pod names or IPs. Labels are the glue that connects Kubernetes objects.


What Happens When You Run kubectl apply

  1. kubectl sends the Deployment manifest to the kube-apiserver.
  2. The API server authenticates, authorizes, validates, and stores the Deployment in etcd.
  3. The Deployment controller sees the new Deployment and creates a ReplicaSet.
  4. The ReplicaSet controller sees that 0 of 3 Pods exist and creates 3 Pod objects (with no node yet).
  5. The scheduler notices the unscheduled Pods and assigns each one to a node.
  6. The kubelet on each chosen node sees a Pod assigned to it and asks the container runtime to pull the image and start the container.
  7. The kubelet runs the readiness probe. Once it passes, the Pod is marked Ready.
  8. The EndpointSlice controller adds the Ready Pods to the Service, and kube-proxy updates the node’s routing rules so traffic can reach them.

No component tells another what to do directly. Each one watches the API server and reacts to changes. This loosely coupled, reconciliation-based design is what makes Kubernetes resilient and extensible.


Common kubectl Commands

Command Purpose
kubectl get pods -A List Pods in all namespaces
kubectl describe pod <name> Show details and recent events for a Pod
kubectl logs <pod> [-c container] [-f] View (or follow) container logs
kubectl exec -it <pod> -- sh Open a shell inside a container
kubectl apply -f <file.yaml> Create or update resources declaratively
kubectl delete -f <file.yaml> Delete resources defined in a file
kubectl scale deployment <name> --replicas=5 Change the replica count
kubectl rollout status deployment/<name> Watch a rolling update
kubectl rollout undo deployment/<name> Roll back to the previous version
kubectl port-forward svc/<name> 8080:80 Forward a local port to a Service
kubectl config get-contexts List clusters/contexts in your kubeconfig

Scaling and Resilience


The Kubernetes Ecosystem

Kubernetes is intentionally a platform for building platforms. Much of what teams use day-to-day comes from the ecosystem:

Area Popular Tools
Packaging Helm, Kustomize
GitOps / CD Argo CD, Flux
Networking (CNI) Cilium, Calico, AWS VPC CNI
Service mesh Istio, Linkerd
Observability Prometheus, Grafana, OpenTelemetry
Policy & security OPA Gatekeeper, Kyverno, Falco
Local clusters kind, minikube, k3d, Docker Desktop

Kubernetes is also extensible through Custom Resource Definitions (CRDs) and the Operator pattern: you define new object types (such as PostgresCluster) and write controllers that manage them using the same reconciliation loop as the built-in controllers.


When Kubernetes Is (and Isn’t) the Right Choice

Good fit:

Maybe overkill:

Managed Kubernetes services (EKS, GKE, AKS) remove much of the control plane burden, but you still own node management, upgrades, networking, and security configuration.


Summary

Kubernetes turns a group of machines into one platform for running containers:

Once you understand Pods, Deployments, Services, and the reconciliation loop, the rest of Kubernetes builds naturally on those ideas.