Kubernetes Resource Allocation: Requests, Limits, QoS, and Quotas

Kubernetes does not simply "run containers" — it makes precise placement and enforcement decisions based on resource declarations. Getting these declarations right is the difference between a stable cluster and a cascade of OOMKills and evictions.

---

Node Allocatable Capacity

Before any pod is scheduled, each node publishes how much resource is actually available for workloads:

Node Capacity
  − kube-reserved   (kubelet, container runtime)
  − system-reserved (OS daemons, sshd, journald)
  − eviction-threshold
= Node Allocatable

kubectl describe node <node-name> | grep -A 8 Allocatable

Allocatable:
  cpu:               94
  memory:            755Gi
  nvidia.com/gpu:    8
  pods:              110

The scheduler only places pods on nodes where allocatable − Σ(existing pod requests) ≥ new pod requests.

---

Requests vs Limits

Every container should declare both values for each resource:

resources:
  requests:
    cpu: "500m"       # 0.5 core — scheduling currency
    memory: "256Mi"   # scheduling currency
  limits:
    cpu: "2"          # runtime cap — excess is throttled
    memory: "512Mi"   # runtime cap — exceed this → OOMKilled

	Requests	Limits
Used by	Scheduler (placement)	Kubelet (enforcement)
CPU exceed	—	Throttled (cgroup CFS)
Memory exceed	—	OOMKilled
Set too low	Pod can't schedule	Normal
Set too high	Wastes capacity	Prevents burst

CPU: Compressible

CPU is compressible — a container that exceeds its limit is throttled, not killed. The kernel's CFS scheduler enforces the quota.

# Check CPU throttling per container
kubectl exec -it <pod> -- cat /sys/fs/cgroup/cpu/cpu.stat | grep throttled

Memory: Incompressible

Memory is incompressible — a container that exceeds its limit is immediately killed by the OOM killer. The pod then restarts according to its restartPolicy.

# Check recent OOMKills
kubectl get events --field-selector reason=OOMKilling

---

Quality of Service (QoS) Classes

Kubernetes automatically assigns a QoS class to every pod based on its resource declarations. This class determines eviction order when a node runs out of memory.

Evicted first ──────────────────────────────────── Evicted last
BestEffort          Burstable              Guaranteed

Guaranteed

Every container in the pod has requests == limits for both CPU and memory:

resources:
  requests:
    cpu: "1"
    memory: "1Gi"
  limits:
    cpu: "1"
    memory: "1Gi"

Use for: production workloads, inference servers, databases.

Burstable

At least one container has requests < limits (or only requests set):

resources:
  requests:
    cpu: "200m"
    memory: "128Mi"
  limits:
    cpu: "2"
    memory: "1Gi"

Use for: most general workloads. Gets to burst when capacity is available, but yields under pressure.

BestEffort

No requests or limits set on any container:

# no resources block at all

Use for: throwaway batch jobs, dev scratchpads. Never for production.

# Check QoS class of a pod
kubectl get pod <pod> -o jsonpath='{.status.qosClass}'

---

The Scheduling Decision

When a pod is created, the scheduler runs two phases:

1. Filtering

Eliminates nodes that cannot satisfy the pod:

• Node has enough allocatable CPU/memory for the requests
• Node has required GPU type and count
• Taints/tolerations match
• Node affinity/anti-affinity satisfied
• Topology spread constraints met

2. Scoring

Ranks the remaining nodes. Key scoring plugins:

Plugin	Prefers
LeastAllocated	Node with most remaining capacity
BalancedAllocation	Even CPU/memory ratio
NodeResourcesFit	Closest fit to request
ImageLocality	Node that already has the image

The highest-scoring node wins. On a tie, the scheduler picks randomly.

---

Namespace-Level Controls

ResourceQuota

Caps the total resource consumption across all pods in a namespace:

apiVersion: v1
kind: ResourceQuota
metadata:
  name: team-quota
  namespace: ml-team
spec:
  hard:
    requests.cpu: "32"
    requests.memory: 128Gi
    limits.cpu: "64"
    limits.memory: 256Gi
    requests.nvidia.com/gpu: "8"
    pods: "50"
    persistentvolumeclaims: "20"

When a namespace hits its quota, new pods are rejected with a clear error. Check usage:

kubectl describe resourcequota -n ml-team

LimitRange

Sets per-container defaults and bounds — automatically injects requests/limits when containers don't specify them:

apiVersion: v1
kind: LimitRange
metadata:
  name: default-limits
  namespace: ml-team
spec:
  limits:
  - type: Container
    default:           # applied when no limits set
      cpu: "500m"
      memory: "512Mi"
    defaultRequest:    # applied when no requests set
      cpu: "100m"
      memory: "128Mi"
    max:               # hard ceiling per container
      cpu: "8"
      memory: "16Gi"
    min:               # floor per container
      cpu: "50m"
      memory: "64Mi"
  - type: PersistentVolumeClaim
    max:
      storage: 100Gi

LimitRange + ResourceQuota together:

LimitRange   → enforces per-pod sensible defaults
ResourceQuota → enforces total namespace budget

---

GPU Resource Allocation

GPUs use the extended resource model. Requests must equal limits (no fractional allocation at the device level):

resources:
  requests:
    nvidia.com/gpu: "2"
  limits:
    nvidia.com/gpu: "2"   # must match requests

MIG — Multi-Instance GPU

NVIDIA MIG (A100, H100) partitions a physical GPU into isolated slices with dedicated memory and compute:

# Request a 1g.5gb MIG slice (1/7 of an A100 80GB)
resources:
  requests:
    nvidia.com/mig-1g.5gb: "1"
  limits:
    nvidia.com/mig-1g.5gb: "1"

Available MIG profiles on A100 80GB:

Profile	Memory	Compute
mig-1g.5gb	5GB	1/7
mig-2g.10gb	10GB	2/7
mig-3g.20gb	20GB	3/7
mig-4g.20gb	20GB	4/7
mig-7g.40gb	40GB	7/7

Time-Slicing (non-MIG)

Allows oversubscription — multiple pods share one GPU via time-slicing. No memory isolation:

# In device plugin ConfigMap
version: v1
sharing:
  timeSlicing:
    resources:
    - name: nvidia.com/gpu
      replicas: 4   # 4 pods can claim 1 "GPU" each

---

Vertical Pod Autoscaler (VPA)

VPA automatically adjusts requests based on observed usage — useful when you don't know the right values upfront:

apiVersion: autoscaling.k8s.io/v1
kind: VerticalPodAutoscaler
metadata:
  name: my-app-vpa
spec:
  targetRef:
    apiVersion: apps/v1
    kind: Deployment
    name: my-app
  updatePolicy:
    updateMode: "Auto"   # or "Off" to just recommend
  resourcePolicy:
    containerPolicies:
    - containerName: my-app
      minAllowed:
        cpu: 100m
        memory: 128Mi
      maxAllowed:
        cpu: 4
        memory: 8Gi

# See VPA recommendations
kubectl describe vpa my-app-vpa

---

Common Pitfalls

No Requests Set (BestEffort)

Pod gets evicted first under any memory pressure. Always set requests.

Requests = 0, Limit > 0

Scheduler thinks the pod needs zero resources, places it anywhere. Pod evicted as BestEffort under pressure despite having a limit.

Memory Limit Too Low

Sudden OOMKills under load. Profile first with VPA in Off mode, then set limits at p99 + 20% headroom.

CPU Limit Too Low

Causes CPU throttling even when the node has spare capacity. For latency-sensitive workloads, consider not setting a CPU limit while keeping a CPU request — this makes the pod Burstable but prevents artificial throttling.

Requests >> Actual Usage

Reserves node capacity that's never used. Run:

kubectl top pods -A --sort-by=cpu

and compare against declared requests. Use VPA recommendations to right-size.

---

Production Checklist

• Set CPU and memory requests on every container
• Set memory limits on every container (prevent runaway processes)
• Consider omitting CPU limits for latency-sensitive workloads
• Use Guaranteed QoS for inference servers and databases
• Apply LimitRange to every namespace to catch containers with no declarations
• Apply ResourceQuota to team namespaces for budget enforcement
• Monitor throttling with container_cpu_cfs_throttled_seconds_total
• Monitor OOMKills with container_oom_events_total

---

Related Posts

• Kubernetes Scheduler Internals
• GPU Scheduling in Kubernetes
• GPU Autoscaling: KEDA, HPA, Cluster Autoscaler
• Kueue: Job Queuing and Quota Management
• Kubernetes Topology Manager