Tech Stack
Summary
A Go CLI that spins up a fully containerised SLURM cluster locally — so HPC and AI batch jobs can be developed, tested, and debugged without access to real cluster hardware.
What I Built
Project Overview
Developing SLURM workloads normally requires access to an HPC cluster — which is slow to provision, expensive, and often shared. Caravan bundles a complete SLURM cluster (controller node + compute nodes) into a single self-contained Go binary backed by Docker Compose containers.
Developers can run caravan cluster up, submit real sbatch jobs, and iterate locally — then ship the same scripts to production SLURM clusters without changes.
Caravan uses SLURM — it doesn't replace it. SLURM stays the scheduler; Caravan is the control plane.
Key Features
One-Command Cluster
caravan cluster up # start controller + compute nodes
caravan submit train.sh # submit an sbatch job
caravan cluster down # tear down cleanly
Real or Fake GPU Support
- Default: fake GPUs — any machine can test GPU-targeted jobs
CARAVAN_GPU=real: bind-mounts actual NVIDIA GPUs into compute containers for full CUDA testing
Cross-Platform
Runs on Linux, macOS, and Windows (via WSL2). Auto-detects Docker or Podman as the container backend.
Self-Contained Binary
Single binary with embedded cluster configuration — no Helm charts, no Kubernetes, no external dependencies beyond a container runtime.
Why It Matters
| Without Caravan | With Caravan |
|---|---|
| Need HPC cluster access to test a batch job | caravan cluster up in seconds on laptop |
| Cluster queue wait times slow iteration | Immediate job execution locally |
| GPU nodes scarce and shared | Fake GPU mode lets anyone test GPU job scripts |
| Debugging requires SSH into remote nodes | Local logs, local attach |
For AI teams that run training pipelines on SLURM (common in HPC-heavy environments like national labs and NVIDIA DGX clusters), Caravan collapses the dev → test → submit cycle.
Technical Highlights
Embedded SLURM Cluster
Docker Compose topology mirrors a real SLURM deployment: one slurmctld controller and configurable slurmd compute nodes, networked and configured automatically.
GPU Passthrough
When CARAVAN_GPU=real, the compute containers use the NVIDIA Container Toolkit to expose host GPUs — enabling actual CUDA workloads inside the local cluster.
Go CLI Design
Built with Go for a single-binary distribution and fast startup. GoReleaser handles cross-platform release artifacts for Linux, macOS, and Windows.
Technology Stack
Language: Go
Scheduler: SLURM (slurmctld + slurmd)
Container Runtime: Docker, Podman (auto-detected)
Orchestration: Docker Compose
GPU Support: NVIDIA Container Toolkit (optional)
Packaging: GoReleaser
Platforms: Linux, macOS, Windows (WSL2)