Lambda Labs GPU Cloud

Comprehensive guide to running ML workloads on Lambda Labs GPU cloud with on-demand instances and 1-Click Clusters.

When to use Lambda Labs

Use Lambda Labs when:

Need dedicated GPU instances with full SSH access
Running long training jobs (hours to days)
Want simple pricing with no egress fees
Need persistent storage across sessions
Require high-performance multi-node clusters (16-512 GPUs)
Want pre-installed ML stack (Lambda Stack with PyTorch, CUDA, NCCL)

Key features:

GPU variety: B200, H100, GH200, A100, A10, A6000, V100
Lambda Stack: Pre-installed PyTorch, TensorFlow, CUDA, cuDNN, NCCL
Persistent filesystems: Keep data across instance restarts
1-Click Clusters: 16-512 GPU Slurm clusters with InfiniBand
Simple pricing: Pay-per-minute, no egress fees
Global regions: 12+ regions worldwide

Use alternatives instead:

Modal: For serverless, auto-scaling workloads
SkyPilot: For multi-cloud orchestration and cost optimization
RunPod: For cheaper spot instances and serverless endpoints
Vast.ai: For GPU marketplace with lowest prices

Quick start

Account setup

Create account at https://lambda.ai
Add payment method
Generate API key from dashboard
Add SSH key (required before launching instances)

Launch via console

Go to https://cloud.lambda.ai/instances
Click "Launch instance"
Select GPU type and region
Choose SSH key
Optionally attach filesystem
Launch and wait 3-15 minutes

Connect via SSH

Get instance IP from console

ssh ubuntu@<INSTANCE-IP>

Or with specific key

ssh -i ~/.ssh/lambda_key ubuntu@<INSTANCE-IP>

GPU instances

Available GPUs

GPU VRAM Price/GPU/hr Best For

B200 SXM6 180 GB $4.99 Largest models, fastest training

H100 SXM 80 GB $2.99-3.29 Large model training

H100 PCIe 80 GB $2.49 Cost-effective H100

GH200 96 GB $1.49 Single-GPU large models

A100 80GB 80 GB $1.79 Production training

A100 40GB 40 GB $1.29 Standard training

A10 24 GB $0.75 Inference, fine-tuning

A6000 48 GB $0.80 Good VRAM/price ratio

V100 16 GB $0.55 Budget training

Instance configurations

8x GPU: Best for distributed training (DDP, FSDP) 4x GPU: Large models, multi-GPU training 2x GPU: Medium workloads 1x GPU: Fine-tuning, inference, development

Launch times

Single-GPU: 3-5 minutes
Multi-GPU: 10-15 minutes

Lambda Stack

All instances come with Lambda Stack pre-installed:

Included software

Ubuntu 22.04 LTS
NVIDIA drivers (latest)
CUDA 12.x
cuDNN 8.x
NCCL (for multi-GPU)
PyTorch (latest)
TensorFlow (latest)
JAX
JupyterLab

Verify installation

Check GPU

nvidia-smi

Check PyTorch

python -c "import torch; print(torch.cuda.is_available())"

Check CUDA version

nvcc --version

Python API

Installation

pip install lambda-cloud-client

Authentication

import os import lambda_cloud_client

Configure with API key

configuration = lambda_cloud_client.Configuration( host="https://cloud.lambdalabs.com/api/v1", access_token=os.environ["LAMBDA_API_KEY"] )

List available instances

with lambda_cloud_client.ApiClient(configuration) as api_client: api = lambda_cloud_client.DefaultApi(api_client)

# Get available instance types
types = api.instance_types()
for name, info in types.data.items():
    print(f"{name}: {info.instance_type.description}")

Launch instance

from lambda_cloud_client.models import LaunchInstanceRequest

request = LaunchInstanceRequest( region_name="us-west-1", instance_type_name="gpu_1x_h100_sxm5", ssh_key_names=["my-ssh-key"], file_system_names=["my-filesystem"], # Optional name="training-job" )

response = api.launch_instance(request) instance_id = response.data.instance_ids[0] print(f"Launched: {instance_id}")

List running instances

instances = api.list_instances() for instance in instances.data: print(f"{instance.name}: {instance.ip} ({instance.status})")

Terminate instance

from lambda_cloud_client.models import TerminateInstanceRequest

request = TerminateInstanceRequest( instance_ids=[instance_id] ) api.terminate_instance(request)

SSH key management

from lambda_cloud_client.models import AddSshKeyRequest

Add SSH key

request = AddSshKeyRequest( name="my-key", public_key="ssh-rsa AAAA..." ) api.add_ssh_key(request)

List keys

keys = api.list_ssh_keys()

Delete key

api.delete_ssh_key(key_id)

CLI with curl

List instance types

curl -u $LAMBDA_API_KEY:
https://cloud.lambdalabs.com/api/v1/instance-types | jq

Launch instance

curl -u $LAMBDA_API_KEY:
-X POST https://cloud.lambdalabs.com/api/v1/instance-operations/launch
-H "Content-Type: application/json"
-d '{ "region_name": "us-west-1", "instance_type_name": "gpu_1x_h100_sxm5", "ssh_key_names": ["my-key"] }' | jq

Terminate instance

curl -u $LAMBDA_API_KEY:
-X POST https://cloud.lambdalabs.com/api/v1/instance-operations/terminate
-H "Content-Type: application/json"
-d '{"instance_ids": ["<INSTANCE-ID>"]}' | jq

Persistent storage

Filesystems

Filesystems persist data across instance restarts:

Mount location

/lambda/nfs/<FILESYSTEM_NAME>

Example: save checkpoints

python train.py --checkpoint-dir /lambda/nfs/my-storage/checkpoints

Create filesystem

Go to Storage in Lambda console
Click "Create filesystem"
Select region (must match instance region)
Name and create

Attach to instance

Filesystems must be attached at instance launch time:

Via console: Select filesystem when launching
Via API: Include file_system_names in launch request

Best practices

Store on filesystem (persists)

/lambda/nfs/storage/ ├── datasets/ ├── checkpoints/ ├── models/ └── outputs/

Local SSD (faster, ephemeral)

/home/ubuntu/ └── working/ # Temporary files

SSH configuration

Add SSH key

Generate key locally

ssh-keygen -t ed25519 -f ~/.ssh/lambda_key

Add public key to Lambda console

Or via API

Multiple keys

On instance, add more keys

echo 'ssh-rsa AAAA...' >> ~/.ssh/authorized_keys

Import from GitHub

On instance

ssh-import-id gh:username

SSH tunneling

Forward Jupyter

ssh -L 8888:localhost:8888 ubuntu@<IP>

Forward TensorBoard

ssh -L 6006:localhost:6006 ubuntu@<IP>

Multiple ports

ssh -L 8888:localhost:8888 -L 6006:localhost:6006 ubuntu@<IP>

JupyterLab

Launch from console

Go to Instances page
Click "Launch" in Cloud IDE column
JupyterLab opens in browser

Manual access

On instance

jupyter lab --ip=0.0.0.0 --port=8888

From local machine with tunnel

ssh -L 8888:localhost:8888 ubuntu@<IP>

Open http://localhost:8888

Training workflows

Single-GPU training

SSH to instance

ssh ubuntu@<IP>

Clone repo

git clone https://github.com/user/project cd project

Install dependencies

pip install -r requirements.txt

Train

python train.py --epochs 100 --checkpoint-dir /lambda/nfs/storage/checkpoints

Multi-GPU training (single node)

train_ddp.py

import torch import torch.distributed as dist from torch.nn.parallel import DistributedDataParallel as DDP

def main(): dist.init_process_group("nccl") rank = dist.get_rank() device = rank % torch.cuda.device_count()

model = MyModel().to(device)
model = DDP(model, device_ids=[device])

# Training loop...

if name == "main": main()

Launch with torchrun (8 GPUs)

torchrun --nproc_per_node=8 train_ddp.py

Checkpoint to filesystem

import os

checkpoint_dir = "/lambda/nfs/my-storage/checkpoints" os.makedirs(checkpoint_dir, exist_ok=True)

Save checkpoint

torch.save({ 'epoch': epoch, 'model_state_dict': model.state_dict(), 'optimizer_state_dict': optimizer.state_dict(), 'loss': loss, }, f"{checkpoint_dir}/checkpoint_{epoch}.pt")

1-Click Clusters

Overview

High-performance Slurm clusters with:

16-512 NVIDIA H100 or B200 GPUs
NVIDIA Quantum-2 400 Gb/s InfiniBand
GPUDirect RDMA at 3200 Gb/s
Pre-installed distributed ML stack

Included software

Ubuntu 22.04 LTS + Lambda Stack
NCCL, Open MPI
PyTorch with DDP and FSDP
TensorFlow
OFED drivers

Storage

24 TB NVMe per compute node (ephemeral)
Lambda filesystems for persistent data

Multi-node training

On Slurm cluster

srun --nodes=4 --ntasks-per-node=8 --gpus-per-node=8
torchrun --nnodes=4 --nproc_per_node=8
--rdzv_backend=c10d --rdzv_endpoint=$MASTER_ADDR:29500
train.py

Networking

Bandwidth

Inter-instance (same region): up to 200 Gbps
Internet outbound: 20 Gbps max

Firewall

Default: Only port 22 (SSH) open
Configure additional ports in Lambda console
ICMP traffic allowed by default

Private IPs

Find private IP

ip addr show | grep 'inet '

Common workflows

Workflow 1: Fine-tuning LLM

1. Launch 8x H100 instance with filesystem

2. SSH and setup

ssh ubuntu@<IP> pip install transformers accelerate peft

3. Download model to filesystem

python -c " from transformers import AutoModelForCausalLM model = AutoModelForCausalLM.from_pretrained('meta-llama/Llama-2-7b-hf') model.save_pretrained('/lambda/nfs/storage/models/llama-2-7b') "

4. Fine-tune with checkpoints on filesystem

accelerate launch --num_processes 8 train.py
--model_path /lambda/nfs/storage/models/llama-2-7b
--output_dir /lambda/nfs/storage/outputs
--checkpoint_dir /lambda/nfs/storage/checkpoints

Workflow 2: Batch inference

1. Launch A10 instance (cost-effective for inference)

2. Run inference

python inference.py
--model /lambda/nfs/storage/models/fine-tuned
--input /lambda/nfs/storage/data/inputs.jsonl
--output /lambda/nfs/storage/data/outputs.jsonl

Cost optimization

Choose right GPU

Task Recommended GPU

LLM fine-tuning (7B) A100 40GB

LLM fine-tuning (70B) 8x H100

Inference A10, A6000

Development V100, A10

Maximum performance B200

Reduce costs

Use filesystems: Avoid re-downloading data
Checkpoint frequently: Resume interrupted training
Right-size: Don't over-provision GPUs
Terminate idle: No auto-stop, manually terminate

Monitor usage

Dashboard shows real-time GPU utilization
API for programmatic monitoring

Common issues

Issue Solution

Instance won't launch Check region availability, try different GPU

SSH connection refused Wait for instance to initialize (3-15 min)

Data lost after terminate Use persistent filesystems

Slow data transfer Use filesystem in same region

GPU not detected Reboot instance, check drivers

References

Advanced Usage - Multi-node training, API automation
Troubleshooting - Common issues and solutions

Resources

lambda-labs-gpu-cloud

Safety Notice

Copy this and send it to your AI assistant to learn

Get instance IP from console

Or with specific key

Included software

Check GPU

Check PyTorch

Check CUDA version

Configure with API key

Add SSH key

List keys

Delete key

Mount location

Example: save checkpoints

Store on filesystem (persists)

Local SSD (faster, ephemeral)

Generate key locally

Add public key to Lambda console

Or via API

On instance, add more keys

On instance

Forward Jupyter

Forward TensorBoard

Multiple ports

On instance

From local machine with tunnel

Open http://localhost:8888

SSH to instance

Clone repo

Install dependencies

Train

train_ddp.py

Launch with torchrun (8 GPUs)

Save checkpoint

On Slurm cluster

Find private IP

1. Launch 8x H100 instance with filesystem

2. SSH and setup

3. Download model to filesystem

4. Fine-tune with checkpoints on filesystem

1. Launch A10 instance (cost-effective for inference)

2. Run inference

Source Transparency

Related Skills

ml-paper-writing

faiss

mlflow