Skip to main content

DGX Spark environment

What's on an NVIDIA DGX Spark: DGX OS, the CUDA toolkit, Docker with GPU access, and the memory to leave for the system.

On this page

DGX Spark machines are coming soon. This page describes the environment they'll have, so you can prepare your workflow.

The NVIDIA DGX Spark is a compact computer built around the GB10 Grace Blackwell superchip: a 20-core Arm CPU and a Blackwell GPU sharing 128 GB of LPDDR5x unified memory at 273 GB/s. It runs NVIDIA's full AI software stack, so CUDA code and GPU containers built for Arm64 run as they do on NVIDIA's data-centre systems.

What's installed

  • DGX OS, NVIDIA's Ubuntu-based Linux, with the usual command-line tools and Python 3.
  • The CUDA toolkit (nvcc and libraries such as cuBLAS) and the NVIDIA driver. nvidia-smi shows the GPU.
  • Docker with the NVIDIA Container Toolkit, so containers can use the GPU.

Check the GPU:

bash
nvidia-smi

It's an Arm machine

The GB10's CPU is Arm64 (aarch64), not x86. Python wheels, Docker images and binaries must be built for linux/arm64. Most major projects publish them — NVIDIA's NGC containers, PyTorch, vLLM — but check before relying on an older or niche package.

Running models with containers

NVIDIA and the major inference projects publish containers for DGX Spark. Our Download a model template uses vLLM's image. To serve a model from the Hugging Face cache on port 8000:

bash
docker run -d --name gtl-serve --ipc=host -p 8000:8000 \
  -v ~/.cache/huggingface:/root/.cache/huggingface \
  -e NVIDIA_VISIBLE_DEVICES=all vllm/vllm-openai:latest \
  Qwen/Qwen3-8B --gpu-memory-utilization 0.85 --max-num-seqs 4

Reach it from your computer with port forwarding rather than exposing the port.

Memory

The GPU and the operating system share the 128 GB. Leave headroom for the system: with vLLM, keep --gpu-memory-utilization around 0.85 rather than close to 1.0, which can make the machine unresponsive. The same applies to other frameworks that pre-allocate GPU memory.

Unified memory also means a CUDA program can allocate with cudaMallocManaged and share data between CPU and GPU without explicit copies.

Performance notes

  • The GB10 is rated at up to 1 PFLOP of FP4 compute, so prompt processing and batch work are fast.
  • Token generation is bound by the 273 GB/s memory bandwidth, like any large-model inference. Quantised models (FP8, FP4 or 4-bit) generate faster because each token reads fewer bytes.

DGX Spark and the GB10, explained goes into what fits and how fast it runs.