Skip to main content

Mac Studio environment

What's on a Mac Studio when you log in: macOS, Python 3, Metal and how to set up MLX for running models.

On this page

A Mac Studio reservation is a whole Apple silicon Mac running macOS, reached over SSH. There's no desktop session: you work in the terminal, like on a Linux server.

What's installed

  • macOS with its standard command-line tools: zsh (the default shell), bash, ssh, scp, rsync and curl.
  • Python 3 (3.10 or newer) as python3, with venv for virtual environments.
  • Metal, Apple's GPU interface, which MLX and PyTorch's mps backend use.

Install everything else yourself, in your home directory. Each reservation starts from a clean machine, so a startup script is the easiest way to set up the same tools every time.

Setting up MLX

MLX is Apple's array framework for Apple silicon, and mlx-lm runs and serves language models with it. Install both in a virtual environment:

bash
python3 -m venv ~/.venvs/mlx
source ~/.venvs/mlx/bin/activate
pip install --upgrade pip mlx-lm

Generate text from a 4-bit model (downloaded from Hugging Face on first use):

bash
mlx_lm.generate --model mlx-community/Qwen3-8B-4bit --prompt "Explain unified memory in one paragraph."

Chat with it interactively:

bash
mlx_lm.chat --model mlx-community/Qwen3-8B-4bit

Or serve an OpenAI-compatible API on port 8080:

bash
mlx_lm.server --model mlx-community/Qwen3-8B-4bit --port 8080

The server listens on the machine only; reach it from your computer with port forwarding. Our blog post Running Qwen3 and Llama with MLX covers model choice, quantisation and speed.

Memory

The CPU and GPU share the unified memory, and so does macOS itself. By default macOS lets the GPU use roughly three quarters of it — about 96 GB on a 128 GB machine — which is the practical budget for a model's weights plus its KV cache. Watch memory use with:

bash
memory_pressure | tail -1
top -l 1 -s 0 | grep PhysMem

If a model barely fits, pick a smaller quantisation (4-bit instead of 8-bit) or a shorter maximum context.

Other frameworks

  • PyTorch runs on the GPU through its mps device: pip install torch, then use device="mps".
  • llama.cpp and tools built on it support Metal and GGUF models.
  • Hugging Face Transformers works with PyTorch on mps, but MLX is usually faster for inference on a Mac.

Things that work differently from Linux

  • There's no apt: install tools with pip, or download them.
  • nvidia-smi and CUDA don't exist on a Mac. The GPU is used through Metal.
  • The shell is zsh; your scripts can still use #!/bin/bash.
  • Paths such as /home don't exist; your home directory is under /Users.