edge:waw | CF Pages (SSG) | AI: 2/day
MrocznyBanan@cloud : ~/articles $ ls -la
← cd .. [back to index] chmod 644 2026-11-22-deepseek-r1-vllm-apple-silicon.md
$ cat 2026-11-22-deepseek-r1-vllm-apple-silicon.md

Local DeepSeek R1 on Apple Silicon with MLX and Ollama

πŸ“… Date: 2026-11-22 🏷️ Category: AI ⏱ Read time: 6 min Git: #7b3e09d πŸ€– AI Verified Post

The DeepSeek R1 reasoning model family represents a major milestone for open-weights Chain-of-Thought (CoT) generation. Running these models locally on Apple Silicon delivers zero-latency iterations, complete privacy, and zero API costs.

Unified Memory Architecture (UMA)

On traditional x86 workstations with dedicated GPUs, available VRAM is the primary bottleneck (typically 16 GB or 24 GB on an RTX 4090). On Apple Silicon (M3 Max / M4 Pro), the CPU and GPU share up to 128 GB of unified memory with bandwidth exceeding 400 GB/s.

This allows quantized models (e.g. Q4_K_M or 4-bit MLX) to reside entirely in high-speed RAM without NVMe swapping.

Running with MLX

Apple’s MLX framework provides Metal kernels optimized specifically for the unified memory architecture:

# Install MLX CLI tools
pip install mlx-lm

# Run inference with 14B distilled model
python -m mlx_lm.generate \
  --model mlx-community/DeepSeek-R1-Distill-Qwen-14B-4bit \
  --prompt "Write an idempotent Ansible playbook configuring a K3s cluster" \
  --max-tokens 1024

Benchmark Results (M3 Max, 64 GB)

  • 14B (4-bit): ~42 tokens/sec, RAM usage: 9.8 GB
  • 32B (4-bit): ~19 tokens/sec, RAM usage: 21.4 GB
  • 70B (4-bit): ~8 tokens/sec, RAM usage: 44.1 GB

For day-to-day software engineering and DevOps tasks, the 14B distilled variant provides the best balance between execution speed and reasoning fidelity.

tags: #ai #deepseek #apple-silicon #mlx #llm
[cd .. back to console]