Document Utilities
CUDA & MPS Acceleration
DEQLYNK can run on multiple hardware paths: CPU-only, NVIDIA GPU acceleration via CUDA, and Apple Silicon acceleration via MPS (Metal Performance Shaders). The model logic stays the same, but throughput and latency characteristics change substantially based on device capability and memory bandwidth.
CPU vs GPU vs Apple Silicon
- CPU: Universally available, easiest setup, lowest throughput.
- CUDA (NVIDIA GPU): Highest token throughput, best for heavy sessions and experimentation.
- MPS (Apple Silicon): Excellent efficiency for Mac workflows, faster than CPU, typically below high-end CUDA cards.
Expected Throughput (Illustrative)
Real-world numbers vary with quantization, prompt length, model batching, and thermal conditions. The table below provides educational baseline expectations for a 21M-class local model workload.
| Hardware Type | Typical Environment | Expected TPS | Best Use Case |
|---|---|---|---|
| CPU | Laptop/Desktop without discrete GPU | 8-20 TPS | Learning, low-traffic local testing |
| NVIDIA GPU (CUDA) | RTX-class desktop or cloud GPU | 45-140 TPS | Production-like inference and fine-tuning experiments |
| Apple Silicon (MPS) | M1/M2/M3 MacBook or Mac Mini | 25-75 TPS | Portable development with strong efficiency |
Deployment Strategy Guidance
- Start on CPU for deterministic baseline behavior and easier debugging.
- Move to CUDA when throughput or concurrent sessions become limiting.
- Use MPS for Mac-native workflows where energy efficiency matters.
- Benchmark with your own prompts before choosing production hardware.
Important: TPS is only one metric. Also measure time-to-first-token, consistency under load,
and memory headroom for longer context windows.
Educational takeaway: Acceleration changes user experience more than model architecture changes in many practical deployments.