Document Utilities

CUDA & MPS Acceleration

DEQLYNK can run on multiple hardware paths: CPU-only, NVIDIA GPU acceleration via CUDA, and Apple Silicon acceleration via MPS (Metal Performance Shaders). The model logic stays the same, but throughput and latency characteristics change substantially based on device capability and memory bandwidth.

CPU vs GPU vs Apple Silicon

  • CPU: Universally available, easiest setup, lowest throughput.
  • CUDA (NVIDIA GPU): Highest token throughput, best for heavy sessions and experimentation.
  • MPS (Apple Silicon): Excellent efficiency for Mac workflows, faster than CPU, typically below high-end CUDA cards.

Expected Throughput (Illustrative)

Real-world numbers vary with quantization, prompt length, model batching, and thermal conditions. The table below provides educational baseline expectations for a 21M-class local model workload.

Hardware Type Typical Environment Expected TPS Best Use Case
CPU Laptop/Desktop without discrete GPU 8-20 TPS Learning, low-traffic local testing
NVIDIA GPU (CUDA) RTX-class desktop or cloud GPU 45-140 TPS Production-like inference and fine-tuning experiments
Apple Silicon (MPS) M1/M2/M3 MacBook or Mac Mini 25-75 TPS Portable development with strong efficiency

Deployment Strategy Guidance

  1. Start on CPU for deterministic baseline behavior and easier debugging.
  2. Move to CUDA when throughput or concurrent sessions become limiting.
  3. Use MPS for Mac-native workflows where energy efficiency matters.
  4. Benchmark with your own prompts before choosing production hardware.
Important: TPS is only one metric. Also measure time-to-first-token, consistency under load, and memory headroom for longer context windows.
Educational takeaway: Acceleration changes user experience more than model architecture changes in many practical deployments.