Document Utilities

Knowledge Distillation Explained

Knowledge Distillation is a Teacher-Student training strategy where a smaller model learns behavior from stronger, larger models. In DEQLYNK, the 21M local model is the Student; external providers such as Gemini/OpenAI act as Teachers when BYOK keys are enabled.

Teacher-Student Concept

Instead of training only on raw human text, the Student learns from curated teacher outputs that encode richer reasoning patterns, better formatting, and more robust completion style. Distillation does not clone the teacher. It transfers useful response behavior into a smaller deployable model.

  • Teacher: stronger external model via BYOK provider routing.
  • Student: local DEQLYNK 21M model optimized for controllability and cost.
  • Objective: capture high-quality response patterns under local constraints.

DEQLYNK Nightly Distillation Workflow

During nightly cron jobs, queued hard prompts and selected production samples are replayed through teacher models. Responses are normalized, filtered, and transformed into supervised training pairs for student updates.

Nightly job sequence:
1) Read pending distillation queue
2) Route prompts to teacher APIs (Gemini/OpenAI/OpenRouter priority)
3) Validate and sanitize outputs
4) Build student training dataset
5) Run update/fine-tune pass
6) Evaluate checkpoint and promote if quality threshold passes

Why BYOK Is Critical

BYOK keeps control with the operator. You decide which teacher providers are active, how often they are used, and how costs are managed. This enables high-quality transfer learning without surrendering platform ownership.

Quality Safeguards in Distillation

  • Reject teacher responses that violate formatting or safety policy.
  • Keep traceability between source prompt, teacher response, and student checkpoint.
  • Use hold-out evaluation prompts to detect regressions before deployment.
  • Promote checkpoints only when both quality and latency targets are met.
Distillation insight: the student wins when data curation is strict. Weak teacher samples create weak student behavior.

When to Distill vs When to Retrieve

Distillation improves general response behavior over time; RAG improves factual grounding at runtime. Mature systems combine both: distill stable reasoning patterns into weights, and retrieve dynamic facts on demand.