Skip to main content
SkillJury
Back to registry

knowledge-distillation

davila7/claude-code-templates

Installs229
Install command
npx skills add https://github.com/davila7/claude-code-templates --skill knowledge-distillation
Security audits
Gen Agent Trust HubPASS
SocketPASS
SnykPASS
About this skill
Use Knowledge Distillation when you need to: Key Techniques : Temperature scaling, soft targets, reverse KLD (MiniLLM), logit distillation, response distillation Papers : Hinton et al. 2015 (arXiv 1503.02531), MiniLLM (arXiv 2306.08543), KD Survey (arXiv 2402.13116) Source : arXiv 2306.08543 (2024) Innovation : Use reverse KLD instead of forward KLD for better generative model distillation. Why reverse KL? Purpose : Soften probability distributions to expose teacher's uncertainty. Rule : Use T=2-5 for distillation (2 is common default). - Compress models from 70B → 7B while retaining 90%+ performance - Transfer capabilities from proprietary models (GPT-4) to open-source (LLaMA, Mistral) - Reduce inference costs by deploying smaller student models - Create specialized models by distilling domain-specific knowledge - Improve small models using synthetic data from large teachers - Forward KL (standard): Student learns to match teacher's mean - Reverse KL (MiniLLM): Student learns to cover all teacher's modes - Better for diverse text generation - Hinton et al. 2015 (Foundational) : - MiniLLM (Reverse KLD) : - KD Survey for LLMs (2024) : - MiniLLM GitHub :

Source description provided by the upstream skill listing. Community reviews and install context appear in the sections below.

Community Reviews

Latest reviews

Sign in to review

No community reviews yet. Be the first to review.

Browse this skill in context
FAQ
What does knowledge-distillation do?

knowledge-distillation is listed in SkillJury, but the source summary is still sparse.

Is knowledge-distillation good?

knowledge-distillation does not have approved reviews yet, so SkillJury cannot publish a community verdict.

What agent does knowledge-distillation work with?

knowledge-distillation currently lists compatibility with codex, gemini-cli, opencode, cursor, github-copilot, claude-code.

What are alternatives to knowledge-distillation?

Skills in the same category include telegram-bot-builder, flutter-app-size, sharp-edges, iterative-retrieval.

How do I install knowledge-distillation?

npx skills add https://github.com/davila7/claude-code-templates --skill knowledge-distillation

telegram-bot-builderflutter-app-sizesharp-edgesiterative-retrieval
Related skills

More from davila7/claude-code-templates

senior-frontend

by davila7/claude-code-templates

925

Complete toolkit for senior frontend with modern tools and best practices.

Software EngineeringFrontend and DesignFirst seen Jan 19, 2026

market-research-reports

by davila7/claude-code-templates

738

Market research reports are comprehensive strategic documents that analyze industries, markets, and competitive landscapes to inform business decisions, investment strategies, and strategic planning. This skill generates professional-grade reports of 50+ pages with extensive visual content, modeled after deliverables...

Software EngineeringFrontend and DesignFirst seen Jan 20, 2026

senior-data-engineer

by davila7/claude-code-templates

693

World-class senior data engineer skill for production-grade AI/ML/Data systems.

Software EngineeringFrontend and DesignFirst seen Jan 19, 2026

senior-devops

by davila7/claude-code-templates

666

Complete toolkit for senior devops with modern tools and best practices.

Software EngineeringFrontend and DesignFirst seen Jan 19, 2026
Related skills

Alternatives in Software Engineering

telegram-bot-builder

by sickn33/antigravity-awesome-skills

998

Source details, install context, and public review data are available on the full page.

Software EngineeringFrontend and DesignFirst seen Jan 18, 2026

flutter-app-size

by flutter/skills

996

Analyzes and optimizes Flutter application size by measuring build artifacts, generating size analysis reports, utilizing Dart DevTools for component breakdown, and implementing specific size reduction strategies such as debug info splitting, resource compression, and platform-specific tree-shaking. Assumes a...

Software EngineeringFrontend and Design

sharp-edges

by trailofbits/skills

994

Evaluates whether APIs, configurations, and interfaces are resistant to developer misuse. Identifies designs where the "easy path" leads to insecurity.

Software EngineeringFrontend and DesignFirst seen Jan 18, 2026

iterative-retrieval

by affaan-m/everything-claude-code

993

Solves the "context problem" in multi-agent workflows where subagents don't know what context they need until they start working.

Software EngineeringFrontend and DesignFirst seen Jan 25, 2026