How to Train a Classifier Using an LLM as Teacher

TL;DR

You can train a classifier without starting from hand-labelled data by using an LLM as a teacher. The process: generate diverse examples, train a task-specific model, evaluate it on held-out cases, and export it to ONNX. Sparkient's four controlled synthetic domains report 0.886–0.951 macro F1 and 33–42ms batch-average time per item.

The Problem: LLMs Are Smart but Slow and Expensive

Large language models can be effective classification teachers, but quality depends on the policy, prompt, model, and test set. Establish the teacher's per-class performance before using its labels.

The issue is the production profile:

| | Live LLM | Sparkient compiled classifier | |---|---|---| | Latency | Depends on model, prompt, and provider | 33–42ms batch-average time per item in four controlled synthetic runs | | Cost model | Provider token usage on each call | Plan credits for decisions and supporting operations | | Runtime path | LLM provider on every request | Compiled model normally; optional cloud escalation | | Update path | Change the prompt | Add examples, retrain, evaluate, and deploy |

The teacher-student pattern resolves this: use the LLM's intelligence once (during training), then deploy a fast model that carries that intelligence forward.

How Teacher-Student Training Works

The pattern has three phases:

Phase 1: TEACH         Phase 2: TRAIN         Phase 3: DEPLOY
┌─────────────┐       ┌──────────────┐       ┌──────────────┐
│  LLM Teacher │──────▶│  Train Small  │──────▶│  ONNX Model  │
│  generates   │       │  Classifier   │       │  in prod     │
│  examples    │       │  on synthetic │       │  <100ms      │
│              │       │  data         │       │              │
└─────────────┘       └──────────────┘       └──────────────┘
   (offline)             (offline)             (production)

In a fully local student-model deployment, the LLM is used offline and does not run on the request path. Sparkient cloud deployments can optionally escalate configured low-confidence cases to a live model, so measure that path separately.

Phase 1: Generating Synthetic Training Data

The quality of your teacher determines the quality of your student. Here's what matters:

What makes a good teacher prompt:

A good prompt gives the LLM enough context to generate realistic, diverse examples. It needs:

1. Clear category definitions — Exactly what each label means, with boundary cases
2. Domain context — What kind of platform, what kind of users, what tone is normal
3. Diversity instructions — Vary length, tone, topic, and style
4. Edge case emphasis — Explicitly ask for borderline examples

teacher_prompt = """You are generating training data for a content moderation
classifier on a gaming community forum.

Categories:
- "approve": Normal gaming discussion, strategy talk, trash talk that stays
  playful, opinions about games, memes, questions. Most content falls here.
- "review": Borderline content that needs human judgement — heated arguments
  that might cross a line, mild personal attacks, possible doxxing attempts,
  content that could be harassment depending on context.
- "reject": Clear violations — slurs, explicit threats, doxxing (sharing
  personal info), sexual content involving minors, coordinated harassment,
  real-world violence threats.

Generate {n} examples. For each, provide:
- "text": The forum post content (realistic gaming community language)
- "label": The correct moderation decision
- "reasoning": Brief explanation of why this label is correct

Requirements:
- Include gaming-specific edge cases: competitive trash talk vs actual toxicity
- Vary post length from one-liners to multi-paragraph
- Include examples that could be misclassified by simple keyword matching
- Mix of English dialects and informal writing styles
- {class_distribution} approximate distribution across labels
"""

How many examples you need:

There is no universal examples-per-class threshold. Start with enough diversity to cover each policy class and known edge case, hold out a representative evaluation set, then add examples where the confusion matrix exposes gaps. Generation credit usage depends on the teacher model and actual tokens.

Handling class imbalance:

Real-world data is rarely balanced. In content moderation, ~90% of content is "approve," ~7% is "review," and ~3% is "reject." But training on that distribution means the model barely sees the minority classes.

Two strategies:

1. Oversample during generation — Generate more examples for rare classes. Ask the LLM for 500 "reject" examples even though they represent 3% of real traffic.

2. Augment after initial generation — Generate a balanced initial set, evaluate performance, then generate more examples specifically for classes with low recall.

Sparkient's training pipeline does both: it generates a balanced initial set, runs gap analysis to identify weak areas, then uses the LLM to augment underperforming classes.

Phase 2: Training the Student Model

The student model needs to be fast (sub-100ms), small (deploys anywhere), and accurate (matches the teacher as closely as possible).

The DIY approach with scikit-learn:

from sklearn.feature_extraction.text import TfidfVectorizer
from sklearn.linear_model import LogisticRegression
from sklearn.model_selection import cross_val_score
from sklearn.pipeline import Pipeline
import json
import numpy as np

# Load synthetic data
with open("teacher_data.json") as f:
    data = json.load(f)

texts = [d["text"] for d in data]
labels = [d["label"] for d in data]

# Simple baseline: TF-IDF + Logistic Regression
pipeline = Pipeline([
    ("tfidf", TfidfVectorizer(max_features=10000, ngram_range=(1, 2))),
    ("clf", LogisticRegression(max_iter=1000, class_weight="balanced"))
])

scores = cross_val_score(pipeline, texts, labels, cv=5, scoring="f1_macro")
print(f"Baseline F1: {np.mean(scores):.3f} ± {np.std(scores):.3f}")
# Typical result: 0.78 ± 0.03

This gets you a working baseline, but TF-IDF misses semantic meaning. Two sentences with completely different words but the same intent — "I'll find where you live" and "Your address is about to become public" — won't cluster together.

The better architecture: Text Encoder + Classifier

Sparkient's training pipeline uses a two-stage approach:

1. Text encoder creates high-dimensional semantic embeddings from text
2. Classifier makes predictions based on those embeddings plus any structured features (account age, previous violations, etc.)
3. Automated hyperparameter tuning optimizes the model configuration

# Conceptual outline — this is what Sparkient automates
import numpy as np

# Step 1: Encode text with a semantic text encoder
# The encoder converts text into dense vector representations
# that capture meaning, not just word frequency
def encode_texts(texts, batch_size=32):
    embeddings = []
    for batch in chunks(texts, batch_size):
        # Tokenize and encode each batch
        batch_embeddings = text_encoder.encode(batch)
        embeddings.append(batch_embeddings)
    return np.vstack(embeddings)

X_text = encode_texts(texts)

# Step 2: Combine with structured features
X_structured = np.array([[d.get("account_age_days", 0),
                           d.get("previous_violations", 0)]
                          for d in data])
X = np.hstack([X_text, X_structured])

# Step 3: Train classifier with automated hyperparameter tuning
# The tuning process explores different model configurations
# to find the best-performing combination
best_model = train_with_hyperparameter_optimization(
    X, labels,
    n_trials=50,
    scoring="f1_macro"
)

In the public content-moderation run, the full compilation pipeline achieved 0.900 macro F1 versus 0.606 for the strongest tested traditional baseline. The result combines teacher relabelling, augmentation, the text encoder, and the gradient-boosted classifier; it does not isolate one component's causal effect.

Phase 3: Export and Deploy

The trained model needs to run fast in production. ONNX (Open Neural Network Exchange) is the deployment format:

# Export to ONNX for fast inference
# The trained model is converted to ONNX format for
# portable, high-performance inference
onnx_model = convert_to_onnx(best_model, input_shape=X.shape[1])
save_onnx_model(onnx_model, "moderation_model.onnx")

The ONNX model avoids a network call to an LLM on the compiled path. Sparkient targets under 100ms including text encoding; its four controlled synthetic domains report 33–42ms batch-average time per item, not per-request p95. Each new model still needs full-path measurement in its deployment environment.

With Sparkient, all of this is one API call:

import httpx

# Trigger training — synthetic data generation, model training, and
# ONNX export all happen automatically
response = httpx.post(
    f"https://api.sparkient.ai/api/v1/decision-types/{decision_type_id}/train",
    headers={"Authorization": "Bearer YOUR_API_KEY"}
)
# Training runs asynchronously. Poll the status endpoint or wait for
# the webhook notification.

# Once deployed, call /decide
result = httpx.post(
    "https://api.sparkient.ai/api/v1/decide",
    headers={"Authorization": "Bearer YOUR_API_KEY"},
    json={
        "decision_type": "content-moderation",
        "input": {"text": "gg ez noob uninstall", "user_id": "player_42"}
    }
).json()
# {"decision": "approve", "confidence": 0.82, "latency_ms": 36, "stage": "classifier"}

Validating the Student Against the Teacher

How do you know the student learned the right lessons? You need a validation strategy.

Hold-out validation: Set aside 20% of synthetic data as a test set. Train on 80%, evaluate on 20%. This measures how well the student learned the teacher's patterns — but not whether those patterns are correct.

Teacher agreement: Run a sample of real-world inputs through both the teacher LLM and the student classifier. Measure agreement rate. For a well-trained student, you should see 85-95% agreement with the teacher on the same inputs.

Confidence calibration: Check that the student's confidence scores are meaningful. When the model says 0.95 confidence, it should be right ~95% of the time. When it says 0.6, expect ~60% accuracy. Poorly calibrated models are dangerous because you can't trust the escalation threshold.

Edge case audit: Manually curate 50-100 tricky examples and check the student's decisions. This is the most important validation step because it catches systematic blind spots — categories of content the teacher didn't generate enough examples for.

When to Use This Pattern (and When Not To)

Use teacher-student training when:

• You need sub-100ms decisions (payment flows, content submission, real-time routing)
• You're making the same type of decision millions of times
• You have a clear set of output categories (2-10 classes)
• Observed token cost or operating overhead is material

Use direct LLM calls when:

• You need free-form reasoning, not classification
• Decision types change frequently (weekly policy updates)
• The current path already meets quality, latency, cost, reliability, privacy, and maintenance requirements
• You need the LLM to explain its reasoning in natural language

Use rules engines when:

• Logic is purely deterministic (rate limits, geo-blocking, age verification)
• You need sub-1ms latency
• The decision tree is small and well-defined

The best systems combine all three — which is exactly what the compiled decision pipeline does.

FAQ

Does the student ever surpass the teacher? Rarely in raw accuracy, but often in consistency. LLMs can give different answers to the same input depending on temperature, prompt phrasing, and API version. A compiled classifier gives the exact same answer every time. For production systems, determinism is often more valuable than a marginal accuracy improvement.

How do I handle concept drift? When your domain changes — new types of spam, new slang, new attack patterns — the student's accuracy will degrade. Monitor confidence distributions over time. If average confidence drops or the "review" rate spikes, it's time to retrain. Generate new synthetic data that covers the emerging patterns, mix with your existing examples, and retrain. Sparkient tracks these metrics in the dashboard.

Can I use a different LLM as the teacher? Yes. Any LLM that can generate structured labelled examples works as a teacher. The quality of the student depends on the quality and diversity of the training data, not the specific teacher model. The architecture is model-agnostic — what matters is the quality of the generated examples.

What if I already have some labelled data? Use it. Even 50-100 real examples significantly improve the student, especially for edge cases specific to your domain. Upload them alongside the synthetic data — real examples anchor the model in your actual distribution, while synthetic data provides breadth and diversity.

The teacher-student pattern isn't new, but LLMs can lower the cost of creating a candidate dataset. A trustworthy production classifier still needs a clear policy, representative held-out cases, a sound training pipeline, per-class evaluation, and operational monitoring.

Try it in the playground, or start with the free tier — 5,000 credits, no credit card.