I Need a Classifier but I Don't Have Training Data or an ML Team

TL;DR

You don't need labelled data or ML expertise to build a production classifier. The LLM-as-teacher approach uses a large language model to generate synthetic training data offline, then trains a lightweight classifier that runs at sub-100ms latency with 89-95% F1 accuracy. Sparkient automates this full pipeline — define your decision options in plain English, and the system generates examples, trains the model, and deploys it. No data science. No GPU clusters. No manual labelling.

The Problem: The ML Cold Start

You need a classifier. Maybe it's content moderation for your community platform. Maybe it's ticket routing for your support queue. Maybe it's lead scoring for your sales pipeline.

The traditional path looks like this:

1. Collect data — months of historical examples, manually labelled by domain experts
2. Hire ML talent — data scientist or ML engineer ($150K-250K/year)
3. Build infrastructure — training pipelines, model serving, monitoring
4. Iterate — feature engineering, hyperparameter tuning, evaluation
5. Deploy and maintain — model versioning, retraining schedules, drift detection

Timeline: 3-6 months. Cost: $50K+ before the model makes its first production decision.

Most startups don't have 3-6 months or $50K to allocate to a classification feature. They need something that works this week.

The Four Options

There are four practical approaches to getting a classifier running without existing training data or ML expertise. Each has real tradeoffs.

Option 1: Zero-Shot LLM Classification

The simplest approach. Send each input to an LLM with a classification prompt:

import openai

response = openai.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{
        "role": "user",
        "content": f"""Classify this support ticket into one of: billing, technical, account, general.

Ticket: {ticket_text}

Respond with only the category name."""
    }]
)
category = response.choices[0].message.content.strip()

Pros: Works immediately, no training data needed, handles nuance well. Cons: $0.0003-$0.004 per call, 500-1500ms latency, rate limits at scale, occasional inconsistency. Best for: Prototyping, low-volume use cases (under 1,000/day), when you're still figuring out the categories.

Option 2: Few-Shot Prompting

Improve accuracy by including examples in the prompt:

prompt = """Classify this support ticket. Here are examples:

"My card was charged twice" → billing
"App crashes when I upload photos" → technical
"I need to change my email address" → account
"What's your return policy?" → general

Now classify: {ticket_text}

Respond with only the category name."""

Pros: Better accuracy than zero-shot, easy to iterate by editing examples. Cons: Same cost and latency as zero-shot (or worse — more input tokens), prompt grows linearly with examples. Best for: When zero-shot accuracy isn't good enough and you have a handful of representative examples.

Option 3: Fine-Tuning a Foundation Model

Fine-tune a smaller model on your specific classification task:

# OpenAI fine-tuning
training_data = [
    {"messages": [{"role": "user", "content": ticket}, {"role": "assistant", "content": label}]}
    for ticket, label in labelled_examples
]

openai.fine_tuning.jobs.create(
    training_file=upload_file(training_data),
    model="gpt-4o-mini-2024-07-18"
)

Pros: Lower per-call cost than base models, potentially better accuracy on your specific domain. Cons: You need labelled data (hundreds to thousands of examples), fine-tuning costs money, you're still paying per-call in production, still 200-500ms latency, vendor lock-in. Best for: When you have labelled data and want better accuracy than prompting, but can tolerate per-call costs.

Option 4: LLM-as-Teacher Compilation

Use the LLM to generate labelled training data, then compile a standalone classifier:

1. Define decision options → "billing", "technical", "account", "general"
2. LLM generates hundreds of synthetic training examples
3. System trains a lightweight classifier (compiled model)
4. Model exports to ONNX, deploys to production
5. Production decisions: sub-100ms, no LLM dependency

Pros: No existing labelled data needed, no ML expertise needed, sub-100ms latency in production, fixed cost regardless of volume, deterministic outputs. Cons: Less flexible than a prompt (need to retrain to change logic), requires that the decision has fixed options, 89-95% accuracy (not 100%). Best for: Production classification workloads with fixed options, high volume, and latency or cost constraints.

Comparing the Options

| | Zero-Shot LLM | Few-Shot LLM | Fine-Tuning | LLM-as-Teacher | |--|:---:|:---:|:---:|:---:| | Training data needed | None | 5-20 examples | 500-5,000+ examples | None | | ML expertise needed | None | None | Some | None | | Setup time | Minutes | Minutes | Days-weeks | Under an hour | | Per-call latency | 500-1500ms | 500-1500ms | 200-500ms | <100ms | | Per-call cost | $0.0003-$0.004 | $0.0004-$0.005 | $0.0001-$0.001 | Included in plan | | Cost at 50K/day | $450-$6,000/mo | $600-$7,500/mo | $150-$1,500/mo | $199/mo | | LLM dependency | Every call | Every call | Every call | Training only | | Deterministic | No | No | No | Yes |

The Sparkient Workflow: Zero to Production Classifier

Here's the actual workflow for building a classifier without data or ML expertise using Sparkient:

Step 1: Describe Your Decision (5 minutes)

You don't write code for this part — you describe what the decision means in plain English:

import httpx

API = "https://api.sparkient.ai/api/v1"
HEADERS = {"Authorization": "Bearer YOUR_API_KEY"}

response = httpx.post(f"{API}/decision-types", headers=HEADERS, json={
    "name": "content_moderation",
    "description": (
        "Decide whether user-generated content should be approved, sent for "
        "human review, or rejected. Approve content that is on-topic, constructive, "
        "and follows community guidelines. Send for review if the content is "
        "borderline — potentially offensive but context-dependent. Reject content "
        "that contains hate speech, explicit threats, spam, or illegal content."
    ),
    "options": ["approve", "review", "reject"],
    "reason_codes": [
        "on_topic", "constructive", "borderline_language", "needs_context",
        "hate_speech", "threats", "spam", "illegal_content"
    ]
})
decision_type_id = response.json()["id"]

The description is your prompt logic, translated into a decision definition. This is what the LLM teacher uses to understand your classification criteria.

Step 2: Generate Training Data (Automatic)

Sparkient's LLM teacher generates diverse, realistic training examples:

response = httpx.post(
    f"{API}/decision-types/{decision_type_id}/examples/generate",
    headers=HEADERS,
    json={"count": 200}
)
print(response.json())
# {"generated": 200, "by_option": {"approve": 80, "review": 60, "reject": 60}}

The teacher generates examples across all options, including edge cases and ambiguous scenarios. It's producing the labelled dataset that would normally take weeks of manual work.

Step 3: Train the Model (Automatic)

One API call triggers the full training pipeline:

response = httpx.post(
    f"{API}/decision-types/{decision_type_id}/training/train",
    headers=HEADERS,
    json={"preset": "balanced", "auto_deploy": True}
)
print(f"Training policy: {response.json()['id']}")

Behind the scenes:

• Text encoder creates semantic embeddings from text features
• Classifier trains on the combined feature set
• Hyperparameters are tuned automatically
• The model exports to ONNX format
• If auto_deploy is set, it goes live immediately

Step 4: Use It (Sub-100ms)

response = httpx.post(f"{API}/decide", headers=HEADERS, json={
    "decision_type_id": "content_moderation",
    "input": {
        "text": "Has anyone else noticed the new update breaks dark mode?",
        "user_id": "user_456"
    }
})

result = response.json()
# {"decision": "approve", "confidence": 0.92, "latency_ms": 38, "stage": "classifier"}

You now have a production classifier. No training data was collected manually. No ML engineer was hired. The total time from start to deployed model is under an hour.

Adding Your Own Data Later

The LLM-as-teacher approach doesn't lock you out of using real data. As your product runs and you collect real examples:

1. Export decision logs — every decision Sparkient makes is logged with the input and output
2. Add real examples — upload actual classified examples from your production data
3. Retrain — the model trains on a mix of synthetic and real data, improving accuracy on your specific distribution

This is a natural progression: start with synthetic data when you have nothing, then enrich with real data as it becomes available.

When You Actually Need an ML Team

Be realistic about when the LLM-as-teacher approach isn't enough:

• Custom model architectures — if your problem requires a specialized neural network (image classification, time-series prediction, recommendation systems), you need ML expertise.
• Sub-1% error rates — if your use case requires 99.9% accuracy (medical diagnosis, autonomous vehicles), you need domain experts, extensive validation, and likely a dedicated ML team.
• Non-classification problems — if you need generation, ranking, or regression, a classifier won't help. Compilation works specifically for structured decisions with fixed output options.

For the vast majority of classification use cases in production software — content moderation, ticket routing, lead scoring, fraud flags, approval workflows — 89-95% F1 is more than sufficient, and the time-to-production advantage of not needing an ML team is significant.

FAQ

Q: How accurate is a classifier trained on synthetic data vs real data? Across benchmarked domains, Sparkient's compiled models achieve 89-95% F1 scores using LLM-generated synthetic data. Content moderation specifically hits 91.5% F1. These are comparable to models trained on high-quality human-labelled data for the same tasks. The LLM escalation fallback handles the remaining edge cases.

Q: Can I define the decision options myself, or does the system choose them? You define them. The options, description, reason codes, and any hard rules are all specified by you. The system generates training data and trains a model to match your definition — it doesn't decide what the categories should be.

Q: What if my first model isn't accurate enough? You have several options: add more specific examples (manual or generated), refine your decision type description with clearer criteria, add CEL rules for cases that should always have a deterministic outcome, or upload real production examples to supplement the synthetic data. Each retraining run costs 2,000 credits and takes a few minutes.

Q: Do I need to know Python to use Sparkient? No. The API is a standard REST API — any language that can make HTTP requests works. There's also an MCP server for Claude, Cursor, and VS Code that lets you create decision types, generate examples, and train models from your IDE without writing any API integration code.

You don't need labelled data or an ML team to build a production classifier. Start with the free tier — 5,000 credits, no credit card — and deploy your first model in under an hour.