I Need a Classifier but I Don't Have Training Data or an ML Team

TL;DR

You do not need an existing labelled dataset to evaluate a classifier. An LLM can generate examples offline, then Sparkient trains and deploys a lightweight model. Four controlled synthetic domains report 0.886–0.951 macro F1 and 33–42ms batch-average time per item. Production use still requires a clear policy, representative evaluation, and monitoring.

The Problem: The ML Cold Start

You need a classifier. Maybe it's content moderation for your community platform. Maybe it's ticket routing for your support queue. Maybe it's lead scoring for your sales pipeline.

The traditional path looks like this:

1. Collect data — months of historical examples, manually labelled by domain experts
2. Allocate ML capability — internal specialists, a partner, or founder time
3. Build infrastructure — training pipelines, model serving, monitoring
4. Iterate — feature engineering, hyperparameter tuning, evaluation
5. Deploy and maintain — model versioning, retraining schedules, drift detection

The timeline and cost depend on the existing data, team, evaluation burden, model, and deployment. The practical question is whether the decision deserves a custom ML project at all.

The Four Options

There are four practical approaches to getting a classifier running without existing training data or ML expertise. Each has real tradeoffs.

Option 1: Zero-Shot LLM Classification

The simplest approach. Send each input to an LLM with a classification prompt:

import openai

response = openai.chat.completions.create(
    model="gpt-4o-mini",
    messages=[{
        "role": "user",
        "content": f"""Classify this support ticket into one of: billing, technical, account, general.

Ticket: {ticket_text}

Respond with only the category name."""
    }]
)
category = response.choices[0].message.content.strip()

Pros: Works immediately, no training data needed, handles nuance well. Cons: Current provider pricing and limits; model-, prompt-, and load-dependent latency; outputs can change across model versions. Best for: Prototyping or production where the measured quality, runtime, and operating cost fit.

Option 2: Few-Shot Prompting

Improve accuracy by including examples in the prompt:

prompt = """Classify this support ticket. Here are examples:

"My card was charged twice" → billing
"App crashes when I upload photos" → technical
"I need to change my email address" → account
"What's your return policy?" → general

Now classify: {ticket_text}

Respond with only the category name."""

Pros: Better accuracy than zero-shot, easy to iterate by editing examples. Cons: Same cost and latency as zero-shot (or worse — more input tokens), prompt grows linearly with examples. Best for: When zero-shot accuracy isn't good enough and you have a handful of representative examples.

Option 3: Fine-Tuning a Foundation Model

Fine-tune a smaller model on your specific classification task:

# OpenAI fine-tuning
training_data = [
    {"messages": [{"role": "user", "content": ticket}, {"role": "assistant", "content": label}]}
    for ticket, label in labelled_examples
]

openai.fine_tuning.jobs.create(
    training_file=upload_file(training_data),
    model="gpt-4o-mini-2024-07-18"
)

Pros: Lower per-call cost than base models, potentially better accuracy on your specific domain. Cons: You need a task-specific dataset, pay training and inference or hosting costs, and take on provider or model operations. Data needs and latency are model-dependent. Best for: When you have labelled data and want better accuracy than prompting, but can tolerate per-call costs.

Option 4: LLM-as-Teacher Compilation

Use the LLM to generate labelled training data, then compile a standalone classifier:

1. Define decision options → "billing", "technical", "account", "general"
2. LLM generates hundreds of synthetic training examples
3. System trains a lightweight classifier (compiled model)
4. Model exports to ONNX, deploys to production
5. Production decisions: compiled normal path, with optional cloud escalation

Pros: Can start without an existing labelled dataset, provides a managed training workflow, targets a sub-100ms compiled path, avoids a live LLM call on the normal runtime path, and returns structured outputs. You still own evaluation, policy review, and production monitoring. Cons: Less flexible than a prompt, requires fixed options and retraining for learned-policy changes, and quality is task-specific. Best for: Repeated classifications with fixed options and a measured quality, latency, cost, privacy, reliability, or offline constraint.

Comparing the Options

| | Zero-Shot LLM | Few-Shot LLM | Fine-Tuning | LLM-as-Teacher | |--|:---:|:---:|:---:|:---:| | Training data needed | None | 5-20 examples | 500-5,000+ examples | None | | ML expertise needed | None | None | Some | None | | Setup time | Project-dependent | Project-dependent | Project-dependent | Train and evaluate before integration | | Per-call latency | Model and prompt dependent | Model and prompt dependent | Model and hosting dependent | Not measured; controlled synthetic runs report 33–42ms batch-average time per item | | Cost model | Token usage | Token usage | Provider or hosting usage | Plan credits | | Runtime LLM dependency | Every call | Every call | Depends on serving model | Normal path no; optional cloud escalation | | Output contract | Prompt constrained | Prompt constrained | Schema constrained | Structured decision response |

The Sparkient Workflow: Zero to Candidate Classifier

Here's the actual workflow for building a classifier without data or ML expertise using Sparkient:

Step 1: Describe Your Decision

You don't write code for this part — you describe what the decision means in plain English:

import httpx

API = "https://api.sparkient.ai/api/v1"
HEADERS = {"Authorization": "Bearer YOUR_API_KEY"}

response = httpx.post(f"{API}/decision-types", headers=HEADERS, json={
    "name": "content_moderation",
    "description": (
        "Decide whether user-generated content should be approved, sent for "
        "human review, or rejected. Approve content that is on-topic, constructive, "
        "and follows community guidelines. Send for review if the content is "
        "borderline — potentially offensive but context-dependent. Reject content "
        "that contains hate speech, explicit threats, spam, or illegal content."
    ),
    "options": ["approve", "review", "reject"],
    "reason_codes": [
        "on_topic", "constructive", "borderline_language", "needs_context",
        "hate_speech", "threats", "spam", "illegal_content"
    ]
})
decision_type_id = response.json()["id"]

The description is your prompt logic, translated into a decision definition. This is what the LLM teacher uses to understand your classification criteria.

Step 2: Generate Training Data (Automatic)

Sparkient's LLM teacher generates diverse, realistic training examples:

response = httpx.post(
    f"{API}/decision-types/{decision_type_id}/examples/generate",
    headers=HEADERS,
    json={"count": 200}
)
print(response.json())
# {"generated": 200, "by_option": {"approve": 80, "review": 60, "reject": 60}}

The teacher generates examples across all options, including edge cases and ambiguous scenarios. It's producing the labelled dataset that would normally take weeks of manual work.

Step 3: Train the Model (Automatic)

One API call triggers the full training pipeline:

response = httpx.post(
    f"{API}/decision-types/{decision_type_id}/train",
    headers=HEADERS,
    json={"auto_deploy": True}
)
print(f"Training policy: {response.json()['policy_id']}")

Behind the scenes:

• Text encoder creates semantic embeddings from text features
• Classifier trains on the combined feature set
• Hyperparameters are tuned automatically
• The model exports to ONNX format
• If auto_deploy is set, the completed policy deploys automatically when any configured quality gate is met

Step 4: Use It (Sub-100ms)

response = httpx.post(f"{API}/decide", headers=HEADERS, json={
    "decision_type": "content_moderation",
    "input": {
        "text": "Has anyone else noticed the new update breaks dark mode?",
        "user_id": "user_456"
    }
})

result = response.json()
# {"decision": "approve", "confidence": 0.92, "latency_ms": 38, "stage": "classifier"}

You now have a deployed candidate classifier trained from generated examples. It becomes production-ready only after it passes the project's representative evaluation and operational checks.

Adding Your Own Data Later

The LLM-as-teacher approach doesn't lock you out of using real data. As your product runs and you collect real examples:

1. Export decision logs — every decision Sparkient makes is logged with the input and output
2. Add real examples — upload actual classified examples from your production data
3. Retrain and compare — train on the reviewed mix, then check whether the fixed evaluation set actually improves

This is a natural progression: start with synthetic data when you have nothing, then enrich with real data as it becomes available.

When You Actually Need an ML Team

Be realistic about when the LLM-as-teacher approach isn't enough:

• Custom model architectures — if your problem requires a specialized neural network (image classification, time-series prediction, recommendation systems), you need ML expertise.
• High-consequence decisions — medical, safety-critical, legal, or other consequential uses require domain experts, extensive validation, and controls beyond a generic classifier workflow.
• Non-classification problems — if you need generation, ranking, or regression, a classifier won't help. Compilation works specifically for structured decisions with fixed output options.

Required quality depends on the class and consequence. Sparkient's public 0.886–0.951 macro-F1 range justifies evaluation in several domains, but it is not automatically sufficient for a production decision.

FAQ

Q: How accurate is a classifier trained on synthetic data vs real data? Across four controlled synthetic domains, Sparkient's compiled models report 0.886–0.951 macro F1. Content moderation reports 0.900 macro F1 and 91.5% accuracy. The results do not prove equivalence to a human-labelled or customer-production model; optional escalation must be evaluated separately.

Q: Can I define the decision options myself, or does the system choose them? You define them. The options, description, reason codes, and any hard rules are all specified by you. The system generates training data and trains a model to match your definition — it doesn't decide what the categories should be.

Q: What if my first model isn't accurate enough? You can add specific examples, refine the decision definition, add CEL rules for deterministic outcomes, or upload reviewed production examples. Each retraining run costs 2,000 credits; duration varies with data and configuration, and every new model should be checked on a fixed evaluation set.

Q: Do I need to know Python to use Sparkient? No. The API is a standard REST API — any language that can make HTTP requests works. There's also an MCP server for Claude, Cursor, and VS Code that lets you create decision types, generate examples, and train models from your IDE without writing any API integration code.

You do not need an existing labelled dataset to test the approach. Start with the free tier, train one candidate, and decide from measured quality, latency, credits, and effort.