How to Audit Your LLM Spending: A Developer's Guide

TL;DR

Most teams don't know exactly what their LLM API calls cost or which calls could be replaced with cheaper alternatives. This guide walks through a five-step audit: find all LLM calls in your codebase, categorize them by purpose, calculate monthly cost for each, identify classification workloads that are candidates for compilation, and estimate the savings. Classification workloads — decisions with fixed output options — are the best candidates because they can be compiled into sub-100ms models at 89-95% F1 accuracy, removing the per-call LLM cost entirely.

Why Audit Now?

LLM costs are distributed across your codebase — embedded in different services, called from different endpoints, often without centralized tracking. Many teams don't realize how much they're spending until the bill arrives, and by then they don't know which call is responsible for which portion.

The most common pattern: a team starts with one or two LLM integrations, then engineers add more as they discover how useful they are. Six months later, there are 15 different LLM calls across 8 services, and the monthly bill has gone from $200 to $8,000 without anyone tracking the growth.

An audit gives you the data to make decisions: which calls are worth the cost, which should be optimized, and which should be replaced.

Step 1: Find Every LLM Call in Your Codebase

Search for all LLM API client usage. Here's what to grep for:

# OpenAI
grep -rn "openai\." --include="*.py" --include="*.ts" --include="*.js" .
grep -rn "chat.completions.create" --include="*.py" .

# Anthropic
grep -rn "anthropic\." --include="*.py" --include="*.ts" .
grep -rn "messages.create" --include="*.py" .

# Google (Gemini)
grep -rn "genai\.\|GenerativeModel" --include="*.py" .

# Generic HTTP calls to LLM APIs
grep -rn "api.openai.com\|api.anthropic.com\|generativelanguage.googleapis.com" -r .

# LangChain / LlamaIndex wrappers
grep -rn "ChatOpenAI\|ChatAnthropic\|ChatGoogleGenerativeAI" --include="*.py" .
grep -rn "LLMChain\|ConversationChain" --include="*.py" .

For each match, log:

• File and function: Where is the call?
• Model used: Which model (GPT-4o, Claude Sonnet, Gemini Flash)?
• Endpoint: Is it part of an API handler, background job, or internal service?

Step 2: Categorize Each Call by Purpose

Not all LLM calls are equal. Categorize each one:

| Category | Description | Example | Compilation Candidate? | |----------|-------------|---------|:----------------------:| | Classification | Pick one of N fixed options | Content moderation, ticket routing, intent detection | ✅ Yes | | Extraction | Pull structured data from text | Entity extraction, key-value parsing | ⚠️ Sometimes | | Generation | Create free-form text | Email drafts, summaries, chatbot responses | ❌ No | | Transformation | Rewrite or translate text | Tone adjustment, translation | ❌ No | | Analysis | Open-ended reasoning | Sentiment analysis (if fixed scale), root cause analysis | ⚠️ Sometimes |

Classification calls are your best compilation targets. They have fixed output options, structured inputs, and the decision logic is learnable by a smaller model.

Build a table like this for your codebase:

| # | File | Function | Model | Category | Volume/Day |
|---|------|----------|-------|----------|------------|
| 1 | moderation.py | check_content() | GPT-4o | Classification | 25,000 |
| 2 | routing.py | route_ticket() | GPT-4o-mini | Classification | 8,000 |
| 3 | support.py | draft_reply() | Claude Sonnet | Generation | 2,000 |
| 4 | analytics.py | extract_topics() | GPT-4o-mini | Extraction | 5,000 |
| 5 | onboarding.py | classify_lead() | GPT-4o | Classification | 3,000 |

Step 3: Calculate Monthly Cost for Each Call

Use your provider's pricing and your actual token usage. If you don't have per-call token counts, estimate based on your prompts:

def estimate_monthly_cost(
    daily_volume: int,
    avg_input_tokens: int,
    avg_output_tokens: int,
    model: str
) -> dict:
    """Estimate monthly LLM cost for a single call type."""
    pricing = {
        "gpt-4o":          {"input": 2.50 / 1_000_000, "output": 10.00 / 1_000_000},
        "gpt-4o-mini":     {"input": 0.15 / 1_000_000, "output": 0.60 / 1_000_000},
        "claude-sonnet":   {"input": 3.00 / 1_000_000, "output": 15.00 / 1_000_000},
        "claude-haiku":    {"input": 0.25 / 1_000_000, "output": 1.25 / 1_000_000},
        "gemini-flash":    {"input": 0.075 / 1_000_000, "output": 0.30 / 1_000_000},
    }

    if model not in pricing:
        return {"error": f"Unknown model: {model}"}

    p = pricing[model]
    cost_per_call = (avg_input_tokens * p["input"]) + (avg_output_tokens * p["output"])
    monthly_cost = cost_per_call * daily_volume * 30

    return {
        "model": model,
        "cost_per_call": round(cost_per_call, 6),
        "daily_cost": round(cost_per_call * daily_volume, 2),
        "monthly_cost": round(monthly_cost, 2),
        "daily_volume": daily_volume,
    }

# Run for each call in your audit
calls = [
    {"name": "content_moderation", "daily": 25_000, "in": 600, "out": 40, "model": "gpt-4o"},
    {"name": "ticket_routing", "daily": 8_000, "in": 400, "out": 30, "model": "gpt-4o-mini"},
    {"name": "draft_reply", "daily": 2_000, "in": 1200, "out": 500, "model": "claude-sonnet"},
    {"name": "extract_topics", "daily": 5_000, "in": 800, "out": 100, "model": "gpt-4o-mini"},
    {"name": "classify_lead", "daily": 3_000, "in": 500, "out": 30, "model": "gpt-4o"},
]

total = 0
for call in calls:
    result = estimate_monthly_cost(call["daily"], call["in"], call["out"], call["model"])
    total += result["monthly_cost"]
    print(f"{call['name']:25s} {result['monthly_cost']:>10,.2f}/mo  ({result['cost_per_call']:.6f}/call)")

print(f"\n{'TOTAL':25s} {total:>10,.2f}/mo")

Example output:

content_moderation         1,560.00/mo  (0.002080/call)
ticket_routing                16.56/mo  (0.000069/call)
draft_reply                  666.00/mo  (0.011100/call)
extract_topics                27.00/mo  (0.000180/call)
classify_lead                135.00/mo  (0.001500/call)

TOTAL                      2,404.56/mo

Step 4: Identify Compilation Candidates

Now you can see where the money goes. Filter for calls that meet all of these criteria:

The Compilation Checklist

For each LLM call, ask:

• [ ] Fixed output options? Does it return one of a defined set of labels (approve/reject, high/medium/low, billing/technical/account)?
• [ ] Structured input? Is the input a defined set of fields (text, category, user attributes), not an open-ended conversation?
• [ ] Repetitive pattern? Is the same type of decision made thousands of times with similar (not identical) inputs?
• [ ] Stable logic? Has the decision logic been stable for at least a few weeks (not changing daily)?
• [ ] High volume? More than 1,000 decisions per day?
• [ ] Latency-sensitive? Is this in a hot path where 500ms+ matters?

If a call checks 4+ of these boxes, it's a strong compilation candidate.

Applying this to the audit example:

| Call | Fixed Options | Structured Input | Repetitive | Stable | High Volume | Latency-Sensitive | Candidate? | |------|:---:|:---:|:---:|:---:|:---:|:---:|:---:| | content_moderation | ✅ | ✅ | ✅ | ✅ | ✅ | ✅ | Yes | | ticket_routing | ✅ | ✅ | ✅ | ✅ | ✅ | ⚠️ | Yes | | draft_reply | ❌ | ✅ | ⚠️ | ✅ | ⚠️ | ❌ | No | | extract_topics | ⚠️ | ✅ | ✅ | ✅ | ✅ | ❌ | Maybe | | classify_lead | ✅ | ✅ | ✅ | ✅ | ✅ | ⚠️ | Yes |

Three of five calls are clear compilation candidates. One (extract_topics) might be compilable if the topic set is fixed.

Step 5: Estimate Savings

For each compilation candidate, calculate the cost difference:

| Call | Current Cost/Mo | Sparkient Plan | Savings/Mo |
|------|----------------:|---------------:|-----------:|
| content_moderation | $1,560 | $199 (Starter) | $1,361 |
| ticket_routing | $16.56 | (included) | $16.56 |
| classify_lead | $135 | (included) | $135 |
| **Total saveable** | **$1,711.56** | **$199** | **$1,512.56** |

The content moderation call alone justifies the compilation — $1,560/month drops to a fraction of the $199 Starter plan. The other classification calls ride on the same plan at no additional cost (within credit limits).

The non-compilable calls (draft_reply, extract_topics) stay on the LLM. You're not trying to eliminate LLM usage — you're removing it from the workloads where it's overkill.

The Ongoing Audit: What to Track

After the initial audit, set up ongoing monitoring:

Track Per-Call Costs

Most LLM client libraries support logging token usage:

# OpenAI — log token usage
response = await openai.chat.completions.create(
    model="gpt-4o",
    messages=[...],
)
logger.info("llm_call", extra={
    "model": "gpt-4o",
    "input_tokens": response.usage.prompt_tokens,
    "output_tokens": response.usage.completion_tokens,
    "function": "moderate_content",
    "estimated_cost": calculate_cost(response.usage, "gpt-4o"),
})

Set Budget Alerts

Every provider has usage dashboards, but they notify you after the fact. Set proactive alerts:

# Pseudo-code: daily cost check
daily_cost = sum_llm_costs_today()
if daily_cost > DAILY_BUDGET * 0.8:
    alert(f"LLM spend at {daily_cost}/{DAILY_BUDGET} — 80% of daily budget")
if daily_cost > DAILY_BUDGET:
    alert(f"LLM spend OVER budget: {daily_cost}/{DAILY_BUDGET}")

Review Monthly

Schedule a monthly review of your LLM audit spreadsheet. Look for:

• New LLM calls that engineers added
• Volume increases on existing calls
• Calls that have stabilized and are now compilation candidates
• Compiled calls that might need retraining (accuracy drift)

The Full Audit Template

Here's a template you can copy into a spreadsheet or markdown file:

# LLM Spending Audit — [Date]

## Summary
- Total LLM calls: [X] distinct call types
- Total daily volume: [Y] calls/day
- Total monthly cost: $[Z]/month
- Compilation candidates: [N] calls
- Estimated monthly savings: $[S]/month

## Call Inventory

| # | Service | Function | Model | Category | Tokens (in/out) | Daily Vol | Monthly Cost | Compilable? |
|---|---------|----------|-------|----------|-----------------|-----------|--------------|:-----------:|
| 1 | | | | | / | | $ | |
| 2 | | | | | / | | $ | |
| 3 | | | | | / | | $ | |

## Compilation Candidates

| Call | Current Cost | Est. Compiled Cost | Savings | Priority |
|------|-------------:|-------------------:|--------:|----------|
| | $ | $ | $ | |

## Action Items
- [ ] Compile [highest-cost classification call]
- [ ] Set up per-call cost logging
- [ ] Configure daily budget alerts
- [ ] Schedule next audit: [date]

Common Findings

After running this audit across multiple codebases, a few patterns emerge consistently:

1. One call dominates: Usually one or two LLM calls account for 60-80% of the total spend. Those are your highest-leverage optimization targets.

2. Classification is everywhere: Content moderation, intent detection, routing, scoring, approval — these are all classification tasks hiding behind LLM prompts. They're the easiest to compile.

3. Token waste is real: Many prompts include long system instructions or few-shot examples that get sent on every call. Even without compilation, trimming prompts can save 20-40% on token costs.

4. Nobody tracks it: The most common finding is that nobody in the organization has a complete picture of LLM usage. The audit itself is the most valuable deliverable.

FAQ

Q: My provider dashboard already shows costs. Why do I need a code-level audit? Provider dashboards show aggregate spending by model, not by use case. You can see that GPT-4o costs $5,000/month, but not that $3,000 of that is content moderation and $2,000 is response generation. The code-level audit maps costs to business functions so you can make informed optimization decisions.

Q: How do I estimate token counts for calls I haven't instrumented? Count the characters in your prompt template and divide by 4 (rough token estimate for English text). Add the typical input length. For output, classification calls typically use 10-50 tokens; generation calls use 100-1,000+. Or instrument one day of traffic and measure actual usage.

Q: What if my LLM costs are low enough that compilation isn't worth it? If you're spending under $100/month on LLM classification calls and the volume is under 1,000/day, the operational simplicity of a direct LLM call likely outweighs the savings. Focus the audit on understanding your cost structure — the compilation decision can wait until volume grows.

Q: How often should I re-audit? Quarterly at minimum. Monthly if you're actively developing new features that use LLMs. The audit is most valuable when it catches new high-volume calls early, before they've been running for months unchecked.

Know where your LLM budget goes before you optimize it. When you're ready to compile your classification workloads, start with the free tier — 5,000 credits, no credit card.