Stop Routing Every Text Task Through an LLM

Somewhere in the last two years, "call the LLM" quietly became the default answer to every text problem, including the ones that never needed one. Classifying a support ticket into one of eight categories, deduplicating a lead list, tagging a document by topic — these are NLP problems with decades of solved tooling behind them, and routing them through a general-purpose chat model is often slower, more expensive, and no more accurate than the boring alternative.

The cost-accuracy trade isn't what people assume

The intuition that a bigger model is always a safer bet doesn't hold up under actual measurement. More capable frontier models do score higher on open-ended reasoning, but on narrow, well-defined classification tasks the accuracy gap over a fine-tuned classical model often closes to nearly nothing — while the cost gap does the opposite. A lightweight text classifier or embedding-similarity lookup running in single-digit milliseconds costs a fraction of a per-token API call, and at the request volumes a real product sees, that difference compounds into a real line item, not a rounding error.

Where classical NLP still wins outright

Text classification, keyword and entity extraction, deduplication via embedding similarity, and topic clustering are all tasks where a purpose-built model — trained or fine-tuned on your actual label distribution — outperforms a zero-shot LLM prompt on both speed and consistency. LLMs are non-deterministic by default; ask the same borderline ticket to be classified twice and you can get two different answers, which is a real problem for anything feeding a downstream automation or a metric someone's going to report on. A classifier trained on your data doesn't have that failure mode.

• Classification, tagging, and routing: a fine-tuned classifier beats a zero-shot LLM prompt on cost, latency, and consistency.
• Deduplication and similarity search: embedding models plus a vector index, not a model call per comparison.
• Anything at high volume with a fixed, known label set: classical NLP is the default, not the fallback.
• Open-ended generation, summarization with judgment calls, and anything requiring real reasoning: that's where the LLM budget should go.

The architecture that's actually winning in production

The teams getting this right aren't choosing between classical NLP and LLMs — they're routing. A lightweight classifier sits in front of the pipeline and handles the large majority of incoming requests directly; only the genuinely ambiguous or open-ended cases get escalated to a frontier model. That single design decision is usually the biggest lever available for cutting blended AI cost without touching the two things users actually notice: accuracy and response time. If your architecture diagram has every text request flowing through one large model, the router is the missing box, not a nice-to-have.