Cheap text classification at scale: not one model, but a cost-ordered, calibrated cascade
Asked:
“Give me a report on the best methods to do cheap text classifiers for classifying a lot of text without having to run the text through a full LLM”
Eight methods and production strategies — from deterministic rules and sparse linear models to fastText, SetFit, distilled transformers, and cascade architectures — each scored 1–5 on inference economy, semantic flexibility, and label efficiency (5 is better on every axis), plus a six-stage implementation roadmap. Published metrics come from heterogeneous benchmarks and are not head-to-head comparable.
Method chooser: three scores per method
Dot size and colour encode the 1–5 score; hover a row for cost profile, best fit, and tradeoff. Recommended default path highlighted.
Executive recommendation. Start with rules for exact cases and a TF-IDF n-gram + logistic-regression baseline (0.772 accuracy on scikit-learn's sparse-feature benchmark); wrap whatever wins in a confidence cascade that auto-accepts easy cases and escalates the rest — one cascade study reports up to 90% lower inference cost and 20× faster large-scale knowledge mining.
arxiv.org
For extreme throughput, many labels, or multilingual noisy text, fastText's paper reports accuracy often on par with deep classifiers while training and evaluating many orders of magnitude faster.
arxiv.orgWhen paraphrase is the failure mode and labels are scarce, SetFit with a 110M sentence encoder and linear head reaches 66.9 RAFT accuracy with no generative decoding and no prompt tokens.
huggingface.coReach for a distilled transformer only after cheaper baselines fail on a representative error set: DistilBERT retains 97% of BERT's capability with 40% fewer parameters and 60% higher speed, but tokenization and attention still dominate CPU cost.
ar5iv.labs.arxiv.orgTraining-data strategies are not inference models: weak supervision manufactures labels at reported 10–100× less effort, and active learning spends the labeling budget where the classifier is least certain.
snorkel.ai
Winner by situation
Research synthesis from the linked sources; the shaded card is the recommended production architecture.
Staged architecture: six gates from gold set to cascade
Selection must rest on a representative immutable test set, per-class metrics, calibration, coverage, end-to-end throughput, and the cost of escalations.
Evaluation and production checklist
- Benchmark on target hardware, end to end — include preprocessing and tokenization, never model forward time alone.
- Batch offline workloads; record documents/second, p95 latency, model bytes, and preprocessing time.
- Judge with macro-F1, per-class recall, and a confusion matrix — never accuracy alone on imbalanced classes.
- Calibrate confidence and set per-class abstention thresholds from risk–coverage curves.
- Cascade budget = cheap inference on all items + fallback cost × escalation fraction.
- Monitor drift, selective accuracy, escalation rate, calibration error, and fallback spend by class and source.
- Lower labeling cost with weak supervision and active learning; keep immutable test slices and control sampling bias.
- Distinguish training-data strategies (weak supervision, active learning) from inference models — they combine, not compete.
All eight methods, with published metrics
Scores are 1–5, 5 better. Published metrics come from different datasets and setups; do not read them as a ranking.
| Method | Family | Econ | Sem | Label | Published metric | Source |
|---|
Research synthesis over 8 methods/strategies (three 1–5 scores each: inference economy, semantic flexibility, label efficiency — 5 is better) and a 6-stage implementation roadmap, compiled from the linked sources. Scores and recommendations are the synthesis; published metrics are source-specific figures from heterogeneous datasets and are not directly comparable. Full cost-profile and tradeoff text appears in the cards; the table trims long prose for space.