StockDuty ← Dashboard Nov 17, 2026 07:40 PM ET
SignalTech/AI

Not Everything Needs an LLM

High-priority alert based on source significance and keyword relevance.

What happened: PostHog published findings on using decision models, which return typed answers with probability scores, instead of LLMs for simpler tasks. Several use cases have reached production, including gating expensive LLM calls and routing requests. Key numbers: Decision models are up to 444.6x cheaper and 193.6x faster than frontier Why it matters: The post argues decision models make uncertainty explicit, since a calibrated 0.8 score is right about 80% of the time. It also warns thresholds do not carry over between tasks and models, so they must be set per task and per model.
PostHog published findings on using decision models, which return typed answers with probability scores, instead of LLMs for simpler tasks. Several use cases have reached production, including gating expensive LLM calls and routing requests.
The post argues decision models make uncertainty explicit, since a calibrated 0.8 score is right about 80% of the time. It also warns thresholds do not carry over between tasks and models, so they must be set per task and per model.
Sources