Knowing When To Think: A Training-Free Controller For Adaptive Reasoning Budgets in Large Language Models

Abstract

Reasoning models allocate extended chain-of-thought computation to nearly every query they receive, regardless of whether the query needs it. This paper argues that reasoning budget should function as a decision rather than a fixed setting, and it proposes a training-free controller that estimates query difficulty from two cheap, black-box-compatible signals: surface prompt features and short reasoning prefixes sampled before full generation begins. The controller routes each query into a direct-answer, brief-reasoning, or full-reasoning regime, defaulting to caution when signals conflict. Building on behavioral-signal uncertainty estimation methods from prior work, this paper extends that approach from post-hoc trust assessment to upfront budget allocation. An evaluation framework is outlined for characterizing the accuracy-token frontier and testing whether adaptive routing can reduce token expenditure without a proportional loss of accuracy, with implications for cost governance in deployed reasoning systems.