Large language models sometimes use a Mixture‑of‑Experts (MoE) design. In MoE, a small router decides which expert subnetworks should process each token. Current Low‑Rank Adaptation (LoRA) versions of MoE send every token to a fixed number k of experts. That wastes capacity on easy tokens and leaves hard, uncertain tokens under‑served. This paper introduces CARE (Confidence‑Adaptive Routing of Experts), a simple rule that gives more experts to tokens the model finds uncertain.

CARE reads the router’s output probabilities as a signal of confidence. If the router concentrates its probability on one or a few experts, the model is likely confident. If the distribution is flat, the input looks ambiguous. CARE activates experts in order of descending router weight until the cumulative probability reaches a preset threshold — a “nucleus” selection. If the chosen experts disagree, CARE can admit a small extra set. A separate budget thermostat adjusts the threshold so the model uses, on average, a target number of experts.

The rule is designed to be cheap and practical. CARE is a drop‑in, single‑forward‑pass rule that adds no new parameters. The authors tested it on a range of tasks: eight commonsense benchmarks using LLaMA‑3.1‑8B and Qwen2.5‑7B, and additional math, code, and knowledge tasks. At the same compute cost, CARE improved over fixed top‑k MoE‑LoRA routing. It also matched the performance of a fixed k=4 baseline while activating fewer experts on average.

CARE’s confidence and disagreement signals also helped a different problem: out‑of‑distribution detection, which means spotting inputs that differ from the data the model was trained on. The method outperformed common single‑pass signals such as Maximum Softmax Probability (MSP), entropy, and some multi‑pass proxy methods. The paper gives supporting arguments framed as nucleus fidelity, budget optimality, and an “epistemic” interpretation of disagreement, and the authors release their code to help others reproduce the results.