nowait-reasoning-optimizer

Implements the NOWAIT technique from the paper "Wait, We Don't Need to 'Wait'! Removing Thinking Tokens Improves Reasoning Efficiency" (Wang et al., 2025). NOWAIT is a training-free inference-time intervention that suppresses self-reflection tokens (e.g., "Wait", "Hmm", "Alternatively") during generation, reducing chain-of-thought (CoT) trajectory length by 27-51% without compromising model utility. Important : NOWAIT works best with RL-based models. Distilled models (Qwen3-4B/8B/14B) show degraded performance when reflection tokens are suppressed. See references/keywords.md for the complete list. Core keywords: Distilled models rely heavily on CoT structure from training data—removing reflection tokens disrupts their reasoning patterns. - Deploying R1-style reasoning models with limited compute - Reducing inference latency for production systems - Optimizing token costs for reasoning tasks - Working with verbose CoT outputs that need streamlining - Initialize Keywords : Identify reflection keywords from empirical analysis - Expand to Token Variants : Map keywords to all token variants in vocabulary (e.g., "wait" → " wait", "Wait", " Wait", ".wait", "WAIT") - Suppress During Inference : Set logits of reflection tokens to large negative values during decoding - NOWAIT doesn't eliminate self-reflection entirely—it guides models to skip unnecessary "waiting" reasoning - Models...