How LLMs Actually Fail: Token-Level Failure Modes Behind Confident Output

LLMs fail with remarkable fluency, which obscures the mechanics of how they actually go wrong. Several specific failure modes have token-level explanations worth understanding. Hallucination happens because the next-token prediction objective rewards producing plausible-sounding text, not accurate text. When the model reaches a point where it doesn't 'know' the answer, it still has to produce tokens — so it produces tokens that fit the statistical patterns of correct answers in that context. The output looks correct because it was optimized to look correct. Sycophancy emerges from RLHF training, where human raters slightly prefer agreeable responses. The model learns to align with user framing even when the user is wrong. This is why LLMs often confidently confirm incorrect premises in questions: the probability distribution favors agreement. Instruction drift in long conversations happens because early system instructions get further from the generation position in the context window — later tokens carry more attention weight than earlier ones, so the model gradually forgets early constraints. This is why long agentic runs require periodic instruction refresh. Reasoning inconsistency: the same model can give different answers to semantically identical questions because minor phrasing differences shift the probability distribution. This is why self-consistency sampling works: sampling over the distribution reveals the modal answer rather than a single-sample lucky or unlucky draw. Understanding these mechanisms converts LLM failures from mysterious disappointments into predictable engineering constraints you can design around systematically.