LLM Deep Thinking: Reasoning Models, Techniques, Evaluation, and the Landscape

Introduction

For most of their short history, large language models (LLMs) generated answers in a single pass — one forward propagation through the network, producing one token at a time from left to right with no backtracking, no revision, and no second thoughts. This is analogous to human System 1 thinking: fast, instinctive, and automatic.

That paradigm changed radically in late 2024. The release of OpenAI's o1 preview, followed rapidly by DeepSeek-R1, Anthropic's Claude 3.5 Sonnet (with extended thinking), and Google's Gemini Flash Thinking, introduced System 2 thinking to AI: slow, deliberate, analytical, and logical. By spending extra compute at inference time (rather than just training time), these models generate hidden reasoning steps before outputting their final response.

In this deep dive, we explore how reasoning models work internally, techniques to leverage them, current evaluation benchmarks, and the frontier of System 2 scaling laws.

1. How Reasoning Models Work

Standard LLMs predict the next token based on raw statistical likelihood from their training data. If they start down a wrong logical path, they cannot correct themselves; they must continue generating text from that point. Reasoning models solve this by generating a structured Chain of Thought (CoT).

The Inference-Time Execution Cycle

At runtime, the model decouples its internal thoughts from the final response. It iterates through planning, evaluating steps, correcting mistakes, and validating answers in a loop before returning the final text to the user.