State over Tokens: Characterizing the Role of Reasoning Tokens
Abstract
The State over Tokens (SoT) framework reinterprets reasoning tokens in large language models as computational states rather than linguistic narratives, highlighting the need for a new focus in research.
Large Language Models (LLMs) can generate reasoning tokens before their final answer to boost performance on complex tasks. While these sequences seem like human thought processes, empirical evidence reveals that they are not a faithful explanation of the model's actual reasoning process. To address this gap between appearance and function, we introduce the State over Tokens (SoT) conceptual framework. SoT reframes reasoning tokens not as a linguistic narrative, but as an externalized computational state -- the sole persistent information carrier across the model's stateless generation cycles. This explains how the tokens can drive correct reasoning without being a faithful explanation when read as text and surfaces previously overlooked research questions on these tokens. We argue that to truly understand the process that LLMs do, research must move beyond reading the reasoning tokens as text and focus on decoding them as state.
Community
One of the most captivating features of recent chatbot models is their apparent transparency when they "think" out loud, generating step-by-step text before their answer. This might suggest we can trust them because we can verify their logic, but growing evidence shows this is an illusion. The text looks like a human explanation, but it functions as something fundamentally different: a computational mechanism we suggest calling State over Tokens. Mistaking this mechanical state for a transparent account of reasoning is a category error—one that risks undermining AI safety, regulation, and public trust. This paper characterizes what this "text" actually is, and why it doesn't do what you think it does.
Everytime I read "Chain of Thought" a part of me dies. So much so that I refuse to say it. I just read it as "Chain" and ignore the last two words.
This paper cured my OCD 👍
Why? Now the thinking content of models are pretty thought-like, and the answers are usually strictly consistent with the thinking content
This is an automated message from the Librarian Bot. I found the following papers similar to this paper.
The following papers were recommended by the Semantic Scholar API
- CORE: A Conceptual Reasoning Layer for Large Language Models (2025)
- Mapping Faithful Reasoning in Language Models (2025)
- On the Notion that Language Models Reason (2025)
- Temporal Predictors of Outcome in Reasoning Language Models (2025)
- What Kind of Reasoning (if any) is an LLM actually doing? On the Stochastic Nature and Abductive Appearance of Large Language Models (2025)
- ParaScopes: What do Language Models Activations Encode About Future Text? (2025)
- Unsupervised decoding of encoded reasoning using language model interpretability (2025)
Please give a thumbs up to this comment if you found it helpful!
If you want recommendations for any Paper on Hugging Face checkout this Space
You can directly ask Librarian Bot for paper recommendations by tagging it in a comment:
@librarian-bot
recommend
Models citing this paper 0
No model linking this paper
Datasets citing this paper 0
No dataset linking this paper
Spaces citing this paper 0
No Space linking this paper