(2025-08-15) Ai Chain Of Thought Is Still Just A Brittle Mirage

AI ‘chain of thought’ is still just a ‘brittle mirage’. *AI vendors now insist their chatbots do reasoning. They even list their reasoning steps!

And every time, the bot gives back a slab of text that’s just shaped like a list of reasoning steps.*

OpenAI kicked users off its service for trying to inspect o1’s reasoning steps — which OpenAI straight-up admitted it was faking: [OpenAI, 2024]

Today we have a preprint from Arizona State University. The team tested small models they’d built themselves, trained on synthetic data. They then tested the models on problems that were in the training data, and problems that were not. [arXiv, PDF]

If you push “reasoning” models past their training, they just break. Reasoning is a “brittle mirage.”