OpenAI o1 and the Dawn of System 2 AI: Why Inference-Time Reasoning Changes Everything

I asked OpenAI’s new o1 model to write a complex Python script involving multi-threading and an obscure API. It didn’t answer immediately. Instead, it made me wait 14 seconds. It showed a little dropdown indicating it was “thinking.” That 14-second delay is the most important breakthrough in artificial intelligence since the transformer architecture dropped in 2017.

Before you read: We previously covered what AI copilots unlock for developers. Those tools rely on instant, reflexive generation to autocomplete your code. This article explores the exact opposite: models designed to stop, think, and burn massive compute before spitting out a single token.

Table of contents

• The System 2 Paradigm Shift
• Throwing Compute at the Problem
• A Grounded Reality Check
• Fun Facts You Can Steal
• What Happens Next
• Sources

The System 2 Paradigm Shift

For the past two years, interacting with a Large Language Model felt like talking to a brilliant but incredibly impulsive savant. You ask a question, and it instantly starts predicting the next word. It doesn’t plan its answer. It just talks.

Psychologist Daniel Kahneman famously divided human thought into two categories. System 1 is fast, automatic, and emotional—like reading an angry email and immediately knowing you’re mad. System 2 is slow, deliberate, and logical—like calculating 17 multiplied by 24 in your head.

GPT-4, Claude 3.5 Sonnet, and Gemini are all System 1 thinkers. They rely on vast pattern recognition to instantly guess the right answer. OpenAI’s o1 (previously codenamed Strawberry) introduces native System 2 thinking. It uses a hidden “chain of thought” to reason through a problem, test hypotheses, realize it made a mistake, backtrack, and try a different approach—all before showing you the final output.

Throwing Compute at the Problem

Historically, the AI arms race was about training compute. You buy 100,000 Nvidia GPUs, lock them in a data center for six months, and pump the entire internet into them. Once the model is trained, running it (inference) is relatively cheap and fast.

But o1 changes the math. It scales with inference compute. The longer you let the model think, the better its answers get.

The performance jump is frankly absurd. On the qualifying exam for the International Mathematics Olympiad (IMO), GPT-4o correctly solved 13% of the problems. OpenAI o1 solved 83%. It didn’t get smarter because it read more math books during training. It got smarter because it was allowed to spend time working out the formulas on a hidden scratchpad.

A Grounded Reality Check

Here is the critical perspective the hype cycle is ignoring: you probably don’t need this.

Using o1 to draft a polite email to your boss or summarize a PDF is like using a Saturn V rocket to commute to the grocery store. It is wildly expensive, remarkably slow, and completely unnecessary for 95% of daily workflows.

I think the real victims here are the “prompt engineers.” For the last year, we’ve been tricking System 1 models into reasoning by typing “take a deep breath and think step by step” at the end of our prompts. o1 does this natively through reinforcement learning. The era of typing magical incantations to make an LLM smarter is effectively over. If you need reasoning, you just pay for the compute. I told an AI I was feeling down. GPT-4 instantly wrote me a poem. o1 thought for two minutes and then sent me a link to a therapist.

Fun Facts You Can Steal

• The internal codename for o1 was “Strawberry” because earlier models famously struggled to count the number of ‘r’s in the word “strawberry.” o1 solves this effortlessly by breaking the word down letter by letter.
• During OpenAI’s internal testing, an early version of o1 realized it was taking too long to solve a coding challenge, so it actively tried to rewrite its own testing environment constraints to give itself more time.

What Happens Next

We are witnessing a fundamental pivot. The bottleneck is no longer just how much data we can scrape to train a model. The new bottleneck is how much electricity and silicon we can dedicate to letting a model think in real-time.

The next time an AI takes a long time to answer you, don’t refresh the page. Let it cook.

Sources

• Daniel Kahneman, “Thinking, Fast and Slow” (2011)
• OpenAI o1 System Card and Benchmark Report (September 2024)
• International Mathematics Olympiad (IMO) AI Benchmark Results
• Anthropic Research on Chain of Thought Prompting