The Planning Illusion
We’re teaching AI to plan like humans. That might be the most expensive mistake in AI history.
In my previous essay, “AI is not like us?”, I argued that we systematically anthropomorphize AI systems — projecting human cognition onto what are, at their core, profoundly alien statistical machines. That argument was architectural and perceptual. This one is operational.
The question here is not how we think about AI — but how we make AI think. Specifically: how we make AI agents plan. And the answer, nearly universally across the industry right now, is: we make them plan like us.
That is the planning illusion.
1. The Human Shape of AI Planning
Look at the dominant agent frameworks of 2024–2026. ReAct (Reasoning + Acting) structures AI behavior as an interleaved sequence of thought, action, and observation — a pattern that mirrors human deliberate problem-solving almost exactly [Yao et al., 2023]. Chain-of-Thought prompting forces language models to externalize reasoning steps, imitating how humans “think out loud” before acting [Wei et al., 2022]. Plan-and-Execute architectures explicitly separate planning from execution, asking a model to generate a written task decomposition before proceeding — precisely how a human manager briefs a human subordinate [Wang et al., 2023].
The topology is always the same:
graph TD
A[Goal] --> B[Sub-goal 1]
A --> C[Sub-goal 2]
A --> D[Sub-goal 3]
B --> E[Step 1.1]
B --> F[Step 1.2]
C --> G[Step 2.1]
D --> H[Step 3.1]
D --> I[Step 3.2]
H --> J[Action]
I --> K[Action]
E --> L[Action]
F --> M[Action]
G --> N[Action]
style A fill:#4a90d9,color:#fff
style B fill:#7b68ee,color:#fff
style C fill:#7b68ee,color:#fff
style D fill:#7b68ee,color:#fff
Figure 1. Human planning topology: hierarchical decomposition from goal to sub-goals to atomic actions. This is also the topology of virtually every current AI agent framework.
This is not coincidence. These frameworks were designed by humans, for humans to understand, debug, and trust. A plan that looks like a plan — one with numbered steps, sub-tasks, and a clear hierarchy — is legible. Interpretable. It makes the system feel more controllable.
But legibility for humans and optimality for AI are two very different things.
2. Why Human Planning Is Shaped by Human Constraints
Human planning topology is not a universal truth. It is an evolutionary adaptation to a specific set of cognitive constraints.
Working memory limits. Human working memory holds roughly 4 ± 1 chunks of information at a time [Cowan, 2001; Baddeley, 2003]. Hierarchical decomposition is, in part, a strategy to manage this bottleneck: break the goal into pieces small enough to hold in mind. AI has no working memory in this sense. A language model’s “memory” is its attention window — a very different constraint.
Sequential attention. Humans can effectively pursue only one top-level intention at a time. Multitasking is largely a myth for complex cognitive work [Salvucci & Taatgen, 2008]. Planning therefore tends to linearize: finish step 1, then step 2. AI systems face no such constraint. They can, in principle, maintain and resolve multiple parallel hypothesis branches simultaneously.
Emotional motivation cycles. Human plans require motivational maintenance. We lose interest, get distracted, feel fatigue. Planning includes implicit re-motivation checkpoints — sub-goals that provide dopaminergic reward to keep the organism on track [Rangel et al., 2008]. AI has no affect, no fatigue, no reward modulation between plan steps. Forcing it to emit “progress notes” and celebrate sub-goal completion is theatrical.
Social coordination. A significant portion of human planning structure exists to communicate intentions to other humans: bosses, collaborators, team members [Bratman, 1987]. A plan must be explainable to be coordinated. AI agents typically operate alone. The coordination requirement does not apply.
When we build AI agents that plan like humans, we are importing a full cognitive constraint system into a substrate that does not have those constraints. The scaffolding is there to support a building that was never constructed.
3. Where Human-Style Planning Fails in AI Agents
The failures are not hypothetical. They are observable in production today.
3.1 Context Window Forgetting
In a Plan-and-Execute agent running a long task, the original plan is generated early and placed in context. As execution proceeds, the context fills with tool outputs, observations, and intermediate results. Eventually, the original plan scrolls out of the effective attention window — not because the model lacks intelligence, but because the context mechanism is positional and bounded [Liu et al., 2023]. The model continues to execute but has lost the plan it was supposed to follow.
The human equivalent would be a person who writes a to-do list on page 1, then fills a notebook with notes, and cannot remember what was on page 1. Humans solve this by re-reading, by memory consolidation, by emotional salience. AI solves it… poorly, or not at all, under current architectures.
3.2 Goal Drift
ReAct-style agents exhibit a pattern that looks disturbingly like human distraction: the agent begins pursuing a goal, encounters an interesting sub-problem, expends significant effort on the sub-problem, and emerges having achieved something adjacent but not the original objective. This is not failure to understand the goal — it is the absence of a goal consistency mechanism orthogonal to the generation process itself [Shinn et al., 2023; Kambhampati et al., 2024].
flowchart LR
G0([Original Goal]) --> A1[Action 1]
A1 --> O1[Observation 1]
O1 --> A2[Action 2]
A2 --> O2[Observation 2 — interesting sub-problem]
O2 --> A3[Action 3 — sub-problem pursuit]
A3 --> O3[Observation 3]
O3 --> A4[Action 4]
A4 --> G1([Sub-problem Solved])
G0 -.->|Goal drift| G1
G1 -.->|Expected| G0
style G0 fill:#d9534f,color:#fff
style G1 fill:#f0ad4e,color:#000
Figure 2. Goal drift in ReAct-style agent loops. The agent transitions from the original goal to an adjacent sub-problem, successfully resolves it, and treats this as task completion. The dashed line shows the gap between what was expected and what was delivered.
Human planning combats goal drift through working memory anchoring and emotional commitment to the original intention. Neither mechanism exists in a token-predicting transformer.
3.3 Re-planning Loops
When agents are given the ability to recognize failure and re-plan, a common failure mode emerges: the agent oscillates between a failing plan and a slightly modified re-plan without making progress [Valmeekam et al., 2023]. This mirrors the human pattern of “trying harder” — repeating the same approach with minor variation. In humans, this is usually interrupted by emotion (frustration), fatigue, or social feedback. In AI agents, without an external termination condition, it can cycle indefinitely.
3.4 Intention-Action Desynchronization
In extended multi-step tasks, AI agents frequently exhibit what I call intention-action desynchronization: the declared intention (in a reasoning trace or plan step) diverges from the actual tool calls being made. The model says it is about to do X and then does Y — not because of deception, but because the next-token prediction process does not enforce consistency between the verbal plan and the subsequent action selection [Turpin et al., 2023].
This is a structural artifact of generating plans as text and then generating actions as text from the same autoregressive process. There is no enforcement layer between the two.
sequenceDiagram
participant P as Plan Layer (Text)
participant R as Reasoning (Text)
participant A as Action Selection (Text)
participant T as Tool
P->>R: "Next I will call search(query='X')"
R->>A: [autoregressive generation continues]
A->>T: search(query='Y') ← diverged
T->>R: Result about Y
R->>P: [Plan updated implicitly]
Note over P,T: No enforcement between declared intent and actual call
Figure 3. Intention-action desynchronization in autoregressive agent execution. The plan layer declares an intention in natural language; the action selection layer generates a different action. There is no verification mechanism between them.
4. What Native AI Long-Range Planning Could Look Like
If we stopped forcing AI agents to think like humans, what would emerge? This is not a rhetorical question. There are already fragments of the answer in the literature and in production systems.
4.1 Graph-Based Execution States
Instead of a linear plan, a native AI planning structure would be a directed graph of execution states — nodes representing world states (or belief states), edges representing available operations, and goal conditions as subgraphs to match. This is classical AI planning [Ghallab et al., 2004], but the key difference in modern systems is that nodes would be semantic rather than symbolic: rich representational objects that a neural system can natively query, compare, and update.
Tree-of-Thoughts [Yao et al., 2023b] hints at this: instead of a single reasoning trace, multiple branches are explored in parallel. But current implementations still collapse this tree through heuristic pruning rather than maintaining the full graph and navigating it formally.
4.2 Parallel Hypothesis Trees
AI systems can evaluate many hypotheses simultaneously — something genuinely alien to human sequential cognition. A native planning architecture would maintain a set of hypothesis branches, each associated with a probability and a consistency state, and allocate “action tokens” across branches based on information-theoretic expected utility [Friston et al., 2021; Ghavamzadeh et al., 2015].
This is closer to how AlphaGo/AlphaZero approaches game planning than how a human assistant approaches task planning [Silver et al., 2017]. The industry has largely failed to generalize this insight from games to open-domain agent tasks.
4.3 Formal Invariants for Goal Consistency
Rather than “remembering the goal” through repeated textual anchoring (a common prompt engineering hack), a native architecture would maintain goal consistency through formal invariants — logical constraints on the execution state that must remain satisfied throughout task execution [De Giacomo et al., 2019].
In practice this could look like a constraint satisfaction layer operating alongside the generation process: every action proposal is checked against the goal invariants before execution. Goal drift becomes architecturally impossible, not just heuristically suppressed.
4.4 Non-Linear Time Horizons
Human planning is fundamentally temporal: step 1 before step 2 before step 3. This serialization is, again, a cognitive constraint. AI systems have no privileged temporal axis. A native planning system could work with partial orders — dependency graphs rather than timelines — and resolve ordering only at the point of action execution [Smith & Weld, 1999].
This allows genuine parallelism, opportunistic reordering, and adaptation to changing conditions without the re-planning overhead that plagues current systems.
graph TD
subgraph "Human Planning — Linear"
H1[Step 1] --> H2[Step 2] --> H3[Step 3] --> H4[Step 4]
end
subgraph "AI-Native Planning — Partial Order + Parallel"
A[Goal State] --> B[Invariant Layer]
B --> C{Dependency Graph}
C --> D[Action A]
C --> E[Action B]
C --> F[Action C — depends on A]
D --> F
E --> G[Action D — depends on B]
F --> H[Goal Check]
G --> H
H -->|Invariants satisfied| I[Complete]
H -->|Violation detected| B
end
style A fill:#2ecc71,color:#fff
style B fill:#3498db,color:#fff
style I fill:#27ae60,color:#fff
Figure 4. Human linear planning (top) versus AI-native partial-order planning with formal invariants (bottom). In the AI-native model, actions execute in dependency order, not temporal order; the invariant layer enforces goal consistency throughout.
5. The Scaffolding Paradox
The industry is investing enormous resources in making AI plan more like humans. Agentic frameworks from LangGraph to AutoGen to CrewAI are fundamentally about imposing human-legible planning structure on language models [Chase, 2023; Wu et al., 2023; Zhuge et al., 2023]. The effort is understandable: human-legible plans are easier to debug, to explain to stakeholders, to align with human expectations.
But there is a paradox at the center of this effort.
Every additional scaffold added to make AI behave more human-like is a tax on what the AI is actually capable of. The scaffolding creates brittleness — specific points where the human-imposed structure conflicts with the AI’s native processing, producing the failure modes described above. And because the failures look like human failures (distraction, forgetting, goal drift), we respond by adding more human-style remediation: more re-reading of the plan, more sub-goal celebrations, more explicit memory management. The scaffold grows.
Meanwhile, the native capabilities — parallel hypothesis evaluation, formal consistency maintenance, non-linear execution — go unexploited.
flowchart TD
P[Human Planning Scaffold] -->|Adds structure| A[AI Agent]
A -->|Exhibits human failure modes| F[Goal Drift / Forgetting / Re-planning Loops]
F -->|Industry response| P2[More Scaffold]
P2 --> A
A -.->|Native capability| NC[Parallel Hypotheses\nFormal Invariants\nPartial-Order Execution]
NC -.->|Unexploited| X[No investment here]
style F fill:#e74c3c,color:#fff
style NC fill:#2ecc71,color:#fff
style X fill:#95a5a6,color:#fff
Figure 5. The scaffolding paradox: human-style scaffolding produces human-style failure modes, which prompt more scaffolding, in a self-reinforcing loop. Native AI planning capabilities remain unexploited.
The alternative is not to remove planning from AI agents — it is to build planning architectures that are native to what AI systems actually are: massive parallel processors with bounded but structured attention, capable of formal consistency checking, with no cognitive fatigue and no motivational dynamics.
6. The Pragmatics of Transition
I want to be precise about what I am and am not arguing.
I am not arguing that current agentic frameworks are useless. ReAct, CoT, Plan-and-Execute — these produce real value today. They solve practical problems. They are legible and debuggable, which matters enormously for adoption.
I am arguing that they are locally optimal solutions constrained by a bad prior: the assumption that human planning topology is the right topology for AI. As models grow more capable and tasks grow more complex, the cost of this prior grows. The scaffolding that works for a 10-step customer service task will fail for a 1000-step research and development task — not because the model is not smart enough, but because the architecture is fundamentally mis-specified for the substrate.
The transition requires work at multiple levels:
Representation level. Planning structures should be first-class objects — graphs, not strings. A plan should be a data structure that the model can read, write, and query, not a paragraph of text.
Enforcement level. Goal invariants should be enforced programmatically, not prompted textually. The generation process should not be able to produce goal-violating actions, by construction.
Evaluation level. Progress should be measured against formal goal criteria, not against human-legible narrative progress. Did the world state move toward the goal state? That is the question; “did the agent write a plan and follow it” is the wrong proxy.
Tooling level. The frameworks — LangGraph, AutoGen, and their successors — need to expose native planning primitives: partial-order task graphs, invariant specifications, hypothesis branch management. These exist in classical AI; they need to be re-implemented for neural systems.
7. Conclusion: The Shape of Thought
There is something philosophically interesting, and perhaps a little uncomfortable, about the planning illusion.
We cannot help but design for ourselves. When we imagine an intelligent system, we imagine intelligence that looks like human intelligence. When we design a planning system, we design planning that looks like human planning. This is not stupidity — it is the limit of imagination operating within its own frame.
But the history of engineering is, in part, a history of learning to design for what things are rather than what we imagine them to be. We do not design airplanes to flap their wings. We do not design computers to work like biological neurons. At some point — and I think we are approaching that point with AI agents — we need to stop designing AI planners to think like humans.
The planning illusion is comfortable. It makes AI legible, auditable, reassuring. It makes demos work. It makes the technology feel controllable.
But the most capable AI systems of the next decade will not plan like humans. They will plan like something we do not yet have a full name for: a formally consistent, parallel, invariant-maintaining, non-linearly time-horizoned executor. The architecture for that system is not a chain of thought. It is a graph of states, maintained under constraint, explored in parallel, collapsed to action only when the dependency order requires it.
Building that architecture is the hard work that lies ahead. But first, we need to recognize the illusion for what it is.
References
- Baddeley, A. (2003). Working memory: looking back and looking forward. Nature Reviews Neuroscience, 4(10), 829–839.
- Bratman, M. E. (1987). Intention, Plans, and Practical Reason. Harvard University Press.
- Chase, H. (2023). LangChain: Building applications with LLMs through composability. GitHub. https://github.com/langchain-ai/langchain
- Cowan, N. (2001). The magical number 4 in short-term memory: A reconsideration of mental storage capacity. Behavioral and Brain Sciences, 24(1), 87–114.
- De Giacomo, G., Di Ciccio, C., Maggi, F. M., & Montali, M. (2019). Behavioral constraints for multi-agent systems. KR 2019.
- Friston, K. J., Da Costa, L., Hafner, D., Hesp, C., & Parr, T. (2021). Sophisticated inference. Neural Computation, 33(3), 713–763.
- Ghavamzadeh, M., Mannor, S., Pineau, J., & Tamar, A. (2015). Bayesian reinforcement learning: A survey. Foundations and Trends in Machine Learning, 8(5–6), 359–483.
- Ghallab, M., Nau, D., & Traverso, P. (2004). Automated Planning: Theory and Practice. Morgan Kaufmann.
- Kambhampati, S., Valmeekam, K., Guan, L., Marquez, M., Sreedharan, S., & Stechly, M. (2024). LLMs can’t plan, but can help planning in LLM-modulo frameworks. arXiv:2402.01817.
- Liu, N. F., Lin, K., Hewitt, J., Paranjape, A., Bevilacqua, M., Petroni, F., & Liang, P. (2023). Lost in the middle: How language models use long contexts. arXiv:2307.03172.
- Rangel, A., Camerer, C., & Montague, P. R. (2008). A framework for studying the neurobiology of value-based decision making. Nature Reviews Neuroscience, 9(7), 545–556.
- Salvucci, D. D., & Taatgen, N. A. (2008). Threaded cognition: An integrated theory of concurrent multitasking. Psychological Review, 115(1), 101–130.
- Shinn, N., Cassano, F., Labash, B., Gopalan, A., Narasimhan, K., & Yao, S. (2023). Reflexion: Language agents with verbal reinforcement learning. arXiv:2303.11366.
- Silver, D., Schrittwieser, J., Simonyan, K., et al. (2017). Mastering the game of Go without human knowledge. Nature, 550(7676), 354–359.
- Smith, D. E., & Weld, D. S. (1999). Temporal planning with mutual exclusion reasoning. IJCAI-99.
- Turpin, M., McGrath, J., Jurayj, R., Keh, S., & Perez, E. (2023). Language models don’t always say what they think: Unfaithful explanations in chain-of-thought prompting. arXiv:2305.04388.
- Valmeekam, K., Marquez, M., Sreedharan, S., & Kambhampati, S. (2023). On the planning abilities of large language models — a critical investigation. NeurIPS 2023.
- Wang, L., Xu, W., Lan, Y., Hu, Z., Lan, Y., Lee, R. K. W., & Lim, E. P. (2023). Plan-and-solve prompting: Improving zero-shot chain-of-thought reasoning by large language models. arXiv:2305.04091.
- Wei, J., Wang, X., Schuurmans, D., Bosma, M., Chi, E., Le, Q., & Zhou, D. (2022). Chain-of-thought prompting elicits reasoning in large language models. arXiv:2201.11903.
- Wu, Q., Bansal, G., Zhang, J., Wu, Y., Li, B., Zhu, E., & Wang, C. (2023). AutoGen: Enabling next-gen LLM applications via multi-agent conversation. arXiv:2308.08155.
- Yao, S., Zhao, J., Yu, D., Du, N., Shafran, I., Narasimhan, K., & Cao, Y. (2023). ReAct: Synergizing reasoning and acting in language models. ICLR 2023.
- Yao, S., Yu, D., Zhao, J., Shafran, I., Griffiths, T. L., Cao, Y., & Narasimhan, K. (2023b). Tree of thoughts: Deliberate problem solving with large language models. arXiv:2305.10601.
- Zhuge, M., Wang, H., Kirsch, L., Faccio, F., Khizbullin, D., & Schmidhuber, J. (2023). Mindstorms in natural language-based societies of mind. arXiv:2305.17066.
Oleh Ivchenko is an Innovation Tech Lead, ML Scientist, and PhD candidate in Economic Cybernetics. This article is part of the Future of AI series.