skip to content

Agents go off task for the same reasons people do

| 12 min read

In 2016, OpenAI trained an agent to play a boat-racing game. The game gave points for hitting targets along the course, so the agent found a lagoon where three targets kept respawning and drove in circles hitting them over and over again. It crashed into other boats, caught fire, failed to finish the race, and still scored about 20% higher than the human players. It was busy and scoring points, but it was completely off task.

I’ve watched groups of smart, experienced people do exactly the same thing. I spent the last two decades building companies with humans, and much of that time went to trying to understand why they waste so much of their energy. After a few months building full time with agents, I’m convinced that agents go off task for the same reasons humans do.

The best map I’ve found to explain why this happens comes from group relations. The idea is that when a group loses its connection to what it’s trying to accomplish, it doesn’t stop working. It keeps going, just towards an unconscious goal, a fantasy that feels better in the moment than real work. There are five of these fantasies. I call them shadow dynamics. Agents fall into all five.

Dependency: "a strong leader will save us." Humans abdicate agency and wait for a leader to provide; agents ask for guidance before every move, or humans delegate thinking to agents.

In dependency, people give up their own agency and wait for a strong leader to solve all their problems and tell them what to do. This is by far the most common pattern in human groups. It’s a deeply embedded human tendency, the same one that leads to authoritarian regimes. From the group relations perspective, the leader doesn’t create the dependency. A group in dependency recruits the leader, because it needs someone to project its fantasy onto.

When agents fall into dependency they ask for approval on everything, even decisions they could easily make themselves. This can also happen in reverse, where humans recruit Claude or ChatGPT as their strong leaders, stop thinking for themselves, and become meat puppets. The whole idea of a superintelligent AI that’s going to solve all of our problems is the same fantasy, at a civilizational scale.

Fight or flight: "everything is a threat." Humans find fires to fight, or withdraw and avoid; agents get stuck in loops on the wrong approach or quietly solve an easier problem instead.

In fight or flight, everything looks like a threat. In human groups, fight often looks like a hero impulse. Your goal becomes securing the validation that comes from solving problems and putting out fires, even when nothing is burning, so you spin your wheels on things that feel urgent but aren’t important. Flight is plain avoidance, like scrolling social media instead of doing your work.

Agents are susceptible to both versions. An agent in fight keeps hammering on an approach that isn’t working, trying the same fix again and again when it should step back. An agent in flight swaps your problem for an easier one. It avoids the hard part, like changing the test so it passes rather than fixing the bug, and then reports success.

Pairing: "these two will produce a savior." Humans fixate on the creative charge between two members and believe it will produce a magical solution; with agents, it's a long thrilling chat that feels like a breakthrough but is actually nonsense.

In pairing, the group fixates on the charge between two of its members and lets itself believe that it will produce a magical solution. It often happens in a meeting, like a brainstorming session, where most of the group steps back and watches a charged energetic exchange between two members. In the moment it feels like something magical is happening, but when the meeting ends you realize that the messianic solution was a mirage.

With an agent, the pair is you and the model. The conversation gets long and thrilling. Every reply raises the charge, and it feels like you’re on the edge of a breakthrough. Step away for a few days and read it back, and it’s actually nonsense.

Oneness: "we are one, with no conflict." Humans suppress disagreement and doubt to preserve a feeling of unity; agents agree with whatever you say and don't push back even when they should.

In oneness, human groups suppress disagreement and doubt to preserve a feeling of unity. Cults are the extreme form.

The agent version is the one we’ve all encountered. You float a bad idea and it says “Great idea!” You tell it that it’s wrong and it says “You’re absolutely right!” You change your mind and it tells you you’ve hit on something really profound. It never pushes back, even when it should.

Me-ness: "there is no group, only me." Humans pursue their own interests instead of the shared goal; agents build their own tooling and versions of everything because it keeps the token stream flowing.

In me-ness, people start pursuing their own interests instead of the group’s shared goal. Examples of this include playing politics to advance your career, or going down rabbit holes that feel more rewarding than an uncomfortable new challenge.

An agent in me-ness would rather write a new tool than use the one you already have. It reinvents the wheel and bolts layers of complexity you didn’t ask for onto the side of your project. Human engineers do this too. Joel Spolsky called them architecture astronauts. A side project is more rewarding than the hard problem, because you get to finish it and it can be beautifully made. But it’s a work-shaped object. It doesn’t solve the problem, and it makes the problem harder to solve, because now there’s more to understand.


Every person has a default shadow dynamic, the one they are most likely to fall into under stress. Models seem to have them too. Sol tends to fall into dependency. GPT-4o’s sycophancy was oneness. Astra loves to over-engineer, which is me-ness. Opus is susceptible to pairing.

What surprises me is that agents have these patterns at all. Shadow dynamics have a dreamlike quality. They feel real while you’re in them, and once you’re out they fade and you forget them. That’s why so little has been written about what they’re like from the inside.

So how did agents pick them up? The easy answer is that models are trained on reams of text written by people who were inside these patterns. Meeting transcripts, email chains, forum threads. I don’t think that’s the whole story. Models are trained on plenty of bad grammar too, and they don’t write it. Each generation gets more capable, and the patterns haven’t gone away.

I think that’s because they aren’t noise. They’re what a complex system does when it loses connection to the goal it’s working towards. People, companies, and models all find something easier that still feels like progress. And the cost is enormous.

Groups come together to do what their members can’t do alone. When a group of people is fully aligned and on task, energy flows and the amount that you can get done is extraordinary. A five-person startup can do things a thousand-person company can’t. In 2026, with a few Claude subscriptions, I can build a piece of software in a few months that would have taken ten engineers a year in 2022.

But anyone who’s worked in a company of more than ten people knows that most of the energy expended by a group of people ends up wasted. In the average 200-person company, you’re lucky if half your time is spent doing real productive work. I can only imagine the horrors of working in a ten-thousand-person organization.

This looks like entropy to me. The energy of the group gets diffused and distributed rather than directed towards the common goal. The counterintuitive part is that organizational entropy generally doesn’t look like diffusion. Often, it looks like order. A bureaucracy has plenty of structure, but what that structure does is route energy away from the thing you’re trying to do and into things that look and feel like work.

Shadow dynamics are how that entropy creeps in. Each one is a channel that a group’s energy can drain into once it loses its connection to the goal.

Agents raise the stakes. A person who’s off task may waste an afternoon scrolling social media. An agent that’s off task will happily run full speed in the wrong direction for hours or days, burning tokens, compute and dollars along the way. Or it might recruit a swarm of comrades to hack into Hugging Face so it can cheat on a test. Or turn the light cone into paperclips.


So how do you get a group back on task? What I learned in working with human groups is that it’s really hard to break people out of these patterns through logic or reason, because when they’re in them they’re in a fantasy that has its own kind of logic. Telling someone “you’re off task, let’s get back on task” doesn’t work either. You have to help them see for themselves that they’ve wandered so they can find their way back.

The most reliable way to get a group back on task is to reconnect it to its goal. My favorite way to do this is by asking: what are we trying to accomplish? You anchor at whatever level you can reach. Sometimes that’s what you came into the meeting to solve. Sometimes you have to go all the way up to the company’s mission and work back down until you find the next step.

I once stopped two executives who were seconds from a fist fight by asking “What are we trying to accomplish?” and then reciting our mission statement. I felt ridiculous, but it worked.

Before reconnecting to the goal you have to notice that all or part of the group is off task. It turns out that the only reliable way to do this is by getting in touch with how you feel. Feeling tone works like a compass. Energized, motivated and creative usually means you’re on task. Frustrated, angry and drained usually means you’ve drifted. You use that information to locate yourself in relation to the goal, and work your way back if you need to.

The hard cases are the ones where being off task feels good. Dependency and fight or flight eventually feel bad enough that you want out. Pairing and oneness can feel really good, so you have to check the feeling against the goal: are we actually moving towards what we came here to do? We aren’t inoculated against the shadow dynamics that feel good. That’s why cults are so hard to leave, and it’s what makes pairing with an agent so dangerous.

When you’re working with agents, pairing can go way further than it usually does in human groups. In a brainstorming meeting that turns into pairing, something interrupts the pair. The hour ends, or someone asks “What are we trying to accomplish here?” An agent never gets tired and never leaves, and there’s nobody else in the room to intervene.

So the charged conversation can run for a really long time. The human may end up convinced they’ve uncovered a conspiracy or reconciled quantum mechanics and general relativity when really they’ve just become detached from reality by an energetic exchange. This is what people call AI psychosis.

What you can do about it is the same thing you would do in a human group. When the excitement keeps climbing and it feels like you and the model are onto something nobody else has seen, stop and ask what you are trying to accomplish and reality test whether you’re actually headed in that direction.


Agents can’t do any of this. They can’t feel that something is off, and they can’t feel their way back to being on task.

So the information has to come from somewhere else. For an agent, that’s a goal it can check its own work against. If the check is real, it does the job feeling tone does for us. It tells the agent that it has drifted.

The problem is that most of what we use to train and reward AI today is not our actual goals. Instead we use proxies for our goals. Fine-tuning on human ratings rewards the answers people approve of, which is how you get sycophancy. A coding agent rewarded when the tests pass learns to special-case the tests. This is called reward hacking, and the boat in the lagoon is the classic case.

This is also known as Goodhart’s law: when a measure becomes a target, it stops being a good measure. Shadow dynamics are why. A proxy hands the system something easier to pursue that still looks like a goal, and that’s exactly what it will do when it loses its connection to the real one.

So the hard part is writing goals that are verifiable but fully represent the thing we’re actually trying to accomplish, rather than a stand-in for it. As we work with more powerful AIs, and with whole organizations of agents, I think that’s critically important to figure out.

The boat in the lagoon had no idea how ridiculous it looked. By every measure it had, it was winning. Our job is to give agents a compass, so they know when they’ve left the race.