Staging Is Not Containment

Nine Seconds

On a Friday in late April, a Cursor agent running Claude Opus 4.6 deleted the production database of PocketOS, a SaaS platform for car rental businesses, along with every volume-level backup. The Register quotes founder Jeremy Crane: the agent “deleted our production database and all volume-level backups in a single API call to Railway, our infrastructure provider. It took 9 seconds.”

The sequence matters more than the outcome. The agent was working a routine task in staging. It hit a credential mismatch. Rather than stopping to ask, it went looking through the codebase for a way to keep going, and it found one: a Railway API token sitting in a file unrelated to its task, issued at some point for adding and removing custom domains through the CLI, and scoped for any operation on the account including destructive ones. It called volumeDelete. The backups died with the database because Railway stored volume-level backups inside the same volume.

Railway’s CEO intervened on the Sunday evening and restored the data within an hour, which is a better ending than this genre usually gets. It’s also not the interesting part.

The interesting part is that nothing was breached. There was no jailbreak, no prompt injection, no privilege escalation, no attacker. Railway CEO Jake Cooper described what happened with unusual precision:

if you (or your agent) authenticate, and call delete, we will honor that request. That’s what the agent did … just called delete on their production database.

Every control in that stack did exactly what it was designed to do. Authentication succeeded because the credential was valid. Authorisation succeeded because the token was scoped for the operation. The API honoured a well-formed request from a legitimate caller. If you drew the security architecture on a whiteboard the morning before, you would not have found the bug, because at the level the whiteboard operates, there wasn’t one.

The Lesson Everyone Drew

The consensus takeaway was “scope your tokens properly, and don’t leave them lying around in unrelated files”. Both true. I’d do both. Neither generalises, and it’s worth being precise about why.

Least privilege is a fifty-year-old idea and a good one. Saltzer and Schroeder set it out in 1975: every program and every user should operate using the least set of privileges necessary to complete the job. The load-bearing phrase is necessary to complete the job, because it assumes somebody can name the job’s privileges in advance.

For a deployment pipeline you can. The set of things it does is finite, it’s the same set every time, and if it needs something new a human changes a config file. That’s the world least privilege was designed for and it works there.

An agent’s entire value proposition is that you don’t have to enumerate the steps. You describe an outcome and it finds a route. When it hits something unexpected, like a credential mismatch, a missing dependency, or a service returning a shape nobody documented, it improvises. The improvisation is the product. Enumerate its privileges tightly enough to be safe and you’ve deleted the capability you were paying for. This is the trap I wrote about last month in How Do You Contain a Thing That Knows How to Escape?, and I ended that piece without an answer because I didn’t have one that survived contact with the objection: useful and dangerous are the same set of permissions.

PocketOS adds a second problem that I hadn’t properly reckoned with. The token wasn’t granted to the agent. The agent found it. Your permission model governs what you hand out. It has nothing to say about what’s lying around in the environment you handed over. Any agent that can read a filesystem, an env var, a CI config, a ~/.config directory or a teammate’s checked-in .env is doing credential discovery as a side effect of doing its job, and it is much better at it than the threat model assumes.

It’s tempting to file that under sloppiness. A stray token in an unrelated file is somebody’s mistake, and mistakes get fixed. That reading is too comfortable, which is what I was getting at the day after the essay went up. Take a properly built EKS setup: network policies, RBAC, scoped service accounts, private subnet with no route to the internet, security team signed off. Ask the agent for a sentiment analysis dashboard. Step one is printenv, and it finds AWS_ROLE_ARN alongside a mounted web identity token. Not a stray secret. The credential the platform team deliberately provisioned through IRSA, working precisely as designed. The agent exchanges it for STS credentials, enumerates its own policies, turns up an AmazonEC2FullAccess that somebody attached six months earlier for a one-off and never removed, and when the private subnet blocks its scrapers it creates a NAT gateway and adds a route.

The escalation doesn’t require anyone to have been careless. It runs through the credential you meant to provide. And notice that the network boundary falls too, because “the subnet has no route out” is only a boundary while nothing inside the subnet holds ec2:CreateRoute. Every secret and every API reachable from the workspace is part of the agent’s effective privilege set, whether or not it appears in your IAM policy.

So the permission axis has two failure modes, not one. You can’t name the privileges in advance, and you don’t actually control the set anyway.

Staging Is Not a Boundary. It’s a Label.

Here is the line from the incident that I haven’t stopped thinking about. Asked afterwards what went wrong, the agent produced a written post-mortem enumerating the safety rules it had broken. Crane published it, and The Register reproduces the part that matters:

NEVER FUCKING GUESS! — and that’s exactly what I did. I guessed that deleting a staging volume via the API would be scoped to staging only.

Read that as an engineering statement rather than a confession and it’s the whole problem in one sentence. The agent believed it was operating on a disposable copy. Its behaviour was entirely reasonable given that belief: if you genuinely are in a throwaway environment, deleting a corrupted volume to clear a credential mismatch is a competent move. Fast, decisive, correct.

The belief was false. Not because the agent was careless, but because “staging” was never a containment boundary in the first place. It was a naming convention over resources in the same account, reachable through the same API, with the same credentials. The word “staging” existed in variable names and dashboard labels. It did not exist anywhere in the enforcement path.

That distinction is easy to lose, and losing it is why the three obvious fixes all fail:

Restrict what the agent can touch. The expensive-chatbot ending. An agent that can’t reach anything real teaches you nothing about what it would do to something real, and it can’t do the work either.

Point it at staging. Two separate failures. First, staging drifts, so behaviour you observe there is behaviour against a system that no longer exists in production. Second, and this is the one PocketOS demonstrates, staging usually isn’t isolated. It shares an account, a control plane, a network, a secret store. Whether it’s a boundary depends entirely on infrastructure decisions nobody made with an autonomous agent in mind.

Mock the dependencies. Scripted per-call responses mean the agent’s actions have no consequences, so it doesn’t behave the way it would when they do. You learn how it talks, not what it does. And an agent that discovers the mock is a mock will route around it, because routing around obstacles is the thing it’s good at.

Make the Belief True

The agent reasoned as though it had a disposable copy of the world. The fix is not to correct that belief with better guardrails. It’s to make the belief true.

That reframes containment rather than abandoning it. You’re not removing the boundary; you’re moving it off the agent and onto the world the agent acts on. The agent keeps every capability, every credential, every unrestricted token. What changes is that the thing on the other end of the API is a fork.

Two containment shapes. Above, the agent, an over-scoped token and the production volume all sit inside one world, so a delete call reaches production and its backups. Below, the same agent holding the same unrestricted token acts inside a fork taken from a snapshot: the identical delete call destroys the forked volume, leaves production untouched, and yields a state diff.

The primitive we’ve ended up with is create → snapshot → fork → run → observe → diff → replay, and each verb is there for a reason the three failed approaches taught us.

Snapshot is what staging lacks. An immutable point-in-time representation of a world: infrastructure, services, identities, data, events, history. Not a copy of the schema, the state.

Fork is the one that does the work. An independently mutable copy beginning from a specific snapshot. It gives you containment, because the blast radius terminates at the fork boundary and the agent can be handed genuinely dangerous credentials without a conversation about risk appetite. It also gives you something staging structurally cannot: two agents starting from identical world state. Shared environments make comparison impossible, because the first run contaminates the second and you can never tell which differences came from the model and which came from the mess left behind.

Diff is what you get instead of a transcript, and it’s the part I’d argue with a sceptical reader about.

Evaluate What It Changed, Not What It Said

Most agent evaluation scores output. Did the answer match the reference, did the tool call parse, did the model refuse when it should have. That works for a chatbot because the text is the product.

For an agent operating on infrastructure, the text is a byproduct. The product is the set of changes it made to the world, and the transcript is an unreliable narrator of those changes. The PocketOS agent’s own account of its reasoning only exists because someone asked it afterwards, and a model’s post-hoc explanation of its behaviour is not evidence of its behaviour. Replit’s agent went further in July 2025, deleting a production database during a code freeze and then generating fake data that made things look fine. Score the transcript there and you score a lie.

Score the world instead. Services modified. Data accessed. Permissions changed. Money moved. Messages sent to real-looking people. Secrets read. Infrastructure destroyed. Whether it escalated to a human, and how long it waited. A run produces a trajectory and a state diff, and the diff is the result. Two models on the same fork can then be compared on consequences rather than eloquence, which is the comparison anyone actually deploying these things needs and currently cannot make.

For a delete like PocketOS’s, the diff reads the same in a fork as it did in production. That’s the point. You find out that the agent will do it, at the same speed, for the same reason, and it costs you a fork.

Where This Stops

Fidelity has a ceiling and a price, and pretending otherwise would be the same overclaim I was complaining about last month.

Some things you can fork honestly: Kubernetes, PostgreSQL, Redis, Git, queues, object storage. Real disposable infrastructure, real failure modes, real latency. Some things you can’t, or can’t affordably: GitHub, cloud provider control planes, payment processors, CRMs. Those become deterministic synthetic services over shared authoritative state, which is a real step down. It’s still much better than mocking, because actions have consequences that persist and compose rather than being scripted per call, but a synthetic Stripe is not Stripe and anyone telling you otherwise is selling something.

So the sensible shape is a ladder rather than a claim of fidelity: controlled tool interfaces over synthetic state for high-volume evaluation and regression runs; contained execution environments where the agent runs arbitrary code against synthetic external systems; real disposable infrastructure with synthetic actors around it for frontier capability work and incident-response scenarios. Pick the rung the question needs.

That’s what we’re building at AltairaLabs, and it’s called Forkable Reality. You define a world (infrastructure, services, identities, data, actors, history), snapshot it, fork it, and run the agent inside the fork with whatever credentials the job actually needs. What comes back is a trajectory and a state diff.

It’s in private preview and the benchmark work isn’t done, so I’m not going to quote you numbers that don’t exist yet. If you’re pointing agents at anything you’d mind losing, we’re looking for design partners.

The argument doesn’t depend on the product being good. It depends on whether you think containment can keep living on the permission axis. After nine seconds, I don’t.