AI

Sandbox the code. Govern the workflow.

Andre Elizondo

By: Andre Elizondo

8 min read

Isolation can constrain an agent’s environment. Production autonomy requires control over what happens next. 

[For more news, click here]

Imagine an agent investigating a checkout outage. It runs in a hardened sandbox. Its credentials are scoped to the affected service. Its tools are approved. After reviewing the evidence, it rolls back the latest deployment. 

The rollback request succeeds. Checkout gets worse. 

Since the previous application version was deployed, the database schema has changed. The version the agent restored can no longer serve the application correctly. 

The agent never escaped its sandbox. It used legitimate credentials, called an approved tool, and acted on the service it was assigned to fix. Every access boundary held. The failure was in the conditions under which the decision had been delegated. 

The cloud native community is already examining the limits of sandboxing agents. A July CNCF article on why sandboxing alone is not enough argues that secure isolation is only part of the runtime problem, with lifecycle management, efficiency, and scale also requiring attention. 

Production agents introduce another problem once they are allowed to act on real systems: their authority has to remain valid throughout the workflow. 

Cloud native systems already provide strong mechanisms for identity, isolation, policy, and observation. Kubernetes can tell us whether an identity is allowed to reach an API or modify a resource. Those controls do not determine on their own whether a permitted action still makes sense given the available evidence, changes that occurred during the investigation, or actions already taken elsewhere. 

This is the challenge of production autonomy. An objective tells the agent what outcome to pursue. It does not define all the conditions under which the agent remains authorized to pursue it. 

A sandbox defines where an agent can operate. Governance defines the conditions under which its authority remains valid. 

Containment has limits 

A sandbox is an essential part of the answer. Filesystem restrictions, network controls, scoped credentials, and workload isolation constrain what untrusted code can reach and affect. Those protections matter whether an agent makes a mistake, encounters hostile input, or behaves unexpectedly. 

Useful production agents eventually have to interact with production, and production does not stand still while they reason. Once those interactions are permitted, isolation answers only part of the question. Evidence becomes stale. Another workflow modifies a dependency. An action that satisfied policy at the beginning of an investigation may no longer be appropriate when the agent is ready to execute it. 

The same problem appears across agents. Two recovery agents may each be authorized to restart one replica. Each sees the other replica as healthy. Both act. Neither exceeds its individual permission, yet together they remove the service’s available capacity. 

This is why production autonomy has to be governed as a workflow rather than as a collection of individually permitted actions. Authority must account for sequence, current state, competing changes, cumulative impact, and what has already happened. 

The harness controls delegated authority 

An instruction such as “restore checkout” supplies a goal. It does not supply unlimited authority to achieve it. 

Operator-defined policy establishes the boundaries: which systems are in scope, which operations are permitted, what limits apply, what evidence is required, and which decisions remain with people. The harness enforces those conditions as agent decisions become production effects. 

A proposed rollback, for example, might require a compatible target version, evidence that the database schema supports it, and confirmation that no conflicting change is underway. The model can investigate, reason, and propose the action. The harness determines whether the conditions for taking it have been satisfied. 

If they have not, the workflow should not work around the constraint. It must either continue along a permitted path or stop and ask for what is missing. 

Escalation should preserve the boundary 

Return to the checkout incident. The agent discovers that a database migration occurred after the target version was deployed, but it cannot establish backward compatibility. That is not simply a reason to ask someone to approve the rollback. A prerequisite is missing. 

The harness should hold the action and surface the specific gap, along with the evidence gathered so far, the proposed action, and the input required to continue. If the issue is missing authority, an authorized operator may grant a bounded permission through policy. If the issue is missing evidence or expertise, that gap must be resolved before the action becomes eligible. 

While the workflow waits, its state should be preserved. When new information arrives, the action should be evaluated again against current policy and production state rather than resumed from an outdated decision. If no response arrives, the workflow can remain paused, expire, or escalate according to policy. Silence never becomes approval. 

This distinction matters. Human judgment may be required for some decisions, but approval should not override a safety condition that has not been met. 

Enforcement must reach the production effect 

Governance only works if enforcement covers the action that actually changes production. 

Approving access to a shell or Python interpreter is not enough if code running inside it can make unrestricted production API calls. The same applies to connectors, delegated agents, and other execution paths. If an agent can reach the same system through an unrestricted route, the governance boundary is incomplete. 

The mechanisms that grant production authority therefore need to remain outside the untrusted code’s control. Sandboxes, constrained executors, tool brokers, and resource-side policy can work together to keep every consequential action within the authority granted to the workflow. 

That control cannot end when execution begins. The system may need to recheck conditions that can become stale, coordinate actions that could conflict, and enforce limits on retries or cumulative change. 

It must also verify the result. An accepted API request proves only that the request was accepted. Operational success requires observations tied to explicit criteria: whether checkout recovered, whether error rates fell, and whether the change created unacceptable effects elsewhere. The agent can interpret those observations, but its assessment is not a substitute for them. 

The record must outlive the sandbox 

The workflow also has to remain explainable after the execution environment is gone. 

Sandbox logs can show activity, but they do not necessarily preserve the evidence behind a decision, the policy that authorized it, the exception that required human input, or the outcome observed afterward. If that context disappears with an ephemeral environment, operators are left reconstructing the incident after the fact. 

The durable record should connect the workflow and agent identity with the evidence considered, applicable policy, approvals and exceptions, actions executed, tool receipts, failures, and observed outcomes. The agent’s own summary can contribute to that record, but it should not be able to rewrite the authoritative evidence of what occurred. 

This is more than an audit requirement. It creates a useful operational history. Future operators and agents should be able to distinguish between a remediation that demonstrably restored the service and one that merely completed without a known outcome. 

That history can inform future delegation, but it does not grant authority. A remediation that worked once does not become universally safe, nor does prior success give an agent permission to repeat it. Memory can make previous experience available. Evidence can show what happened under particular conditions. Policy still determines whether those conditions justify action now. 

As confidence and evidence grow, people may choose to expand an agent’s authority, keep it fixed, or narrow it as conditions change. The agent does not promote itself. 

This is the broader architectural requirement for production agents: actions pass through enforceable controls, authority remains bounded throughout the workflow, outcomes are verified, and the evidence survives execution. 

The goal is not to put a person in front of every routine action. It is to let people define the boundaries, let agents work independently within them, and bring operators back in when the workflow reaches the edge of its authority. 

Putting the ideas into practice 

These principles guide our work on AURA, an Apache 2.0 open-source agentic harness, and the Mezmo platform. The goal is to keep reasoning, authority, execution, and evidence connected throughout a workflow rather than treating each tool call as an independently authorized event. 

The architecture matters more than any particular implementation. Whether teams use AURA or another agent system, production actions should pass through enforceable controls, authority should remain bounded as conditions change, and the evidence should survive the execution environment. 

Contributed by Andre Elizondo, Head of Innovation and AI 

Andre is a recovering SRE who runs Innovation & Strategy at Mezmo, the creator of AURA. He's spent time running and developing production infrastructure at companies like Big Fish Games and Adobe, as well as product, engineering, and solutions roles at companies like Datadog, Chef, and Wiz. Outside of work, Andre can be found hanging out with his dogs (2 border collies) or enjoying a well-pulled shot of espresso. 

KubeCon + CloudNativeCon is taking place in Salt Lake City from November 9 to 12. Register Now! 


Related Tech Revolt Articles 

AI Agent Governance Is Falling Behind Enterprise Adoption 

Bringing Attestation, Provenance, and Tamper-Evident Execution History to Workflows and AI Agents 

Red Hat Wants AI Agents to Prove Their Work Before They Get the Keys to the Business 


 


 

Share this article

Related Articles