What Is an AI Agent Harness? A Recovery Example
Learn what an AI agent harness does, then run a Python recovery example that separates committed tool effects from receipts and exposes unsafe replay behavior.

What is an AI agent harness? It is the software around a model that manages context, calls tools, records results, and decides whether work should continue, stop, or go to a person. The model proposes actions; the harness connects those proposals to an execution environment and a recoverable record of what happened.
A useful way to evaluate a harness is to interrupt it between a tool action and its saved receipt. Can it determine whether the action already happened, or will it repeat the effect? That question reveals more than a diagram with a box labelled memory.
We built a small Python and SQLite exercise that exposes this failure window. Across three separate processes, an interrupted operation resumes with one stored effect, then returns a cached receipt on another run. A naive replay creates two effects. The action is scripted and local: this is a synthetic recovery example, not an AI model benchmark, remote-service guarantee, or production performance result.
What does an agent harness contain?
A harness connects the model to the state and tools needed to pursue a task. It typically includes a control loop, context assembly, tool dispatch, progress storage, termination conditions, and review boundaries. Implementations differ, and a library, coding application, or custom runner can supply several of these functions together.
The labels are less important than the behavior you can inspect:
| Component | Responsibility | Question to ask |
|---|---|---|
| Model interface | Send context and receive a proposed next action | Which provider response and errors are recorded? |
| Context assembly | Select instructions, task state, and relevant observations | What survives a new session? |
| Tool dispatcher | Validate and execute a supported operation | What happens after a timeout with an unknown result? |
| Durable state | Preserve progress and operation identity | Can a new process resume without guessing? |
| Completion check | Compare results with the task's acceptance evidence | Is “done” more than the model saying it is done? |
| Human boundary | Hold decisions that require a person | Can the approved action change before execution? |
Anthropic's long-running agent guidance describes initializer work, persistent progress, and incremental feature checks across sessions. Our example focuses on one narrower mechanism: reconciling an effect that committed before the runner saved its receipt.
Harness, framework, SDK, sandbox, and orchestration
These terms describe related responsibilities rather than a universal hierarchy of products. A framework can provide harness components; an SDK can expose agent loops as well as basic API calls. Read the actual implementation instead of assuming a product's label tells you where its responsibilities stop.
| Term | Practical meaning in this guide |
|---|---|
| Model | Produces a response or proposed action from supplied context. |
| SDK | A client interface or development kit; its scope depends on the provider. |
| Framework | Reusable components for constructing an application or agent workflow. |
| Harness | The running control, tool, context, and state machinery surrounding the model. |
| Sandbox | A restricted execution environment, with boundaries determined by its configuration. |
| Orchestration | Coordination of work, dependencies, and handoffs across workers or steps. |
A harness does not become a sandbox because it has an approval prompt. A sandbox does not decide whether a completed change satisfies the user's task. Likewise, a shared task board cannot infer whether a remote payment succeeded after a lost response.
Our agent orchestration guide covers shared work and dependencies. This article stays with the local execution loop and its recovery record, so you can see which problem each layer actually handles.
Follow one operation through the loop
Consider a task that asks an agent to write one result. The harness gathers the task and prior observations, requests a next action, validates the proposed tool call, and executes it. It then saves the result, evaluates progress, and either continues or stops.
There are several distinct failure points:
- The model request fails before an action is returned.
- The action is rejected before the tool executes.
- The tool fails before changing anything.
- The tool changes something, but its response is lost.
- The harness receives the result but stops before saving it.
- A later process resumes with incomplete evidence.
Retrying the model call is different from retrying the effect. In the fourth and fifth cases, “try again” may duplicate an already completed action. A useful recovery design preserves an operation identifier and checks what can be known before deciding to replay.
Record model identity separately when routing is involved. Our LLM router audit-trail guide explains why a requested alias alone is insufficient execution evidence. The operation receipt serves a different purpose: it connects a specific tool operation to an observed result.
Run a provider-neutral interruption exercise
Save the following as harness_demo.py in a disposable exercise directory. It uses Python's standard library and creates a local SQLite file at the path you supply. It neither contacts a model nor executes arbitrary commands proposed by one.
import sqlite3,sys
def boot(path):
db = sqlite3.connect(path)
db.execute("CREATE TABLE IF NOT EXISTS effects (op TEXT PRIMARY KEY, value INTEGER NOT NULL)")
db.execute("CREATE TABLE IF NOT EXISTS receipts (op TEXT PRIMARY KEY, value INTEGER NOT NULL)")
db.commit()
return db
def step(db, op, crash=False, desired=7):
receipt = db.execute("SELECT value FROM receipts WHERE op=?", (op,)).fetchone()
if receipt:
if receipt[0] != desired:
raise ValueError("operation key reused with different input")
return "cached"
# In this local fixture the tool ledger commits before the harness receipt.
# This is NOT a distributed transaction or a remote API guarantee.
with db:
db.execute("INSERT OR IGNORE INTO effects VALUES (?, ?)", (op, desired))
effect = db.execute("SELECT value FROM effects WHERE op=?", (op,)).fetchone()
if effect[0] != desired:
raise ValueError("operation key reused with different input")
if crash:
raise RuntimeError("simulated crash after tool commit")
with db:
db.execute("INSERT INTO receipts VALUES (?, ?)", (op, effect[0]))
return "recorded"
if __name__ == "__main__":
db = boot(sys.argv[1])
try:
print(step(db, sys.argv[2], crash="--crash" in sys.argv))
finally:
db.close()
The effects table represents our local tool's durable record. The receipts table represents what the harness has acknowledged. Both are local for inspectability, but the example deliberately commits them separately to expose the recovery window.
If your real action and receipt genuinely share one database transaction, using that transaction can be simpler. Two tables and a retry loop are not automatically a better architecture. The interesting case is when the tool owns its effect and the harness cannot commit both records atomically.
Run the first command once. Its nonzero exit and traceback are expected: the demonstration raises an exception after the local effect commits.
python3 harness_demo.py demo.sqlite3 task-1:write-1 --crash
Run the same operation again, using the same database file and operation key:
python3 harness_demo.py demo.sqlite3 task-1:write-1
python3 harness_demo.py demo.sqlite3 task-1:write-1
The second process prints recorded; the third prints cached. Reusing demo.sqlite3 is intentional. Reusing an old file from an unrelated exercise is not: start with a fresh disposable directory when repeating the whole experiment.
What the six checks actually showed
We executed the exercise on Python 3.14.4, including separate processes for the interruption, restart, and completed replay. We also checked payload disagreement, a different operation key, and a naive replay without stable operation identity.
| Check | Observed result |
|---|---|
| Exception after the tool commit | One effect, no receipt; process exited unsuccessfully. |
| Restart with the same operation key | One effect, one receipt; output recorded. |
| Repeat the completed operation | One effect, one receipt; output cached. |
| Reuse the key with a different requested value | ValueError; the conflicting input was not accepted. |
| Use a different operation key | A second effect and a second receipt. |
| Naive replay without stable operation identity | Two effects for the repeated local action. |
The operation key is supplied as persistent input. Generating a fresh key every time the process starts would describe a different operation and defeat this example's deduplication. In an application, attach it to durable task state before attempting the effect.
The simulated crash is a deliberate Python exception after a committed transaction. We did not test power failure, corrupt disks, network partitions, a concurrent worker race, or a real provider's behavior. A correct result for these six checks supports the mechanism shown here, not a blanket claim of reliable autonomous execution.
Why this does not prove remote exactly-once execution
The local tool record makes reconciliation possible because the harness can look up the stable key. A remote service may offer idempotency keys, a status lookup, both, or neither. Its documented semantics determine what a retry can safely mean.
If a remote write times out and you cannot determine whether it committed, record the outcome as unknown. Stop or ask for reconciliation instead of relabelling the timeout a failed action. A generic harness cannot manufacture an atomic transaction with an external system.
Also separate successful transport from successful work. A tool can return HTTP 200 while producing the wrong artifact. Store the response and verify the task's acceptance criteria. A receipt documents an observation; it does not replace the check that the observation satisfies the request.
For concurrent workers, this demonstration needs additional design. Two workers could both reach the receipt insertion before either sees a completed receipt. Database constraints expose the collision, but our small script does not handle every interleaving. A production runner must define ownership, retry behavior, and conflict handling for that situation rather than copying the example and assuming concurrency is solved.
Keep recovery state separate from conversation memory
Conversation notes can explain why a task took a particular path. They are a poor substitute for a precise operation record. A summary saying “probably wrote the result” cannot establish which operation key was used or whether the effect exists.
For the separate problem of access scope, see our agent identity and project-access guide. Keep the minimum recovery fields explicit: task identity, operation identity, expected input identity, execution state, observed result, and unresolved outcome. Add timestamps and relevant revisions so a reviewer can reconstruct the sequence. Avoid placing secrets or unnecessary tool output into records that more people can read.
If a context window is compacted or a new session starts, rebuild the next step from those records. Do not ask the model to infer whether a costly action already occurred from conversational tone. The harness should expose the known state, including uncertainty, as structured input.
This is also why changing models and changing harness behavior are different experiments. A better response may improve planning, while a stable operation key prevents one particular replay failure. Measure each against the failure it is supposed to address rather than attributing every improvement to the model.
Put approval and stopping rules beside the loop
Recovery is only one harness responsibility. The runner also needs bounded work: a maximum number of steps, a budget convention, and a stop condition for repeated nonprogress. Our script performs one operation and exits; it does not demonstrate a general autonomous loop or a spending cap.
When an action requires a person's decision, bind that approval to the actual action under consideration. If its material inputs change, reconsider the approval. Our AI guardrails guide examines approval boundaries and action controls in a separate synthetic lab. Those checks complement recovery; they do not make an unknown remote outcome known.
Keep failure states visible. “Blocked pending review,” “unknown tool outcome,” and “completed with verified evidence” mean different things. Flattening all three into a generic finished status deprives the next session and the human reviewer of the information needed to continue.
Recovery and acceptance answer separate questions. After reconciling whether an action happened, use the AI agent testing procedure to check whether its artifacts satisfy the current requirement rather than a stale version.
A multi-agent harness adds another question: whether separate workers coordinate and produce a correct joint result. The multi-agent benchmark comparison maps coding, dependency, conflict, resource-contention, and communication suites to the failure each one can actually test.
Where shared coordination belongs
Each harness can retain its own execution records while agents share the repository, acceptance criteria, and decisions. The overlap problem appears when two otherwise competent runners both plan changes to the same file or assume they own the same unfinished task.
SyncStaff gives connected coding agents a shared board, declared intents, file leases, conflict arbitration, and human decisions. Use that shared record to make ownership and overlapping changes visible across tools. Keep tool receipts, provider-specific idempotency behavior, and acceptance tests in the systems that actually implement them.
Those roles are complementary. SyncStaff is not a substitute for the recovery mechanism in this exercise and does not turn a remote tool into an exactly-once service. Its relevant benefit is helping people and agents coordinate the work surrounding those operations.
Before evaluating another harness feature, run one interruption exercise on a harmless local action. Identify the operation key, the committed effect, the saved receipt, and the next step after an unknown result. If those four things are hard to explain, improve that path before adding a longer autonomous run.
Frequently asked questions
What is an AI agent harness?
An AI agent harness is the software surrounding a model that assembles context, dispatches tool calls, records progress, and controls continuation or stopping. It connects proposed actions to an execution environment and task evidence. The exact division between an SDK, framework, application, and harness varies by implementation.
How is a harness different from a framework?
A framework provides reusable building blocks; a harness is the running machinery around the model. Framework components may implement much of that machinery. An SDK can also include agent functionality, so inspect actual responsibilities rather than assuming that these labels form one fixed product hierarchy.
Does a durable receipt prevent every duplicate action?
No. An action may commit before the harness saves its receipt. Recovery needs a stable operation identity and a way to reconcile the effect. The local example supplies that lookup, but a remote service must expose suitable behavior before a replay can be treated as safe.
Does this example call an AI model?
No. It uses a scripted action and Python SQLite storage to isolate one recovery mechanism. The interruption, restart, and repeated run use separate processes. The checks demonstrate local behavior; they do not measure a model, an external API, concurrency reliability, or production task success.
What should a harness do after an unknown tool outcome?
Preserve the operation identity and record the uncertainty. Use the tool provider's supported status lookup or idempotency mechanism when available. If the effect cannot be determined, stop or ask for reconciliation. A timeout alone does not establish that the action failed without changing anything.