The harness architecture
Do not just improve the prompt.
Improve the conditions for success.
A harness is the runtime system surrounding an AI model: how it receives context, accesses capabilities, executes a task, retains progress and gets feedback. A framework can supply primitives; harness engineering decides how those primitives behave together under real constraints.
Model
Interprets information and proposes text, decisions or tool calls. Its outputs are probabilistic, not authoritative state transitions.
Harness
Owns context assembly, execution policy, state, resource budgets, recovery and verification. Controls that matter are enforced here or at the tool boundary.
Environment
The files, APIs, applications and records the agent can observe or affect. Clear interfaces and actionable evidence make this environment legible.
Evaluation harness
Runs test tasks and scores behaviour. It measures the execution harness and model together; it is not a substitute for runtime controls.
01 · Assemble
Load the task contract, scoped context, durable state and current tool capabilities.
02 · Propose
Let the model choose a next step or propose an artifact, without granting it execution authority.
03 · Validate & execute
Check permissions, arguments, budgets and approvals; run the authorised tool in its scoped environment.
04 · Observe & checkpoint
Capture the actual outcome, reconcile uncertain effects and persist progress at a safe boundary.
05 · Verify & route
Check acceptance evidence. Continue with useful feedback, wait for a person, or terminate with the actual outcome.
Start with one controlled loop and narrow tools. Add delegation only when it improves measured outcomes; more agents introduce handoffs, duplicated work and additional permission boundaries.
01 / Task contracts
Define done before the agent starts.
A model can propose the next action. The harness must know the task, its boundaries and the evidence that counts as completion. Start with an explicit task contract, not an open-ended instruction to keep trying.
Inputs and acceptance criteria
Record the requested outcome, permitted data sources, required artifacts and measurable acceptance checks. Separate hard requirements from preferences and flag missing inputs before execution.
Authority is not intent
A request describes what the user wants; authenticated permissions determine what the agent may do. Carry tenant, actor and permitted capabilities through every tool call.
A completion predicate
Require external evidence such as a passing test, confirmed record or reconciled result. A fluent final answer is not proof that work succeeded.
Version the whole run
Pin the harness, prompt, policy, tool schema and model configuration. Store their versions with the task so failures can be attributed and regressions reproduced.
For an invoice agent, done means a validated draft exists with the correct customer, line items and source references. Sending the invoice is a separate authorised action, not an inferred next step.
Submit an ambiguous request, a forbidden action and a plausible but incorrect artifact. Each must be held, rejected or failed by the contract rather than accepted by the model.
02 / Context engineering
Give the model the right world, not the whole world.
The context window is a working set, not a database. Assemble a small, relevant view from authoritative records, the current task and recent observations. Preserve provenance and treat retrieved content as untrusted data.
Progressive disclosure
Start with a concise map of available sources and capabilities. Retrieve detailed instructions, files or tool descriptions when needed instead of loading every document into every turn.
Source and freshness
Attach source identifiers, access scope, retrieval time and policy versions to evidence. Refresh volatile facts before consequential decisions; stale availability or pricing should not become a commitment.
Compaction with invariants
Summarise completed work while preserving unresolved questions, decisions, approvals and artifact references. Store durable facts outside the model context and validate a checkpoint after compaction.
Trust boundaries
Keep system policy, user intent and retrieved material distinct. A web page or tool response cannot grant permission, overwrite the task contract or instruct the harness to expose secrets.
A support agent receives the current ticket, the relevant policy section and customer-scoped order facts. It does not receive the entire customer database or unrelated conversations.
Force a context rollover mid-task. Confirm the next turn retains the original constraints and can cite the same evidence without reconstructing permissions from a generated summary.
03 / Tool contracts
Make actions narrow, typed and inspectable.
Tools turn model intent into real effects. Their interfaces should make valid work easy, invalid work difficult and every outcome explicit. Descriptions help the model choose; server-side validation controls execution.
Small capabilities
Prefer domain actions such as create_invoice_draft over unrestricted database queries or shell access. Use clear input schemas, units, required identifiers and compact structured results.
Validate at execution
Check tenant ownership, permission, object state and business constraints immediately before acting. Schema-valid arguments can still be unauthorised or commercially wrong.
Explicit result states
Return success, rejected, retryable failure or unknown outcome with safe diagnostic context. An HTTP timeout does not prove that a remote write failed.
Defensive output handling
Bound output size, redact credentials and mark externally supplied content. Store large artifacts separately and return references so tool responses do not overwhelm the working context.
A payment tool accepts an approved payment intent identifier, not a free-form account number and amount assembled by the model. The executor resolves and validates the intent.
Fuzz tool inputs, cross-tenant identifiers and oversized responses. Check that invalid requests never reach the external service and errors remain understandable without leaking private data.
04 / Execution control
Own the loop outside the model.
The harness coordinates observation, decision, validation, execution and feedback. The model proposes actions; a deterministic controller decides whether to continue, wait, stop or escalate.
An explicit state machine
Use states such as queued, running, waiting_for_approval, verifying, succeeded, failed and cancelled. Persist transitions and reject invalid ones. Waiting for a person is not a tool failure.
Budgets and deadlines
Track elapsed time, model usage, cost, tool calls and concurrency outside the prompt. Enforce task-specific budgets and cancellation signals at dispatch and between steps.
Detect no progress
Compare meaningful artifacts and state changes, not just generated text. Repeated identical calls or unchanged errors should trigger a repair path or escalation, not an endless reasoning loop.
Independent stop checks
Require acceptance evidence before success. Stop safely on denied authority, budget exhaustion or unrecoverable failure, with completed work and unresolved items reported separately.
A coding agent that repeatedly runs the same failing test stops to surface the unchanged failure. It cannot mark the task complete merely because its final message says the issue is resolved.
Inject a never-ending tool, a repeated error and a cancellation during a slow call. Confirm the controller terminates safely and records the actual outcome.
05 / Durable execution
Resume work, not side effects.
Long-running agents outlive context windows, connections and worker processes. Persist task state and artifacts so another worker can resume. A checkpoint alone does not provide exactly-once effects across external systems.
Checkpoints and artifacts
Store the task contract, completed steps, pending actions, tool results and artifact references at explicit boundaries. Distinguish durable state from a conversational summary.
Idempotency and reconciliation
Reuse the same intent identifier when retrying a side effect. Use provider idempotency where available; otherwise query the external outcome or reconcile before retrying an ambiguous write.
Concurrency ownership
Use leases, fencing or version checks to prevent two workers from acting on the same run. A restarted worker must not silently compete with the original one.
Recovery and compensation
Define which actions can be retried, which require human reconciliation and which have a compensating action. Cancelling a task cannot undo a message already sent or a payment already settled.
If a booking request times out after the supplier accepts it, the resumed agent checks the original booking intent. It does not submit a second booking under a new identifier.
Crash execution before and after each side-effect boundary. Resume the same task and prove there are no duplicate commitments; ambiguous outcomes must remain visibly unresolved until reconciled.
06 / Execution environments
Separate reasoning from reach.
The agent should have enough access to do its task and no more. Isolate execution, mediate credentials and constrain network reach. A sandbox is one layer, not a complete security boundary for every integration.
Scoped workspaces
Give each task or tenant an isolated workspace with explicit read/write paths and cleanup rules. Restrict subprocesses, resource use and persistence where arbitrary code runs.
Credential mediation
Keep secrets out of prompts and model-visible files. Where possible, execute authenticated API calls through a broker that applies caller identity and scoped, short-lived credentials.
Network and data boundaries
Allow only necessary destinations and operations. Apply tenant checks at data access, not merely in the user interface; validate outbound payloads before sending.
Delegation cannot expand power
Sub-agents receive the parent task's scoped capabilities or fewer. Their output is evidence to validate, not a new instruction that can change policy.
A research agent can read approved sources and write a report artifact. It cannot publish the report, access production secrets or call a payment endpoint.
Attempt cross-workspace reads, unapproved network calls and privilege escalation through a sub-agent response. Record and reject them at the enforcement boundary.
07 / Human checkpoints
Approve the exact action, not a vague plan.
Approval is part of the execution protocol. It must describe the proposed effect, bind to its exact parameters and survive pauses without becoming a blanket permission for later actions.
Bind the proposal
Store an immutable proposal with target, amount or payload, relevant evidence and an action digest. Material changes invalidate the approval and require a fresh review.
Authorised reviewers
Verify reviewer identity and authority independently of the model. Apply separation of duties where required; agents cannot approve their own proposals.
Freshness and single use
Expire approvals when policy, object state or time-sensitive facts change. Revalidate at execution and atomically consume the grant where appropriate.
Honest reviewer context
Present expected effect, uncertainty, source evidence and a clear reject or edit path. Record the decision without collecting hidden chain-of-thought.
A reviewer approves sending a specific proposal to a named client. If the destination, price or attachment changes, that approval cannot authorise the revised send.
Approve an action, mutate its payload, revoke the reviewer's access and attempt grant reuse. The executor must refuse the altered, revoked or consumed authorisation.
08 / Verification & feedback
Give the agent evidence it can act on.
Good harnesses turn errors into useful feedback. Check the actual artifact, environment or transaction rather than grading only the final answer. Independence helps, but a second model is not proof of correctness.
Deterministic checks first
Use schema validation, tests, reconciliations and invariant checks for properties that can be computed. Model graders are useful for judgement-based criteria, with calibration and human review.
Observe the real surface
For coding agents, expose test output and browser evidence. For business agents, inspect the saved draft or external confirmation. Verify the same outcome the user relies on.
Actionable failures
Return which requirement failed, the observed evidence and a safe repair scope. Avoid a generic try harder instruction or exposing unrelated sensitive records.
Bound repair attempts
Allow repair when new evidence can change the result. If the same failure persists, stop and escalate with the artifact, attempted fixes and remaining issue.
A generated dashboard passes syntax checks but shows no data. A browser check catches the empty state; the harness sends that observable failure back for repair before accepting completion.
Seed plausible artifacts with wrong totals, missing records and inaccessible controls. Acceptance checks must catch these defects even when a model-generated explanation sounds convincing.
09 / Harness evaluations
Test the environment, not just the model.
A harness changes what the model can see, do and verify. Evaluate the combined system with representative tasks, injected failures and fixed acceptance criteria. A test harness measures behaviour; the execution harness controls the live run.
Representative trajectories
Include routine work, ambiguous requests, long tasks, adversarial inputs and interrupted execution. Check intermediate tool choices and effects as well as final artifacts.
Controlled comparisons
Compare harness versions on the same task set with pinned tool fixtures and documented model settings. Repeat stochastic runs and report uncertainty rather than treating one pass as a guarantee.
Failure attribution
Classify missing context, poor tool design, permission rejection, model error, provider failure and verifier error separately. Fix the causal layer instead of adding every incident to the system prompt.
Release and rollback
Gate changes on quality, safety, latency and cost evidence. Canary a version to a scoped population and keep a rollback path for the harness as well as the model.
A new context compactor reduces token usage but drops unresolved approvals. The rollout fails the interrupted-task suite even though short-task answer quality improves.
Run the same tasks through old and candidate harnesses with recorded fixtures, then fault-inject tool outages and worker restarts. Review both artifact quality and forbidden side effects.
10 / Production operations
Keep the harness understandable as it evolves.
Production adds queue pressure, model changes, connector outages and new policies. Keep the harness observable, modular and versioned so operators can diagnose a run without guessing what the model was thinking.
Trace decisions and effects
Connect run, step, tool, artifact, approval and provider identifiers. Record safe inputs, observed outcomes and policy decisions; redact sensitive data at capture and avoid private chain-of-thought.
Meaningful operating metrics
Track verified completion, escalation, retries, unresolved effects, latency and cost per completed task. High message volume or tool-call count is not a business success metric.
Backpressure and degradation
Limit concurrency, honour provider throttling and queue work predictably. Degraded mode may mean a draft for a person, not silent routing to a weaker or more privileged path.
Remove obsolete scaffolding
Retest prompts, recovery logic and special cases as models and tools change. Keep deterministic security controls; remove workarounds only when evaluation evidence supports doing so.
A connector outage shifts affected tasks to a waiting state with visible ownership. The harness does not multiply retries across workers or label queued tasks as successfully completed.
Rehearse an outage and rollback. An operator should locate the affected runs, distinguish failed from uncertain effects and resume safe work with the original contract intact.
Worked examples / harness failures
Find the missing layer.
Three illustrative scenarios isolate harness failures that a longer prompt cannot reliably fix. These are design examples, not live executions or reported customer incidents.
- Symptom
- A long task resumes with an attractive summary but forgets an unresolved approval.
- Harness layer
- Context assembly and durable task state
- Response
- Restore the task contract and pending approval from durable records. Do not infer permission from the generated summary.
- Proof
- Restart the task after forced compaction; the action remains blocked until the exact proposal is approved.
# Illustrative design contract — not executable configuration
run:
task_ref: task/charter-proposal
actor_ref: authenticated_actor
tenant_ref: verified_tenant
versions: [harness, prompt, tools, policy, model]
context:
sources: [current_request, approved_operator_quotes]
preserve: [constraints, open_questions, pending_approvals]
tools:
capabilities: [read_quote, create_proposal_draft]
validate: [schema, tenant, authority, current_state]
effects:
intent_ref: durable_intent_identifier
on_unknown_outcome: reconcile_before_retry
approval:
external_send: exact_payload_digest_and_recipient
on_change: invalidate_and_request_review
completion:
requires: [saved_draft, source_checks, acceptance_checks]
model_claim_alone: insufficient
recovery:
resume_from: last_committed_checkpoint
stop_on: [cancelled, budget_exhausted, no_progress]
# Set budgets, expiry and retry limits for your workload.
# Implement and test enforcement outside the model.The example describes desired controls. Build the controller, storage, authorisation and verification mechanisms, then prove their behaviour with fault-injection tests. Writing these requirements in a prompt does not enforce them.
Before production
Ten checks before you trust the loop.
Attach a test result, run trace or recovery record to each answer. This review is session-only and is not a certification of readiness.
0 / 10 reviewed in this session
Further reading
Follow the engineering evidence.
These sources explore agent-first environments, long-running execution, context design and security. Many examples concern coding agents; the mechanisms here are adapted to broader business workflows, not claims that every architecture needs the same implementation.
Security boundaries
AI agent securityRelease evidence
Agent evaluation & testingMeaningful approvals
Human-in-the-loop designAccountable operations
Observability & audit trailsTechnical guidance and illustrative patterns, not a deployed agent platform, security guarantee or regulatory certification. Choose controls and operating limits against the risks and constraints of your own workload.
Build the system
around the intelligence.
Bring a workflow, an agent prototype or a run that keeps failing. We can help identify the missing harness layer and design a measured path to production.