What Is an Agent Harness?

Overview

The harness is invisible when things work and the first thing you debug when they don't. It is the runtime layer between agent architectures and execution — the operational machinery that turns a clever design into a running system. When a production agent crashes halfway through a 30-step research task, when it calls the same failing API in an infinite loop, or when you cannot trace why it chose tool X over tool Y, you are debugging the harness, not the model.

This lesson defines the harness as a distinct layer in the agent stack, introduces the eight concerns every harness must solve, and walks through what breaks when you skip the harness and run agents as raw API loops.

By the end of this lesson, you will understand where the harness fits, what it is responsible for, and why evaluating agent frameworks requires evaluating their harness capabilities first.

We will introduce a simple "research assistant" as a case study that threads through this module. In this lesson, we will see it built naively as a raw API loop. In later lessons, we will progressively add harness concerns until it becomes production-ready.

The Harness Layer

When you build an agent, you are working with at least five distinct layers of abstraction:

1. Application Layer — your product's UX, business logic, and workflow
2. Orchestration Layer — coordination across agents, services, deployment
3. Harness / Runtime Layer — the execution environment for a single agent
4. Model API — the LLM service that produces outputs and tool-call requests
5. Tool Layer — the actual capabilities, functions, and APIs the agent invokes

The harness sits between orchestration and the model API. It is responsible for running the agent — managing the loop, validating and dispatching tool calls, assembling context each turn, persisting state, tracing execution, and gating human intervention.

graph TB
  APP["Application Layer<br/>(UX, business logic, product workflow)"]
  ORCH["Orchestration Layer<br/>(multi-agent coordination, deployment)"]
  HARNESS["Harness / Runtime Layer<br/>(loop, tools, state, control flow,<br/>persistence, observability, HITL)"]
  MODEL["Model API<br/>(LLM calls, tool-call requests)"]
  TOOLS["Tool Layer<br/>(functions, APIs, external services)"]
  MEM["Memory / Knowledge<br/>(vector stores, databases)"]
  ENV["Environment<br/>(file system, network, user)"]

  APP --> ORCH
  ORCH --> HARNESS
  HARNESS --> MODEL
  HARNESS --> TOOLS
  HARNESS --> MEM
  HARNESS --> ENV

  style HARNESS fill:#9f7aea,stroke:#6b46c1,color:#fff

Think of the harness like an operating system for a single agent. An OS manages process lifecycle, memory allocation, I/O, error handling, and scheduling. A harness manages agent lifecycle, context assembly, tool dispatch, error recovery, and termination. Just as you would not write a web server by manually managing memory and syscalls, you should not write production agents as raw API loops.

The Five Adjacent Layers

Before we dive into the harness itself, it helps to clarify what the harness is not responsible for:

Model API — This is the LLM service (Anthropic, OpenAI, Google, etc.). It receives messages and tools, produces text outputs or structured tool-call requests, and returns a stop reason. The model API does not run the agent — it processes individual turns. The harness calls the model API, not the other way around.

Tool Layer — These are your agent's capabilities: search_web, read_file, send_email, etc. Tools are functions or API wrappers that take arguments and return results. The harness validates tool-call requests, dispatches them to the correct implementation, handles errors, and formats results. But the tools themselves are standalone.

Harness / Runtime — This is what we are studying. It controls:

• Loop execution — when to call the model, when to terminate, how many steps to allow
• Tool dispatch — validating tool-call requests, invoking tools, wrapping results
• Context management — assembling the message history, system prompt, and tool schemas each turn
• State and goals — tracking agent objectives, progress, and working memory
• Control flow — routing between agents or tools, handling conditional logic
• Persistence — checkpointing state so runs can resume after crashes or pauses
• Observability — tracing, logging, cost tracking, replay
• Human intervention — approval gates, escalation, manual overrides

Application Layer — This is your product. The UI that users interact with, the business logic that defines the workflow, the integration with your backend services. The application calls the harness to run an agent, but it does not implement the agent loop. In our research assistant example introduced later, the Application Layer is the user-facing interface that accepts a research question and displays the final synthesized findings.

Orchestration Layer — When you have multiple agents working together, or agents deployed across services, you need orchestration: workflow management, inter-agent communication, deployment infrastructure. This sits above the harness. The harness runs one agent; orchestration coordinates many. In our research assistant, the Orchestration Layer would manage handoffs if we split research across specialized sub-agents (one for web search, one for data analysis, one for synthesis).

To make this concrete for the research assistant we will build in this module:

• Tool Layer: The actual capabilities — web_search to query sources, calculator to compute statistics, read_document to parse PDFs.
• Model API: The LLM service (Claude, GPT-4, etc.) that receives the research question and decides which tools to call and how to synthesize findings.
• Harness / Runtime: The execution environment that runs the agent loop, validates and dispatches tool calls to web_search and calculator, assembles the growing context of sources found so far, checkpoints progress after each search, and tracks how close the agent is to the user's research goal.
• Orchestration Layer: Not needed for a single-agent research assistant, but would coordinate if we decomposed the task into specialized agents (search → extract → synthesize).
• Application Layer: The chat UI where the user types "What are the top 5 trends in AI agent deployment in 2026?" and sees the final synthesized report with citations.

The Eight Harness Concerns

Every production agent harness must solve eight operational concerns. These are not optional features — they are the difference between an agent that runs and an agent that ships.

1. Loop Execution — Who controls the agent's execution cycle? The harness decides when to call the model, when to stop (max steps, goal reached, error), and how to handle termination. Without this, agents run forever or stop arbitrarily.

2. Tool Dispatch — How do tool-call requests become actual function invocations? The harness validates the request (is the tool registered? are the arguments valid?), routes to the correct implementation, executes it, handles errors, and packages the result back to the model. Without this, you have no boundary between the model's intent and system execution.

3. Context Management — What does the model see each turn? The harness assembles the message history, system prompt, tool schemas, and any injected context (knowledge, state, user input). It enforces token limits, prunes history when needed, and manages conversational continuity. Without this, the agent forgets what it was doing or exceeds context windows.

4. State and Goals — What is the agent trying to do, and how much progress has it made? The harness tracks the agent's objective, working memory (variables, intermediate results), and progress indicators. This is separate from conversational context — it is the agent's internal state. Without this, you cannot resume interrupted runs or debug why an agent diverged from its goal.

5. Control Flow — What happens after a tool call completes? Does the agent continue in a loop, hand off to another agent, wait for human input, or branch based on conditions? The harness implements the control flow model — whether that is a simple loop, a state machine, a directed graph, or a workflow definition. Without this, you are stuck with one fixed execution pattern.

6. Persistence — Can the agent survive a crash, a deploy, or a pause? The harness checkpoints state to durable storage so runs can resume from the last known step, not the beginning. This is critical for long-running agents and human-in-the-loop workflows. Without this, every failure is catastrophic.

7. Observability — Can you see what the agent did and why? The harness emits traces (structured logs of every turn, tool call, and decision), tracks cost and latency, supports replay (re-running a trace to debug), and integrates with monitoring systems. Without this, debugging is guesswork.

8. Human Intervention — When should a human approve, reject, or redirect the agent? The harness defines intervention points: approval gates before risky actions, escalation triggers when confidence is low, and manual override mechanisms. This couples persistence (to pause execution), control flow (to resume or redirect), and observability (to show the human what is happening). Without this, you cannot safely deploy agents that take consequential actions.

graph TB
  subgraph HARNESS["Harness / Runtime Layer"]
      direction TB
      LOOP["1. Loop Execution<br/><i>when to run, when to stop</i>"]
      DISPATCH["2. Tool Dispatch<br/><i>request → execution</i>"]
      CONTEXT["3. Context Management<br/><i>what the model sees</i>"]
      STATE["4. State and Goals<br/><i>progress, variables, objectives</i>"]
      CONTROL["5. Control Flow<br/><i>loops, graphs, handoffs</i>"]
      PERSIST["6. Persistence<br/><i>checkpoints, resumability</i>"]
      OBS["7. Observability<br/><i>traces, logs, cost, replay</i>"]
      HITL["8. Human Intervention<br/><i>approval, escalation, override</i>"]
  end

  LOOP -.-> DISPATCH
  DISPATCH -.-> CONTEXT
  CONTEXT -.-> STATE
  STATE -.-> CONTROL
  CONTROL -.-> PERSIST
  PERSIST -.-> OBS
  OBS -.-> HITL

  style HARNESS fill:#9f7aea,stroke:#6b46c1,color:#fff

These concerns are deeply interconnected. Persistence requires state to checkpoint. Human intervention requires persistence to pause and observability to surface context. Control flow determines when dispatch happens. A production harness must address all eight, not just the ones that feel urgent.

The Harness Spectrum

Not every agent needs the same level of harness sophistication. The spectrum runs from "no harness" (raw API calls) to "full runtime" (managed execution with durable state). Understanding where an approach falls on this spectrum helps you choose the right tool for the job.

No Harness — Raw API Loop
You write a while True loop, call the model API directly, manually parse tool calls, invoke functions yourself, and manage state in local variables. This works for prototypes and simple scripts. It breaks the moment you need error recovery, persistence, tracing, or human oversight.

Thin SDK — Helper Functions
You use a client library (Anthropic SDK, OpenAI SDK) that wraps the API and provides utilities for tool-call parsing and message formatting. This eliminates boilerplate but does not manage the loop, state, or lifecycle. You still own execution control.

Agent Loop SDK — Managed Loop
The SDK provides a run_agent() function that manages the loop, dispatches tools, and handles basic termination. You define tools and a goal; the SDK executes until done. This covers concerns 1-2, sometimes 3. State, control flow, persistence, and observability are still on you.

Graph Runtime — Declarative Control Flow
You define the agent as a graph of nodes (LLM calls, tool executions, conditional branches) and edges (transitions). The runtime executes the graph, manages state transitions, and supports cyclic workflows. This handles concerns 1-5 well. Persistence and observability may be partial. LangGraph and Burr are examples.

Workflow Runtime — Durable Execution
You define the agent as a workflow with explicit checkpoints, retries, and compensation logic. The runtime persists state at every step, supports long-running tasks, and survives crashes. This handles concerns 1-7 comprehensively. Temporal, Replit Agent, and Inngest are examples. Human intervention is often manual.

Full Agent Platform — Production Harness
The platform manages everything: loop, tools, state, control flow, persistence, tracing, cost tracking, approval gates, and deployment. You define the agent's behavior; the platform runs it safely. Examples include Claude Code, Devin, and enterprise agent platforms. These address all eight concerns but at the cost of flexibility.

The following table summarizes what each level of the spectrum provides:

| Level | Concerns Covered | You Still Own | Best For | |-------|-----------------|---------------|----------| | No Harness | None | Everything | Quick prototypes, scripts | | Thin SDK | Message formatting | Loop, state, persistence, observability, HITL | Simple single-turn agents | | Agent Loop SDK | 1-2 (loop, dispatch) | State, control flow, persistence, observability, HITL | Multi-turn agents with simple flow | | Graph Runtime | 1-5 (loop through control flow) | Full persistence, advanced observability | Complex agents with branching logic | | Workflow Runtime | 1-7 (loop through observability) | Human intervention patterns | Long-running, fault-tolerant agents | | Full Platform | All 8 | Application-level customization | Production agents with all requirements |

Where you land on this spectrum depends on your constraints: How critical is reliability? How long do runs take? Do you need human oversight? Can you afford to debug opaque failures?

When You Need a Harness

Not every agent needs a harness. The decision depends on the task's complexity, duration, risk, and reliability requirements. A quick decision framework:

graph TD
  Q1{"Does the agent run<br/>for more than 60 seconds?"}
  Q2{"Does the agent take<br/>consequential actions?"}
  Q3{"Do you need to debug<br/>failures post-hoc?"}
  Q4{"Can the user cheaply<br/>retry on failure?"}

  RAW["Raw API loop is fine"]
  SDK["Use an Agent Loop SDK"]
  HARNESS["You need a harness"]

  Q1 -->|No| Q2
  Q1 -->|Yes| HARNESS
  Q2 -->|Yes| HARNESS
  Q2 -->|No| Q3
  Q3 -->|Yes| SDK
  Q3 -->|No| Q4
  Q4 -->|Yes| RAW
  Q4 -->|No| SDK

  style HARNESS fill:#9f7aea,stroke:#6b46c1,color:#fff
  style RAW fill:#68d391,stroke:#38a169,color:#000
  style SDK fill:#fbbf24,stroke:#f59e0b,color:#000

The key signals that push you toward a harness are duration (long runs need persistence), risk (consequential actions need approval gates), and debuggability (complex workflows need observability).

Here are four scenarios that illustrate the spectrum:

Scenario 1: FAQ Chatbot — No harness needed
You are building a chatbot that answers frequently asked questions from a company knowledge base. Each query is independent, responses take 2-3 seconds, and the agent has read-only access to a vector store. A raw API loop is fine here. If the agent crashes, the user retries. No state to lose, no long-running tasks, no risky actions. The absence of harness concerns is acceptable because failures are cheap and recovery is instant.

Scenario 2: Code Review Agent — Needs harness (persistence, observability, step limits)
You are building an agent that reviews pull requests by reading 20+ files, checking style, running static analysis, and generating a report. This takes 10-15 minutes and involves dozens of tool calls. Without a harness, a crash at minute 12 means restarting from scratch. Without observability, you cannot trace why the agent flagged a false positive. Without step limits, a circular import could cause infinite recursion. The harness provides persistence (resume from the last file), observability (trace every file read and analysis step), and loop execution controls (max 100 steps, 20-minute timeout).

Scenario 3: Customer Support Agent — Needs harness (HITL approval gates, audit logging)
You are building an agent that handles refund requests. It reads the user's account history, determines eligibility, and can issue refunds up to $500. This agent takes consequential actions on behalf of the company. Without a harness, there is no approval gate — the agent could issue refunds incorrectly or be exploited. Without observability, you have no audit trail of why a refund was approved. The harness provides human intervention (require approval for refunds over $100), persistence (checkpoint the decision context for later review), and observability (structured logs for compliance audits).

Scenario 4: Research Agent — Needs harness (context management, state tracking, error recovery)
You are building an agent that researches a question by searching 20+ sources, extracting key facts, cross-referencing claims, and synthesizing a report with citations. This involves 50+ tool calls, takes 5-10 minutes, and accumulates a large context of intermediate findings. Without context management, the agent will exceed token limits after 30 sources. Without state tracking, it cannot remember which sources it has already checked or what facts it has found. Without error recovery, a single search API timeout crashes the entire run. The harness provides context management (prune old search results, keep only key facts), state tracking (maintain a list of sources checked and facts extracted), and tool dispatch error handling (retry failed searches with exponential backoff).

The pattern: notice that each scenario adds more concerns. The FAQ bot needs zero. The code reviewer needs three. The support agent needs three different ones. The research agent needs four. Production agents almost always need a harness — the question is which concerns matter most for your use case.

Case Study: The Missing Concerns

Let us make this concrete. Here is a "research assistant" implemented as a raw API loop using the Anthropic SDK:

This is 20 lines and it works — for the happy path. Now let us audit it against the eight harness concerns. Each missing concern would manifest as a specific production failure that blocks deployment or damages user trust.

1. Loop Execution — The while True loop runs forever until stop_reason == "end_turn". What if the model never returns end_turn? What if the API call times out? There is no step limit, no max duration, no way to interrupt. In production, this means a single bad API response can lock the process indefinitely, consuming compute and blocking the user with no way to cancel the research task.

2. Tool Dispatch — call_tool() is a placeholder. What if block.name is not a registered tool? What if block.input is malformed? What if the tool raises an exception? The loop crashes. In production, this means the first tool error — a search API rate limit, a malformed JSON input, a network timeout — crashes the entire agent, forcing the user to restart and losing all progress.

3. Context Management — The messages list grows unbounded. If the agent takes 50 tool calls, you will exceed the context window and the API will reject the request. There is no history pruning. In production, this means research tasks that require more than ~30 sources will fail with a cryptic "context length exceeded" error, and users will not understand why some questions work while others fail unpredictably.

4. State and Goals — There is no state. If you wanted to track progress ("how many sources have I checked?") or save intermediate results, where would you put them? Local variables are lost on restart. In production, this means you cannot show the user a progress indicator, cannot resume a partially completed research task, and cannot debug why the agent stopped searching before finding enough sources.

5. Control Flow — The flow is hardcoded: loop until done. What if you wanted to pause for human approval before calling a destructive API? What if you wanted to hand off to a specialized summarization agent? No mechanism. In production, this means you cannot implement workflows like "research 10 sources, then ask the user if they want more detail" or "hand off to a fact-checking agent before returning results."

6. Persistence — None. If the process crashes at step 15 of 20, you restart from step 1. If you need to resume later, you cannot. In production, this means any transient failure — a deploy, a container restart, a rate limit error — loses all accumulated research, forcing the agent to re-search sources it already processed and wasting both time and API costs.

7. Observability — You have no trace of what the agent did. No structured logs, no cost tracking, no replay. Debugging requires adding print statements and re-running. In production, this means when a user reports "the agent gave me wrong information," you have no way to audit which sources it searched, what facts it extracted, or why it chose to cite source X over source Y — debugging becomes archeology.

8. Human Intervention — None. The agent runs autonomously until it finishes or crashes. If it is about to call an expensive API or take a risky action, you have no way to intervene. In production, this means if the agent is about to call a paid data API 100 times (burning $500 in credits) or is about to search a restricted internal database, you have no approval gate to stop it — users will discover these issues only after the damage is done.

What breaks first in production?
Context overflow. The agent will run for a few dozen turns, exceed the token limit, and fail. You will patch it by truncating messages. Then tool dispatch will fail when a tool times out. You will add exception handling. Then you will realize you cannot resume interrupted runs. You will add manual checkpointing. You will repeat this for every missing concern until you have rebuilt a harness from scratch.

This is the cycle every team goes through when they deploy raw-loop agents. The harness is not optional — you either choose one or build one yourself.

Exercise: Classify Agent Failures

For each scenario below, identify whether the failure is a harness-layer, model-layer, or tool-layer problem.

Scenario 1: An agent searches the web for "top competitors in market X." The search API returns a 500 error. The agent retries the same query infinitely until it exhausts the API rate limit.

Scenario 2: An agent is tasked with summarizing a 100-page PDF. Midway through, the process is killed due to a deploy. When restarted, the agent begins summarizing from page 1 again, re-reading pages it already processed.

Scenario 3: An agent is instructed to "write a comprehensive report on climate change." It produces a 50-page document citing 200 sources. When you review it, 40% of the citations are hallucinated — the papers do not exist.

Scenario 4: An agent is integrated into a customer support workflow. It correctly identifies that a user's account needs a refund, but instead of escalating to a human for approval, it issues the refund immediately and the user exploits this by requesting refunds repeatedly.

Scenario 5: An agent is instructed to "analyze sales data for Q4." It queries the database, processes the results, and generates a chart. When you ask it to refine the chart based on feedback, it has forgotten what data it queried and starts over from scratch.

Answers:

Scenario 1: Harness-layer (Tool Dispatch + Loop Execution). The tool layer returned an error correctly. The harness failed to handle it — no retry backoff, no circuit breaking, no step limit. This is a dispatch and termination failure.

Scenario 2: Harness-layer (Persistence). The agent had no checkpoint mechanism. A production harness checkpoints state so runs can resume from the last step, not the beginning.

Scenario 3: Model-layer. The model hallucinated citations. This is a model capability issue. A harness concern would be observability (to detect hallucinations through citation checks) or human intervention (to require approval for high-stakes claims), but the root cause is model behavior.

Scenario 4: Harness-layer (Human Intervention). The agent had no approval gate for risky actions. A production harness defines intervention points and enforces them before execution.

Scenario 5: Harness-layer (State and Context Management). The agent did not persist intermediate state (the queried data) or manage conversational continuity. A harness would checkpoint the data and maintain it across turns.

Notice how most failures are harness failures, not model or tool failures. The model reasoned correctly. The tools executed correctly. But the runtime layer failed to manage execution safely.

Module Roadmap

This lesson introduced the harness layer and its eight concerns. The rest of this module builds on this foundation in three phases:

Phase 1: Deep Dives (Lessons 2–8) — One lesson per concern area. You will trace the agent loop step-by-step (Lesson 2), implement tool dispatch with validation and error handling (Lesson 3), design context assembly strategies (Lesson 4), model control flow from loops to graphs (Lesson 5), build checkpoint-based persistence (Lesson 6), design human-in-the-loop intervention patterns (Lesson 7), and instrument agents with structured traces and cost tracking (Lesson 8).

Phase 2: Build Labs (Lessons 9–10) — You will build a complete harness from scratch using a deterministic model stub called FakeModelClient that makes tests reproducible without real API calls. Lesson 9 builds the minimal harness (loop + dispatch + context). Lesson 10 extends it with persistence, human intervention, observability, and MCP tool discovery. By the end, you will have a working harness that addresses all eight concerns.

Phase 3: Ecosystem and Selection (Lessons 11–12) — With your custom harness as a reference point, you will compare it against production frameworks (LangGraph, Temporal, CrewAI, Claude Agent SDK, and others) using the eight-concern framework as your evaluation lens. You will learn a decision framework for choosing the right harness for your use case, and explore frontier developments like MCP-native runtimes and agent-to-agent protocols.

graph LR
  subgraph Phase1["Phase 1: Deep Dives"]
      L2["L2: Loop"] --> L3["L3: Tools"]
      L3 --> L4["L4: Context"]
      L4 --> L5["L5: Control"]
      L5 --> L6["L6: Persist"]
      L6 --> L7["L7: HITL"]
      L7 --> L8["L8: Observe"]
  end

  subgraph Phase2["Phase 2: Build Labs"]
      L9["L9: Build"] --> L10["L10: Extend"]
  end

  subgraph Phase3["Phase 3: Ecosystem"]
      L11["L11: Compare"] --> L12["L12: Choose"]
  end

  L8 --> L9
  L10 --> L11

  style Phase1 fill:#e9d5ff,stroke:#9f7aea
  style Phase2 fill:#fef3c7,stroke:#f59e0b
  style Phase3 fill:#d1fae5,stroke:#10b981

Summary

The harness is the runtime layer between agent architectures and execution. It is responsible for eight critical concerns:

1. Loop execution — controlling when the agent runs and stops
2. Tool dispatch — validating and executing tool calls
3. Context management — assembling what the model sees each turn
4. State and goals — tracking progress and objectives
5. Control flow — routing through loops, branches, or handoffs
6. Persistence — checkpointing state for resumability
7. Observability — tracing, cost tracking, replay
8. Human intervention — approval gates, escalation, overrides

Raw API loops skip all eight. Thin SDKs solve one or two. Graph runtimes handle five. Full platforms address all eight. Choosing a harness means evaluating how well it solves these concerns for your use case.

In the next lesson, The Agent Loop Internals, we will open the loop and trace a single turn through all decision points: when does the harness call the model? How does it decide what to do next? Where do errors surface? You will see runs, turns, steps, and the control flow decisions that determine agent behavior.