> For the complete documentation index, see [llms.txt](https://handbook.modular.com/llms.txt).
> Markdown versions of all pages are available by appending .md to any URL.

# Agent harnesses

An agent harness is a system that wraps one or more AI models and enables them
to interact with the user and their environment. It can hold long-running
conversations with a model and carry out actions with other programs when the
model requests them.

The harness maintains conversation state, exposes available tools, executes
approved actions, returns observations, and decides whether to prompt the model
again or stop. This work typically takes place in an iterative control loop. The
harness doesn't make the model more capable as the weights are unchanged, but it
gives the model information about its environment, so that useful work can
happen safely and repeatedly. The relation is often summarized as:

> Agent = Model + Harness

## Why do you need an agent harness?

Many models today accept more than text, including images, audio, or video.
However, without support from the surrounding application, they can't run a
tool, execute code, set up an environment, retain state across API calls, or
verify whether the larger task is finished.

[Function calling](https://handbook.modular.com/model-interaction/function-calling.md) closes part of that gap
by letting a model request an action, but a request isn't an execution. The
[Model Context Protocol (MCP)](https://handbook.modular.com/model-interaction/model-context-protocol.md)
standardizes how tools and data sources connect, but it doesn't define the agent
loop. Something still has to run the tools, feed the results back, and decide
whether to continue or stop.

Without a deliberate harness, a model fails in predictable ways:

- **Unbounded loops**. It repeats an action or keeps revising without making
  progress.
- **Context overload**. Logs, files, and tool results crowd out the instructions
  that matter.
- **Unsafe actions**. A valid tool call reaches a system the model shouldn't
  have direct authority over.
- **False completion**. It reports success with no test or external check
  confirming the result.
- **Brittle recovery**. A timeout or malformed response ends the task because no
  checkpoint or retry policy exists.
- **Opaque failures**. The answer is wrong, but no trace shows which call or
  tool result caused it.

These are mainly runtime problems. A stronger model can reduce some mistakes,
but a model upgrade doesn't replace control flow, permissions, or verification.

## Agent harness use cases

Agent harnesses work best when a task needs several model calls, external
actions, and feedback from the environment. Common use cases include:

- **Coding agents**. Read a repository, edit files, run tests, and iterate on
  failures. This is a common harness workload and among the most demanding, with
  large
  [context windows](https://handbook.modular.com/llm-inference-basics/how-does-llm-inference-work.md#what-is-a-context-window-and-how-does-it-work-in-llm-inference)
  and tool calls on multiple steps.
- **Deep research**. Search, read, cross-check, and synthesize sources. These
  runs can last a long time, fan out to parallel sub-agents, and fill context
  with retrieved documents rather than reasoning from the agent itself.
- **Customer support services**. A support agent can read account data, apply
  policy, draft a response, and request approval before a refund or account
  change.
- **Browser and computer use**. The harness can translate model decisions into
  clicks or keystrokes, enforcing domain restrictions and confirmation rules.
- **Multi-agent pipelines**. A planner delegates to specialized sub-agents and
  merges their results, which is the agentic form of the patterns covered in
  [multi-model inference pipelines](https://handbook.modular.com/infrastructure-and-operations/multi-model-inference-pipelines.md).

## Popular agent harnesses

The boundary between a harness, an agent framework, and an end-user product
isn't sharp. The following projects are representative rather than ranked, and
they fall into two rough groups.

### Coding agent harnesses

These harnesses are generally invoked per task, run in a terminal or editor, and
scope their actions to a repository and the commands needed to build and test
it.

- **Claude Code**. The harness from Anthropic, available in terminals, IDE
  integrations, desktop clients, and on the web, with MCP and subagent support.
- **Codex**. The open-source CLI from OpenAI is a sandbox-first terminal
  harness; Codex also runs in IDE, app, and cloud workflows.
- **OpenCode**. An open-source terminal coding agent with provider selection,
  permissions, and specialized agents.
- **Cline**. Started as an editor extension and grew a standalone open-source AI
  coding agent, with support for parallel agents and SDK workflows.
- **Letta Code**. An open-source, model-agnostic harness built around persistent
  memory. Agents persist across sessions rather than starting fresh each time,
  with subagents, searchable conversation history, and skills.

Some model providers ship harnesses optimized for their own models, such as Qwen
Code, ZCode, and Kimi Code.

### General-purpose assistant harnesses

These often run continuously rather than per task. They reach users through
messaging apps instead of a terminal and act across a whole environment (e.g.,
email, calendar, browser, and filesystem) rather than a single repository. That
wider scope is also the risk. A prompt injection or a malicious skill can
trigger real actions on the host or expose sensitive data.

- **OpenClaw**. An open-source personal assistant that runs on hardware you
  control. A local gateway acts as the control plane for sessions, tools,
  events, and channel connections. A primary agent spawns subagents as needed.
  Plugins and skills extend the agent to different channels, such as email,
  browser control, calendar, and the filesystem.
- **Hermes Agent**. An open-source, self-hosted assistant harness from Nous
  Research, built around a learning loop. The agent creates skills from
  completed tasks, then refines them in use. Curated memory persists across
  sessions. A single gateway carries one conversation across a terminal UI,
  Telegram, Discord, Slack, WhatsApp, and Signal.

## How to use an agent harness with open-source LLMs

The best-known harnesses ship pointed at models from the vendor that built them,
but the harness and the model are separable. Many harnesses allow you to connect
to an open-source or open-weight model through a local or self-hosted inference
endpoint. This provides more privacy and customization, and helps control your
cost.

API compatibility makes the connection possible, but protocol compatibility
alone doesn't guarantee good agent behavior. To connect a harness to a model
and verify that the pairing works, work through the following steps:

1. **Choose a model trained for tool use**. Confirm that the model can select
   tools, produce valid arguments, and recover after a tool error.
2. **Match the API protocol**. A harness may require
   [OpenAI-compatible](https://handbook.modular.com/model-interaction/openai-compatible-api.md) Chat
   Completions or the Responses API, an
   [Anthropic-compatible](https://handbook.modular.com/model-interaction/anthropic-compatible-api.md) Messages
   API, or a provider SDK.
3. **Configure the server for the model**. Use the correct chat template,
   tool-call parser, reasoning parser, stop tokens, and maximum context length.
4. **Point the harness at the endpoint**. Configure the base URL, model name,
   and required credentials (or a placeholder). You can often do this by
   starting a server with an
   [inference framework](https://handbook.modular.com/getting-started/choosing-the-right-inference-framework.md)
   like MAX or vLLM, and point your harness at that server.
5. **Test the full loop**. Measure tool call accuracy, task completion,
   recovery, context growth, and permission behavior rather than only text
   quality.
6. **Optimize repeated prefixes**. Keep stable instructions and tool definitions
   near the front of the prompt, then monitor prefix cache reuse.

An OpenAI-compatible API or Anthropic-compatible API often provides the easiest
integration path for harnesses. Some vendor harnesses require a different or
larger API surface, so check the current integration documentation before
choosing a server.

Open models also change capacity planning. Long context and several concurrent
agent sessions can consume more GPU memory than the model weights suggest. Test
with realistic trajectories and tool output before setting production limits.

## FAQs

### What is harness engineering?

Harness engineering is the practice of improving agent results by changing the
system around the model instead of the model itself. This includes the loop, the
tool definitions, what enters context on each turn, the stopping and retry
rules, the verification steps, and the permission boundary. It treats those
pieces as things you measure and iterate on, evaluated end to end on real tasks
rather than on single-response quality. This is important because the same model
can score very differently on the same benchmark depending on the harness it
runs in.

### What is the difference between an agent harness and an agent framework?

A harness is the runtime layer that surrounds the model. A framework is a
software package used to build some or all of that layer. A ready-to-use agent
product may contain a harness without exposing the harness as a library.

### Can any open-source LLM work with any harness?

No. The API shape must match, and the model needs the tool use behavior expected
by the harness. Chat templates, parsers, context limits, and reasoning formats
can also differ. Test the combination rather than assuming that a compatible
endpoint makes every model interchangeable.