Agent Harness Comparison

A side-by-side breakdown of how each agent harness is built, what it can do, and where it falls short. Click any harness column to see full evidence and citations.

Confidence levels high Confirmed in source · medium Inferred from patterns · low Best guess
Information Field Codex CLI OpenCode Pi Coding Agent
Core Metadata
License license

Apache-2.0

high

MIT

high

MIT

high
Primary Language primary_language

Rust

high

TypeScript

medium

TypeScript

high
Platforms platforms

linux, macos, windows

high

linux, macos, windows

high

linux, macos, windows

high
Edit
Mechanism mechanism

apply_patch

high

apply_patch, str_replace, rewrite_file

high

str_replace

high
Anchoring anchoring

Context-anchored diff hunks (`@@` + context/old lines) matched with seek-sequence; mismatch errors if context or expected lines cannot be found.

high

`apply_patch` anchors updates using `@@` sections plus context/old-line sequence matching; `edit` falls back through block/context/whitespace/indentation matchers.

high

Anchored by matching oldText in file content: exact match first, then fuzzy-normalized fallback; replacement must be unique.

high
Verification verification

Reject-on-mismatch: patch input is reparsed/verified before apply; parse/context errors fail the call; no fuzzy merge path.

medium

Reject-on-mismatch. Patch input is parsed and verified before writes; edit fails when oldString is missing or non-unique.

high

Rejects on missing match, non-unique match, or no-op replacement; applies single replacement and writes full file (no partial merge).

high
Retry Loop retry_loop

No automatic apply_patch retry loop in the handler; a single verified parse/apply attempt returns success or error.

medium

No external auto-retry loop for failed edits; each call is single-shot. `edit` performs internal matcher fallbacks before failing.

medium

No edit-specific auto-retry loop in harness code; edit errors are surfaced to the agent as tool failures.

medium
Failure Modes failure_modes

implicit_invocation_rejected: Raw patch bodies without explicit apply_patch invocation are rejected.; parse_or_payload_error: Unsupported payload and invalid/non-apply_patch input produce model-visible errors.; context_…

high

patch_parse_or_empty: Invalid patch envelope or empty patch is rejected.; patch_target_missing: Update fails if target file cannot be read.; patch_context_mismatch: Update fails when expected context/old lines are not f…

high

file_not_found; old_text_not_found; non_unique_old_text; no_effect_change; aborted

high
Mechanism Default mechanism_default

apply_patch

medium

Tools
Available available

exec_command, write_stdin, shell, apply_patch, read_file, list_dir, grep_files, list_mcp_resources, list_mcp_resource_templates, read_mcp_resource, update_plan, request_user_input, js_repl, js_repl_reset, search_tool_bm…

high

bash, read, list, glob, grep, edit, write, apply_patch, task, skill, todowrite, webfetch, websearch, codesearch, question, lsp (experimental), custom tools from config dirs/plugins, MCP tools

high

read, write, edit, bash, grep, find, ls, extension_tools

high
Schema Style schema_style

Mixed: JSON-schema function tools, freeform grammar tools (Lark), native web_search tool type, plus MCP-to-tool conversion and dynamic tools.

high

Zod-defined parameters converted to JSON Schema for AI SDK function tools; MCP tools converted from MCP JSON schema; one freeform-style patch payload field (`patchText`).

high

TypeBox JSON-schema-like parameter objects, validated before tool execution.

high
Sandbox sandbox

Command execution is sandbox-policy driven (none/macOS seatbelt/Linux seccomp/Windows restricted token) with approval gating and execpolicy allow/prompt/forbid decisions.

high

No hard OS sandbox is implemented in tool runner. Boundaries are permission rules (`allow`/`ask`/`deny`) plus explicit `external_directory` checks; shell runs via local spawn.

medium

No built-in permission popup/approval sandbox; bash runs in configured cwd, and file tools resolve relative-to-cwd or absolute paths.

medium
Config
Locations locations

admin managed preferences (macOS), system `/etc/codex/config.toml` (Unix) or `%ProgramData%\OpenAI\Codex\config.toml` (Windows), user `${CODEX_HOME}/config.toml`, cwd `${PWD}/config.toml` (trust-gated), tree parent `./.…

high

remote org config: <provider>/.well-known/opencode, global config: ~/.config/opencode/opencode.json|jsonc, custom config file: $OPENCODE_CONFIG, project config: opencode.json|jsonc (searches upward), .opencode directori…

high

path=~/.pi/agent/settings.json, scope=global, precedence=lower; path=.pi/settings.json, scope=project, precedence=higher; path=~/.pi/agent (base dir, overridable via env), scope=global, precedence=base

high
Env Vars env_vars

name=CODEX_HOME, meaning=Overrides Codex state/config home directory (default `~/.codex`).; name=RUST_LOG, meaning=Controls Codex logging verbosity.; name=OPENAI_BASE_URL, meaning=Overrides default OpenAI provider base …

high

name=OPENCODE_CONFIG, meaning=Path to custom config file loaded between global and project configs.; name=OPENCODE_CONFIG_DIR, meaning=Additional config directory scanned for agents/commands/modes/plugins.; name=OPENCOD…

high

name=PI_CODING_AGENT_DIR, meaning=Override agent config directory (default ~/.pi/agent); name=PI_PACKAGE_DIR, meaning=Override package asset directory; name=PI_SKIP_VERSION_CHECK, meaning=Skip startup version check; nam…

high
Extensions
Supported skills.supported

true

high

true

high

true

high
Locations skills.locations

project config folders: `<layer-config-folder>/skills`, deprecated user location: `$CODEX_HOME/skills`, user-installed: `$HOME/.agents/skills`, embedded system cache: `$CODEX_HOME/skills/.system`, repo-local: directorie…

high

.opencode/skills/<name>/SKILL.md, ~/.config/opencode/skills/<name>/SKILL.md, .claude/skills/<name>/SKILL.md, ~/.claude/skills/<name>/SKILL.md, .agents/skills/<name>/SKILL.md, ~/.agents/skills/<name>/SKILL.md, additional…

high

~/.pi/agent/skills/, .pi/skills/, package skills/ or pi.skills in package.json, settings skills[] paths, --skill <path>

high
Format skills.format

Skill body in `SKILL.md` with required YAML frontmatter; optional metadata in `.agents/agents/openai.yaml`.

high

Markdown `SKILL.md` with YAML frontmatter (`name`, `description` required).

high

Markdown SKILL.md with frontmatter (Agent Skills spec).

high
Plugins plugins

Dynamic tool injection plus app connector tools; no separate package registry was found in this scan.

medium

Supported. Loads local TS/JS plugin files and npm plugins, auto-installs dependencies with Bun, and exposes hook/event APIs plus plugin-defined tools.

high

TypeScript extension API plus pi package ecosystem; extensions can register tools/commands/flags/providers and intercept tool calls/results.

high
Mcp mcp

Supports MCP client integration (configured servers/tools/resources) and an experimental MCP server mode (`codex mcp-server`).

high

Supported MCP client for local+remote servers, OAuth flows, dynamic MCP tool registration, and MCP prompts/resources consumption.

high

No built-in MCP in core harness; MCP integration is expected via extensions.

medium
Providers
Supported supported

openai, ollama, lmstudio, user_defined_via_model_providers

high

models.dev provider catalog (75+ providers), opencode, anthropic, openai, amazon-bedrock, azure, google, google-vertex, google-vertex-anthropic, openrouter, xai, mistral, groq, deepinfra, cerebras, cohere, gateway, toge…

medium

amazon-bedrock, anthropic, google, google-gemini-cli, google-antigravity, google-vertex, openai, azure-openai-responses, openai-codex, github-copilot, xai, groq, cerebras, openrouter, vercel-ai-gateway, zai, mistral, mi…

high
Auth auth

Provider auth supports OpenAI login/auth token flow (`requires_openai_auth`) and environment-based API keys via `env_key`/`api_key()` lookup.

high

Supports API keys, OAuth tokens, and well-known tokens; credentials are stored in `~/.local/share/opencode/auth.json`; `/connect` and `opencode auth` workflows are documented.

high

Supports OAuth login and API keys. Resolution order: --api-key runtime override, auth.json entry, environment variable, then models.json fallback resolver.

high
Model Selection model_selection

Model/provider can be selected by CLI flags (`--model`, `--oss`, `--local-provider`) and config (`model_provider`, profile, fallback default `openai`).

high

Model can be selected by CLI flag (`--model`), config (`model`/`small_model`), and interactive `/models`; default selection falls back to recent-model state then provider defaults.

high

Interactive `/model` command and CLI flags (`--provider`, `--model`); defaults and cycling are configurable in settings.

high
Ux
Interface interface

CLI multitool with interactive TUI default and non-interactive `exec` mode.

high

CLI with interactive TUI by default, non-interactive run mode, and optional headless server.

high

CLI/TUI interactive interface, plus non-interactive print, JSON stream, RPC, and SDK embeddings.

high
Streaming streaming

Yes. Streaming is exposed in app-server event notifications and SDK `runStreamed()` event generator.

high

Yes. `opencode run --format json` streams structured events; server exposes SSE event streams.

high

Yes. Assistant/token/tool events stream; JSON mode emits event stream lines.

high
Review Controls review_controls

Approval controls include `--ask-for-approval` policies, sandbox policy selection, and a dangerous bypass mode; runtime execpolicy can allow/prompt/forbid commands.

high

Permission system supports `allow`/`ask`/`deny` with prompt-time approvals (`once`, `always`, `reject`) and per-agent overrides.

high

No built-in permission popup approvals in core; confirmation gates can be added by extensions.

medium
Sessions sessions

Session rollouts persist as JSONL under `$CODEX_HOME/sessions` with archived sessions and append-only `session_index.jsonl`; `--ephemeral` disables persistence.

high

Session/message state is persisted in SQLite at `~/.local/share/opencode/opencode.db`; CLI supports continue/session/fork flows.

high

Sessions persist as JSONL trees under ~/.pi/agent/sessions (unless --no-session/in-memory mode).

high
Context
Compaction compaction

Auto-compaction is token-threshold driven (`model_auto_compact_token_limit`), run pre-sampling and mid-turn; implementation chooses inline local or remote compaction task.

high

Automatic compaction is token-threshold based (`reserved` buffer) and configurable (`compaction.auto`, `compaction.prune`, `compaction.reserved`); compaction can auto-insert a synthetic continue message.

high

Auto-compaction summarizes old context when threshold exceeded; manual /compact is also supported; settings control reserve/keep token budgets.

high
Overflow Handling overflow_handling

On normal sampling overflow, the turn errors with `ContextWindowExceeded`; during compaction, overflow triggers trimming oldest history items and retrying until fit or terminal failure.

high

When near context limit, session loop enqueues compaction and continues; context-overflow errors are treated as non-retryable.

high

On context overflow, pi removes the overflow error message from active context, auto-compacts, then retries the agent run.

high
Reliability
Auto Retry auto_retry

Automatic retries exist for retryable stream failures (with backoff and provider-configured retry budget) and HTTP transport failures (5xx/network/timeout) via request retry policy.

high

Automatic retry is enabled for retryable provider/API failures with exponential backoff and `Retry-After` header support; no explicit user-facing disable switch was found in scanned docs/code.

medium

Built-in transient-error retry with exponential backoff; configurable (`retry.enabled`, `maxRetries`, delays) and disable-able.

high
Recovery Loops recovery_loops

Recovery loops include stream transport fallback (WebSocket->HTTPS), invalid-image sanitization+continue, and compaction retry-by-trimming when compact prompts overflow context.

high

Includes a doom-loop guard (same tool call repeated 3 times) that requests permission, plus auto-compaction loop handoff (`process` returns `compact`).

high

Two recovery loops: (1) overflow-triggered auto-compaction + continue; (2) transient-error auto-retry loop. Both emit start/end events.

high
Integration
Sdk sdk

Supported TypeScript SDK package: `@openai/codex-sdk` wrapping the CLI.

high

Supported JS/TS SDK package `@opencode-ai/sdk` (createOpencode server+client or createOpencodeClient to attach).

high

Supported TypeScript/Node SDK in @mariozechner/pi-coding-agent (createAgentSession/AgentSession API).

high
Modes modes

interactive_tui_default, non_interactive_exec, jsonl_event_output_mode, mcp_server_mode, app_server_mode

high

Interactive TUI, non-interactive `run`, headless HTTP `serve`, and ACP subprocess mode.

high

interactive, print, json_event_stream, rpc, sdk_embedding

high
Protocol protocol

Exec JSON mode uses JSONL event framing; app-server uses bidirectional JSON-RPC 2.0 over stdio JSONL or websocket text frames.

high

HTTP/OpenAPI server APIs with SSE event streams; ACP uses JSON-RPC over stdio with NDJSON framing.

high

JSON lines protocol over stdio. RPC commands on stdin, responses/events on stdout; JSON mode streams AgentSession events as JSONL.

high
Customization
System Prompt system_prompt

System/developer instructions are customizable via config (`instructions`, `developer_instructions`, `model_instructions_file`), global AGENTS files under CODEX_HOME, and project AGENTS hierarchy (root->cwd).

high

System instructions come from agent prompt + built-in model prompt, project/global `AGENTS.md` (with `CLAUDE.md` compatibility), and `instructions` files/URLs in config.

high

System prompt can be replaced by SYSTEM.md (project over global), appended via APPEND_SYSTEM.md, and overridden/appended by CLI flags.

high
Prompt Templates prompt_templates

Built-in instruction templates are Markdown files embedded via `include_str!`; user custom prompts load from `$CODEX_HOME/prompts` as `.md` files with optional frontmatter keys (`description`, `argument-hint`).

high

Prompt templates are defined as custom commands: JSON `command.<name>.template` or Markdown files in `commands/**/*.md` with frontmatter + body template.

high

Markdown prompt files loaded from global/project/package/settings/CLI paths, invoked as slash commands with argument substitution.

high
Keybindings keybindings

No keybinding config file/setting location was found; shortcuts appear implemented in TUI code.

medium

Configurable via `opencode.json` `keybinds` object (string key chords, `none` to disable).

high

Customizable via JSON at ~/.pi/agent/keybindings.json; user config overrides defaults.

high
Themes themes

No explicit theme configuration file/setting was found in this scan; TUI color handling references terminal palette behavior.

medium

Theme selection via config (`theme`) with built-ins and JSON custom themes loaded from user/project `.opencode/themes` directories.

high

Themes are JSON files loaded from built-ins, global/project dirs, packages, settings, or CLI paths; active custom theme hot-reloads.

high
Distribution
Artifacts artifacts

npm_global_package=@openai/codex; homebrew_cask=codex; platform_specific_release_binaries

high

install script (`curl ... | bash`), npm global package (`opencode-ai`), Homebrew formula, Scoop and Chocolatey packages, Arch Linux and AUR packages, Nix package, desktop binaries on GitHub releases

high

npm package: @mariozechner/pi-coding-agent; CLI bin=pi (dist/cli.js); optional compiled binary via bun build --compile

high
Packages
Install Locations install_locations

global_npm_install, global_homebrew_cask_install

medium

$OPENCODE_INSTALL_DIR, $XDG_BIN_DIR, $HOME/bin, $HOME/.opencode/bin

high

global npm install root and ~/.pi/agent/git/<host>/<path>, project-local .pi/npm/node_modules and .pi/git/<host>/<path>, temporary extension installs under tmpdir for one-off -e/--extension

high
Extras
integration.app_server_protocol_versioning extras.integration.app_server_protocol_versioning

App-server protocol outputs and generated TypeScript bindings are specific to the Codex version used.

high

reliability.transport_fallbacks extras.reliability.transport_fallbacks

Codex can switch from WebSockets to HTTPS as a fallback transport during retry handling.

high

skills.system_skill_cache extras.skills.system_skill_cache

Embedded system skills are installed into `CODEX_HOME/skills/.system`.

high

ux.event_opt_out_controls extras.ux.event_opt_out_controls

App-server clients can suppress selected notifications per connection by method name.

high

tools.model_conditional_exposure extras.tools.model_conditional_exposure

Tool registry toggles `apply_patch` versus `edit`/`write` exposure based on model ID checks.

high

integration.local_rpc_mode extras.integration.local_rpc_mode

TUI worker thread supports direct RPC communication without HTTP.

high

reliability.loop_guard_doom_loop extras.reliability.loop_guard_doom_loop

Session processing uses a doom-loop threshold of three repeated tool calls before asking permission.

high

config.variable_substitution extras.config.variable_substitution

Config interpolation replaces `{env:...}` variables and resolves `{file:...}` references.

high

messaging.queue_controls extras.messaging.queue_controls

Pi has explicit steering vs follow-up queue semantics and modes.

high
extensions.ui_bridge_modes extras.extensions.ui_bridge_modes

RPC mode maps extension UI calls into protocol messages; some UI capabilities are unavailable in RPC.

high
packages.filtering_controls extras.packages.filtering_controls

Pi supports include/exclude and exact force include/exclude patterns (`!`, `+`, `-`) for package resources.

high