feat(agent-runner): an effort argument next to model on run and review (swamp-club #3570) #513

Merged

stack72 merged 2 commits from 3570 into main

2026-10-16 23:50:47 +00:00

stack72 commented

2026-10-16 23:47:24 +00:00

Owner

Resolves swamp-club #3570.

Summary

@swamp/agent-runner could choose an agent's model but not its effort. The runner clears the environment, so CLAUDE_CODE_EFFORT_LEVEL can't get through, and that stays as it is.

run and review now take an optional top-level effort, next to model: one of low, medium, high, xhigh or max. It is top-level rather than provider config, so it carries over when a caller's fallback switches provider.

Provider	Passed as
`claude`	`--effort <level>`
`codex`	`-c model_reasoning_effort=<level>` (still applied under `--ignore-user-config`)

When effort is unset, no flag is sent and the CLI's default applies.
The effort used is recorded on runResult and reviewResult (as a data field and an effort tag), and on the agent.run span (agent_runner.effort).
The recorded effort only ever comes from the run's argument. The agent's own review JSON is validated with a separate AgentReviewOutputSchema that has no effort field. That way an agent can't set the recorded effort, and a stray value from it can't fail a review.

Changes

_lib/schemas.ts: EffortSchema; effort on RunArgsSchema, ReviewArgsSchema, RunResultSchema and ReviewResultSchema; AgentReviewOutputSchema for the agent's JSON
_lib/types.ts, _lib/runner.ts, _lib/tracing.ts: thread effort through the run, plus a span attribute and a log line
_lib/providers/claude.ts, _lib/providers/codex.ts: the flags above
agent_runner.ts, _lib/review.ts: record effort in data and tags
Model version 2026.10.17.1 with an identity upgrade; manifest bumped
README: "Set the Effort Per-Invocation" section

Testing

113/113 unit tests pass:
- Each level for both providers, and no flag when unset
- Schema accepts every level and rejects unknown values
- run and review executed against a fake cached claude binary, checking the argv it receives and the recorded data and tags
- An effort the agent writes into its review JSON is ignored
Manual end-to-end through swamp (scratch repo loading this branch, claude CLI 2.1.302 downloaded by the runner, auth=cli):
- run with effort=low: replied OK, recorded effort: low in data and tag
- review with effort=medium: completed, effort recorded
- effort=extreme: rejected by argument validation
Codex 0.157.1 accepts -c model_reasoning_effort=max with the runner's flags; its API lists none, minimal, low, medium, high, xhigh, max as valid.

Verification attestation: bbba041e-ef5b-4954-8368-b3a2a3ea14b4 (commit 46e0a327, 12/14 passed, 2 guarded skips). An earlier verification round passed, but both reviews flagged that the agent's own JSON could set the recorded effort; that is fixed in the second commit.

Known limitations

The claude CLI accepts any --effort string, so the recorded effort is the one the runner sent, not proof the CLI used it.
A pinned claude version older than the CLI's --effort support fails when effort is set. The cutoff version is not documented here.
Whether a level is accepted depends on the model; the CLI reports one it doesn't support.
Codex levels outside the shared set (none, minimal) are not exposed.

🤖 Generated with Claude Code

Resolves swamp-club #3570. ## Summary `@swamp/agent-runner` could choose an agent's model but not its effort. The runner clears the environment, so `CLAUDE_CODE_EFFORT_LEVEL` can't get through, and that stays as it is. `run` and `review` now take an optional top-level `effort`, next to `model`: one of `low`, `medium`, `high`, `xhigh` or `max`. It is top-level rather than provider config, so it carries over when a caller's fallback switches provider. | Provider | Passed as | | -------- | --------- | | `claude` | `--effort <level>` | | `codex` | `-c model_reasoning_effort=<level>` (still applied under `--ignore-user-config`) | - When `effort` is unset, no flag is sent and the CLI's default applies. - The effort used is recorded on `runResult` and `reviewResult` (as a data field and an `effort` tag), and on the `agent.run` span (`agent_runner.effort`). - The recorded effort only ever comes from the run's argument. The agent's own review JSON is validated with a separate `AgentReviewOutputSchema` that has no effort field. That way an agent can't set the recorded effort, and a stray value from it can't fail a review. ## Changes - `_lib/schemas.ts`: `EffortSchema`; `effort` on `RunArgsSchema`, `ReviewArgsSchema`, `RunResultSchema` and `ReviewResultSchema`; `AgentReviewOutputSchema` for the agent's JSON - `_lib/types.ts`, `_lib/runner.ts`, `_lib/tracing.ts`: thread `effort` through the run, plus a span attribute and a log line - `_lib/providers/claude.ts`, `_lib/providers/codex.ts`: the flags above - `agent_runner.ts`, `_lib/review.ts`: record effort in data and tags - Model version `2026.10.17.1` with an identity upgrade; manifest bumped - README: "Set the Effort Per-Invocation" section ## Testing - 113/113 unit tests pass: - Each level for both providers, and no flag when unset - Schema accepts every level and rejects unknown values - `run` and `review` executed against a fake cached claude binary, checking the argv it receives and the recorded data and tags - An effort the agent writes into its review JSON is ignored - Manual end-to-end through swamp (scratch repo loading this branch, claude CLI 2.1.302 downloaded by the runner, `auth=cli`): - `run` with `effort=low`: replied `OK`, recorded `effort: low` in data and tag - `review` with `effort=medium`: completed, effort recorded - `effort=extreme`: rejected by argument validation - Codex 0.157.1 accepts `-c model_reasoning_effort=max` with the runner's flags; its API lists `none, minimal, low, medium, high, xhigh, max` as valid. Verification attestation: `bbba041e-ef5b-4954-8368-b3a2a3ea14b4` (commit `46e0a327`, 12/14 passed, 2 guarded skips). An earlier verification round passed, but both reviews flagged that the agent's own JSON could set the recorded effort; that is fixed in the second commit. ## Known limitations - The claude CLI accepts any `--effort` string, so the recorded effort is the one the runner sent, not proof the CLI used it. - A pinned claude `version` older than the CLI's `--effort` support fails when `effort` is set. The cutoff version is not documented here. - Whether a level is accepted depends on the model; the CLI reports one it doesn't support. - Codex levels outside the shared set (`none`, `minimal`) are not exposed. 🤖 Generated with [Claude Code](https://claude.com/claude-code)

stack72 added 2 commits

2026-10-16 23:47:24 +00:00

feat(agent-runner): an effort argument next to model on run and review (swamp-club #3570 ) d334f7709b

run and review take an optional effort (low, medium, high, xhigh, max).
It is top-level rather than provider config, so it carries across a
claude-to-codex fallback. claude receives --effort <level>; codex receives
-c model_reasoning_effort=<level>, which still applies under
--ignore-user-config. When effort is unset, no flag is sent. The
environment stays cleared, so CLAUDE_CODE_EFFORT_LEVEL is still not
forwarded.

The applied effort is recorded on runResult and reviewResult, as a data
field and as a tag, and on the agent.run span.

Model version 2026.10.17.1 with an identity upgrade.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

fix(agent-runner): take the recorded review effort from the run, not the agent's JSON (swamp-club #3570 )

CI / Review Integrity (pull_request) Successful in 26s

Details

CI / Validate Attestation (pull_request) Successful in 31s

Details

46e0a32791

parseResult validated the agent's output with ReviewResultSchema, which
now has an effort field. So an agent could set the recorded effort itself,
and an off-enum value would fail the review. Agent output is now validated
with AgentReviewOutputSchema, which has no effort; effort comes only from
args.effort.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

stack72 merged commit 11c995de39 into main

2026-10-16 23:50:47 +00:00

stack72 deleted branch 3570

2026-10-16 23:50:47 +00:00

stack72 referenced this pull request from a commit

2026-10-16 23:50:47 +00:00

feat(agent-runner): an effort argument next to model on run and review (swamp-club #3570) (#513)