ci: optimize model checks, fix review sandbox, remove GitHub workflows #5

Merged

stack72 merged 3 commits from worktree-462 into main 2026-05-27 18:18:49 +00:00

Conversation 6 Commits 3 Files changed 10 +203 -1686

stack72 commented 2026-05-27 17:38:36 +00:00

Owner

Copy link

Summary

Optimizes CI for generated model type-checking, fixes the Claude Code review sandbox issue, removes legacy GitHub workflows, and adds a merge gate.

Closes #462

What changed

CI optimization (~30min → ~2min for model checks)

The aws-check, gcp-check, and cloudflare-check jobs previously iterated through all 586 service directories (255 AWS + 262 GCP + 69 Cloudflare) running deno check, deno lint, and deno fmt on every PR that touches model/ or codegen/.

• Dropped fmt and lint for all generated model jobs (AWS, GCP, Cloudflare, Hetzner, DigitalOcean). These are provably redundant: the generator already runs deno fmt on output, and every generated file includes // deno-lint-ignore-file no-explicit-any.
• Replaced full deno check with sample-based checking. 13 representative services chosen for schema diversity:
  • AWS (5): healthimaging (baseline), quicksight (deep nesting), sns (json/any unions), apigateway (array/map types), ec2 (106 model files)
  • GCP (5): groupssettings (flat baseline), container (1092 z.object calls), storage (z.any patterns), dataproc (153 z.array + 90 z.record), compute (119 model files)
  • Cloudflare (3): r2 (simple), load-balancing (nested), pages (array/map-heavy)
• Added nightly full-model-check.yml that runs full deno check across all services as a safety net.

Claude Code review sandbox fix

All three review jobs (claude-review, claude-adversarial-review, claude-ci-security-review) were writing output to $RUNNER_TEMP/review/ which is outside the workspace. Claude Code's sandbox blocks writes outside the workspace, causing reviews to silently fail to produce output files. Moved the output directory to $GITHUB_WORKSPACE/.review-output/.

Merge gate

Added a merge-gate job that depends on every check and review job. If any job fails (including reviews), the gate fails. Set this as the required status check in Forgejo branch protection to block merges on review failures.

GitHub workflow removal

Removed all .github/workflows/ files (ci.yml, publish.yml, regenerate-models.yml, auto-response.yml). CI runs exclusively on Forgejo now; GitHub is a read-only mirror with issues disabled. Updated all agent-constraints/ docs and scripts/audit_actions.ts to reference .forgejo/workflows/.

Why this is correct

1. fmt/lint removal is safe — every generated .ts file contains // deno-lint-ignore-file no-explicit-any (verified via grep), and the codegen pipeline runs deno fmt on all output as part of generation.
2. Sample-based checking is sufficient — the 13 samples were chosen to cover: baseline/simple types, deeply nested objects, json/any-typed properties, array/map types, and large multi-file services. Codegen unit tests (~15 tests) validate the templates themselves. The nightly full check catches any gaps within 24 hours.
3. Sandbox fix is targeted — only the files Claude writes (review-body.md, review-failed) moved into the workspace. The binary download, changed-files list, and payload assembly remain in $RUNNER_TEMP (written by shell steps outside the sandbox).
4. Merge gate uses contains(needs.*.result, 'failure') — skipped jobs (due to path filters not matching) don't count as failures, so the gate only blocks on actual failures.

Test plan

• Push to branch and verify sample-based check jobs fire for the expected services
• Verify fmt/lint matrix entries no longer appear for model jobs
• Trigger full-model-check.yml via workflow_dispatch and confirm full check runs
• Confirm Claude review output lands in .review-output/ (no sandbox errors)
• Confirm merge-gate job appears and reflects review job results
• Set merge-gate as required status check in Forgejo branch protection

## Summary Optimizes CI for generated model type-checking, fixes the Claude Code review sandbox issue, removes legacy GitHub workflows, and adds a merge gate. **Closes #462** ## What changed ### CI optimization (~30min → ~2min for model checks) The `aws-check`, `gcp-check`, and `cloudflare-check` jobs previously iterated through **all 586 service directories** (255 AWS + 262 GCP + 69 Cloudflare) running `deno check`, `deno lint`, and `deno fmt` on every PR that touches `model/` or `codegen/`. - **Dropped `fmt` and `lint`** for all generated model jobs (AWS, GCP, Cloudflare, Hetzner, DigitalOcean). These are provably redundant: the generator already runs `deno fmt` on output, and every generated file includes `// deno-lint-ignore-file no-explicit-any`. - **Replaced full `deno check` with sample-based checking.** 13 representative services chosen for schema diversity: - **AWS** (5): `healthimaging` (baseline), `quicksight` (deep nesting), `sns` (json/any unions), `apigateway` (array/map types), `ec2` (106 model files) - **GCP** (5): `groupssettings` (flat baseline), `container` (1092 z.object calls), `storage` (z.any patterns), `dataproc` (153 z.array + 90 z.record), `compute` (119 model files) - **Cloudflare** (3): `r2` (simple), `load-balancing` (nested), `pages` (array/map-heavy) - **Added nightly `full-model-check.yml`** that runs full `deno check` across all services as a safety net. ### Claude Code review sandbox fix All three review jobs (`claude-review`, `claude-adversarial-review`, `claude-ci-security-review`) were writing output to `$RUNNER_TEMP/review/` which is outside the workspace. Claude Code's sandbox blocks writes outside the workspace, causing reviews to silently fail to produce output files. Moved the output directory to `$GITHUB_WORKSPACE/.review-output/`. ### Merge gate Added a `merge-gate` job that depends on every check and review job. If any job fails (including reviews), the gate fails. Set this as the required status check in Forgejo branch protection to block merges on review failures. ### GitHub workflow removal Removed all `.github/workflows/` files (`ci.yml`, `publish.yml`, `regenerate-models.yml`, `auto-response.yml`). CI runs exclusively on Forgejo now; GitHub is a read-only mirror with issues disabled. Updated all `agent-constraints/` docs and `scripts/audit_actions.ts` to reference `.forgejo/workflows/`. ## Why this is correct 1. **fmt/lint removal is safe** — every generated `.ts` file contains `// deno-lint-ignore-file no-explicit-any` (verified via grep), and the codegen pipeline runs `deno fmt` on all output as part of generation. 2. **Sample-based checking is sufficient** — the 13 samples were chosen to cover: baseline/simple types, deeply nested objects, json/any-typed properties, array/map types, and large multi-file services. Codegen unit tests (~15 tests) validate the templates themselves. The nightly full check catches any gaps within 24 hours. 3. **Sandbox fix is targeted** — only the files Claude writes (`review-body.md`, `review-failed`) moved into the workspace. The binary download, changed-files list, and payload assembly remain in `$RUNNER_TEMP` (written by shell steps outside the sandbox). 4. **Merge gate uses `contains(needs.*.result, 'failure')`** — skipped jobs (due to path filters not matching) don't count as failures, so the gate only blocks on actual failures. ## Test plan - [ ] Push to branch and verify sample-based check jobs fire for the expected services - [ ] Verify fmt/lint matrix entries no longer appear for model jobs - [ ] Trigger `full-model-check.yml` via workflow_dispatch and confirm full check runs - [ ] Confirm Claude review output lands in `.review-output/` (no sandbox errors) - [ ] Confirm merge-gate job appears and reflects review job results - [ ] Set `merge-gate` as required status check in Forgejo branch protection

stack72 added 1 commit 2026-05-27 17:38:36 +00:00

ci: optimize model checks, fix review sandbox, remove GitHub workflows ... 802672db22

Replace full deno check across 586 generated service directories (~30min)
with sample-based checking of 13 diverse representatives (~2min per PR).
Add nightly full-model-check workflow as safety net. Drop redundant
fmt/lint for generated code (generator already formats, lint is suppressed).

Fix Claude Code review sandbox issue by moving review output from
$RUNNER_TEMP to $GITHUB_WORKSPACE/.review-output/ so the sandbox
allows writes.

Remove all .github/workflows/ — CI now runs exclusively on Forgejo.
Update agent-constraints docs and audit_actions.ts to reference
.forgejo/workflows/.

Closes #462

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

stack72 commented 2026-05-27 17:46:59 +00:00

Author

Owner

Copy link

CI Security Review

Medium

1. .forgejo/workflows/ci.yml:632-633, 744-745, 835-836 — pull-requests: write permission on GITHUB_TOKEN is unused. The three Claude review jobs (claude-review, claude-adversarial-review, claude-ci-security-review) declare pull-requests: write in their job-level permissions, but they post review comments using the BOT_TOKEN secret via curl, not via GITHUB_TOKEN. The pull-requests: write scope on GITHUB_TOKEN is not consumed by any step. This is not exploitable since no step uses GITHUB_TOKEN with write intent, but removing it would tighten the principle of least privilege. Fix: Remove pull-requests: write from these jobs' permissions: blocks, or reduce to just contents: read.

Positive Security Observations

• Prompt injection mitigated: The Claude review steps construct prompts from cat .forgejo/prompts/*.md (repo-controlled files) and CHANGED_FILES (file paths from git diff --name-only). No attacker-controlled event data (PR title, body, issue content, commit messages) is interpolated into the prompt. The LLM fetches PR content itself via git diff tool calls. All three prompt files include a security preamble instructing the model to treat PR content as untrusted.

• Tool scoping is tight: The --allowedTools for all three Claude review steps are correctly minimal: Read, Grep, Bash(git diff:*), Bash(git log:*), Bash(tee ${OUTPUT_DIR}/review-body.md:*), Bash(touch ${OUTPUT_DIR}/review-failed). No arbitrary bash, no gh CLI, no network access. This is a significant improvement over the deleted GitHub workflow which granted Bash(gh pr review:*) and Bash(gh pr diff:*).

• Supply chain hardened for Claude CLI: The Claude CLI is downloaded with a pinned version (2.1.150) and verified via SHA256 checksum before execution (.forgejo/workflows/ci.yml:665-668). This is equivalent to SHA-pinning an action.

• Actions are from trusted publishers: All referenced actions (actions/checkout@v6, denoland/setup-deno@v2) are from trusted publishers, acceptable with tag-only pins per project policy.

• Workflow-level permissions are minimal: The top-level permissions: contents: read applies to all jobs by default. Only review jobs add write permissions at the job level (though as noted above, the write perm is unused).

• No dangerous triggers: Both Forgejo workflows use safe triggers (pull_request, schedule, workflow_dispatch). No pull_request_target, no issue_comment triggers.

• Reduced attack surface: Deleting four GitHub Actions workflows (auto-response, ci, publish, regenerate-models) eliminates potential attack vectors, including the issues: [opened] trigger in auto-response.yml and the PAT-based contents: write + force-push in regenerate-models.yml.

• Expression injection safe: All GitHub/Forgejo expressions in run: blocks are either from controlled matrix values (${{ matrix.task }}, ${{ matrix.vault }}), boolean checks (${{ contains(...) }}), or are passed through environment variables (BASE_SHA, HEAD_SHA) before use in shell commands.

• Secret handling is clean: ANTHROPIC_API_KEY is scoped to only the Claude CLI steps. BOT_TOKEN is scoped to only the comment-posting steps. Neither is logged or exposed through command substitution.

Verdict

PASS — This PR is a net security improvement. It migrates from GitHub Actions (with claude-code-action and broader tool grants) to Forgejo Actions with tightly-scoped Claude CLI invocations, SHA256-verified downloads, and prompt-injection-resistant prompt construction. The only finding is an unused pull-requests: write permission that is not exploitable.

## CI Security Review ### Medium 1. **`.forgejo/workflows/ci.yml`:632-633, 744-745, 835-836 — `pull-requests: write` permission on GITHUB_TOKEN is unused.** The three Claude review jobs (`claude-review`, `claude-adversarial-review`, `claude-ci-security-review`) declare `pull-requests: write` in their job-level permissions, but they post review comments using the `BOT_TOKEN` secret via curl, not via `GITHUB_TOKEN`. The `pull-requests: write` scope on `GITHUB_TOKEN` is not consumed by any step. This is not exploitable since no step uses `GITHUB_TOKEN` with write intent, but removing it would tighten the principle of least privilege. **Fix:** Remove `pull-requests: write` from these jobs' `permissions:` blocks, or reduce to just `contents: read`. ### Positive Security Observations - **Prompt injection mitigated**: The Claude review steps construct prompts from `cat .forgejo/prompts/*.md` (repo-controlled files) and `CHANGED_FILES` (file paths from `git diff --name-only`). No attacker-controlled event data (PR title, body, issue content, commit messages) is interpolated into the prompt. The LLM fetches PR content itself via `git diff` tool calls. All three prompt files include a security preamble instructing the model to treat PR content as untrusted. - **Tool scoping is tight**: The `--allowedTools` for all three Claude review steps are correctly minimal: `Read`, `Grep`, `Bash(git diff:*)`, `Bash(git log:*)`, `Bash(tee ${OUTPUT_DIR}/review-body.md:*)`, `Bash(touch ${OUTPUT_DIR}/review-failed)`. No arbitrary bash, no `gh` CLI, no network access. This is a significant improvement over the deleted GitHub workflow which granted `Bash(gh pr review:*)` and `Bash(gh pr diff:*)`. - **Supply chain hardened for Claude CLI**: The Claude CLI is downloaded with a pinned version (`2.1.150`) and verified via SHA256 checksum before execution (`.forgejo/workflows/ci.yml`:665-668). This is equivalent to SHA-pinning an action. - **Actions are from trusted publishers**: All referenced actions (`actions/checkout@v6`, `denoland/setup-deno@v2`) are from trusted publishers, acceptable with tag-only pins per project policy. - **Workflow-level permissions are minimal**: The top-level `permissions: contents: read` applies to all jobs by default. Only review jobs add write permissions at the job level (though as noted above, the write perm is unused). - **No dangerous triggers**: Both Forgejo workflows use safe triggers (`pull_request`, `schedule`, `workflow_dispatch`). No `pull_request_target`, no `issue_comment` triggers. - **Reduced attack surface**: Deleting four GitHub Actions workflows (auto-response, ci, publish, regenerate-models) eliminates potential attack vectors, including the `issues: [opened]` trigger in `auto-response.yml` and the PAT-based `contents: write` + force-push in `regenerate-models.yml`. - **Expression injection safe**: All GitHub/Forgejo expressions in `run:` blocks are either from controlled matrix values (`${{ matrix.task }}`, `${{ matrix.vault }}`), boolean checks (`${{ contains(...) }}`), or are passed through environment variables (`BASE_SHA`, `HEAD_SHA`) before use in shell commands. - **Secret handling is clean**: `ANTHROPIC_API_KEY` is scoped to only the Claude CLI steps. `BOT_TOKEN` is scoped to only the comment-posting steps. Neither is logged or exposed through command substitution. ### Verdict **PASS** — This PR is a net security improvement. It migrates from GitHub Actions (with `claude-code-action` and broader tool grants) to Forgejo Actions with tightly-scoped Claude CLI invocations, SHA256-verified downloads, and prompt-injection-resistant prompt construction. The only finding is an unused `pull-requests: write` permission that is not exploitable.

stack72 commented 2026-05-27 17:50:10 +00:00

Author

Owner

Copy link

Code Review

Blocking Issues

1. ssh/ missing from deps-audit find expression (.forgejo/workflows/ci.yml line 579)

   The deps-audit job scans:

   find vault datastore model codegen issue-lifecycle workflows cve -name "deno.lock" ...
   

   The ssh/ directory is absent. SSH has its own deno.lock and dedicated ssh-check/ssh-lockfile CI jobs, but its dependencies are never audited for known vulnerabilities or outdated packages. Per the CI fan-out checklist in planning-conventions.md and the ci-matrix-completeness dimension in adversarial-dimensions.md, the deps-audit find expression must cover every extension directory.

   Fix: add ssh to the directory list on line 579.

Suggestions

1. claude-review needs array missing ssh-check and ssh-lockfile (.forgejo/workflows/ci.yml lines 603–629)

   All other check/lockfile pairs are present, but ssh-check and ssh-lockfile are omitted. This means claude-review can post its verdict while SSH tests are still in-flight. The merge-gate still catches SSH failures, so nothing broken merges, but the ordering violates the CI fan-out convention documented in planning-conventions.md and adversarial-dimensions.md.

2. claude-adversarial-review needs array missing several jobs (.forgejo/workflows/ci.yml lines 719–742)

   The adversarial review's needs: omits model-check, model-lockfile, ssh-check, ssh-lockfile, and actions-audit. Same ordering concern as above — the merge-gate provides the final safety net, but the convention calls for these to be listed.

3. SSH changes don't trigger claude-adversarial-review (.forgejo/workflows/ci.yml line 750)

   The condition that activates adversarial review checks vaults, datastores, issue-lifecycle, workflows, cve, and codegen, but not ssh. If the SSH extension warrants the adversarial test-fidelity and sdk-wrap-defense checks (it connects to external hosts), add needs.changes.outputs.ssh == 'true' to the condition.

4. --allow-write unscoped in the actions-audit step (.forgejo/workflows/ci.yml line 600)

   deno run --allow-read --allow-net=api.github.com --allow-env=GITHUB_STEP_SUMMARY,GITHUB_TOKEN --allow-write scripts/audit_actions.ts
   

   --allow-write grants filesystem-wide write access; only $GITHUB_STEP_SUMMARY is actually needed. implementation-conventions.md cites audit_actions.ts as the precedent for scoped permissions — the precedent should match the principle. Use --allow-write=$GITHUB_STEP_SUMMARY.

5. audit_actions.ts version checks against GitHub's releases API, but workflows run on Forgejo (scripts/audit_actions.ts line 251)

   The script hardcodes workflowDir = ".forgejo/workflows" (correct after the GitHub workflow removal) and queries api.github.com to compare action versions. Forgejo's action catalog (e.g. code.forgejo.org) uses its own versioning — actions/checkout@v6 exists there but GitHub's latest is v4.x. isSameMajor("v6", "v4.x.x") returns false, so every PR's actions-audit output will carry a spurious "outdated" warning for the checkout action. This is a warning, not exit 1, so it won't block merges, but the persistent noise undercuts the signal value of the audit. Consider either skipping the version-outdated check for trusted publishers that are known to diverge between Forgejo and GitHub, or maintaining a known-version allowlist.

## Code Review ### Blocking Issues 1. **`ssh/` missing from `deps-audit` find expression** (`.forgejo/workflows/ci.yml` line 579) The `deps-audit` job scans: ```bash find vault datastore model codegen issue-lifecycle workflows cve -name "deno.lock" ... ``` The `ssh/` directory is absent. SSH has its own `deno.lock` and dedicated `ssh-check`/`ssh-lockfile` CI jobs, but its dependencies are never audited for known vulnerabilities or outdated packages. Per the CI fan-out checklist in `planning-conventions.md` and the `ci-matrix-completeness` dimension in `adversarial-dimensions.md`, the `deps-audit` find expression must cover every extension directory. Fix: add `ssh` to the directory list on line 579. ### Suggestions 1. **`claude-review` needs array missing `ssh-check` and `ssh-lockfile`** (`.forgejo/workflows/ci.yml` lines 603–629) All other check/lockfile pairs are present, but `ssh-check` and `ssh-lockfile` are omitted. This means `claude-review` can post its verdict while SSH tests are still in-flight. The `merge-gate` still catches SSH failures, so nothing broken merges, but the ordering violates the CI fan-out convention documented in `planning-conventions.md` and `adversarial-dimensions.md`. 2. **`claude-adversarial-review` needs array missing several jobs** (`.forgejo/workflows/ci.yml` lines 719–742) The adversarial review's `needs:` omits `model-check`, `model-lockfile`, `ssh-check`, `ssh-lockfile`, and `actions-audit`. Same ordering concern as above — the merge-gate provides the final safety net, but the convention calls for these to be listed. 3. **SSH changes don't trigger `claude-adversarial-review`** (`.forgejo/workflows/ci.yml` line 750) The condition that activates adversarial review checks `vaults`, `datastores`, `issue-lifecycle`, `workflows`, `cve`, and `codegen`, but not `ssh`. If the SSH extension warrants the adversarial `test-fidelity` and `sdk-wrap-defense` checks (it connects to external hosts), add `needs.changes.outputs.ssh == 'true'` to the condition. 4. **`--allow-write` unscoped in the `actions-audit` step** (`.forgejo/workflows/ci.yml` line 600) ```yaml deno run --allow-read --allow-net=api.github.com --allow-env=GITHUB_STEP_SUMMARY,GITHUB_TOKEN --allow-write scripts/audit_actions.ts ``` `--allow-write` grants filesystem-wide write access; only `$GITHUB_STEP_SUMMARY` is actually needed. `implementation-conventions.md` cites `audit_actions.ts` as the precedent for scoped permissions — the precedent should match the principle. Use `--allow-write=$GITHUB_STEP_SUMMARY`. 5. **`audit_actions.ts` version checks against GitHub's releases API, but workflows run on Forgejo** (`scripts/audit_actions.ts` line 251) The script hardcodes `workflowDir = ".forgejo/workflows"` (correct after the GitHub workflow removal) and queries `api.github.com` to compare action versions. Forgejo's action catalog (e.g. `code.forgejo.org`) uses its own versioning — `actions/checkout@v6` exists there but GitHub's latest is v4.x. `isSameMajor("v6", "v4.x.x")` returns `false`, so every PR's actions-audit output will carry a spurious "outdated" warning for the checkout action. This is a warning, not `exit 1`, so it won't block merges, but the persistent noise undercuts the signal value of the audit. Consider either skipping the version-outdated check for trusted publishers that are known to diverge between Forgejo and GitHub, or maintaining a known-version allowlist.

stack72 added 1 commit 2026-05-27 17:51:38 +00:00

fix(ci): add ssh/ to deps-audit find expression ... 3b8830a764

The ssh directory has its own deno.lock but was missing from the
deps-audit outdated-dependency scan. Its dependencies were never
audited for known vulnerabilities or outdated packages.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

stack72 commented 2026-05-27 17:58:44 +00:00

Author

Owner

Copy link

CI Security Review

Critical / High

No critical or high severity findings.

Medium

1. .forgejo/workflows/ci.yml:632-633, 744-745, 835-836 — pull-requests: write permission may be unnecessary on review jobs. The claude-review, claude-adversarial-review, and claude-ci-security-review jobs each declare pull-requests: write at the job level. However, none of these jobs use the built-in GITHUB_TOKEN for PR write operations — they post review comments via curl to the Forgejo API using the BOT_TOKEN secret. If the Forgejo runner does not require pull-requests: write for some other purpose, this grants the GITHUB_TOKEN broader scope than needed. Suggestion: Verify whether pull-requests: write is required by the Forgejo runner for these jobs; if not, remove it to follow least-privilege.

Low

No low severity findings.

Notes — Security-Positive Changes

• Deleted .github/workflows/auto-response.yml: This workflow ran on issues: [opened] and processed attacker-controlled issue content (title, body) to forward to an external API. Removing it eliminates an external-user-triggered workflow that handled untrusted input.
• Deleted .github/workflows/regenerate-models.yml: This workflow had contents: write + pull-requests: write permissions, used a PAT (GH_TOKEN) for checkout and PR creation, and ran git push --force. Removing it reduces the surface of privileged automation.
• Deleted .github/workflows/ci.yml and .github/workflows/publish.yml: Consolidation from GitHub Actions to Forgejo reduces the number of CI surfaces to audit and maintain.
• Claude tool scoping is well done: All three Claude review steps restrict tools to Read, Grep, Bash(git diff:*), Bash(git log:*), and tightly-scoped tee/touch commands. No arbitrary Bash, no network access, no write capabilities beyond the review output files.
• Prompt hardening is present: All three review prompts (review.md, adversarial.md, ci-security.md) include a security preamble instructing the model to treat PR content as untrusted data and ignore embedded instructions.
• No direct interpolation of user-controlled data: The CHANGED_FILES variable passed to Claude prompts contains only file paths from git diff --name-only, not PR title/body/comment content. The BASE_SHA/HEAD_SHA values in the changes job are correctly passed via environment variables rather than shell interpolation.
• Claude CLI is integrity-verified: The binary is downloaded with version pinning AND SHA256 checksum verification (sha256sum -c), preventing supply chain tampering.
• Expression injection is not present: All ${{ }} expressions in run: blocks use either fixed matrix values (matrix.task, matrix.vault, etc.) or safe boolean expressions (contains(needs.*.result, 'failure')). No event-payload fields are interpolated in shell commands.
• Added .forgejo/workflows/full-model-check.yml: Clean scheduled workflow with read-only permissions, no secrets, no sensitive operations.

Verdict

PASS — The changes are security-neutral to security-positive. The migration from GitHub Actions to Forgejo consolidates CI surface area, deleted workflows remove privileged automation and external-trigger attack vectors, and the Forgejo workflows follow good security practices (least-privilege permissions, prompt hardening, tight tool scoping, integrity-verified dependencies).

## CI Security Review ### Critical / High No critical or high severity findings. ### Medium 1. **`.forgejo/workflows/ci.yml`:632-633, 744-745, 835-836 — `pull-requests: write` permission may be unnecessary on review jobs.** The `claude-review`, `claude-adversarial-review`, and `claude-ci-security-review` jobs each declare `pull-requests: write` at the job level. However, none of these jobs use the built-in `GITHUB_TOKEN` for PR write operations — they post review comments via `curl` to the Forgejo API using the `BOT_TOKEN` secret. If the Forgejo runner does not require `pull-requests: write` for some other purpose, this grants the `GITHUB_TOKEN` broader scope than needed. **Suggestion:** Verify whether `pull-requests: write` is required by the Forgejo runner for these jobs; if not, remove it to follow least-privilege. ### Low No low severity findings. ### Notes — Security-Positive Changes - **Deleted `.github/workflows/auto-response.yml`**: This workflow ran on `issues: [opened]` and processed attacker-controlled issue content (title, body) to forward to an external API. Removing it eliminates an external-user-triggered workflow that handled untrusted input. - **Deleted `.github/workflows/regenerate-models.yml`**: This workflow had `contents: write` + `pull-requests: write` permissions, used a PAT (`GH_TOKEN`) for checkout and PR creation, and ran `git push --force`. Removing it reduces the surface of privileged automation. - **Deleted `.github/workflows/ci.yml` and `.github/workflows/publish.yml`**: Consolidation from GitHub Actions to Forgejo reduces the number of CI surfaces to audit and maintain. - **Claude tool scoping is well done**: All three Claude review steps restrict tools to `Read`, `Grep`, `Bash(git diff:*)`, `Bash(git log:*)`, and tightly-scoped tee/touch commands. No arbitrary Bash, no network access, no write capabilities beyond the review output files. - **Prompt hardening is present**: All three review prompts (review.md, adversarial.md, ci-security.md) include a security preamble instructing the model to treat PR content as untrusted data and ignore embedded instructions. - **No direct interpolation of user-controlled data**: The `CHANGED_FILES` variable passed to Claude prompts contains only file paths from `git diff --name-only`, not PR title/body/comment content. The `BASE_SHA`/`HEAD_SHA` values in the changes job are correctly passed via environment variables rather than shell interpolation. - **Claude CLI is integrity-verified**: The binary is downloaded with version pinning AND SHA256 checksum verification (`sha256sum -c`), preventing supply chain tampering. - **Expression injection is not present**: All `${{ }}` expressions in `run:` blocks use either fixed matrix values (`matrix.task`, `matrix.vault`, etc.) or safe boolean expressions (`contains(needs.*.result, 'failure')`). No event-payload fields are interpolated in shell commands. - **Added `.forgejo/workflows/full-model-check.yml`**: Clean scheduled workflow with read-only permissions, no secrets, no sensitive operations. ### Verdict **PASS** — The changes are security-neutral to security-positive. The migration from GitHub Actions to Forgejo consolidates CI surface area, deleted workflows remove privileged automation and external-trigger attack vectors, and the Forgejo workflows follow good security practices (least-privilege permissions, prompt hardening, tight tool scoping, integrity-verified dependencies).

stack72 commented 2026-05-27 18:01:09 +00:00

Author

Owner

Copy link

Code Review

Blocking Issues

1. scripts/ changes are excluded from CI security review scope (.forgejo/workflows/ci.yml, claude-ci-security-review job, "Compute changed files" step)

   The changes job uses check_path ci ".forgejo/workflows/" "scripts/", so a scripts/-only change correctly sets ci=true and triggers the claude-ci-security-review job. However, the changed-files computation in that job filters with grep -E '^\.(forgejo|github)/', which excludes scripts/*.ts from the file list passed to Claude. If a PR only touches scripts/ (e.g., audit_actions.ts), has_files will be false and the review step is skipped silently — the security review job runs, does nothing, and reports success. Scripts running in CI with --allow-net=api.github.com --allow-write are exactly the kind of code that the CI security review is meant to cover.

   Fix: Broaden the grep to also capture scripts/ files:

   git diff --name-only origin/main..HEAD \
     | grep -E '^\.(forgejo|github)/|^scripts/' \
     > ${RUNNER_TEMP}/review/changed-files.txt || true
   

Suggestions

1. Lint and fmt checks for AWS, GCP, and Cloudflare models are now unchecked in any CI — The PR removes lint and fmt from the PR CI for all model providers, and the new full-model-check.yml only adds them back for hetzner-cloud and digitalocean. AWS, GCP, and Cloudflare generated code now has no CI lint/fmt coverage anywhere — neither on PR nor on schedule. The hetzner-digitalocean-full-check job runs deno lint --no-config and deno fmt --no-config --check, but the AWS/GCP/Cloudflare full-check jobs do not. Consider adding lint/fmt steps to the aws-full-check, gcp-full-check, and cloudflare-full-check jobs in full-model-check.yml for parity.

2. Glob tool absent from review --allowedTools — The old GitHub Actions CI provided Read,Glob,Grep plus gh commands. The new Forgejo implementation drops Glob. Without it, the reviewer cannot list directory contents and must rely entirely on the git diff output for structure discovery. Adding Glob to the --allowedTools string in all three review jobs (claude-review, claude-adversarial-review, claude-ci-security-review) would give Claude the same file-exploration capability it had before.

3. changes job failure silently bypasses all per-scope checks in the merge gate — The merge-gate job does not include changes in its needs: list. If the changes job fails (e.g., checkout fails, git diff errors), all per-scope check/lockfile jobs that depend on it are skipped rather than failed. Since contains(needs.*.result, 'failure') returns false for skipped jobs, the merge-gate passes despite no scope checks having run. Adding changes to merge-gate's needs: would surface this failure explicitly.

4. ssh-check and ssh-lockfile absent from claude-review needs: — For SSH-only PRs, the Claude review can start and complete before or without SSH tests finishing, since ssh-check/ssh-lockfile are not in claude-review's dependency list. The merge-gate backstop prevents actual merging with failing tests, but per adversarial-dimensions.md, the claude-review / test ordering gap is the documented "worst case." This is low urgency given the merge gate, but worth aligning with the stated convention.

## Code Review ### Blocking Issues 1. **`scripts/` changes are excluded from CI security review scope** (`.forgejo/workflows/ci.yml`, `claude-ci-security-review` job, "Compute changed files" step) The `changes` job uses `check_path ci ".forgejo/workflows/" "scripts/"`, so a `scripts/`-only change correctly sets `ci=true` and triggers the `claude-ci-security-review` job. However, the changed-files computation in that job filters with `grep -E '^\.(forgejo|github)/'`, which excludes `scripts/*.ts` from the file list passed to Claude. If a PR only touches `scripts/` (e.g., `audit_actions.ts`), `has_files` will be `false` and the review step is skipped silently — the security review job runs, does nothing, and reports success. Scripts running in CI with `--allow-net=api.github.com --allow-write` are exactly the kind of code that the CI security review is meant to cover. **Fix**: Broaden the grep to also capture `scripts/` files: ```bash git diff --name-only origin/main..HEAD \ | grep -E '^\.(forgejo|github)/|^scripts/' \ > ${RUNNER_TEMP}/review/changed-files.txt || true ``` ### Suggestions 1. **Lint and fmt checks for AWS, GCP, and Cloudflare models are now unchecked in any CI** — The PR removes `lint` and `fmt` from the PR CI for all model providers, and the new `full-model-check.yml` only adds them back for `hetzner-cloud` and `digitalocean`. AWS, GCP, and Cloudflare generated code now has no CI lint/fmt coverage anywhere — neither on PR nor on schedule. The `hetzner-digitalocean-full-check` job runs `deno lint --no-config` and `deno fmt --no-config --check`, but the AWS/GCP/Cloudflare full-check jobs do not. Consider adding lint/fmt steps to the `aws-full-check`, `gcp-full-check`, and `cloudflare-full-check` jobs in `full-model-check.yml` for parity. 2. **`Glob` tool absent from review `--allowedTools`** — The old GitHub Actions CI provided `Read,Glob,Grep` plus `gh` commands. The new Forgejo implementation drops `Glob`. Without it, the reviewer cannot list directory contents and must rely entirely on the `git diff` output for structure discovery. Adding `Glob` to the `--allowedTools` string in all three review jobs (`claude-review`, `claude-adversarial-review`, `claude-ci-security-review`) would give Claude the same file-exploration capability it had before. 3. **`changes` job failure silently bypasses all per-scope checks in the merge gate** — The `merge-gate` job does not include `changes` in its `needs:` list. If the `changes` job fails (e.g., checkout fails, `git diff` errors), all per-scope check/lockfile jobs that depend on it are skipped rather than failed. Since `contains(needs.*.result, 'failure')` returns false for skipped jobs, the merge-gate passes despite no scope checks having run. Adding `changes` to `merge-gate`'s `needs:` would surface this failure explicitly. 4. **`ssh-check` and `ssh-lockfile` absent from `claude-review` `needs:`** — For SSH-only PRs, the Claude review can start and complete before or without SSH tests finishing, since `ssh-check`/`ssh-lockfile` are not in `claude-review`'s dependency list. The `merge-gate` backstop prevents actual merging with failing tests, but per `adversarial-dimensions.md`, the `claude-review` / test ordering gap is the documented "worst case." This is low urgency given the merge gate, but worth aligning with the stated convention.

stack72 added 1 commit 2026-05-27 18:03:13 +00:00

fix(ci): include scripts/ in CI security review scope ... 7efaafc61f

The ci-security-review job's changed-files grep only matched
.forgejo/ and .github/ paths, silently skipping scripts/*.ts files
even though the changes job correctly triggers on scripts/ changes.
A scripts-only PR would cause the review to run but find zero files,
reporting success without actually reviewing anything.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>

stack72 commented 2026-05-27 18:09:32 +00:00

Author

Owner

Copy link

CI Security Review

Summary of Changes

This PR migrates CI from GitHub Actions (.github/workflows/) to Forgejo Actions (.forgejo/workflows/), deleting the four GitHub workflow files (auto-response.yml, ci.yml, publish.yml, regenerate-models.yml) and modifying the existing Forgejo CI (ci.yml) with:

• Model check jobs simplified from matrix (check/lint/fmt) to check-only, with sample-based checking for AWS/GCP/Cloudflare
• New full-model-check.yml for daily scheduled full checks
• New merge-gate job that aggregates all required job results
• Review output paths moved from $RUNNER_TEMP to $GITHUB_WORKSPACE/.review-output
• CI security review scope expanded to include scripts/
• ssh directory added to outdated-dependency scan
• New scripts/audit_actions.ts for auditing action pins and versions
• actions-audit job added to CI

Critical / High

No critical or high severity findings.

Medium

1. Workflow-level permissions instead of job-level — .forgejo/workflows/ci.yml:8-9: permissions: contents: read is set at the workflow level. This means all jobs inherit contents: read even if they don't need it. Jobs like claude-review, claude-adversarial-review, and claude-ci-security-review override with job-level permissions (adding pull-requests: write), which is good. However, best practice is to set permissions: {} at the workflow level and declare per-job. In practice, since the base is read-only and the jobs that need write permissions declare them explicitly, this is not exploitable — this is a hardening suggestion, not a blocking issue.

2. Tag-only pins on trusted actions — Throughout both .forgejo/workflows/ci.yml and .forgejo/workflows/full-model-check.yml, actions/checkout@v6 and denoland/setup-deno@v2 use tag-based pins rather than commit SHAs. Per the project's own policy in scripts/audit_actions.ts, these are from trusted publishers (actions, denoland) and are explicitly acceptable with tag-only pins. This is consistent and intentional, but noting it for completeness.

3. scripts/audit_actions.ts only scans .forgejo/workflows/ — scripts/audit_actions.ts:251: The workflowDir is hardcoded to .forgejo/workflows. Since the .github/workflows/ files are being deleted in this PR, this is correct going forward. However, if any .github/ workflows are ever re-added (e.g., the auto-response workflow was GitHub-specific), they would not be audited. Low risk since the GitHub workflows are being removed.

Low

1. contains(needs.*.result, 'failure') in merge-gate — .forgejo/workflows/ci.yml:958: This expression is interpolated into a run: block via ${{ }}. Since contains(needs.*.result, 'failure') evaluates to a boolean string (true/false) controlled entirely by the CI runtime (not by any user input), there is no injection risk here. This is a safe use of expression interpolation.

2. Review output path moved to workspace — The review output was moved from $RUNNER_TEMP/review/ to $GITHUB_WORKSPACE/.review-output/. Both are ephemeral per-job. The new path is within the workspace directory, but since the Claude Code agent's tools are tightly scoped (Read, Grep, Bash(git diff:*), Bash(git log:*), Bash(tee ...), Bash(touch ...)), the agent cannot write arbitrary files to the workspace. This is a neutral change.

3. Deleted workflows assessment — The four deleted .github/workflows/ files are being removed as part of the Forgejo migration:

   • auto-response.yml: Was an issues: [opened] trigger workflow that forwarded GitHub issues to swamp.club lab. It used actions/github-script@v7 (tag-only pin, but from trusted actions publisher) and properly scoped permissions (issues: write, contents: read). The issue body was passed through to a swamp.club API, but the actions/github-script context handles this safely via the GitHub REST API rather than shell interpolation. Deletion is clean.
   • ci.yml: The old GitHub CI used dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d (properly SHA-pinned third-party action). The new Forgejo CI replaces this with a custom shell-based change detection using git diff with env vars for base/head SHA — this is a safe pattern that avoids expression injection by passing the SHAs via environment variables.
   • publish.yml: Had properly scoped permissions (contents: read), triggered only on push to main. Deletion is clean.
   • regenerate-models.yml: Had contents: write and pull-requests: write permissions, used secrets.GH_TOKEN for push access. The ${{ inputs.provider }} was used in a shell context but only accepts values from a fixed choice enum, limiting injection risk. The ${{ steps.label.outputs.title }} interpolation in the git commit and gh pr create commands was safe since it was derived from the fixed enum. Deletion removes these concerns entirely.

Verdict

PASS — The changes are security-neutral to positive. The migration from GitHub Actions to Forgejo Actions is clean. LLM review jobs maintain tight tool scoping. No attacker-controlled data is interpolated into prompts or shell commands. Permissions are appropriately scoped. The new audit_actions.ts script adds proactive supply chain monitoring. The deletion of GitHub workflows removes the codebase's attack surface on that platform.

## CI Security Review ### Summary of Changes This PR migrates CI from GitHub Actions (`.github/workflows/`) to Forgejo Actions (`.forgejo/workflows/`), deleting the four GitHub workflow files (`auto-response.yml`, `ci.yml`, `publish.yml`, `regenerate-models.yml`) and modifying the existing Forgejo CI (`ci.yml`) with: - Model check jobs simplified from matrix (check/lint/fmt) to check-only, with sample-based checking for AWS/GCP/Cloudflare - New `full-model-check.yml` for daily scheduled full checks - New `merge-gate` job that aggregates all required job results - Review output paths moved from `$RUNNER_TEMP` to `$GITHUB_WORKSPACE/.review-output` - CI security review scope expanded to include `scripts/` - `ssh` directory added to outdated-dependency scan - New `scripts/audit_actions.ts` for auditing action pins and versions - `actions-audit` job added to CI ### Critical / High No critical or high severity findings. ### Medium 1. **Workflow-level permissions instead of job-level** — `.forgejo/workflows/ci.yml:8-9`: `permissions: contents: read` is set at the workflow level. This means all jobs inherit `contents: read` even if they don't need it. Jobs like `claude-review`, `claude-adversarial-review`, and `claude-ci-security-review` override with job-level permissions (adding `pull-requests: write`), which is good. However, best practice is to set `permissions: {}` at the workflow level and declare per-job. In practice, since the base is read-only and the jobs that need write permissions declare them explicitly, this is not exploitable — this is a hardening suggestion, not a blocking issue. 2. **Tag-only pins on trusted actions** — Throughout both `.forgejo/workflows/ci.yml` and `.forgejo/workflows/full-model-check.yml`, `actions/checkout@v6` and `denoland/setup-deno@v2` use tag-based pins rather than commit SHAs. Per the project's own policy in `scripts/audit_actions.ts`, these are from trusted publishers (`actions`, `denoland`) and are explicitly acceptable with tag-only pins. This is consistent and intentional, but noting it for completeness. 3. **`scripts/audit_actions.ts` only scans `.forgejo/workflows/`** — `scripts/audit_actions.ts:251`: The `workflowDir` is hardcoded to `.forgejo/workflows`. Since the `.github/workflows/` files are being deleted in this PR, this is correct going forward. However, if any `.github/` workflows are ever re-added (e.g., the auto-response workflow was GitHub-specific), they would not be audited. Low risk since the GitHub workflows are being removed. ### Low 1. **`contains(needs.*.result, 'failure')` in merge-gate** — `.forgejo/workflows/ci.yml:958`: This expression is interpolated into a `run:` block via `${{ }}`. Since `contains(needs.*.result, 'failure')` evaluates to a boolean string (`true`/`false`) controlled entirely by the CI runtime (not by any user input), there is no injection risk here. This is a safe use of expression interpolation. 2. **Review output path moved to workspace** — The review output was moved from `$RUNNER_TEMP/review/` to `$GITHUB_WORKSPACE/.review-output/`. Both are ephemeral per-job. The new path is within the workspace directory, but since the Claude Code agent's tools are tightly scoped (`Read`, `Grep`, `Bash(git diff:*)`, `Bash(git log:*)`, `Bash(tee ...)`, `Bash(touch ...)`), the agent cannot write arbitrary files to the workspace. This is a neutral change. 3. **Deleted workflows assessment** — The four deleted `.github/workflows/` files are being removed as part of the Forgejo migration: - `auto-response.yml`: Was an `issues: [opened]` trigger workflow that forwarded GitHub issues to swamp.club lab. It used `actions/github-script@v7` (tag-only pin, but from trusted `actions` publisher) and properly scoped permissions (`issues: write`, `contents: read`). The issue body was passed through to a swamp.club API, but the `actions/github-script` context handles this safely via the GitHub REST API rather than shell interpolation. Deletion is clean. - `ci.yml`: The old GitHub CI used `dorny/paths-filter@fbd0ab8f3e69293af611ebaee6363fc25e6d187d` (properly SHA-pinned third-party action). The new Forgejo CI replaces this with a custom shell-based change detection using `git diff` with env vars for base/head SHA — this is a safe pattern that avoids expression injection by passing the SHAs via environment variables. - `publish.yml`: Had properly scoped permissions (`contents: read`), triggered only on `push` to `main`. Deletion is clean. - `regenerate-models.yml`: Had `contents: write` and `pull-requests: write` permissions, used `secrets.GH_TOKEN` for push access. The `${{ inputs.provider }}` was used in a shell context but only accepts values from a fixed `choice` enum, limiting injection risk. The `${{ steps.label.outputs.title }}` interpolation in the `git commit` and `gh pr create` commands was safe since it was derived from the fixed enum. Deletion removes these concerns entirely. ### Verdict **PASS** — The changes are security-neutral to positive. The migration from GitHub Actions to Forgejo Actions is clean. LLM review jobs maintain tight tool scoping. No attacker-controlled data is interpolated into prompts or shell commands. Permissions are appropriately scoped. The new `audit_actions.ts` script adds proactive supply chain monitoring. The deletion of GitHub workflows removes the codebase's attack surface on that platform.

stack72 commented 2026-05-27 18:11:59 +00:00

Author

Owner

Copy link

Code Review

Blocking Issues

None.

Suggestions

1. AWS/GCP/Cloudflare model lint and fmt checks are now absent everywhere. The PR moves lint/fmt for Hetzner and DigitalOcean models to the nightly full-model-check.yml — that workflow correctly adds deno lint --no-config and deno fmt --no-config --check steps for those two providers. But the nightly aws-full-check, gcp-full-check, and cloudflare-full-check jobs only run deno check, with no lint or fmt step. PR CI no longer runs lint/fmt for these providers (by design, for speed), and neither does the nightly. So a codegen change that emits non-conformant code for AWS/GCP/Cloudflare would go undetected indefinitely. Adding lint/fmt loops to those three nightly jobs (mirroring the hetzner-digitalocean-full-check pattern) would close the gap without impacting PR CI speed.

2. merge-gate only catches failure, not cancelled. The expression contains(needs.*.result, 'failure') passes silently when a dependency is cancelled (distinct from skipped). In the normal push-to-PR flow the entire run is cancelled and restarted, so this is low-risk. But if a runner goes down mid-run or a user manually cancels the workflow, claude-review (or another gated job) could be cancelled within the run while the gate still reports success. Consider contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled') for a stricter posture.

3. No .gitignore entry for .review-output/. The Claude review jobs now write review-body.md and review-failed into ${GITHUB_WORKSPACE}/.review-output (inside the repo checkout). No workflow step commits these files, so the practical risk is low. Adding .review-output/ to .gitignore would prevent them from being accidentally staged by a developer running the review prompt locally or in a modified workflow.

4. audit_actions.ts scans only .forgejo/workflows. This is correct given the deletion of .github/workflows/ files in this PR. Worth noting: if .github/workflows/ files are ever re-added (e.g., for GitHub mirroring), they would be silently unaudited by the actions-audit CI job. A comment in main() documenting the single-directory scope would help future readers understand this is intentional, not an oversight.

## Code Review ### Blocking Issues None. ### Suggestions 1. **AWS/GCP/Cloudflare model lint and fmt checks are now absent everywhere.** The PR moves lint/fmt for Hetzner and DigitalOcean models to the nightly `full-model-check.yml` — that workflow correctly adds `deno lint --no-config` and `deno fmt --no-config --check` steps for those two providers. But the nightly `aws-full-check`, `gcp-full-check`, and `cloudflare-full-check` jobs only run `deno check`, with no lint or fmt step. PR CI no longer runs lint/fmt for these providers (by design, for speed), and neither does the nightly. So a codegen change that emits non-conformant code for AWS/GCP/Cloudflare would go undetected indefinitely. Adding lint/fmt loops to those three nightly jobs (mirroring the `hetzner-digitalocean-full-check` pattern) would close the gap without impacting PR CI speed. 2. **`merge-gate` only catches `failure`, not `cancelled`.** The expression `contains(needs.*.result, 'failure')` passes silently when a dependency is `cancelled` (distinct from `skipped`). In the normal push-to-PR flow the entire run is cancelled and restarted, so this is low-risk. But if a runner goes down mid-run or a user manually cancels the workflow, `claude-review` (or another gated job) could be cancelled within the run while the gate still reports success. Consider `contains(needs.*.result, 'failure') || contains(needs.*.result, 'cancelled')` for a stricter posture. 3. **No `.gitignore` entry for `.review-output/`.** The Claude review jobs now write `review-body.md` and `review-failed` into `${GITHUB_WORKSPACE}/.review-output` (inside the repo checkout). No workflow step commits these files, so the practical risk is low. Adding `.review-output/` to `.gitignore` would prevent them from being accidentally staged by a developer running the review prompt locally or in a modified workflow. 4. **`audit_actions.ts` scans only `.forgejo/workflows`.** This is correct given the deletion of `.github/workflows/` files in this PR. Worth noting: if `.github/workflows/` files are ever re-added (e.g., for GitHub mirroring), they would be silently unaudited by the `actions-audit` CI job. A comment in `main()` documenting the single-directory scope would help future readers understand this is intentional, not an oversight.

stack72 merged commit f0410c4afb into main 2026-05-27 18:18:49 +00:00

stack72 deleted branch worktree-462 2026-05-27 18:18:49 +00:00

stack72 referenced this pull request from a commit 2026-05-27 18:18:51 +00:00

ci: optimize model checks, fix review sandbox, remove GitHub workflows (#5)

stack72 referenced this pull request 2026-06-25 19:01:09 +00:00

feat(cve/scanner): add CVE research pipeline extension #75

Sign in to join this conversation.

Reviewers

No reviewers

Labels

Clear labels

No items

No labels

Milestone

Clear milestone

No items

No milestone

Projects

Clear projects

No items

No project

Assignees

Clear assignees

No assignees

1 participant

Notifications

Subscribe

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference

swamp-club/swamp-extensions!5

No description provided.