6
1
Fork 0
mirror of https://github.com/systeminit/swamp.git synced 2026-07-14 21:09:25 +00:00
Stinemates, Watson, Adam, Mahir, Paul
  • TypeScript 99.7%
  • Python 0.1%
  • Shell 0.1%
Find a file
Paul Stack 6fa872c583
fix(serve): bust ES module cache on hot-reload so updated extension code executes (swamp-club#1140) (#1849)
## Summary

- `swamp serve` hot-reload (SIGHUP) reported success but kept executing
stale model code — Deno's `import()` caches modules by URL within the
process, so re-importing the same file URL returned the old module
- Added a reload generation counter that appends `&gen=N` to import
URLs, forcing `import()` to treat each reload as a new module
- `reloadPulledExtensions` now re-bundles only changed source files
(fingerprint comparison) before re-importing — unchanged files skip the
expensive `deno bundle` subprocess entirely
- Failed re-bundles log a warning and keep the old bundle (graceful
degradation)

### What this fixes

The hot-reload path: source updated → SIGHUP → serve now executes the
new code. Verified with model, vault, datastore, and report extension
kinds.

### What this does NOT fix (related issues)

- Stale code surviving full process restart — could not reproduce; fresh
processes pick up new code correctly
- Lockfile path mismatch with `extensionsDir` — tracked in
swamp-club#1133
- `findByExtension` returning 0 rows for large pulled extensions (e.g.
`@swamp/aws/ec2`) — filed as swamp-club#1149

### Performance

Tested with `@swamp/aws/ec2` (108 model types, 107 bundles):
- Serve startup: no regression
- Hot-reload with 0 changed files: instant (fingerprint match → skip)
- Hot-reload with changed files: only changed files re-bundle (~200ms
each)

## Test plan

- [x] Reproduced bug: serve V1 → update source → SIGHUP → still V1
(before fix)
- [x] Verified fix: serve V1 → update source → SIGHUP → V2 executes
(after fix)
- [x] Tested all 4 extension kinds (model, vault, datastore, report) —
all reload V1→V2
- [x] `@swamp/aws/ec2` (108 types): no startup regression, no
unnecessary re-bundling on reload
- [x] Confirmed Deno `import()` busts cache with different query params
- [x] Failed re-bundle keeps old bundle (graceful degradation)
- [x] Full process restart works correctly (was not broken)
- [x] 8907 unit/integration tests pass
- [x] `integration/bundle_cache_freshness_test.ts` (8 tests) pass — CLI
path unaffected

Closes swamp-club#1140

🤖 Generated with [Claude Code](https://claude.com/claude-code)

---------

Co-authored-by: Blake Irvin <bixu@users.noreply.github.com>
Co-authored-by: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
2026-07-14 21:51:06 +01:00
.agents/skills feat: warn when model definitions use env var expressions (#784) 2026-03-19 23:05:48 +00:00
.claude docs(vault): document auto-refresh in skill docs (#1847) 2026-07-14 19:44:59 +01:00
.github fix(ci): skip re-running review jobs that already approved the current diff (#1804) 2026-07-09 02:00:12 +02:00
agent-constraints feat(model): add code conformance review to issue lifecycle (#1705) 2026-06-29 12:09:26 +02:00
design feat(vaults): add native secret-read access audit trail (swamp-club#1109) (#1830) 2026-07-13 18:52:43 +01:00
evals/promptfoo fix(skill): split guide files for progressive reveal (swamp-club#824) (#1686) 2026-06-26 01:53:25 +01:00
extensions/models fix(issue-lifecycle): truncate lifecycle entry summaries exceeding API limit (#1761) 2026-07-05 18:23:46 +02:00
integration feat(telemetry): record telemetry globally (user-level), not gated on a repo (#1842) 2026-07-14 02:08:58 +01:00
packages fix(data): resolve model names in dataRepository methods (#1770) 2026-07-06 21:06:42 +02:00
scripts feat(auth): add first-run nudge and installer account prompt (#1833) 2026-07-13 22:07:01 +01:00
src fix(serve): bust ES module cache on hot-reload so updated extension code executes (swamp-club#1140) (#1849) 2026-07-14 21:51:06 +01:00
.gitattributes ci: add .gitattributes to pin line endings to LF (#1234) 2026-04-28 15:14:54 +01:00
.gitignore feat(sbom): add CycloneDX SBOM generator for license compliance (#1665) 2026-06-24 01:53:45 +01:00
banner.png docs: Refresh README for open alpha (#307) 2026-02-12 23:51:15 +00:00
CLAUDE.md chore: streamline CLAUDE.md (#1569) 2026-06-12 11:09:22 +01:00
CODE_OF_CONDUCT.md chore: update copyright and branding to Elder Swamp Club (#1504) 2026-06-03 23:23:43 +01:00
CONTRIBUTING.md chore: update copyright and branding to Elder Swamp Club (#1504) 2026-06-03 23:23:43 +01:00
COPYING license: Add AGPLv3 copyright headers to all TypeScript files (#287) 2026-02-11 23:55:22 +00:00
COPYING-EXCEPTION Update COPYING-EXCEPTION (#1703) 2026-06-29 12:28:35 +02:00
COPYRIGHT docs: add OSS FAQ and update licensing/trademark docs (#1595) 2026-06-19 18:03:09 +01:00
deno.json feat(telemetry): record telemetry globally (user-level), not gated on a repo (#1842) 2026-07-14 02:08:58 +01:00
deno.lock fix(expressions): resolve vault references before CEL validation (swamp-club#1141) (#1851) 2026-07-14 20:29:53 +01:00
Dockerfile fix: pin Deno Docker image and pass release version via job outputs (#883) 2026-03-26 18:32:28 +00:00
FILE-LICENSE-TEMPLATE.md chore: update copyright and branding to Elder Swamp Club (#1504) 2026-06-03 23:23:43 +01:00
LICENSE chore: update copyright and branding to Elder Swamp Club (#1504) 2026-06-03 23:23:43 +01:00
logo.png docs: add README with project logo (#1) 2026-01-28 10:40:57 -07:00
main.ts feat(error): surface lock_timeout as exit code 75 and document error codes (swamp-club#874) (#1718) 2026-06-30 10:06:53 +02:00
main_test.ts chore: update copyright and branding to Elder Swamp Club (#1504) 2026-06-03 23:23:43 +01:00
OSS-FAQ.md docs: add OSS FAQ and update licensing/trademark docs (#1595) 2026-06-19 18:03:09 +01:00
README.md feat(sbom): add CycloneDX SBOM generator for license compliance (#1665) 2026-06-24 01:53:45 +01:00
sc-mark.png docs(trademarks): use Swamp Club (SC) mark for reserved trademark (#1624) 2026-06-20 00:03:39 +01:00
SECURITY.md chore: update copyright and branding to Elder Swamp Club (#1504) 2026-06-03 23:23:43 +01:00
swamp-club-wordmark.png docs(trademarks): add SWAMP CLUB wordmark to reserved trademarks (swamp-club#802) (#1671) 2026-06-24 17:24:21 +01:00
TRADEMARKS.md docs(trademarks): add SWAMP CLUB wordmark to reserved trademarks (swamp-club#802) (#1671) 2026-06-24 17:24:21 +01:00

Swamp — AI Automation for Hackers

Swamp

Deterministic Automation for AI Agents.

Swamp is a CLI that supercharges AI agents to create operational workflows that are reviewable, shareable, and accurate. Built for agents, there to empower humans. All the data lives in the .swamp/ (the swamp).

Come join the swamp party on discord.

Getting Started

curl -fsSL https://swamp-club.com/install.sh | sh

Quick Start

swamp repo init                    # Claude Code (default)
swamp repo init --tool cursor      # Cursor
swamp repo init --tool opencode    # OpenCode
swamp repo init --tool codex       # Codex

Start your AI agent in the repo and tell it what you want to do. Just ask:

  • "Manage my EC2 fleet — inventory every instance across all regions and flag anything without a cost-center tag"
  • "Set up a workflow to check my bare metal Minecraft servers are online and under 80% memory"
  • "Audit our DNS records and compare them against what's actually running"
  • "Build a workflow that rotates database credentials and stores them in the vault"

The agent will create models, wire up workflows, and run them — all reviewable in .swamp/ before anything touches production.

Learn More

You can learn more about swamp by reading the manual. How Swamp Works is the best place to start - it provides an overview of the entire system.

Core Concepts

  • Models — Typed representations of external systems (cloud resources, CLI tools, APIs). Each model type defines metadata, arguments, methods, and inputs.
  • Definitions — YAML files that instantiate a model type with specific configuration. Support CEL expressions for dynamic values and cross-model references.
  • Workflows — Orchestrate model method executions across parallel jobs and steps, with dependency ordering and trigger conditions.
  • Data — Versioned, immutable artifacts (resources, logs, files) produced by method runs. Searchable by tags.
  • Vaults — Secure storage for secrets and credentials, referenced in definitions via CEL expressions.
  • Tags — Key-value labels on definitions, workflows, and data. Flow from definitions to produced data, overridable at runtime with --tag.

Everything lives in a .swamp/ directory inside a Git repository, with human-friendly symlink views under /models/ and /workflows/.

Local Execution

Swamp runs entirely on your machine. It picks up the environment variables from the shell you run it in — your AWS credentials, SSH keys, kubeconfig, whatever the task needs. No credentials leave your laptop unless a model explicitly calls an external API.


Update

Swamp can update itself in place:

swamp update

Shell Completions

Tab-completion for commands, model names, and workflow names:

# Bash — add to ~/.bashrc
eval "$(swamp completions bash)"

# Zsh with oh-my-zsh
mkdir -p ~/.oh-my-zsh/completions
swamp completions zsh > ~/.oh-my-zsh/completions/_swamp
rm -f ~/.zcompdump* && exec zsh

# Zsh without oh-my-zsh — add to ~/.zshrc
eval "$(swamp completions zsh)"

# Fish
swamp completions fish > ~/.config/fish/completions/swamp.fish

Completions are directory-dependent — they return names from the current directory's swamp repository.

Using Swamp with AI Agents

Swamp ships first-class skills for four AI coding tools:

Tool Init flag Skills dir Instructions file
Claude Code (default) .claude/skills/ CLAUDE.md
Cursor --tool cursor .cursor/skills/ .cursor/rules/swamp.mdc
OpenCode --tool opencode .agents/skills/ AGENTS.md
Codex --tool codex .agents/skills/ AGENTS.md

The skills are bundled into the swamp binary and written into the appropriate directory so your agent discovers them automatically. Each skill teaches the agent how to work with swamp — search for models, create definitions, run workflows, manage vaults, and more.

You can switch tools later or run multiple tools side-by-side — each tool's skills directory is independent and gitignored:

swamp repo upgrade --tool cursor

User-Defined Models

Extend Swamp with custom TypeScript models. Place them in extensions/models/ (or configure via SWAMP_MODELS_DIR or .swamp.yaml).

See the swamp skill in your skills directory (e.g. .claude/skills/swamp/SKILL.md for Claude Code) for details.

Developer Guide

Prerequisites

Commands

deno run dev          # Run the CLI from source
deno run test         # Run the test suite
deno check            # Type-check
deno lint             # Lint
deno fmt              # Format
deno run compile      # Compile the binary

License Compliance

deno run sbom generates a CycloneDX SBOM of every npm and JSR dependency for license-compliance scanning. See design/license-compliance.md for details on the generator, license resolution, and scanning with FOSSA.

Contributing

Swamp uses an issue-driven contribution model. We don't accept pull requests from external contributors — fork PRs are automatically closed. This isn't about gatekeeping; it's about supply chain security in the age of AI-generated code. When AI agents can produce large, plausible-looking changes, the only way to maintain quality and security is to tightly control the inputs to the development process.

Here's how it works:

  1. You file an issuebug reports and feature requests are very welcome. Be as detailed as you like.
  2. We triage it — A maintainer triages the issue locally using Claude, confirms bugs by tracing through the codebase, and generates a detailed implementation plan. Plans are revised interactively until the approach is solid.
  3. We build it — Elder Swamp Club engineers (with AI agents under our direct control) implement the plan, with full test coverage and code review.
  4. You get credit — We're happy to include you as a co-author on any PR generated from your request.

This means you get the feature you asked for, maintained over time, without having to worry about keeping a fork in sync. See CONTRIBUTING.md for the full details.

Datastores

By default, swamp stores all runtime data (model data, workflow runs, outputs, audit logs, etc.) in the local .swamp/ directory. You can configure a different datastore backend to share state across machines or centralise data.

Default (local filesystem)

When you run swamp repo init, the datastore is .swamp/ inside the repo. No extra configuration needed.

Setting up an external filesystem datastore

Move runtime data to a directory outside the repo (e.g. a shared NFS mount):

swamp datastore setup filesystem --path /mnt/shared/swamp-data

This migrates existing .swamp/ runtime data to the new path and updates .swamp.yaml. A file-based lock prevents concurrent access from multiple processes.

Setting up an S3 datastore

Store runtime data in S3 for team collaboration using the @swamp/s3-datastore extension:

swamp datastore setup extension @swamp/s3-datastore \
  --config '{"bucket":"my-swamp-bucket","prefix":"my-project","region":"us-east-1"}'

This pushes existing local data to S3 and updates .swamp.yaml. Subsequent commands automatically pull changes before execution and push changes after. A distributed lock (S3 conditional writes) prevents concurrent access.

Use --skip-migration on either setup command to skip the initial data migration.

Migrating between datastores

Run swamp datastore setup again with the new backend. For example, to move from a filesystem datastore to S3:

swamp datastore setup extension @swamp/s3-datastore \
  --config '{"bucket":"my-bucket","region":"us-east-1"}'

Or from S3 back to local filesystem:

swamp datastore setup filesystem --path /path/to/data

Each setup command migrates existing data to the new backend.

Checking datastore status

swamp datastore status        # Shows type, health, and config
swamp datastore sync          # Manual bidirectional sync (S3 only)
swamp datastore sync --pull   # Pull-only from S3
swamp datastore sync --push   # Push-only to S3

Stuck locks

If a process crashes without releasing the datastore lock, subsequent commands will wait up to 60 seconds before timing out (locks auto-expire after 30 seconds). To inspect or force-release a stuck lock:

swamp datastore lock status           # Show who holds the lock
swamp datastore lock release --force  # Force-release the lock

Environment variable override

For CI/CD, override the datastore without modifying .swamp.yaml:

export SWAMP_DATASTORE=s3:my-bucket/my-prefix
export SWAMP_DATASTORE=filesystem:/tmp/swamp-data

Repository Directory

Every command runs against a swamp repository. By default this is the current working directory. Pass --repo-dir to point at a different repo per invocation, or set SWAMP_REPO_DIR to persist the override across a shell session or CI job:

swamp model search --repo-dir /path/to/repo
export SWAMP_REPO_DIR=/path/to/repo
swamp model search

Priority order (highest to lowest): --repo-dir flag → SWAMP_REPO_DIR env var → current working directory.

Log Level

By default, swamp outputs at the info level. You can change this once rather than repeating a flag on every command.

Per-invocation flags (highest priority):

swamp -q workflow run my-workflow               # error level only
swamp --log-level debug workflow run my-workflow

Via environment variable (useful for CI/CD):

export SWAMP_LOG_LEVEL=warning
swamp workflow run my-workflow

Permanently for a repository — add to .swamp.yaml:

logLevel: error

Valid levels: trace, debug, info, warning, error, fatal.

Priority order (highest to lowest): -q / --log-level flag → SWAMP_LOG_LEVEL env var → .swamp.yaml logLevel → default (info).

Tracing

Swamp has native OpenTelemetry tracing for diagnosing slow or failing operations. Tracing is opt-in and has zero overhead when disabled.

Set OTEL_EXPORTER_OTLP_ENDPOINT to enable:

# Send traces to a local Jaeger instance
OTEL_EXPORTER_OTLP_ENDPOINT=http://localhost:4318 swamp workflow run my-workflow

# Quick debug: print spans to stderr (no collector needed)
OTEL_TRACES_EXPORTER=console swamp workflow run my-workflow

Traces capture the full execution hierarchy — CLI command, workflow, job, step, model method, and driver execution — with automatic context propagation to in-process extensions and Docker containers via TRACEPARENT.

Local development

Run Jaeger for a local trace UI:

docker run -d --name jaeger -p 16686:16686 -p 4318:4318 jaegertracing/all-in-one:latest

Then open http://localhost:16686 and search for the swamp service.

Configuration

Variable Purpose Default
OTEL_EXPORTER_OTLP_ENDPOINT Collector URL (tracing off when unset) (unset = off)
OTEL_TRACES_EXPORTER otlp, console, or none otlp
OTEL_SERVICE_NAME Service name in traces swamp
OTEL_EXPORTER_OTLP_HEADERS Auth headers (key=val,key=val) (none)

Telemetry

Swamp collects anonymous usage telemetry to help us understand which commands are used, how long they take, and what errors occur. All user-identifiable values are redacted before transmission — nothing sensitive is ever sent.

Here is a complete example of a telemetry event:

{
  "event": "cli_invocation",
  "distinct_id": "a3f1b2c4-5678-9abc-def0-1234567890ab",
  "properties": {
    "id": "550e8400-e29b-41d4-a716-446655440000",
    "invocation": {
      "command": "model",
      "subcommand": "create",
      "args": ["prompt", "<REDACTED>"],
      "optionKeys": ["--force"],
      "globalOptions": ["--json"]
    },
    "result": {
      "status": "success",
      "exitCode": 0
    },
    "startedAt": "2026-02-16T10:00:00.000Z",
    "completedAt": "2026-02-16T10:00:01.234Z",
    "durationMs": 1234,
    "swampVersion": "0.13.0",
    "denoVersion": "2.1.0",
    "platform": "linux"
  }
}

Note that positional arguments containing user data (model names, file paths, queries) are replaced with <REDACTED>. Only categorical values defined by swamp itself (like model types) are recorded. Option values are never recorded — only the option keys.

Disabling Telemetry

Per-invocation:

swamp --no-telemetry workflow run my-workflow

Via environment variable (useful for CI/UAT environments):

export SWAMP_NO_TELEMETRY=1
swamp workflow run my-workflow

Permanently for a repository — add to .swamp.yaml:

telemetryDisabled: true

Priority order (highest to lowest): --no-telemetry flag → SWAMP_NO_TELEMETRY env var → .swamp.yaml telemetryDisabled: true.

License

Swamp is licensed under the GNU Affero General Public License v3.0 with the Swamp Extension and Definition Exception. No rights are granted for use of our trademarks outside our explicit policy See COPYRIGHT, CONTRIBUTING.md and Open Source FAQ for details.