heavy-duty/cast
7
0
Fork 0
Code Issues 4 Pull requests Projects Releases 3 Packages Wiki Activity Actions

feat: cast capture — adopt a hand-built Coolify into the age secret store (#15)

Browse source 
cast was scoped to the steady state: manifest → Coolify, forever. It had no
adoption path — no way to bootstrap the age store from an instance built by
hand, before any manifest existed. The operator did it by hand: curl the envs,
assemble 17 name=value pairs into /dev/shm/prod.env, age -r, shred. Every input
to that pipeline is something cast already has, so a human was shuffling cast's
own inputs through a terminal, with the leak (scrollback, history, a tmp file
that never got shredded) and the silent miss both live.

  cast capture <org>/<repo> --env <env> [--generated N] [--override N] [--force]

The required set comes from the MANIFEST, not the box: the ${...} refs in that
environment's env templates, read by the same parser apply uses to demand them.
resolveTemplate and templateRefs now share one grammar — a drift between them
would mean capture collects a different set than apply later requires, which is
exactly the "a name silently missed" failure this verb exists to remove.

The mapping is deliberately NOT mechanical. A DATABASE_URL read off the source
points at the SOURCE box's Postgres: confidently wrong, entirely plausible, and
the target's real URL does not exist until Coolify creates the resource. So the
manifest declares `generated_secrets:` and those names are written as the
literal `pending-coolify-generated`. staging's ADMIN_EMAIL must be the operator,
not the source's — staging and prod share a Mailgun domain, so a staging box
carrying the real address can mail real users; that is --override.

A "capture everything" verb would be wrong in ~4 of 17 entries, silently —
worse than being wrong in all of them. So every name is forced into a
disposition, and two of the four stop the run: a name required by a template but
absent from the source REFUSES (an empty substitutes to nothing and the app
boots misconfigured), as does one name carrying different values on two
resources.

generated_secrets is a manifest property rather than a flag the operator must
remember, because the manifest is what knows DATABASE_URL comes from a database
it declares. An entry no template refers to is a hard error: a guard standing
over nothing reads like a guard, and the likeliest cause is a typo whose real
name is then captured from the source instead of placeheld.

Secret hygiene, all covered by tests asserting on real values:
- the plan prints names and provenance, NEVER values
- an --override's value comes from $CAST_CAPTURE_<NAME>, never argv (`ps`)
- plaintext is piped to age on stdin — never a temp file, stdout, or history
- an existing store is not overwritten without --force: it may hold the only
  copy of values the source no longer has (apply's never-delete, applied here)

capture inherits diff's absent-target refusal (D-237) — against a project that
isn't there it would report every secret as missing, an alarming report about
the wrong box — plus the team assert and the --path/--env prod ban. The last
gate is a typed confirmation of the environment's name; there is no --yes.

The end-to-end test decrypts the store cast wrote and asserts on its contents,
so "exactly the names the manifest requires, no more and no fewer" is checked
against real ciphertext rather than against cast's own console output.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
claude-hdb 2026-07-13 16:51:43 +00:00
parent e457261436
commit 3eee70fa77
11 changed files with 1478 additions and 50 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

121
README.md
View file

@ -52,6 +52,7 @@ environments.yaml # bindings: the team each env's token must belon
# GitHub App name, smoke target, guards
secrets/<repo>.<env>.env.age # age-encrypted values for the ${…} placeholders
.coolify.env # COOLIFY_BASE_URL + COOLIFY_ACCESS_TOKEN (never commit)
.coolify/<name>.env # …the same, for a NAMED instance (see below)
```
Pass it with `--state <dir>`, or set `CAST_STATE`. Defaults to the cwd.
@ -59,8 +60,9 @@ Pass it with `--state <dir>`, or set `CAST_STATE`. Defaults to the cwd.
## Commands
```sh
cast apply <org>/<repo> --env <env> [--path <dir>] [--hostname-overlay <file>]
cast diff <org>/<repo> --env <env> [--full]
cast apply <org>/<repo> --env <env> [--path <dir>] [--hostname-overlay <file>]
cast diff <org>/<repo> --env <env> [--full]
cast capture <org>/<repo> --env <env> [--generated <NAME>] [--override <NAME>]
cast server add <name> --ip <ip> --key <file> --env <env> [--user root] [--port 22]
cast smoke --env <env>
cast team [--env <env>]
@ -73,6 +75,9 @@ cast team [--env <env>]
default branch).
- **`diff`** — reports drift, manifest → Coolify. Structural by default; `--full`
also compares env vars. Exits non-zero when dirty, so CI can gate on it.
- **`capture`** — the adoption path: reads a hand-built instance's live env and
writes the environment's age store from it. See *Adopting a hand-built
instance* below.
- **`server add`** — uploads a server's private key and registers it with Coolify.
- **`smoke`** — contract test against `smoke_target`: proves Coolify's bulk env
endpoint still *upserts* rather than replacing. Run it after every Coolify
@ -89,6 +94,118 @@ them first asserts the token's team (below).
`--hostname-overlay` swaps domains for a pre-flight run against temporary
hostnames; re-applying **without** it is the cutover.
## Cloning: cast authenticates, and never prompts
`apply`, `diff` and `capture` clone the product repo (unless `--path` points at a
local checkout — refused for prod, which always reads the default branch). For a
private repo that needs credentials, and cast resolves them itself:
1. **`gh`**, borrowed as a credential helper for that one invocation — it does
not touch your global git config.
2. **`GITHUB_TOKEN` / `GH_TOKEN`** from the environment (the CI path).
3. Whatever git's own credential helper does, if you have one.
Being logged into `gh` is enough. You do **not** need `gh auth setup-git` —
that separate act is what wires git's helper, and not running it is exactly how
you end up at git's interactive username/password prompt, which GitHub no longer
accepts. cast sets `GIT_TERMINAL_PROMPT=0` on every path, so it can never hang
there or hide a credentials failure behind an error about *the repository*. With
no credentials at all it says so, and names the fix.
The token is never put in the clone URL or in `http.extraheader` — both leak it
into `ps`, and the latter persists it into the clone's git config.
## Many Coolifys
`--instance <name>` reads `<state>/.coolify/<name>.env` instead of
`<state>/.coolify.env`. Every verb that reaches Coolify takes it.
```sh
cast diff heavy-duty/incubator --env prod --full --instance legacy
```
An environment can bind one, so `--env` selects the right control plane with no
flag at all:
```yaml
environments:
prod:
server: prod-box
team: { id: 1, name: heavy-duty }
instance: prod-cp # → <state>/.coolify/prod-cp.env
```
An explicit `--instance` still wins, so a one-off read against a legacy box needs
no edit to that file either. **With no flag and no binding, nothing changes** —
`.coolify.env` is read exactly as before.
Two properties, both deliberate:
- **An unknown `--instance` refuses**, and names the instances that do exist.
Falling back to the default is how a diff meant for a legacy box gets run
against production.
- **An instance may declare `COOLIFY_READ_ONLY=true`**, and then `apply`,
`smoke` and `server add` refuse it — *before their first call*, and even
though the token itself would permit the writes. That turns "I pointed the
wrong token at the wrong box" from a live incident into an exit code.
Every command that reaches a Coolify now says which one, next to the team
assert. It is the most consequential input to any run, and the least visible.
## Adopting a hand-built instance
cast is otherwise scoped to the steady state: manifest → Coolify, forever.
`capture` is the one-way-in — it bootstraps an environment's age store from an
instance that was built by hand, before any manifest existed.
```sh
CAST_CAPTURE_ADMIN_EMAIL=me@example.com \
cast capture heavy-duty/incubator --env prod --instance legacy \
--override ADMIN_EMAIL
```
It reads the required secret **names** from the manifest's own env templates (the
`${…}` refs — the manifest already declares exactly this set), reads the live
values off the instance, and classifies every name:
| | |
| --- | --- |
| **captured** | found live, value taken |
| **generated** | the manifest's `generated_secrets` declares it provider-made → written as the literal `pending-coolify-generated`, never the live value |
| **overridden** | supplied by you, for a value that must *not* be carried over |
| **missing** | required by a template, absent live → **refuses** |
Then it prints a plan of **names and provenance — never values** — and waits for
you to type the environment's name.
The mapping is not mechanical, and that is the whole design. A `DATABASE_URL`
copied off the source box points at the *source box's* Postgres: confidently
wrong, entirely plausible, and the target's real URL does not exist until Coolify
creates the resource. So the manifest declares those names, and cast placeholds
them:
```yaml
environments:
prod:
generated_secrets: [DATABASE_URL_PROD, REDIS_URL_PROD, UMAMI_DATABASE_URL]
```
It is a manifest property rather than a flag you have to remember, because the
manifest is what knows `DATABASE_URL` comes from a database it declares. (A
`generated_secrets` entry no template refers to is a schema error — a guard
standing over nothing is worse than no guard, because it reads like one.
`--generated <NAME>` covers a manifest that hasn't declared them yet.)
An **`--override`**'s value is read from `$CAST_CAPTURE_<NAME>`, never from the
command line: argv is visible in `ps` to every process on the box. It exists for
values that must not survive the copy — staging and prod sharing a Mailgun
domain means a staging box carrying the real `ADMIN_EMAIL` can mail real users.
The store is encrypted to the environment's `age_recipient` (add it to
`environments.yaml` — it's the public half, safe to commit). Plaintext goes to
`age` on stdin: it is never a temp file, never on stdout, never in your shell
history. An existing store is not overwritten without `--force`.
**[docs/semantics.md](docs/semantics.md)** is the contract behind those
commands: what `apply` guarantees (never deletes, never recreates a database,
fails loudly rather than recreating on un-updatable drift), the `dockercompose`

126
docs/semantics.md
View file

@ -158,6 +158,132 @@ softened by an implementation detail):
refuses `--path` combined with `--env prod`: prod always reads the default
branch, so a feature-branch checkout can never reach it.
## Instance selection
**The Coolify a command talks to is an explicit, named value** — not a property
of whatever `<state>/.coolify.env` happens to contain at the moment. Resolution
order, highest first:
1. `--instance <name>` → `<state>/.coolify/<name>.env`
2. the environment's `instance:` binding in `environments.yaml`
3. `<state>/.coolify.env` (the default; unchanged when neither of the above is
used)
Two refusals, both fail-closed:
- **An unknown `--instance` aborts**, naming the instances that do exist. It
does *not* fall back to the default — that fallback is how a `--full` diff
meant for a legacy box gets run against production.
- **`COOLIFY_READ_ONLY=true` in an instance file makes it read-only**, and
`apply` / `smoke` / `server add` refuse it *before their first call*. The
guard is the **declaration**, not the token's scope: an instance configured
for inspection must not be writable even when the token it holds would permit
the writes. `diff`, `team` and `capture` still work against it — they read.
Every command that reaches a live Coolify prints which one, next to the team
assert.
## Adoption (`capture`)
`capture` is the only verb that writes *into* the state directory rather than
into Coolify, and the only one that reads a hand-built instance as a **source**
rather than as a target. It exists because cast is otherwise scoped to the
steady state and has no bootstrap path for a box that predates its manifest.
**The required set comes from the manifest, not from the box.** The names are
the `${…}` refs in that environment's env templates, read by the same parser
`apply` uses to demand them (`parseTemplate`, shared by `resolveTemplate` and
`templateRefs` — deliberately one grammar, because a drift between the two
would mean `capture` collects a different set than `apply` will later require,
which is the "a name silently missed" failure it exists to remove). So the store
it writes contains **exactly** the names the manifest requires: a live var
nobody asked for is not the store's business, and a template literal
(`NODE_ENV=production`) is not a secret.
**The mapping is not mechanical, and must not be.** Some entries encode
migration decisions rather than facts about the source box:
- A `DATABASE_URL` / `REDIS_URL` read off the source points at the **source
box's** Postgres/Redis. Copying it is confidently wrong in a way that looks
entirely plausible, and the target's real URL does not exist until Coolify
creates the resource. These are declared `generated_secrets:` in the manifest
environment and written as the literal `pending-coolify-generated`.
- staging's `ADMIN_EMAIL` must be the operator, not the source's value: staging
and prod share a Mailgun domain, so a staging box carrying the real address
can mail real users. That is `--override`.
A "capture everything" verb would therefore be silently wrong in a handful of
entries out of seventeen — worse than being wrong in all of them. So every
required name is **forced into a disposition**, and two of the four stop the
run:
| disposition | source | outcome |
| --- | --- | --- |
| captured | found live | value taken |
| generated | manifest `generated_secrets` (or `--generated`) | `pending-coolify-generated` |
| overridden | `$CAST_CAPTURE_<NAME>` | operator's value |
| **missing** | required by a template, absent live | **refuses** |
| **conflict** | one name, different live values on two resources | **refuses** |
`generated_secrets` is a **manifest** property, not a flag: the manifest is what
knows `DATABASE_URL` comes from a database it declares. An entry naming
something no template refs is a hard error — dead config here is not untidy but
dangerous, because it reads like a guard standing over a name while standing
over nothing, and the likeliest cause is a typo whose real name is then
*captured* from the source box instead of placeheld.
**Secret hygiene**, all enforced by tests against real values:
- The plan prints **names and provenance, never values**. (The one value-shaped
thing it prints is the `pending-coolify-generated` literal, which carries no
information about the source.)
- An `--override`'s value is read from `$CAST_CAPTURE_<NAME>`, **never from
argv** — a command-line value is visible in `ps` to every process on the box.
- Plaintext is piped to `age` on **stdin**: never a temp file, never stdout,
never shell history. The hand-run recipe this replaces wrote
`/dev/shm/prod.env` and relied on remembering to `shred -u` it.
- An existing store is **not overwritten** without `--force`: it may hold the
only copy of values the source box no longer has. Same disposition as apply's
never-delete.
**`capture` takes `diff`'s position on an absent target** (see `LiveLookup`),
and refuses one: against a project or environment that isn't there it would read
back zero live values and report every required secret as *missing* — an
alarming, meaningless report about the wrong box. It also inherits the team
assert (a wrong-team token reads back `null` for everything, producing the same
lie) and the `--path`-with-`--env prod` refusal (a feature-branch manifest must
not decide which names land in the prod store).
The final gate is a **typed confirmation** — the environment's own name, after
the plan. There is no `--yes`: a store written without someone reading the
provenance column is the outcome the verb exists to prevent. A closed stdin
aborts rather than hanging.
## Cloning a private manifest
`resolveCheckout` resolves git credentials **inside cast**, in a fixed order —
`gh` borrowed as a per-invocation credential helper, then
`GITHUB_TOKEN`/`GH_TOKEN`, then the ambient helper — rather than leaving it to
whatever the workstation's git config happens to do.
It matters because `gh auth login` **does not** wire git's credential helper
(that is `gh auth setup-git`, a separate act most people never run), so a
perfectly logged-in operator still fell through to git's interactive
username/password prompt — which GitHub no longer accepts — and got an error
about *the repository* rather than about the missing credentials. There is no
routing around it for prod: `--path` is refused there, so the clone is the only
path and its auth is mandatory.
`GIT_TERMINAL_PROMPT=0` is set on every path, so cast can never hang on or fall
into that prompt. The token is never placed in the clone URL or in
`http.extraheader` — both leak it into `ps`, and the latter persists it into the
clone's `.git/config`; the helper reads it from the environment at run time, so
what lands in argv is the literal text `$CAST_GIT_TOKEN`. Note that the empty
`credential.helper=` reset clears **URL-scoped** helpers
(`credential.https://github.com.helper`, which is what `gh auth setup-git`
writes) as well as generic ones, so cast's chosen credential is genuinely the
one used — verified against a live private clone.
**Known limitations, not defects:**
- **Backup schedules are create-time only.** A manifest database's `backup`

6
src/bindings.ts
View file

@ -47,6 +47,12 @@ const BindingsSchema = z
// An explicit --instance still wins, so a one-off read against a
// legacy box needs no change to this file either.
instance: z.string().optional(),
// The age recipient (public key) this environment's secret store is
// encrypted TO. Only `capture` needs it — decryption resolves an
// identity per keyFileFor, and the state repo deliberately holds
// ciphertext but never the identity that opens it. This is the
// public half, so it is safe to commit here next to the bindings.
age_recipient: z.string().optional(),
s3_destination: z.string().optional(),
// Var-name patterns this environment refuses outright (see
// assertEnvVarPolicy). Operator-owned guard: prod typically bans

189
src/capture.ts Normal file
View file

@ -0,0 +1,189 @@
import type { RequiredSecret } from "./resolve.js";
// What the manifest writes for a provider-generated name. Not the source box's
// value — that points at the SOURCE box's Postgres/Redis — and not an empty
// string, which would boot the app misconfigured. A literal that is obviously
// a placeholder, and that Coolify replaces when it creates the resource.
export const GENERATED_PLACEHOLDER = "pending-coolify-generated";
// Live env vars, per resource: resource name -> (env key -> value).
export type LiveEnvs = Record<string, Record<string, string>>;
export type Provenance = "captured" | "generated" | "overridden";
export type Site = { resource: string; key: string };
export type Disposition = {
ref: string;
provenance: Provenance;
// Never rendered. Kept here so the caller can encrypt it, and nowhere else.
value: string;
sites: Site[];
};
export type Classification = {
plan: Disposition[];
// Required by a template, absent from the source, and not dispositioned
// otherwise. Refuses the run: writing an empty value substitutes to nothing
// and the app boots misconfigured — the exact failure capture exists to
// remove, and one that looks entirely plausible from the outside.
missing: Array<{ ref: string; sites: Site[] }>;
// The same ref carrying DIFFERENT live values on two resources. cast cannot
// pick, and picking wrong is silent, so it refuses.
conflicts: Array<{ ref: string; values: Site[] }>;
};
function groupByRef(required: RequiredSecret[]): Map<string, Site[]> {
const byRef = new Map<string, Site[]>();
for (const { ref, resource, key } of required) {
const sites = byRef.get(ref) ?? [];
sites.push({ resource, key });
byRef.set(ref, sites);
}
return byRef;
}
// Force disposition, never guess. Every name the manifest requires lands in
// exactly one of four buckets, and two of them stop the run.
//
// The mapping is deliberately NOT a mechanical dump of the source box: some
// entries encode migration decisions rather than facts about the source. A
// "capture everything" verb would be wrong in a handful of entries out of
// seventeen, silently — which is worse than being wrong in all of them.
export function classify(
required: RequiredSecret[],
generated: string[],
live: LiveEnvs,
overrides: Record<string, string>,
): Classification {
const generatedSet = new Set(generated);
const plan: Disposition[] = [];
const missing: Classification["missing"] = [];
const conflicts: Classification["conflicts"] = [];
for (const [ref, sites] of groupByRef(required)) {
// The operator's word beats both the manifest and the source box: this is
// the escape hatch for a value that must NOT be carried over (staging's
// ADMIN_EMAIL, where the source's value is a real founder and staging
// shares a Mailgun domain with prod).
if (ref in overrides) {
plan.push({
ref,
provenance: "overridden",
value: overrides[ref],
sites,
});
continue;
}
if (generatedSet.has(ref)) {
plan.push({
ref,
provenance: "generated",
value: GENERATED_PLACEHOLDER,
sites,
});
continue;
}
// Captured: read the live value off whichever resources declare it.
const found = sites
.map((s) => ({ site: s, value: live[s.resource]?.[s.key] }))
.filter((f): f is { site: Site; value: string } => f.value !== undefined);
if (found.length === 0) {
missing.push({ ref, sites });
continue;
}
const distinct = new Set(found.map((f) => f.value));
if (distinct.size > 1) {
conflicts.push({ ref, values: found.map((f) => f.site) });
continue;
}
plan.push({
ref,
provenance: "captured",
value: found[0].value,
sites,
});
}
return { plan, missing, conflicts };
}
const site = (s: Site) => `${s.resource}.${s.key}`;
// Names and provenance. NEVER values.
//
// The one thing printed that looks like a value is GENERATED_PLACEHOLDER,
// which is a literal constant in this file and carries no information about
// the source box. Everything else is a name the manifest already declares in
// plaintext, in a committed file.
export function renderCapturePlan(
c: Classification,
ctx: {
orgRepo: string;
env: string;
instance: string;
store: string;
recipient: string;
},
): string {
const lines = [
`capture plan — ${ctx.orgRepo} ${ctx.env}`,
"",
` source: instance ${ctx.instance} (live values read from it)`,
` store: ${ctx.store}`,
` recipient: ${ctx.recipient}`,
"",
];
const width = Math.max(
0,
...[...c.plan, ...c.missing, ...c.conflicts].map((d) => d.ref.length),
);
for (const d of c.plan) {
const where = d.sites.map(site).join(", ");
const note =
d.provenance === "generated"
? ` → ${GENERATED_PLACEHOLDER}`
: d.provenance === "overridden"
? ` (from CAST_CAPTURE_${d.ref})`
: "";
lines.push(
` ${d.ref.padEnd(width)} ${d.provenance.padEnd(10)} ${where}${note}`,
);
}
for (const m of c.missing) {
lines.push(
` ${m.ref.padEnd(width)} MISSING required by ${m.sites.map(site).join(", ")}, absent from the source`,
);
}
for (const c2 of c.conflicts) {
lines.push(
` ${c2.ref.padEnd(width)} CONFLICT differs between ${c2.values.map(site).join(" and ")}`,
);
}
const counts = (["captured", "generated", "overridden"] as const)
.map((p) => [p, c.plan.filter((d) => d.provenance === p).length] as const)
.filter(([, n]) => n > 0)
.map(([p, n]) => `${n} ${p}`)
.join(", ");
lines.push(
"",
`${c.plan.length} name(s) to write${counts ? `: ${counts}` : ""}`,
);
if (c.missing.length > 0) {
lines.push(
"",
`refusing to write the store: ${c.missing.length} name(s) the manifest requires are not`,
"present on the source. An empty value substitutes to nothing and the app boots",
"misconfigured — plausibly, and silently. Supply each one with --override <NAME>",
"(its value is read from CAST_CAPTURE_<NAME>, never from argv), or fix the source.",
);
}
if (c.conflicts.length > 0) {
lines.push(
"",
`refusing to write the store: ${c.conflicts.length} name(s) carry different values on`,
"different resources of the source. The store holds one value per name, and cast",
"will not pick for you. Reconcile them on the source, or pin one with --override.",
);
}
return lines.join("\n");
}

275
src/cli.ts
View file

@ -1,10 +1,12 @@
#!/usr/bin/env node
import { readFileSync } from "node:fs";
import { existsSync, readFileSync } from "node:fs";
import { join } from "node:path";
import { createInterface } from "node:readline/promises";
import { parseArgs } from "node:util";
import { parse as parseYaml } from "yaml";
import { type Executor, applyHostnameOverlay, applyPlan } from "./apply.js";
import { githubAppNameFor, loadBindings } from "./bindings.js";
import { type LiveEnvs, classify, renderCapturePlan } from "./capture.js";
import {
type CoolifyInstance,
assertWritable,
@ -19,14 +21,24 @@ import {
renderDiff,
} from "./diff.js";
import { assertEnvVarPolicy } from "./envtemplate.js";
import { desiredFromManifest, resolveCheckout } from "./resolve.js";
import { decryptSecrets, keyFileFor, secretsFileFor } from "./secrets.js";
import {
desiredFromManifest,
requiredSecrets,
resolveCheckout,
} from "./resolve.js";
import {
decryptSecrets,
encryptSecrets,
keyFileFor,
secretsFileFor,
} from "./secrets.js";
import { serverAdd } from "./server.js";
import { smoke } from "./smoke.js";
import { assertTeam, formatTeam } from "./team.js";
const USAGE = `usage: cast apply <org>/<repo> --env <env> [--path <dir>] [--project <name>] [--hostname-overlay <file>]
cast diff <org>/<repo> --env <env> [--full] [--project <name>]
const USAGE = `usage: cast apply <org>/<repo> --env <env> [--path <dir>] [--project <name>] [--hostname-overlay <file>]
cast diff <org>/<repo> --env <env> [--full] [--project <name>]
cast capture <org>/<repo> --env <env> [--path <dir>] [--project <name>] [--generated <NAME>] [--override <NAME>] [--force]
cast server add <name> --ip <ip> --key <file> --env <env> [--user root] [--port 22]
cast smoke --env <env>
cast team [--env <env>]
@ -50,7 +62,16 @@ const USAGE = `usage: cast apply <org>/<repo> --env <env> [--path <dir>] [--proj
repo (the default). A project built by hand in the UI is called
whatever someone typed; \`diff\` refuses rather than reporting an
absent project as an empty one, and this is how you point it at
the real name.`;
the real name.
capture (adopt a hand-built instance into the age secret store):
--generated <NAME> force NAME to the \`pending-coolify-generated\` placeholder,
for a manifest that has not declared generated_secrets yet.
Repeatable.
--override <NAME> supply NAME yourself instead of copying the source's value.
The VALUE is read from \$CAST_CAPTURE_<NAME>, never from the
command line — argv is visible in \`ps\`. Repeatable.
--force overwrite an existing store (refused by default).`;
// cast is stateless: every instance-scoped input is read from the state
// directory it is pointed at, never from a location the tool itself knows.
@ -290,21 +311,26 @@ export async function fetchLive(
// exists next to it.
export function renderAbsentTarget(
lookup: Extract<LiveLookup, { found: false }>,
ctx: { orgRepo: string; overridden: boolean },
ctx: { orgRepo: string; overridden: boolean; verb?: string },
): string {
// `capture` takes the same position as `diff`, and for the same reason: it
// is only ever a claim about something that already exists. Against an
// absent target it would read back zero live values and call every required
// secret "missing" — an alarming-but-meaningless report about the wrong box.
const verb = ctx.verb ?? "diff";
const origin = ctx.overridden
? "--project"
: `derived from the repo slug ${ctx.orgRepo}`;
const head =
lookup.missing === "project"
? [
`refusing to diff: no project named "${lookup.project}" exists in this team`,
`refusing to ${verb}: no project named "${lookup.project}" exists in this team`,
"",
` looked for: project "${lookup.project}" (${origin})`,
` exists here: ${lookup.available.join(", ") || "(no projects at all)"}`,
]
: [
`refusing to diff: project "${lookup.project}" has no environment "${lookup.environment}"`,
`refusing to ${verb}: project "${lookup.project}" has no environment "${lookup.environment}"`,
"",
` looked for: environment "${lookup.environment}" in project "${lookup.project}"`,
" note: cast names environments after --env, so a project built by",
@ -316,7 +342,7 @@ export function renderAbsentTarget(
"",
"An absent target reads back exactly like an empty one, so continuing would diff",
'it as "nothing exists — create everything": a clean-looking report that verified',
"nothing. `apply` may create a target; `diff` may only ever describe one that is",
`nothing. \`apply\` may create a target; \`${verb}\` may only ever describe one that is`,
"already there.",
"",
lookup.missing === "project"
@ -325,6 +351,79 @@ export function renderAbsentTarget(
].join("\n");
}
// A live resource's env vars, by key. `real_value` is the decrypted one and
// needs a token with read:sensitive; `value` is what a lesser token sees.
//
// A 404 (a resource we just listed no longer having an envs endpoint — not
// expected in practice, but consistent with treating "gone" as "no env vars")
// collapses to {}; anything else (401, 5xx, network) must surface. Swallowing
// it would make a live resource's env look EMPTY, which turns every one of its
// vars into a spurious create in a diff, and into a spurious "missing" in a
// capture.
async function fetchEnv(
client: CoolifyClient,
l: Live,
): Promise<Record<string, string>> {
const base = l.kind === "database" ? "databases" : `${l.kind}s`;
const envs = (await client.get(`/${base}/${l.uuid}/envs`).catch((err) => {
if (err instanceof HttpError && err.status === 404) return [];
throw err;
})) as Array<{ key: string; real_value?: string; value: string }>;
return Object.fromEntries(envs.map((e) => [e.key, e.real_value ?? e.value]));
}
// The value for an --override, read from the ENVIRONMENT rather than argv.
//
// A secret passed as a command-line argument is visible in `ps` to every
// process on the box, and lands in shell history — the same class of leak the
// clone-auth fix (#13) exists to avoid. So --override names the secret and the
// environment carries it.
function readOverrides(names: string[]): Record<string, string> {
const out: Record<string, string> = {};
for (const name of names) {
const varName = `CAST_CAPTURE_${name}`;
const value = process.env[varName];
if (value === undefined) {
throw new Error(
[
`--override ${name}: no value supplied.`,
"",
`cast reads an override's value from ${varName}, never from the command`,
"line — an argv value is visible in `ps` to every process on this box.",
"",
` ${varName}=… cast capture …`,
].join("\n"),
);
}
out[name] = value;
}
return out;
}
// Typed confirmation, and deliberately NOT a --yes flag.
//
// This verb writes an environment's secret store, once, off a box nobody is
// going to rebuild. The entire reason it exists is that the hand-run version
// was easy to get subtly wrong — so the last gate is a human who has read the
// provenance column typing the environment's own name. Nothing shorter counts:
// not "y", not a flag. Automating it means deliberately echoing the
// environment name into cast, which is an explicit act rather than an absent
// one.
//
// EOF (a closed or empty stdin) resolves to `null` and aborts. Without that
// race, a `< /dev/null` run would hang forever on a question nobody can answer.
async function confirmCapture(envName: string): Promise<boolean> {
const rl = createInterface({ input: process.stdin, output: process.stdout });
const answer = await new Promise<string | null>((resolve) => {
rl.question(
`\ntype the environment name to write this store (${envName}): `,
).then(resolve, () => resolve(null));
rl.once("close", () => resolve(null));
});
rl.close();
return answer?.trim() === envName;
}
async function main(): Promise<number> {
const [command, ...rest] = process.argv.slice(2);
if (command === "-h" || command === "--help" || command === "help") {
@ -419,28 +518,7 @@ async function main(): Promise<number> {
const live = lookup.found ? lookup.live : [];
if (mode === "full") {
for (const l of live) {
const envs = (await client
.get(
`/${l.kind === "database" ? "databases" : `${l.kind}s`}/${l.uuid}/envs`,
)
.catch((err) => {
// Same policy as fetchLive's environment fetch: a 404 (a
// resource we just listed no longer having an envs endpoint —
// not expected in practice, but consistent with treating
// "gone" as "no env vars") collapses to []; anything else
// (401, 5xx, network) must surface. Swallowing it here would
// make a live resource's env look empty and turn every one of
// its vars into a spurious create in the diff.
if (err instanceof HttpError && err.status === 404) return [];
throw err;
})) as Array<{
key: string;
real_value?: string;
value: string;
}>;
l.env = Object.fromEntries(
envs.map((e) => [e.key, e.real_value ?? e.value]),
);
l.env = await fetchEnv(client, l);
}
}
const report = computeDiff(desired, live, mode);
@ -466,6 +544,139 @@ async function main(): Promise<number> {
);
return 0;
}
if (command === "capture") {
const { values, positionals } = parseArgs({
args: rest,
allowPositionals: true,
options: {
env: { type: "string" },
state: { type: "string" },
path: { type: "string" },
project: { type: "string" },
instance: { type: "string" },
generated: { type: "string", multiple: true },
override: { type: "string", multiple: true },
force: { type: "boolean", default: false },
},
});
const orgRepo = positionals[0];
const envName = values.env;
if (!orgRepo || !envName) {
console.error(USAGE);
return 2;
}
const stateDir = stateDirFrom(values.state);
const repoShort = orgRepo.split("/")[1];
const projectName = values.project ?? repoShort;
const store = secretsFileFor(stateDir, repoShort, envName);
// Never overwrite a store by accident. `apply` never deletes; the verb
// that WRITES the store gets the same disposition, because the thing it
// would destroy is the only copy of values that may not exist anywhere
// else any more.
if (existsSync(store) && !values.force) {
console.error(
[
`refusing to capture: ${store} already exists`,
"",
"That store may hold the only copy of values the source box no longer has.",
"Pass --force to overwrite it deliberately, or move it aside first.",
].join("\n"),
);
return 2;
}
const bindings = loadBindings(join(stateDir, "environments.yaml"));
const binding = bindings.environments[envName];
if (!binding) {
console.error(`environment ${envName} not in environments.yaml`);
return 2;
}
const recipient = binding.age_recipient;
if (!recipient) {
console.error(
[
`environment ${envName} has no age_recipient in environments.yaml`,
"",
"capture encrypts the store TO that recipient (the public half of the",
"environment's age key — safe to commit next to the bindings). Add it:",
"",
" environments:",
` ${envName}:`,
" age_recipient: age1…",
].join("\n"),
);
return 2;
}
// Same rule as apply (resolveCheckout enforces it): prod always reads the
// default branch. A feature-branch manifest must not be able to decide
// which names land in the prod store.
const checkout = resolveCheckout(orgRepo, {
env: envName,
path: values.path,
});
const { required, generated } = requiredSecrets(checkout, envName);
const overrides = readOverrides(values.override ?? []);
const { client } = openCoolify(stateDir, values.instance, binding);
// capture READS Coolify and writes only to the local store, so it is
// allowed against a read-only instance — inspecting a legacy box is
// precisely what such an instance is for. It still takes the team assert:
// a wrong-team token reads back nothing, and "nothing" here would render
// as "every secret is missing" against a box that is fine.
const team = await assertTeam(client, binding.team, envName);
console.log(`team ${formatTeam(team)} ✓`);
const lookup = await fetchLive(client, projectName, envName);
if (!lookup.found) {
console.error(
renderAbsentTarget(lookup, {
orgRepo,
overridden: values.project !== undefined,
verb: "capture",
}),
);
return 2;
}
const liveEnvs: LiveEnvs = {};
for (const l of lookup.live) {
// Databases hold no manifest-templated env of their own — their URL is
// what the APPS reference, and that name is generated, not captured.
if (l.kind === "database") continue;
liveEnvs[l.name] = await fetchEnv(client, l);
}
const classification = classify(
required,
[...generated, ...(values.generated ?? [])],
liveEnvs,
overrides,
);
console.log(
renderCapturePlan(classification, {
orgRepo,
env: envName,
instance: values.instance ?? binding.instance ?? "default",
store,
recipient,
}),
);
// Refuse, don't write a wrong store. Both of these are stop conditions,
// and the plan above has already named every offending entry.
if (
classification.missing.length > 0 ||
classification.conflicts.length > 0
)
return 2;
if (!(await confirmCapture(envName))) {
console.error("aborted — nothing written");
return 2;
}
encryptSecrets(
recipient,
store,
Object.fromEntries(classification.plan.map((d) => [d.ref, d.value])),
);
console.log(
`wrote ${store} — ${classification.plan.length} name(s), encrypted to ${recipient}`,
);
return 0;
}
if (command === "server" && rest[0] === "add") {
const { values, positionals } = parseArgs({
args: rest.slice(1),

56
src/envtemplate.ts
View file

@ -2,11 +2,18 @@ export type ResolvedEnv = {
vars: Record<string, { value: string; secret: boolean }>;
};
export function resolveTemplate(
text: string,
secrets: Record<string, string>,
): ResolvedEnv {
const vars: ResolvedEnv["vars"] = {};
// A template line, parsed but not resolved: `ref` is set when the whole RHS is
// a single ${NAME} placeholder.
export type TemplateVar = { key: string; rhs: string; ref?: string };
// ONE grammar, shared by both readers of a template — resolveTemplate (which
// needs the values) and templateRefs (which needs only the names). Keeping
// them on separate parsers would let the two drift, and a drift here is not
// cosmetic: `capture` would collect a different set of names than `apply` will
// later demand, which is precisely the "a name silently missed" failure the
// capture verb exists to remove.
function parseTemplate(text: string): TemplateVar[] {
const vars: TemplateVar[] = [];
const lines = text.split("\n");
for (let i = 0; i < lines.length; i++) {
const line = lines[i].trim();
@ -18,21 +25,42 @@ export function resolveTemplate(
);
const [, key, rhs] = m;
const placeholder = rhs.match(/^\$\{([A-Z][A-Z0-9_]*)\}$/);
if (placeholder) {
const value = secrets[placeholder[1]];
if (value === undefined) {
throw new Error(
`secret ${placeholder[1]} (for ${key}) missing from the age store`,
);
}
vars[key] = { value, secret: true };
} else {
vars.push({ key, rhs, ...(placeholder ? { ref: placeholder[1] } : {}) });
}
return vars;
}
export function resolveTemplate(
text: string,
secrets: Record<string, string>,
): ResolvedEnv {
const vars: ResolvedEnv["vars"] = {};
for (const { key, rhs, ref } of parseTemplate(text)) {
if (ref === undefined) {
vars[key] = { value: rhs, secret: false };
continue;
}
const value = secrets[ref];
if (value === undefined) {
throw new Error(`secret ${ref} (for ${key}) missing from the age store`);
}
vars[key] = { value, secret: true };
}
return { vars };
}
// The ${NAME} refs a template declares: the secret names the manifest requires,
// paired with the env var each one lands on. `capture` reads these to learn
// what to go and fetch — at capture time there is no store to resolve against
// yet, which is the whole point of the verb.
export function templateRefs(
text: string,
): Array<{ key: string; ref: string }> {
return parseTemplate(text).flatMap(({ key, ref }) =>
ref === undefined ? [] : [{ key, ref }],
);
}
// An environment may forbid variables by name pattern, declared as
// `environments.<env>.forbidden_var_patterns` in the state repo. The rule is
// PRESENCE, not value: a forbidden var set to "false" still refuses the apply,

13
src/manifest.ts
View file

@ -82,6 +82,19 @@ const EnvironmentSpecSchema = z
applications: z.record(AppSpecSchema),
databases: z.record(DatabaseSpecSchema).optional(),
services: z.record(ServiceSpecSchema).optional(),
// Secret names whose values the PROVIDER generates — a Coolify-created
// Postgres/Redis URL, a service's own generated credentials. `capture`
// writes these as the literal `pending-coolify-generated` and never copies
// the source box's live value: that value points at the SOURCE box's
// database, so carrying it over would be confidently wrong in a way that
// looks entirely plausible, and the target's real URL does not exist until
// Coolify creates the resource.
//
// It is a manifest property rather than a flag the operator has to
// remember, because the manifest is what knows DATABASE_URL comes from a
// database it declares. Optional: a manifest that names none simply has no
// generated secrets, and `capture` will say so in its plan.
generated_secrets: z.array(z.string()).optional(),
})
.strict();

78
src/resolve.ts
View file

@ -3,7 +3,11 @@ import { existsSync, mkdtempSync, readFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import type { Desired } from "./diff.js";
import { type ResolvedEnv, resolveTemplate } from "./envtemplate.js";
import {
type ResolvedEnv,
resolveTemplate,
templateRefs,
} from "./envtemplate.js";
import { loadManifest } from "./manifest.js";
// How cast authenticated (or failed to authenticate) a clone.
@ -132,8 +136,11 @@ export function resolveCheckout(
opts: { env: string; path?: string },
): string {
if (opts.path && opts.env === "prod") {
// Holds for every verb that reads a manifest (apply, and now capture): a
// feature-branch checkout must not be able to decide what prod runs, nor
// which secret names land in prod's store.
throw new Error(
"apply refuses --path with --env prod: prod always reads the default branch",
"refuses --path with --env prod: prod always reads the default branch",
);
}
if (opts.path) return opts.path;
@ -171,6 +178,73 @@ export function resolveCheckout(
return dir;
}
// One secret the manifest requires: the ${REF} a template names (the key it
// gets in the age store), the resource that needs it, and the env var it lands
// on there. That last pair is what `capture` reads the live value from — the
// store is keyed by REF, but the live box knows it as `resource.key`.
export type RequiredSecret = { ref: string; resource: string; key: string };
// Exactly the set of secret names an environment's manifest demands — the same
// set `apply` will later insist on, read from the same templates by the same
// parser. `capture` uses this to know what to go and fetch; nothing else has to
// be told, and nothing can be silently missed.
//
// Deliberately does NOT take a secrets map: at capture time the store does not
// exist yet. That is the whole point of the verb.
export function requiredSecrets(
checkoutDir: string,
envName: string,
): { required: RequiredSecret[]; generated: string[] } {
const manifest = loadManifest(join(checkoutDir, ".infra", "manifest.yaml"));
const envSpec = manifest.environments[envName];
if (!envSpec) {
throw new Error(
`environment ${envName} not in manifest (has: ${Object.keys(manifest.environments).join(", ") || "none"})`,
);
}
const required: RequiredSecret[] = [];
const collect = (resource: string, template?: string) => {
if (!template) return;
const file = join(checkoutDir, ".infra", "env", template);
if (!existsSync(file))
throw new Error(
`env template missing: ${file} (referenced by ${resource})`,
);
for (const { key, ref } of templateRefs(readFileSync(file, "utf8"))) {
required.push({ ref, resource, key });
}
};
for (const [name, app] of Object.entries(envSpec.applications)) {
collect(name, app.env_template);
}
for (const [name, svc] of Object.entries(envSpec.services ?? {})) {
collect(name, svc.env_template);
}
const generated = envSpec.generated_secrets ?? [];
// A generated_secrets entry naming something no template refs is dead
// config — and dead config in THIS list is not merely untidy, it is
// dangerous: it reads like a guard standing over a name while standing over
// nothing. The likeliest cause is a typo, and the consequence of the typo is
// that the real name gets CAPTURED from the source box instead of placeheld.
const refs = new Set(required.map((r) => r.ref));
const dead = generated.filter((g) => !refs.has(g));
if (dead.length > 0) {
throw new Error(
[
`manifest environment ${envName}: generated_secrets names ${dead.join(", ")}, which no env template refers to`,
"",
` declared: ${generated.join(", ")}`,
` templates: ${[...refs].sort().join(", ") || "(no ${...} refs at all)"}`,
"",
"A generated name that matches nothing guards nothing — and if this is a",
"typo, the name it was meant to guard is being captured from the source",
"box instead of placeheld. Fix the spelling, or drop the entry.",
].join("\n"),
);
}
return { required, generated };
}
export function desiredFromManifest(
checkoutDir: string,
envName: string,

21
src/secrets.ts
View file

@ -22,6 +22,27 @@ export function decryptSecrets(
return secrets;
}
// Write an environment's secret store, encrypted to its recipient.
//
// The plaintext goes to age on STDIN and the ciphertext straight to `file`: it
// is never a temp file, never reaches the terminal, and never lands in shell
// history. The hand-run recipe this replaces assembled /dev/shm/prod.env and
// relied on remembering to `shred -u` it afterwards — a step that is invisible
// when it is skipped.
export function encryptSecrets(
recipient: string,
file: string,
vars: Record<string, string>,
): void {
const plaintext = `${Object.entries(vars)
.map(([k, v]) => `${k}=${v}`)
.join("\n")}\n`;
execFileSync("age", ["-r", recipient, "-o", file], {
input: plaintext,
stdio: ["pipe", "pipe", "pipe"],
});
}
// The age identity for an environment, resolved without cast knowing anything
// about your environment names:
//

334
test/capture-cli.test.ts Normal file
View file

@ -0,0 +1,334 @@
import { execFileSync, spawn } from "node:child_process";
import {
existsSync,
mkdirSync,
mkdtempSync,
readFileSync,
writeFileSync,
} from "node:fs";
import { createServer } from "node:http";
import type { AddressInfo } from "node:net";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { afterEach, beforeAll, describe, expect, it } from "vitest";
import { decryptSecrets } from "../src/secrets.js";
// End-to-end: the real CLI, a real age identity, a stub Coolify holding real
// live values. The point is the store that comes out the other side — it is
// decrypted and asserted on, so "exactly the names the manifest requires, no
// more and no fewer" is checked against the actual ciphertext rather than
// against cast's own console output.
const SECRETS = {
// Points at the SOURCE box. Must NOT be carried over.
DATABASE_URL: "postgres://user:pw@SOURCE-BOX-postgres:5432/app",
MAILGUN_API_KEY: "key-REAL-MAILGUN-SECRET",
OPENROUTER_API_KEY: "sk-or-REAL-OPENROUTER-SECRET",
// A real founder. Must NOT be carried over to staging.
ADMIN_EMAIL: "founder@real-company.com",
// Live on the box, but the manifest never asks for it.
UNRELATED_LIVE_VAR: "nobody-asked-for-this",
};
let keyFile: string;
let recipient: string;
beforeAll(() => {
const dir = mkdtempSync(join(tmpdir(), "cast-age-"));
keyFile = join(dir, "age-staging.key");
execFileSync("age-keygen", ["-o", keyFile], { stdio: "pipe" });
const pub = execFileSync("age-keygen", ["-y", keyFile], { encoding: "utf8" });
recipient = pub.trim();
});
type Stub = { url: string; close: () => Promise<void> };
const stubs: Stub[] = [];
// A Coolify with one project, one environment, one application carrying the
// live env above.
async function stubCoolify(): Promise<Stub> {
const server = createServer((req, res) => {
const path = (req.url ?? "").replace("/api/v1", "");
const json = (body: unknown) => {
res.writeHead(200, { "content-type": "application/json" });
res.end(JSON.stringify(body));
};
if (path === "/teams/current") return json({ id: 0, name: "Root Team" });
if (path === "/projects") return json([{ uuid: "p1", name: "incubator" }]);
if (path === "/projects/p1/staging")
return json({ applications: [{ name: "core", uuid: "a1" }] });
if (path === "/applications/a1/envs")
return json(
Object.entries(SECRETS).map(([key, real_value]) => ({
key,
real_value,
value: "REDACTED",
})),
);
res.writeHead(404);
res.end("{}");
});
await new Promise<void>((r) => {
server.listen(0, "127.0.0.1", r);
});
const stub: Stub = {
url: `http://127.0.0.1:${(server.address() as AddressInfo).port}`,
close: () =>
new Promise<void>((r) => {
server.close(() => r());
}),
};
stubs.push(stub);
return stub;
}
afterEach(async () => {
await Promise.all(stubs.splice(0).map((s) => s.close()));
});
const MANIFEST = `project: incubator
environments:
staging:
generated_secrets: [DATABASE_URL_STAGING]
applications:
core:
source: { repo: heavy-duty/incubator, branch: main }
build: { pack: nixpacks, base_directory: / }
domains: ["http://core.example.com"]
env_template: core.staging.env.template
`;
// NODE_ENV is a literal, not a secret — it must not reach the store.
const TEMPLATE = `NODE_ENV=production
DATABASE_URL=\${DATABASE_URL_STAGING}
MAILGUN_API_KEY=\${MAILGUN_API_KEY}
OPENROUTER_API_KEY=\${OPENROUTER_API_KEY}
ADMIN_EMAIL=\${ADMIN_EMAIL}
`;
function fixture(url: string, opts: { template?: string } = {}) {
const checkout = mkdtempSync(join(tmpdir(), "cast-co-"));
mkdirSync(join(checkout, ".infra", "env"), { recursive: true });
writeFileSync(join(checkout, ".infra", "manifest.yaml"), MANIFEST);
writeFileSync(
join(checkout, ".infra", "env", "core.staging.env.template"),
opts.template ?? TEMPLATE,
);
const state = mkdtempSync(join(tmpdir(), "cast-state-"));
mkdirSync(join(state, "secrets"));
writeFileSync(
join(state, ".coolify.env"),
`COOLIFY_BASE_URL="${url}"\nCOOLIFY_ACCESS_TOKEN="t"\n`,
);
writeFileSync(
join(state, "environments.yaml"),
[
"environments:",
" staging:",
" server: staging-box",
" team: { id: 0, name: Root Team }",
` age_recipient: ${recipient}`,
"github_apps:",
" incubator: hdb-coolify",
"",
].join("\n"),
);
return {
checkout,
state,
store: join(state, "secrets", "incubator.staging.env.age"),
};
}
function runCapture(
args: string[],
opts: { stdin?: string; env?: Record<string, string> } = {},
): Promise<{ code: number; output: string }> {
return new Promise((resolve) => {
const child = spawn("node", ["dist/cli.js", "capture", ...args], {
stdio: ["pipe", "pipe", "pipe"],
env: { ...process.env, ...opts.env },
});
let output = "";
child.stdout.on("data", (d) => {
output += String(d);
});
child.stderr.on("data", (d) => {
output += String(d);
});
child.stdin.end(opts.stdin ?? "");
child.on("close", (code) => resolve({ code: code ?? 0, output }));
});
}
const base = (f: ReturnType<typeof fixture>) => [
"heavy-duty/incubator",
"--env",
"staging",
"--state",
f.state,
"--path",
f.checkout,
];
describe("cast capture (end to end)", () => {
it("writes a store with exactly the manifest's names, and the right provenance", async () => {
const f = fixture((await stubCoolify()).url);
const r = await runCapture([...base(f), "--override", "ADMIN_EMAIL"], {
stdin: "staging\n",
env: { CAST_CAPTURE_ADMIN_EMAIL: "operator@example.com" },
});
expect(r.code).toBe(0);
expect(existsSync(f.store)).toBe(true);
const store = decryptSecrets(f.store, keyFile);
// No more, no fewer: the four ${...} refs. NODE_ENV is a literal, and
// UNRELATED_LIVE_VAR is live but unasked-for — neither belongs here.
expect(Object.keys(store).sort()).toEqual([
"ADMIN_EMAIL",
"DATABASE_URL_STAGING",
"MAILGUN_API_KEY",
"OPENROUTER_API_KEY",
]);
// Captured verbatim.
expect(store.MAILGUN_API_KEY).toBe(SECRETS.MAILGUN_API_KEY);
expect(store.OPENROUTER_API_KEY).toBe(SECRETS.OPENROUTER_API_KEY);
// Generated: the placeholder, NEVER the source box's own database URL.
expect(store.DATABASE_URL_STAGING).toBe("pending-coolify-generated");
expect(store.DATABASE_URL_STAGING).not.toContain("SOURCE-BOX");
// Overridden: the operator's value, not the real founder's address.
expect(store.ADMIN_EMAIL).toBe("operator@example.com");
expect(store.ADMIN_EMAIL).not.toBe(SECRETS.ADMIN_EMAIL);
});
// "No secret value is ever written to stdout" — checked against the real
// values the stub served, on the real console output of a real run.
it("never prints a secret value to the console", async () => {
const f = fixture((await stubCoolify()).url);
const r = await runCapture([...base(f), "--override", "ADMIN_EMAIL"], {
stdin: "staging\n",
env: { CAST_CAPTURE_ADMIN_EMAIL: "operator@example.com" },
});
expect(r.code).toBe(0);
for (const value of Object.values(SECRETS)) {
expect(r.output).not.toContain(value);
}
expect(r.output).not.toContain("operator@example.com");
// It did print the NAMES, though — that is the plan.
expect(r.output).toContain("MAILGUN_API_KEY");
expect(r.output).toContain("captured");
expect(r.output).toContain("generated");
expect(r.output).toContain("overridden");
});
// The plaintext exists only in memory and on age's stdin.
it("leaves no plaintext behind — the store is real ciphertext", async () => {
const f = fixture((await stubCoolify()).url);
await runCapture([...base(f), "--override", "ADMIN_EMAIL"], {
stdin: "staging\n",
env: { CAST_CAPTURE_ADMIN_EMAIL: "operator@example.com" },
});
const raw = readFileSync(f.store, "utf8");
expect(raw).toContain("age-encryption.org");
for (const value of Object.values(SECRETS)) {
expect(raw).not.toContain(value);
}
});
// A name required by the template but absent from the source refuses the run
// — writing an empty would boot the app misconfigured, plausibly.
it("refuses when a required name is absent from the source", async () => {
const f = fixture((await stubCoolify()).url, {
template: `${TEMPLATE}TURNSTILE_SECRET=\${TURNSTILE_SECRET}\n`,
});
const r = await runCapture([...base(f), "--override", "ADMIN_EMAIL"], {
stdin: "staging\n",
env: { CAST_CAPTURE_ADMIN_EMAIL: "operator@example.com" },
});
expect(r.code).not.toBe(0);
expect(r.output).toMatch(/TURNSTILE_SECRET\s+MISSING/);
expect(r.output).toMatch(/refusing to write the store/);
expect(existsSync(f.store)).toBe(false);
});
// The confirmation is the last gate, and it is not "y".
it("aborts, writing nothing, when the confirmation does not name the env", async () => {
const f = fixture((await stubCoolify()).url);
const r = await runCapture([...base(f), "--override", "ADMIN_EMAIL"], {
stdin: "y\n",
env: { CAST_CAPTURE_ADMIN_EMAIL: "operator@example.com" },
});
expect(r.code).not.toBe(0);
expect(r.output).toMatch(/aborted/);
expect(existsSync(f.store)).toBe(false);
});
it("aborts on a closed stdin rather than hanging", async () => {
const f = fixture((await stubCoolify()).url);
const r = await runCapture([...base(f), "--override", "ADMIN_EMAIL"], {
stdin: "",
env: { CAST_CAPTURE_ADMIN_EMAIL: "operator@example.com" },
});
expect(r.code).not.toBe(0);
expect(r.output).toMatch(/aborted/);
});
// An override's VALUE never comes from argv — argv is visible in `ps`.
it("refuses an --override whose CAST_CAPTURE_<NAME> is unset", async () => {
const f = fixture((await stubCoolify()).url);
const r = await runCapture([...base(f), "--override", "ADMIN_EMAIL"], {
stdin: "staging\n",
});
expect(r.code).not.toBe(0);
expect(r.output).toMatch(/CAST_CAPTURE_ADMIN_EMAIL/);
expect(r.output).toMatch(/never from the command/);
});
// The store may hold the only copy of values the source box no longer has.
it("refuses to overwrite an existing store without --force", async () => {
const f = fixture((await stubCoolify()).url);
writeFileSync(f.store, "PRE-EXISTING");
const r = await runCapture([...base(f), "--override", "ADMIN_EMAIL"], {
stdin: "staging\n",
env: { CAST_CAPTURE_ADMIN_EMAIL: "operator@example.com" },
});
expect(r.code).not.toBe(0);
expect(r.output).toMatch(/already exists/);
expect(r.output).toMatch(/--force/);
expect(readFileSync(f.store, "utf8")).toBe("PRE-EXISTING");
});
it("refuses an environment with no age_recipient", async () => {
const f = fixture((await stubCoolify()).url);
writeFileSync(
join(f.state, "environments.yaml"),
[
"environments:",
" staging:",
" server: staging-box",
" team: { id: 0, name: Root Team }",
"github_apps:",
" incubator: hdb-coolify",
"",
].join("\n"),
);
const r = await runCapture(base(f), { stdin: "staging\n" });
expect(r.code).not.toBe(0);
expect(r.output).toMatch(/age_recipient/);
});
// Same position as diff: capture is only ever a claim about something that
// already exists. Against an absent target it would call every secret
// "missing" — an alarming report about the wrong box.
it("refuses an absent target rather than reporting every secret missing", async () => {
const f = fixture((await stubCoolify()).url);
const r = await runCapture([...base(f), "--project", "typo"], {
stdin: "staging\n",
});
expect(r.code).not.toBe(0);
expect(r.output).toMatch(/refusing to capture/);
expect(r.output).toMatch(/no project named "typo"/);
expect(r.output).not.toMatch(/MISSING/);
});
});

309
test/capture.test.ts Normal file
View file

@ -0,0 +1,309 @@
import { mkdirSync, mkdtempSync, writeFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { describe, expect, it } from "vitest";
import {
GENERATED_PLACEHOLDER,
classify,
renderCapturePlan,
} from "../src/capture.js";
import { requiredSecrets } from "../src/resolve.js";
const CTX = {
orgRepo: "heavy-duty/incubator",
env: "prod",
instance: "legacy",
store: "/s/secrets/incubator.prod.env.age",
recipient: "age1abc",
};
// The live case, shrunk: an app whose env template needs a generated database
// URL, a carried-over API key, and an address that must NOT be carried over.
const REQUIRED = [
{ ref: "DATABASE_URL_PROD", resource: "core", key: "DATABASE_URL" },
{ ref: "MAILGUN_API_KEY", resource: "core", key: "MAILGUN_API_KEY" },
{ ref: "ADMIN_EMAIL", resource: "core", key: "ADMIN_EMAIL" },
];
const LIVE = {
core: {
DATABASE_URL: "postgres://SOURCE-BOX-INTERNAL/db",
MAILGUN_API_KEY: "key-abc123-REAL-SECRET",
ADMIN_EMAIL: "founder@real-company.com",
},
};
describe("classify", () => {
it("captures a live value, and records where it came from", () => {
const c = classify([REQUIRED[1]], [], LIVE, {});
expect(c.plan).toEqual([
{
ref: "MAILGUN_API_KEY",
provenance: "captured",
value: "key-abc123-REAL-SECRET",
sites: [{ resource: "core", key: "MAILGUN_API_KEY" }],
},
]);
});
// The failure this verb exists to prevent: the source box's DATABASE_URL
// points at the SOURCE box's Postgres. Copying it over is confidently wrong
// in a way that looks entirely plausible.
it("placeholds a generated name, never copying the source's value", () => {
const c = classify([REQUIRED[0]], ["DATABASE_URL_PROD"], LIVE, {});
expect(c.plan[0]).toMatchObject({
ref: "DATABASE_URL_PROD",
provenance: "generated",
value: GENERATED_PLACEHOLDER,
});
expect(c.plan[0].value).not.toContain("SOURCE-BOX");
});
// staging and prod share a Mailgun domain, so a staging box carrying the
// real ADMIN_EMAIL can mail real users.
it("takes an override from the operator, over the source's value", () => {
const c = classify([REQUIRED[2]], [], LIVE, {
ADMIN_EMAIL: "operator@example.com",
});
expect(c.plan[0]).toMatchObject({
provenance: "overridden",
value: "operator@example.com",
});
});
it("an override beats a generated declaration too", () => {
const c = classify([REQUIRED[0]], ["DATABASE_URL_PROD"], LIVE, {
DATABASE_URL_PROD: "postgres://explicit",
});
expect(c.plan[0]).toMatchObject({
provenance: "overridden",
value: "postgres://explicit",
});
});
// Required by the template, absent from the source: refuse rather than write
// an empty. An empty substitutes to nothing and the app boots misconfigured.
it("refuses on a name required by the template but absent from the source", () => {
const c = classify(
[{ ref: "TURNSTILE_SECRET", resource: "core", key: "TURNSTILE_SECRET" }],
[],
LIVE,
{},
);
expect(c.plan).toEqual([]);
expect(c.missing).toEqual([
{
ref: "TURNSTILE_SECRET",
sites: [{ resource: "core", key: "TURNSTILE_SECRET" }],
},
]);
});
it("a missing name can be rescued by an override", () => {
const c = classify(
[{ ref: "TURNSTILE_SECRET", resource: "core", key: "TURNSTILE_SECRET" }],
[],
LIVE,
{ TURNSTILE_SECRET: "supplied" },
);
expect(c.missing).toEqual([]);
expect(c.plan[0].provenance).toBe("overridden");
});
// One name, two resources, two different live values. The store holds one
// value per name; picking wrong would be silent.
it("refuses when one name carries different values on two resources", () => {
const c = classify(
[
{ ref: "SHARED", resource: "core", key: "SHARED" },
{ ref: "SHARED", resource: "worker", key: "SHARED" },
],
[],
{ core: { SHARED: "a" }, worker: { SHARED: "b" } },
{},
);
expect(c.plan).toEqual([]);
expect(c.conflicts).toEqual([
{
ref: "SHARED",
values: [
{ resource: "core", key: "SHARED" },
{ resource: "worker", key: "SHARED" },
],
},
]);
});
it("is fine when one name carries the SAME value on two resources", () => {
const c = classify(
[
{ ref: "SHARED", resource: "core", key: "SHARED" },
{ ref: "SHARED", resource: "worker", key: "SHARED" },
],
[],
{ core: { SHARED: "same" }, worker: { SHARED: "same" } },
{},
);
expect(c.conflicts).toEqual([]);
expect(c.plan[0].value).toBe("same");
});
// The acceptance criterion: exactly the names the manifest requires, no more
// and no fewer. A live var the manifest does not ask for is not the store's
// business.
it("writes exactly the required names — ignoring live vars nobody asked for", () => {
const c = classify(
REQUIRED,
["DATABASE_URL_PROD"],
{
core: { ...LIVE.core, SOME_OTHER_LIVE_VAR: "not in the manifest" },
},
{},
);
expect(c.plan.map((d) => d.ref).sort()).toEqual([
"ADMIN_EMAIL",
"DATABASE_URL_PROD",
"MAILGUN_API_KEY",
]);
});
});
describe("renderCapturePlan", () => {
// THE invariant. "No secret value is ever written to stdout" — so the plan
// is names and provenance, and the test asserts on the actual live values
// rather than on a pattern that could drift away from them.
it("never prints a secret value", () => {
const c = classify(REQUIRED, ["DATABASE_URL_PROD"], LIVE, {
ADMIN_EMAIL: "operator@example.com",
});
const out = renderCapturePlan(c, CTX);
for (const secret of [
"postgres://SOURCE-BOX-INTERNAL/db",
"key-abc123-REAL-SECRET",
"founder@real-company.com",
"operator@example.com",
]) {
expect(out).not.toContain(secret);
}
});
it("shows every name with its provenance and where it lands", () => {
const c = classify(REQUIRED, ["DATABASE_URL_PROD"], LIVE, {
ADMIN_EMAIL: "operator@example.com",
});
const out = renderCapturePlan(c, CTX);
expect(out).toMatch(/MAILGUN_API_KEY\s+captured\s+core\.MAILGUN_API_KEY/);
expect(out).toMatch(/DATABASE_URL_PROD\s+generated/);
expect(out).toContain(GENERATED_PLACEHOLDER);
expect(out).toMatch(/ADMIN_EMAIL\s+overridden/);
expect(out).toContain("CAST_CAPTURE_ADMIN_EMAIL");
expect(out).toMatch(/3 name\(s\) to write/);
expect(out).toContain("/s/secrets/incubator.prod.env.age");
expect(out).toContain("age1abc");
});
it("names what is missing, and says why an empty would be worse", () => {
const c = classify(
[{ ref: "TURNSTILE_SECRET", resource: "core", key: "TURNSTILE_SECRET" }],
[],
LIVE,
{},
);
const out = renderCapturePlan(c, CTX);
expect(out).toMatch(/TURNSTILE_SECRET\s+MISSING/);
expect(out).toMatch(/refusing to write the store/);
expect(out).toMatch(/--override/);
});
it("names a conflict rather than picking a side", () => {
const c = classify(
[
{ ref: "SHARED", resource: "core", key: "SHARED" },
{ ref: "SHARED", resource: "worker", key: "SHARED" },
],
[],
{ core: { SHARED: "a" }, worker: { SHARED: "b" } },
{},
);
const out = renderCapturePlan(c, CTX);
expect(out).toMatch(/SHARED\s+CONFLICT/);
expect(out).toMatch(/core\.SHARED and worker\.SHARED/);
expect(out).not.toMatch(/\ba\b.*\bb\b/);
});
});
// requiredSecrets is what makes "no more, no fewer" true: the set comes from
// the manifest's own templates, read by the same parser apply uses.
describe("requiredSecrets", () => {
function checkout(manifest: string, templates: Record<string, string>) {
const dir = mkdtempSync(join(tmpdir(), "cast-cap-"));
mkdirSync(join(dir, ".infra", "env"), { recursive: true });
writeFileSync(join(dir, ".infra", "manifest.yaml"), manifest);
for (const [name, body] of Object.entries(templates)) {
writeFileSync(join(dir, ".infra", "env", name), body);
}
return dir;
}
const MANIFEST = `project: incubator
environments:
prod:
generated_secrets: [DATABASE_URL_PROD]
applications:
core:
source: { repo: heavy-duty/incubator, branch: main }
build: { pack: dockercompose, base_directory: /, compose_file: docker-compose.yaml }
service_domains:
api: ["https://api.example.com"]
env_template: core.prod.env.template
services:
umami:
type: umami
env_template: umami.prod.env.template
`;
it("collects the ${...} refs from every app and service template", () => {
const dir = checkout(MANIFEST, {
"core.prod.env.template":
"NODE_ENV=production\nDATABASE_URL=${DATABASE_URL_PROD}\nMAILGUN_API_KEY=${MAILGUN_API_KEY}\n",
"umami.prod.env.template": "APP_SECRET=${UMAMI_APP_SECRET}\n",
});
const { required, generated } = requiredSecrets(dir, "prod");
expect(required).toEqual([
{ ref: "DATABASE_URL_PROD", resource: "core", key: "DATABASE_URL" },
{ ref: "MAILGUN_API_KEY", resource: "core", key: "MAILGUN_API_KEY" },
{ ref: "UMAMI_APP_SECRET", resource: "umami", key: "APP_SECRET" },
]);
expect(generated).toEqual(["DATABASE_URL_PROD"]);
});
// A non-placeholder line (NODE_ENV=production) is not a secret and must not
// land in the store — the store holds the ${...} refs, nothing else.
it("ignores literal template values — only ${...} refs are secrets", () => {
const dir = checkout(MANIFEST, {
"core.prod.env.template":
"NODE_ENV=production\nREPORTING_ENABLED=false\nDATABASE_URL=${DATABASE_URL_PROD}\n",
"umami.prod.env.template": "",
});
const { required } = requiredSecrets(dir, "prod");
expect(required.map((r) => r.ref)).toEqual(["DATABASE_URL_PROD"]);
});
// Dead config in THIS list is dangerous, not merely untidy: it reads like a
// guard standing over a name while standing over nothing, and the likeliest
// cause is a typo whose real name is then CAPTURED from the source box.
it("refuses a generated_secrets entry that no template refers to", () => {
const dir = checkout(
MANIFEST.replace(
"generated_secrets: [DATABASE_URL_PROD]",
"generated_secrets: [DATABASE_URL_TYPO]",
),
{
"core.prod.env.template": "DATABASE_URL=${DATABASE_URL_PROD}\n",
"umami.prod.env.template": "",
},
);
expect(() => requiredSecrets(dir, "prod")).toThrow(
/generated_secrets names DATABASE_URL_TYPO/,
);
});
});