heavy-duty/cast
7
0
Fork 0
Code Issues 4 Pull requests Projects Releases 3 Packages Wiki Activity Actions

feat(capture): --generated-only, the bootstrap's missing pass 2

Browse source 
A manifest that declares `generated_secrets:` bootstraps in two passes by
construction: pass 1 `capture` placeholds those names (their values do not
exist yet), `apply` creates the database and Coolify generates the real URL —
and nothing then taught the store that value. The operator did it by hand:
decrypt a fourteen-name store, edit two lines, re-encrypt to the environment's
age recipient, against production, holding the prod key.

`capture --generated-only` inverts capture's disposition rule and changes
nothing else — it fills the generated names and leaves every other name in the
store exactly as it is, byte for byte. Same verb, same ceremony, same
store-writing code path.

- reads the value from the resource that OWNS it (`internal_db_url` on the
  database), never from a consuming app's env, where a generated URL never
  appears — the app's env holds the placeholder itself at this point.
- resolves the database inside the project+environment via
  GET /projects/{uuid}/{env}, never the instance-wide GET /databases (which
  lists other projects' databases and umami's bundled Postgres — #29's bug in
  another hat). The scoping is structural, not a filter.
- refuses to guess which database a name comes from: nothing in the manifest,
  the templates or the box carries that edge, so it infers only when it cannot
  be wrong (one name, one database) and otherwise hands back `--from`.
- refuses to overwrite a generated name holding a real value without --force
  (a silent credential rotation), a name absent from the store, and a
  placeholder nothing fills.
- asserts the postcondition against the ciphertext on disk: zero
  pending-coolify-generated remain, and the name count is unchanged. That
  assertion was a line in a human runbook.

`apply` deliberately does NOT do this after a create — it would make the verb
that mutates Coolify also mutate the encrypted store, and hence the git repo.

Closes #48.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
This commit is contained in:
claude-hdb 2026-07-14 22:32:07 +00:00
parent 7cc3a3b64e
commit 5e10375837
6 changed files with 1427 additions and 6 deletions
Show stats Download patch file Download diff file Expand all files Collapse all files

69
README.md
View file

@ -68,6 +68,7 @@ cast apply --env <env> --all # no repo: EVERY registered proj
cast diff <org>/<repo> --env <env> [--full]
cast diff --env <env> --all [--full] # no repo: EVERY registered project
cast capture <org>/<repo> --env <env> [--generated <NAME>] [--override <NAME>]
cast capture <org>/<repo> --env <env> --generated-only [--from <NAME>=<db>]
cast inventory <org>/<repo> --env <env>
cast inventory --env <env> [--emit-draft <dir> [--recipient age1…] [--no-secrets]]
cast server add <name> --ip <ip> --key <file> --env <env> [--user root] [--port 22]
@ -104,7 +105,10 @@ cast team [--env <env>]
point-in-time blueprint of a box. See *Drafting a box that was never declared*.
- **`capture`** — the adoption path: reads a hand-built instance's live env and
writes the environment's age store from it. See *Adopting a hand-built
instance* below.
instance* below. With **`--generated-only`** it is instead **pass 2 of a
bootstrap**: run *after* `apply`, it fills the store's provider-generated names
(a Coolify-made `DATABASE_URL`) with the values Coolify generated. See *The
bootstrap is two-pass* below.
- **`server add`** — uploads a server's private key and registers it with Coolify.
- **`smoke`** — contract test against the project's `smoke_target`: proves
Coolify's bulk env endpoint still *upserts* rather than replacing. Run it after
@ -327,6 +331,69 @@ The store is encrypted to the environment's `age_recipient` (add it to
`age` on stdin: it is never a temp file, never on stdout, never in your shell
history. An existing store is not overwritten without `--force`.
## The bootstrap is two-pass: `capture --generated-only`
Those placeheld names are the reason bootstrapping an environment **cannot be one
pass**. The value does not exist until Coolify makes it:
```sh
cast capture heavy-duty/incubator --env prod # 1. the store learns every
# name; generated ones are
# placeheld — nothing has
# created them yet
cast apply heavy-duty/incubator --env prod # 2. Coolify creates the
# database, and generates
# the real URL
cast capture heavy-duty/incubator --env prod \ # 3. the store learns THAT
--generated-only --from DATABASE_URL=incubator-db # value
```
Until step 3 runs, the store says `pending-coolify-generated` while the live value
is real — the exact state in which the next routine `apply` overwrites a working
secret. It is also on the DR path: *rebuild the control plane from state* means
apply-from-nothing, so every generated secret in every store is a placeholder
again. This used to be a hand `age` re-encrypt against production, with the prod
key in a process substitution.
`--generated-only` **inverts** capture's rule and changes nothing else: it fills
the `generated_secrets` names and leaves every other name in the store **exactly as
it is, byte for byte** — never re-read from the box, so a secret you rotated by
hand last month survives it. Same typed confirmation, same names-never-values plan.
The store must already exist: pass 2 *fills* names, it does not create them.
It reads each value from the **database that owns it** (`internal_db_url`) —
never from the consuming app's env, where a generated URL never appears (the app's
env holds what the template resolved to, which at this point is *the placeholder
itself*). And it resolves that database **inside your project and environment
only**, never from the instance-wide `GET /databases` list, which holds every
database on the box — other projects', and umami's own bundled Postgres.
**It will not guess which database a name comes from.** Nothing carries that edge:
`generated_secrets:` is a flat list of names, and the template knows only
`DATABASE_URL=${DATABASE_URL}`. So cast infers it only when it *cannot* be wrong
(one name, one database) and otherwise refuses, printing the flag you need. A pick
made from the *name* (`REDIS_URL` → the redis one) is wrong **silently**, and what
it writes is a well-formed URL to somebody else's database.
Four refusals, each a thing that used to be a step in a runbook:
| | |
| --- | --- |
| **UNMAPPED** | more than one database could be meant → say which, with `--from` |
| **OCCUPIED** | the name already holds a *real* value → filling it silently rotates a live credential. `--force` to mean it |
| **ABSENT** | the name is not in the store at all → pass 1 has not run, or you are pointed at the wrong store |
| **PENDING** | a placeholder in a name nothing here fills → the store would still be a lie |
Afterwards it **asserts the postcondition**, against the ciphertext now on disk: zero
`pending-coolify-generated` remain, and the name count is unchanged. A store that
lost a name re-encrypts perfectly and reads back perfectly — you would find out at
the next `apply`, in an environment whose plaintext nobody has any more.
`apply` deliberately does not do this for you after a create. It would close the
window entirely, but it would make the verb that mutates Coolify also mutate the
encrypted store — and hence the git repo — which is a much bigger blast radius for
a verb people run on a schedule.
**[docs/semantics.md](docs/semantics.md)** is the contract behind those
commands: what `apply` guarantees (never deletes, never recreates a database,
fails loudly rather than recreating on un-updatable drift), the `dockercompose`

73
docs/semantics.md
View file

@ -471,6 +471,79 @@ the plan. There is no `--yes`: a store written without someone reading the
provenance column is the outcome the verb exists to prevent. A closed stdin
aborts rather than hanging.
### Pass 2 (`capture --generated-only`)
An environment that declares `generated_secrets:` **bootstraps in two passes, by
construction** — the value does not exist until Coolify makes it:
capture → apply → capture --generated-only
(placeholds) (Coolify generates) (the store learns the real value)
Without pass 2 the store's value for `DATABASE_URL` stays a placeholder while the
live value is real — which is exactly the state in which the next routine `apply`
overwrites a working secret (#47). **The absence of pass 2 is what leaves that gun
loaded**, and it is on the DR path: *rebuild the control plane from state* means
apply-from-nothing, which means every generated secret in every store is a
placeholder again. This used to be a hand `age` re-encrypt against production —
decrypt a fourteen-name store, edit two lines, re-encrypt to the environment's
recipient, holding the prod key, with a `jq` filter that must not pick the wrong row.
`--generated-only` **inverts** capture's disposition rule and changes nothing else:
the names in `generated_secrets` are the ones it *fills*, and every other name is
left **exactly as the store has it, byte for byte** — never re-read from the box,
which is what makes it safe to run against an environment whose other secrets have
since been rotated by hand. Same verb, same store-writing code path, same typed
confirmation.
| | |
| --- | --- |
| **fill** | a generated name holding the placeholder → the value from the database that owns it |
| **keep** | every other name → carried over from the store, untouched |
| **UNMAPPED** | cast cannot attribute the name to exactly one database → **refuses** |
| **OCCUPIED** | a generated name already holding a real value → **refuses** (without `--force`) |
| **ABSENT** | a generated name the store does not carry at all → **refuses** |
| **PENDING** | a placeholder in a name nothing here fills → **refuses** |
**The value is read from the resource that OWNS it.** A generated URL never appears
on the consuming application's env — the app's env holds whatever the template
resolved to, which at this point in the bootstrap is *the placeholder itself*.
Reading the app back would faithfully capture the lie pass 2 exists to correct. It
lives on the **database**, as `internal_db_url`.
**Resolved inside the project + environment, never instance-wide.** cast reads
`GET /projects/{uuid}/{env}`, whose `postgresqls` / `redis` relations are *this*
environment's and nothing else's. It never calls `GET /databases`, which lists
every database on the box — other projects', and umami's own bundled Postgres —
where picking ours out means matching by name across a list in which a collision
is both possible and silent (#29 in another hat, and the reason the hand-run `jq`
carried a comment about not taking the third row). The scoping is **structural**
rather than a filter cast has to remember to get right.
**cast will not guess which database a name comes from.** Nothing in the system
carries that edge: `generated_secrets:` is a flat list of *names*, the env template
knows only `DATABASE_URL=${DATABASE_URL}`, and the box does not say. So the
inference is made **only when it cannot be wrong** — one generated name, one
database, no other candidate — and otherwise the run refuses and hands back the
flag: `--from DATABASE_URL=incubator-db`. Reading the type out of the *name*
(`REDIS_URL` → the redis one) is precisely the bug this must not have: a
name-directed pick is wrong **silently**, and what it writes is a perfectly
well-formed URL to somebody else's database.
**And it asserts the postcondition it exists for** — against the ciphertext now on
disk, decrypted back, not trusted from memory. Zero `pending-coolify-generated`
remain, and the name count is unchanged. Both claims, because they fail in opposite
directions: a store that still holds a placeholder is still a lie, and a store that
*lost* a name on the way through re-encrypts perfectly, reads back perfectly, and
surfaces at the next `apply` as a missing secret in an environment whose plaintext
nobody has any more. That assertion used to be a line in a human runbook — which is
to say, a step that could be skipped.
`apply` deliberately does **not** do this automatically after a create. It would
close the window entirely, but it would make the verb that mutates Coolify also
mutate the encrypted store and hence the git repo — a much bigger blast radius for
a verb people run on a schedule. A separate, explicit, operator-run verb is the
right first step.
## Drafts (`inventory --emit-draft`)
`inventory` with no repo sweeps an instance. `--emit-draft <dir>` writes that

346
src/capture.ts
View file

@ -239,3 +239,349 @@ export function renderCapturePlan(
}
return lines.join("\n");
}
// ---------------------------------------------------------------------------
// Pass 2 — `capture --generated-only`
//
// Bootstrapping an environment that declares generated_secrets is two-pass BY
// CONSTRUCTION, and the two passes are not variations of one verb:
//
// pass 1 `capture` the store learns every name; the generated
// ones are placeheld, because their values do
// not exist yet — nothing has created them.
// (apply) Coolify creates the database and generates
// the real URL.
// pass 2 `capture --generated-only` the store learns THAT value.
//
// Between the two, the store's value for a generated name is a placeholder while
// the live value is real. Everything below exists to close that window without
// the operator hand-editing decrypted plaintext (which is how it was closed
// before: decrypt, edit two lines, re-encrypt, against prod, holding the key).
//
// The flag INVERTS classify()'s disposition rule and nothing else: the names in
// generated_secrets are the ones it fills, and every other name is left exactly
// as the store has it — byte for byte, never re-read from the box. A generated
// name is the only kind of name whose true value cast can find AFTER the fact,
// because it is the only kind the provider owns.
// ---------------------------------------------------------------------------
// A live database inside the project + environment being filled — the resource
// that OWNS a generated URL.
//
// `url` is `internal_db_url`, and it is read from the DATABASE, never from a
// consuming application's env. It never appears there: the app's env holds
// whatever the template resolved to, which at this point in the bootstrap is
// the placeholder itself. Reading the app back would faithfully capture the
// lie we are here to correct.
export type GeneratedSource = {
resource: string;
type: string;
url: string;
};
// One generated name, filled from the resource that owns it.
export type Fill = {
ref: string;
from: { resource: string; type: string };
// Never rendered. Kept here so the caller can encrypt it, and nowhere else —
// same contract as Disposition.value.
value: string;
};
export type GeneratedPlan = {
fills: Fill[];
// Names the store keeps, byte for byte. Printed as names so the operator can
// see the store is not being rewritten around them.
kept: string[];
// A generated name cast cannot attribute to exactly one database. Refuses:
// picking would be silent, and picking WRONG writes another box's credentials
// into this environment's store.
unmapped: Array<{ ref: string; why: string }>;
// A generated name whose store value is NOT the placeholder. Refuses without
// --force: it is already filled (someone ran pass 2, or set it by hand), and
// overwriting it is a silent rotation of a live credential.
occupied: string[];
// A generated name the store does not carry AT ALL. Refuses: pass 2 fills
// names, it does not invent them. A store missing one did not come from pass
// 1, and the name-count postcondition below could not hold anyway.
absent: string[];
// A store value that is still the placeholder and is NOT in the fill set —
// so this run would leave the store still lying. Refuses. This is the
// postcondition, checked BEFORE the write so it reads as a plan and not as an
// assertion failure over ciphertext already on disk.
stillPending: string[];
};
// Which database does a generated name come from?
//
// The manifest cannot say. `generated_secrets:` is a flat list of NAMES, and
// there is no edge anywhere in the data model from DATABASE_URL to the database
// that owns it — not in the manifest, not in the env template (which knows only
// `DATABASE_URL=${DATABASE_URL}`), and not on the box.
//
// So cast does not guess. It infers ONLY when the inference cannot be wrong —
// one generated name, one database, no other candidate — and otherwise refuses
// and makes the operator state the edge with --from. The temptation is to read
// the type out of the NAME (REDIS_URL → the redis one), and that is precisely
// the bug this verb must not have: a name-directed pick across a list of
// databases is #29 wearing a different hat, and it is wrong SILENTLY. The
// value it would write is a perfectly well-formed URL to somebody else's
// database.
export function resolveGeneratedSources(
generated: string[],
databases: GeneratedSource[],
from: Record<string, string>,
): {
mapping: Record<string, GeneratedSource>;
unmapped: Array<{ ref: string; why: string }>;
} {
const mapping: Record<string, GeneratedSource> = {};
const unmapped: Array<{ ref: string; why: string }> = [];
const byName = new Map(databases.map((d) => [d.resource, d]));
for (const ref of generated) {
const named = from[ref];
if (named !== undefined) {
const hit = byName.get(named);
if (!hit) {
unmapped.push({
ref,
why: `--from ${ref}=${named}, but no database named "${named}" exists in this project+environment`,
});
continue;
}
mapping[ref] = hit;
continue;
}
if (databases.length === 0) {
unmapped.push({
ref,
why: "no database exists in this project+environment to fill it from",
});
continue;
}
// The only inference that cannot be wrong: nothing else it could be.
if (generated.length === 1 && databases.length === 1) {
mapping[ref] = databases[0];
continue;
}
unmapped.push({
ref,
why: `${databases.length} databases here (${databases.map((d) => `${d.resource}:${d.type}`).join(", ")}) — cast will not pick by name`,
});
}
return { mapping, unmapped };
}
// Invert the disposition rule. `store` is the DECRYPTED current store; every
// name in it that is not being filled comes out the other side untouched.
export function planGenerated(
generated: string[],
store: Record<string, string>,
sources: Record<string, GeneratedSource>,
unmapped: Array<{ ref: string; why: string }>,
opts: { force: boolean } = { force: false },
): GeneratedPlan {
const generatedSet = new Set(generated);
const fills: Fill[] = [];
const occupied: string[] = [];
const absent: string[] = [];
for (const ref of generated) {
if (!(ref in store)) {
absent.push(ref);
continue;
}
const source = sources[ref];
// Already reported by resolveGeneratedSources; not also an "occupied".
if (!source) continue;
if (store[ref] !== GENERATED_PLACEHOLDER && !opts.force) {
occupied.push(ref);
continue;
}
fills.push({
ref,
from: { resource: source.resource, type: source.type },
value: source.url,
});
}
const filling = new Set(fills.map((f) => f.ref));
// A placeholder left standing in a name nobody is filling. The run would
// "succeed" and the store would still be a lie — so it refuses instead, and
// names the flag that fixes it. Generated names that failed to map or were
// refused as occupied are NOT reported here: they already have their own row,
// and one problem should be named once.
const stillPending = Object.keys(store)
.filter(
(k) =>
store[k] === GENERATED_PLACEHOLDER &&
!filling.has(k) &&
!generatedSet.has(k),
)
.sort();
// Everything the store keeps byte for byte. A name carrying a refusal above
// is not "kept" — it is the reason the run stops, and printing it in both
// rows would read as though cast had a plan for it.
const flagged = new Set([
...unmapped.map((u) => u.ref),
...occupied,
...stillPending,
]);
const kept = Object.keys(store)
.filter((k) => !filling.has(k) && !flagged.has(k))
.sort();
return { fills, kept, unmapped, occupied, absent, stillPending };
}
export function generatedPlanRefuses(p: GeneratedPlan): boolean {
return (
p.unmapped.length > 0 ||
p.occupied.length > 0 ||
p.absent.length > 0 ||
p.stillPending.length > 0
);
}
// Names, provenance and the resource a value came FROM. Never values.
//
// capture.ts's rule holds unchanged: the only value-shaped thing printed is
// GENERATED_PLACEHOLDER, a literal constant in this file — and here it is
// printed as the thing being REPLACED, which is the one fact about the store's
// current contents an operator needs in order to believe the plan.
export function renderGeneratedPlan(
p: GeneratedPlan,
ctx: {
orgRepo: string;
env: string;
instance: string;
store: string;
recipient: string;
project: string;
environment: string;
},
): string {
const lines = [
`capture --generated-only — ${ctx.orgRepo} ${ctx.env}`,
"",
` source: instance ${ctx.instance}, project "${ctx.project}", environment "${ctx.environment}"`,
` store: ${ctx.store}`,
` recipient: ${ctx.recipient}`,
"",
];
const width = Math.max(
0,
...[
...p.fills.map((f) => f.ref),
...p.kept,
...p.occupied,
...p.absent,
...p.unmapped.map((u) => u.ref),
].map((r) => r.length),
);
for (const f of p.fills) {
lines.push(
` ${f.ref.padEnd(width)} fill ${GENERATED_PLACEHOLDER} → ${f.from.resource} (${f.from.type}) internal_db_url`,
);
}
for (const k of p.kept) {
lines.push(` ${k.padEnd(width)} keep unchanged in the store`);
}
for (const u of p.unmapped) {
lines.push(` ${u.ref.padEnd(width)} UNMAPPED ${u.why}`);
}
for (const o of p.occupied) {
lines.push(
` ${o.padEnd(width)} OCCUPIED already holds a value that is not ${GENERATED_PLACEHOLDER}`,
);
}
for (const a of p.absent) {
lines.push(` ${a.padEnd(width)} ABSENT not in the store at all`);
}
for (const s of p.stillPending) {
lines.push(
` ${s.padEnd(width)} PENDING still ${GENERATED_PLACEHOLDER}, and nothing here fills it`,
);
}
lines.push(
"",
`${p.fills.length} name(s) to fill, ${p.kept.length} left exactly as the store has them`,
);
if (p.unmapped.length > 0) {
lines.push(
"",
`refusing to write the store: ${p.unmapped.length} generated name(s) cannot be attributed`,
"to exactly one database in this project+environment. Nothing in the manifest, the env",
"template or the box says which database a given name comes from, and cast will not",
"pick by name — a wrong pick writes another database's credentials into this store,",
"and it does it silently. State the edge:",
"",
...p.unmapped.map((u) => ` --from ${u.ref}=<database name>`),
);
}
if (p.occupied.length > 0) {
lines.push(
"",
`refusing to write the store: ${p.occupied.length} generated name(s) already hold a real`,
"value. Filling them would rotate a live credential — silently, and against whatever is",
"already running on that value. If Coolify's resource really was recreated and the store",
"is stale, say so with --force.",
);
}
if (p.absent.length > 0) {
lines.push(
"",
`refusing to write the store: ${p.absent.length} generated name(s) are not in the store at`,
"all. --generated-only FILLS names, it does not add them: the store it fills is the one",
"pass 1 wrote, and the name set must come out of this run exactly as it went in. Run",
"`cast capture` first, or check you are pointed at the right store.",
);
}
if (p.stillPending.length > 0) {
lines.push(
"",
`refusing to write the store: ${p.stillPending.length} name(s) would still hold the`,
`${GENERATED_PLACEHOLDER} literal after this run — the store would still be a lie, and`,
"the next apply would push that literal at a live app. If these are generated too,",
"declare them (manifest `generated_secrets:`, or --generated <NAME>).",
);
}
return lines.join("\n");
}
// The postcondition this verb EXISTS for, asserted against the store that was
// actually written — decrypted back off disk, not against the map cast held in
// memory a moment ago. In the hand-run procedure this was a line in a runbook
// ("assert 14 names / zero placeholders"), which is to say it was a step that
// could be, and eventually would be, skipped.
//
// Two claims, and they fail in opposite directions:
// - zero placeholders remain → the store no longer lies about any name
// - the name count is unchanged → and it did not lose one on the way
//
// A store that lost a name re-encrypts perfectly and reads back perfectly; the
// failure surfaces at the next apply, as a missing secret, in an environment
// whose plaintext nobody has any more.
export function assertGeneratedComplete(
before: Record<string, string>,
after: Record<string, string>,
): string[] {
const violations: string[] = [];
const pending = Object.keys(after)
.filter((k) => after[k] === GENERATED_PLACEHOLDER)
.sort();
if (pending.length > 0) {
violations.push(
`${pending.length} name(s) still hold the ${GENERATED_PLACEHOLDER} literal: ${pending.join(", ")}`,
);
}
const beforeNames = Object.keys(before).sort();
const afterNames = Object.keys(after).sort();
if (beforeNames.length !== afterNames.length) {
violations.push(
`the store went in with ${beforeNames.length} name(s) and came out with ${afterNames.length}`,
);
}
const lost = beforeNames.filter((n) => !(n in after));
const gained = afterNames.filter((n) => !(n in before));
if (lost.length > 0) violations.push(`names LOST: ${lost.join(", ")}`);
if (gained.length > 0) violations.push(`names ADDED: ${gained.join(", ")}`);
return violations;
}

279
src/cli.ts
View file

@ -14,11 +14,17 @@ import {
smokeTargetFor,
} from "./bindings.js";
import {
type GeneratedSource,
type LiveEnvs,
absentResources,
assertGeneratedComplete,
classify,
generatedPlanRefuses,
planGenerated,
renderAbsentResources,
renderCapturePlan,
renderGeneratedPlan,
resolveGeneratedSources,
} from "./capture.js";
import {
type CoolifyInstance,
@ -88,6 +94,7 @@ const USAGE = `usage: cast apply <org>/<repo> --env <env> [--path <dir>] [--
cast diff <org>/<repo> --env <env> [--full] [--project <name>] [--environment <name>]
cast diff --env <env> --all [--full] # no repo: EVERY registered project
cast capture <org>/<repo> --env <env> [--path <dir>] [--project <name>] [--environment <name>] [--generated <NAME>] [--override <NAME>] [--force]
cast capture <org>/<repo> --env <env> --generated-only [--from <NAME>=<db>] [--force] # pass 2, AFTER apply
cast inventory <org>/<repo> --env <env> [--path <dir>] [--project <name>] [--environment <name>] [--resource <m>=<l>]
cast inventory --env <env> [--instance <name>] # no repo: SWEEP the whole instance
cast inventory --env <env> --emit-draft <dir> [--recipient age1…] [--no-secrets]
@ -152,6 +159,24 @@ capture (adopt a hand-built instance into the age secret store):
command line — argv is visible in \`ps\`. Repeatable.
--force overwrite an existing store (refused by default).
capture --generated-only (PASS 2 — run it AFTER \`apply\` has created the resources):
a manifest with \`generated_secrets:\` bootstraps in two passes, because the value
does not exist until Coolify makes it: pass 1 \`capture\` placeholds those names,
\`apply\` creates the database, and this fills the store with the URL Coolify then
generated. It INVERTS capture's rule — it fills the generated names and leaves
every other name in the store exactly as it is. The store must already exist.
The value is read from the DATABASE that owns it (\`internal_db_url\`), resolved
inside this project+environment only — never from a consuming app's env, where a
generated URL never appears, and never from the instance-wide database list.
--from <NAME>=<db> which live database NAME is filled from. Required whenever
more than one database could be meant: nothing in the
manifest, the templates or the box says that DATABASE_URL
comes from the postgres one, and cast refuses to guess by
name rather than write another database's credentials into
your store. Repeatable.
--force fill a generated name that already holds a REAL value
(refused by default — it is a silent credential rotation).
inventory --emit-draft (write down what a box has, as a PROPOSAL):
--emit-draft <dir> emit what the sweep saw as a draft of cast's own inputs — a
manifest per project, env templates, an environments.yaml
@ -574,6 +599,48 @@ export function parseResourceAliases(
return alias;
}
// `--from <NAME>=<database>` — the edge nothing else in the system carries.
//
// The right side is the database as COOLIFY names it, in this project and
// environment (which is what `capture --generated-only`'s own refusal prints
// for you). Not the manifest's name: --resource exists to reconcile those two
// vocabularies for the diff, and pass 2 reads its value straight off the live
// resource, so the live name is the one that can be checked.
export function parseFromPairs(
pairs: string[],
generated: string[],
): Record<string, string> {
const out: Record<string, string> = {};
for (const pair of pairs) {
const eq = pair.indexOf("=");
if (eq <= 0 || eq === pair.length - 1) {
throw new Error(`--from expects <NAME>=<database-name>, got "${pair}"`);
}
const ref = pair.slice(0, eq).trim();
const db = pair.slice(eq + 1).trim();
// A --from naming something that is not a generated secret is a no-op that
// LOOKS like it did something: pass 2 fills generated names and nothing
// else, so the flag would be silently ignored and the operator would walk
// away believing they had set a value.
if (!generated.includes(ref)) {
throw new Error(
[
`--from ${ref}=${db}: ${ref} is not a generated secret in this environment`,
"",
` generated: ${generated.join(", ") || "(none declared)"}`,
"",
"--generated-only fills the generated names only. A name that is not one of",
"them is carried over from the store untouched, and --from cannot change that.",
"Declare it in the manifest's `generated_secrets:` (or pass --generated <NAME>)",
"if it really is provider-generated.",
].join("\n"),
);
}
out[ref] = db;
}
return out;
}
// Rename live resources to the manifest's vocabulary, once, at the boundary.
// Everything downstream — computeDiff, classify, reconcile — then matches by
// name as it always has, and none of them needs to know a box was involved.
@ -611,6 +678,62 @@ async function fetchEnv(
return Object.fromEntries(envs.map((e) => [e.key, e.real_value ?? e.value]));
}
// The databases inside ONE project+environment, each carrying the value it
// OWNS. This is `capture --generated-only`'s only read of the box.
//
// Deliberately NOT `GET /databases`. That route lists every database on the
// INSTANCE — other projects', and umami's own bundled Postgres — so finding
// ours in it means matching by name across a list where a collision is both
// possible and silent (#29 in another hat; the hand-run jq this verb replaces
// had a comment warning not to pick the third row). `GET /projects/{uuid}/{env}`
// cannot express that bug: it eager-loads `postgresqls` and `redis` for THIS
// environment and nothing else (ProjectController@environment_details,
// coollabsio/coolify v4.1.2), so the scoping is structural rather than a filter
// cast has to remember to get right.
//
// `internal_db_url` is an appended model attribute — `protected $appends =
// ['internal_db_url', 'external_db_url', 'database_type', 'server_status']` on
// BOTH app/Models/StandalonePostgresql.php and app/Models/StandaloneRedis.php
// @ v4.1.2. Same key on both; only the URL it builds differs
// (`postgres://user:pw@{uuid}:5432/{db}` vs `redis://user:pw@{uuid}:6379/0`).
// environment_details serializes the models whole — serializeApiResponse
// (bootstrap/helpers/api.php) only sorts keys, and unlike DatabasesController
// it calls no removeSensitiveData() — so the field is present here WITHOUT the
// sensitive-read token permission that `GET /databases` gates it behind
// (`can_read_sensitive` → makeHidden(['internal_db_url', …])). The vendored
// OpenAPI documents neither route's body ("Content is very complex. Will be
// implemented later."); the spec's silence is not evidence of absence (#46).
async function fetchGeneratedSources(
client: CoolifyClient,
projectName: string,
envName: string,
): Promise<{ sources: GeneratedSource[]; urlless: string[] }> {
const uuid = await client.projectUuid(projectName);
const raw = (await client.get(
`/projects/${uuid}/${encodeURIComponent(envName)}`,
)) as {
postgresqls?: Array<Record<string, unknown>>;
redis?: Array<Record<string, unknown>>;
} | null;
const sources: GeneratedSource[] = [];
const urlless: string[] = [];
const take = (type: string, items: Array<Record<string, unknown>> = []) => {
for (const i of items) {
const url = i.internal_db_url;
// A database that is THERE but will not tell us its URL. Never a fill of
// "" — that re-encrypts cleanly and boots the app pointed at nothing.
if (typeof url !== "string" || url === "") {
urlless.push(String(i.name));
continue;
}
sources.push({ resource: String(i.name), type, url });
}
};
take("postgresql", raw?.postgresqls);
take("redis", raw?.redis);
return { sources, urlless };
}
// The value for an --override, read from the ENVIRONMENT rather than argv.
//
// A secret passed as a command-line argument is visible in `ps` to every
@ -1030,6 +1153,8 @@ async function main(): Promise<number> {
generated: { type: "string", multiple: true },
override: { type: "string", multiple: true },
force: { type: "boolean", default: false },
"generated-only": { type: "boolean", default: false },
from: { type: "string", multiple: true },
},
});
const orgRepo = positionals[0];
@ -1038,6 +1163,9 @@ async function main(): Promise<number> {
console.error(USAGE);
return 2;
}
// Pass 2 of a two-pass bootstrap. Not a different verb: same ceremony, same
// store-writing code path, one inverted disposition rule. See capture.ts.
const generatedOnly = values["generated-only"];
const stateDir = stateDirFrom(values.state);
const repoShort = orgRepo.split("/")[1];
const projectName = values.project ?? repoShort;
@ -1047,11 +1175,63 @@ async function main(): Promise<number> {
// vocabulary.
const coolifyEnv = values.environment ?? envName;
const store = secretsFileFor(stateDir, repoShort, envName);
// Flag pairings that can never be honored, refused up front — before a state
// file, a store, an age key or a Coolify is opened (same disposition as
// PATH_IN_PROD_REFUSAL). --override supplies a value for a name cast would
// otherwise CAPTURE, and --generated-only captures nothing; --from names the
// database a GENERATED name comes from, and only pass 2 fills those.
if (generatedOnly && (values.override ?? []).length > 0) {
console.error(
"refuses --override with --generated-only: pass 2 fills the generated names and leaves every other name exactly as the store has it — there is nothing for an override to override. Set the value in pass 1 (`cast capture --override`), or edit it there.",
);
return 2;
}
if (!generatedOnly && (values.from ?? []).length > 0) {
console.error(
"refuses --from without --generated-only: --from names the database a generated secret is filled FROM, and plain `capture` never fills one — it placeholds them (that is the point of pass 1).",
);
return 2;
}
// --resource reconciles the MANIFEST's vocabulary with the box's for the
// env-reading pass, and pass 2 reads no env: it takes its value straight off
// the live database, which --from names in the box's own vocabulary. Left
// accepted, the flag would be silently ignored — the exact "the flag missed
// and nothing said so" failure parseResourceAliases refuses for.
if (generatedOnly && (values.resource ?? []).length > 0) {
console.error(
"refuses --resource with --generated-only: pass 2 reads no application env, so there is no manifest-to-box name mapping for it to use. --from names the live database directly, in the box's own vocabulary.",
);
return 2;
}
// The two passes take OPPOSITE positions on the store, and both are the same
// rule: never destroy values that exist nowhere else.
//
// Pass 1 writes the store from nothing, so an existing one is something it
// must not clobber. Pass 2 fills names INTO the store pass 1 wrote, so an
// absent one is not a blank slate — it means this run is pointed somewhere
// unexpected, and writing would produce a store holding two names out of
// fourteen.
if (generatedOnly && !existsSync(store)) {
console.error(
[
`refusing to capture --generated-only: ${store} does not exist`,
"",
"Pass 2 FILLS the generated names in a store that pass 1 already wrote — it does",
"not create one. A store written from here would hold only the generated names,",
"and every other name the manifest requires would be silently absent from it.",
"",
"Run `cast capture` first (pass 1), then `apply`, then this.",
].join("\n"),
);
return 2;
}
// Never overwrite a store by accident. `apply` never deletes; the verb
// that WRITES the store gets the same disposition, because the thing it
// would destroy is the only copy of values that may not exist anywhere
// else any more.
if (existsSync(store) && !values.force) {
// else any more. (Pass 2 is exempt: it REQUIRES the store to exist, and
// reuses --force for the finer refusal — overwriting a generated name that
// already holds a real value. See planGenerated.)
if (!generatedOnly && existsSync(store) && !values.force) {
console.error(
[
`refusing to capture: ${store} already exists`,
@ -1113,6 +1293,101 @@ async function main(): Promise<number> {
);
return 2;
}
if (generatedOnly) {
// Pass 2 needs the age IDENTITY, not just the recipient: it fills names
// into a store it must first read. Everything it does not fill is carried
// over from here byte for byte — never re-read from the box, which is what
// makes this safe to run against a live environment whose other secrets
// have since been rotated by hand.
const keyFile = keyFileFor(envName);
const before = decryptSecrets(store, keyFile);
const generatedNames = [
...new Set([...generated, ...(values.generated ?? [])]),
];
const { sources, urlless } = await fetchGeneratedSources(
client,
projectName,
coolifyEnv,
);
// A database that exists but will not report its URL. The only way this
// happens on this route is a Coolify whose shape we do not know — so it
// stops, rather than filling a name with something that is not a URL.
if (urlless.length > 0) {
console.error(
[
`refusing to capture --generated-only: ${urlless.length} database(s) report no internal_db_url`,
"",
` ${urlless.join(", ")}`,
"",
"`internal_db_url` is an appended attribute on Coolify's StandalonePostgresql /",
"StandaloneRedis models (v4.1.2) and this route serializes them whole, so its",
"absence means this Coolify is not the shape cast knows. Filling a secret with an",
"empty value would re-encrypt cleanly and boot the app pointed at nothing.",
].join("\n"),
);
return 2;
}
const { mapping, unmapped } = resolveGeneratedSources(
generatedNames,
sources,
parseFromPairs(values.from ?? [], generatedNames),
);
const plan = planGenerated(generatedNames, before, mapping, unmapped, {
force: values.force,
});
console.log(
renderGeneratedPlan(plan, {
orgRepo,
env: envName,
instance: values.instance ?? binding.instance ?? "default",
store,
recipient,
project: projectName,
environment: coolifyEnv,
}),
);
if (generatedPlanRefuses(plan)) return 2;
if (plan.fills.length === 0) {
console.log(
"\nnothing to fill — this environment declares no generated secrets, and no name in the store is still pending.",
);
return 0;
}
if (!(await confirmCapture(envName))) {
console.error("aborted — nothing written");
return 2;
}
encryptSecrets(recipient, store, {
...before,
...Object.fromEntries(plan.fills.map((f) => [f.ref, f.value])),
});
// The postcondition this verb exists for, asserted against the ciphertext
// that is now on disk — decrypted back, not trusted from memory. In the
// hand-run procedure this was a line in a runbook, which is to say a step
// that could be skipped, and was only ever as good as the operator's
// attention at the end of a long careful thing.
const after = decryptSecrets(store, keyFile);
const violations = assertGeneratedComplete(before, after);
if (violations.length > 0) {
console.error(
[
"",
`POSTCONDITION FAILED — ${store} was written, and it is not what it should be:`,
"",
...violations.map((v) => ` - ${v}`),
"",
"This store is suspect. Do not apply from it. Restore the previous ciphertext",
"from the state repo (it is committed) and report this — cast wrote a store whose",
"shape it does not itself accept, which is a bug in cast, not in your invocation.",
].join("\n"),
);
return 2;
}
console.log(
`\nwrote ${store} — ${plan.fills.length} name(s) filled, ${Object.keys(after).length} name(s) total (unchanged), zero pending-coolify-generated remaining, encrypted to ${recipient}`,
);
return 0;
}
const aliases = parseResourceAliases(
values.resource ?? [],
manifestResources(checkout, envName).map((r) => r.name),

379
test/capture-cli.test.ts
View file

@ -11,7 +11,7 @@ import type { AddressInfo } from "node:net";
import { tmpdir } from "node:os";
import { join } from "node:path";
import { afterEach, beforeAll, describe, expect, it } from "vitest";
import { decryptSecrets } from "../src/secrets.js";
import { decryptSecrets, encryptSecrets } from "../src/secrets.js";
// End-to-end: the real CLI, a real age identity, a stub Coolify holding real
// live values. The point is the store that comes out the other side — it is
@ -106,10 +106,16 @@ OPENROUTER_API_KEY=\${OPENROUTER_API_KEY}
ADMIN_EMAIL=\${ADMIN_EMAIL}
`;
function fixture(url: string, opts: { template?: string } = {}) {
function fixture(
url: string,
opts: { template?: string; manifest?: string } = {},
) {
const checkout = mkdtempSync(join(tmpdir(), "cast-co-"));
mkdirSync(join(checkout, ".infra", "env"), { recursive: true });
writeFileSync(join(checkout, ".infra", "manifest.yaml"), MANIFEST);
writeFileSync(
join(checkout, ".infra", "manifest.yaml"),
opts.manifest ?? MANIFEST,
);
writeFileSync(
join(checkout, ".infra", "env", "core.staging.env.template"),
opts.template ?? TEMPLATE,
@ -332,3 +338,370 @@ describe("cast capture (end to end)", () => {
expect(r.output).not.toMatch(/MISSING/);
});
});
// ---------------------------------------------------------------------------
// Pass 2 — `cast capture --generated-only` (end to end)
//
// The bootstrap this closes: pass 1 wrote a store in which the generated names
// are placeholders, `apply` then created the database, and Coolify generated the
// real URL. Until this runs, the store's value is a lie while the live value is
// real — which is the state that makes the next routine apply overwrite a
// working secret (#47). Everything here runs against a real age identity and a
// real `dist/cli.js`; the store is decrypted afterwards and asserted on.
// ---------------------------------------------------------------------------
// The values Coolify generated when it created the resources. NOT reachable from
// any application's env — the app's env holds what the template resolved to,
// which at this point in the bootstrap is the placeholder itself.
const PG_URL = "postgres://postgres:GENERATED-PG-PASSWORD@abc123:5432/app";
const REDIS_URL = "redis://default:GENERATED-REDIS-PASSWORD@def456:6379/0";
// Another project's database entirely. `GET /databases` would list it right next
// to ours — it is umami's bundled Postgres, the row the hand-run `jq` had to be
// careful not to pick. A name-directed lookup across the instance eventually
// takes it, and what it writes is a well-formed URL to the wrong database.
const UMAMI_URL = "postgres://umami:UMAMI-PASSWORD@zzz999:5432/umami";
const GEN_MANIFEST = `project: incubator
environments:
staging:
generated_secrets: [DATABASE_URL, REDIS_URL]
applications:
core:
source: { repo: heavy-duty/incubator, branch: main }
build: { pack: nixpacks, base_directory: / }
domains: ["http://core.example.com"]
env_template: core.staging.env.template
databases:
incubator-db: { type: postgresql }
incubator-redis: { type: redis }
`;
const GEN_TEMPLATE = `NODE_ENV=production
DATABASE_URL=\${DATABASE_URL}
REDIS_URL=\${REDIS_URL}
MAILGUN_API_KEY=\${MAILGUN_API_KEY}
`;
// What pass 1 left behind: the two generated names placeheld, the real secret
// captured. Pass 2 must fill the first two and not disturb the third.
const PASS1_STORE = {
DATABASE_URL: "pending-coolify-generated",
REDIS_URL: "pending-coolify-generated",
MAILGUN_API_KEY: "key-REAL-MAILGUN-SECRET",
};
// Records every path the CLI asks for, so a test can assert on what cast did
// NOT call — `GET /databases` is instance-wide, and never reaching for it is the
// point, not an implementation detail.
async function stubCoolifyWithDatabases(): Promise<Stub & { hits: string[] }> {
const hits: string[] = [];
const server = createServer((req, res) => {
const path = (req.url ?? "").replace("/api/v1", "");
hits.push(path);
const json = (body: unknown) => {
res.writeHead(200, { "content-type": "application/json" });
res.end(JSON.stringify(body));
};
if (path === "/teams/current") return json({ id: 0, name: "Root Team" });
if (path === "/projects")
return json([
{ uuid: "p1", name: "incubator" },
{ uuid: "p2", name: "analytics" },
]);
// The project+environment document. Coolify's environment_details eager-loads
// these relations and serializes the models whole, so `internal_db_url` (an
// appended attribute on StandalonePostgresql / StandaloneRedis) rides along.
if (path === "/projects/p1/staging")
return json({
applications: [{ name: "core", uuid: "a1" }],
postgresqls: [
{ name: "incubator-db", uuid: "abc123", internal_db_url: PG_URL },
],
redis: [
{
name: "incubator-redis",
uuid: "def456",
internal_db_url: REDIS_URL,
},
],
});
// Instance-wide: OURS and somebody else's, indistinguishable by name alone.
// Nothing in cast may read this.
if (path === "/databases")
return json([
{ name: "incubator-db", internal_db_url: PG_URL },
{ name: "incubator-redis", internal_db_url: REDIS_URL },
{ name: "incubator-db", internal_db_url: UMAMI_URL },
]);
// The app's env — it holds the PLACEHOLDER, which is exactly why the value
// must be read off the database instead.
if (path === "/applications/a1/envs")
return json([
{
key: "DATABASE_URL",
real_value: "pending-coolify-generated",
value: "REDACTED",
},
{
key: "MAILGUN_API_KEY",
real_value: "key-REAL-MAILGUN-SECRET",
value: "REDACTED",
},
]);
res.writeHead(404);
res.end("{}");
});
await new Promise<void>((r) => {
server.listen(0, "127.0.0.1", r);
});
const stub = {
url: `http://127.0.0.1:${(server.address() as AddressInfo).port}`,
close: () =>
new Promise<void>((r) => {
server.close(() => r());
}),
hits,
};
stubs.push(stub);
return stub;
}
// A fixture whose store is already what pass 1 would have written.
function genFixture(url: string, store: Record<string, string> = PASS1_STORE) {
const f = fixture(url, { manifest: GEN_MANIFEST, template: GEN_TEMPLATE });
encryptSecrets(recipient, f.store, store);
return f;
}
const genBase = (f: ReturnType<typeof fixture>) => [
...base(f),
"--generated-only",
];
// Pass 2 reads the store before it fills it, so it needs the age IDENTITY —
// not just the recipient pass 1 needed.
const withKey = () => ({ CAST_AGE_KEY_FILE_STAGING: keyFile });
const FROM = [
"--from",
"DATABASE_URL=incubator-db",
"--from",
"REDIS_URL=incubator-redis",
];
describe("cast capture --generated-only (end to end)", () => {
it("fills the generated names from the databases that own them, and leaves the rest alone", async () => {
const stub = await stubCoolifyWithDatabases();
const f = genFixture(stub.url);
const r = await runCapture([...genBase(f), ...FROM], {
stdin: "staging\n",
env: withKey(),
});
expect(r.code).toBe(0);
const store = decryptSecrets(f.store, keyFile);
// The postcondition, asserted on the actual ciphertext: zero placeholders,
// and the name count unchanged.
expect(Object.keys(store).sort()).toEqual([
"DATABASE_URL",
"MAILGUN_API_KEY",
"REDIS_URL",
]);
expect(Object.values(store)).not.toContain("pending-coolify-generated");
// Each from the resource that OWNS it — and the redis one is a redis URL,
// not the Postgres URL under a redis name.
expect(store.DATABASE_URL).toBe(PG_URL);
expect(store.REDIS_URL).toBe(REDIS_URL);
// Carried over byte for byte. Pass 2 never re-read it from the box.
expect(store.MAILGUN_API_KEY).toBe("key-REAL-MAILGUN-SECRET");
// Never the other project's database.
expect(store.DATABASE_URL).not.toContain("UMAMI");
expect(r.output).toMatch(/zero pending-coolify-generated remaining/);
});
// The #29 hazard, structurally: the value is resolved inside the project and
// environment, so the instance-wide list is never even consulted.
it("never reads the instance-wide database list", async () => {
const stub = await stubCoolifyWithDatabases();
const f = genFixture(stub.url);
const r = await runCapture([...genBase(f), ...FROM], {
stdin: "staging\n",
env: withKey(),
});
expect(r.code).toBe(0);
expect(stub.hits).toContain("/projects/p1/staging");
expect(stub.hits).not.toContain("/databases");
});
// capture.ts:166's rule holds in pass 2: the only value-shaped thing printed
// is the placeholder literal being replaced.
it("never prints a secret value to the console", async () => {
const stub = await stubCoolifyWithDatabases();
const f = genFixture(stub.url);
const r = await runCapture([...genBase(f), ...FROM], {
stdin: "staging\n",
env: withKey(),
});
expect(r.code).toBe(0);
for (const value of [PG_URL, REDIS_URL, "key-REAL-MAILGUN-SECRET"]) {
expect(r.output).not.toContain(value);
}
expect(r.output).not.toContain("GENERATED-PG-PASSWORD");
// The NAMES, and where each value came from, are the plan.
expect(r.output).toMatch(/DATABASE_URL\s+fill/);
expect(r.output).toContain("incubator-db (postgresql) internal_db_url");
expect(r.output).toMatch(/MAILGUN_API_KEY\s+keep/);
});
// Nothing in the manifest, the templates or the box says DATABASE_URL comes
// from the postgres one. cast will not guess — it hands back the flag.
it("refuses to guess which database a name comes from, and says how to say it", async () => {
const stub = await stubCoolifyWithDatabases();
const f = genFixture(stub.url);
const r = await runCapture(genBase(f), {
stdin: "staging\n",
env: withKey(),
});
expect(r.code).not.toBe(0);
expect(r.output).toMatch(/UNMAPPED/);
expect(r.output).toContain("--from DATABASE_URL=<database name>");
// Refused BEFORE the store was touched.
expect(decryptSecrets(f.store, keyFile)).toEqual(PASS1_STORE);
});
// The refusal that keeps this from being a silent credential rotation.
it("refuses to overwrite a generated name that already holds a real value", async () => {
const stub = await stubCoolifyWithDatabases();
const f = genFixture(stub.url, {
...PASS1_STORE,
DATABASE_URL: "postgres://set:BY-HAND@live:5432/app",
});
const r = await runCapture([...genBase(f), ...FROM], {
stdin: "staging\n",
env: withKey(),
});
expect(r.code).not.toBe(0);
expect(r.output).toMatch(/OCCUPIED/);
expect(r.output).toMatch(/rotate a live credential/);
expect(r.output).toMatch(/--force/);
// Untouched — including the name it COULD have filled. A refusal is a stop,
// not a partial write.
const store = decryptSecrets(f.store, keyFile);
expect(store.DATABASE_URL).toBe("postgres://set:BY-HAND@live:5432/app");
expect(store.REDIS_URL).toBe("pending-coolify-generated");
});
it("--force rotates it deliberately", async () => {
const stub = await stubCoolifyWithDatabases();
const f = genFixture(stub.url, {
...PASS1_STORE,
DATABASE_URL: "postgres://set:BY-HAND@live:5432/app",
});
const r = await runCapture([...genBase(f), ...FROM, "--force"], {
stdin: "staging\n",
env: withKey(),
});
expect(r.code).toBe(0);
expect(decryptSecrets(f.store, keyFile).DATABASE_URL).toBe(PG_URL);
});
// Pass 2 fills a store; it does not create one. A store written from here
// would hold the generated names and nothing else.
it("refuses when the store does not exist — pass 1 has not run", async () => {
const stub = await stubCoolifyWithDatabases();
const f = fixture(stub.url, {
manifest: GEN_MANIFEST,
template: GEN_TEMPLATE,
});
const r = await runCapture([...genBase(f), ...FROM], {
stdin: "staging\n",
env: withKey(),
});
expect(r.code).not.toBe(0);
expect(r.output).toMatch(/does not exist/);
expect(r.output).toMatch(/Pass 2 FILLS the generated names/);
expect(existsSync(f.store)).toBe(false);
});
// A placeholder nobody fills would leave the store still lying — and the next
// apply would push that literal at a live app.
it("refuses to leave a placeholder standing in a name it does not fill", async () => {
const stub = await stubCoolifyWithDatabases();
const f = genFixture(stub.url, {
...PASS1_STORE,
MAILGUN_API_KEY: "pending-coolify-generated",
});
const r = await runCapture([...genBase(f), ...FROM], {
stdin: "staging\n",
env: withKey(),
});
expect(r.code).not.toBe(0);
expect(r.output).toMatch(/MAILGUN_API_KEY\s+PENDING/);
expect(r.output).toMatch(/would still hold the/);
});
// The confirmation is the same last gate as pass 1, and it is not "y".
it("aborts, writing nothing, when the confirmation does not name the env", async () => {
const stub = await stubCoolifyWithDatabases();
const f = genFixture(stub.url);
const r = await runCapture([...genBase(f), ...FROM], {
stdin: "y\n",
env: withKey(),
});
expect(r.code).not.toBe(0);
expect(r.output).toMatch(/aborted/);
expect(decryptSecrets(f.store, keyFile)).toEqual(PASS1_STORE);
});
// A flag pairing that can never be honored: pass 2 captures nothing, so there
// is nothing for an override to override.
it("refuses --override with --generated-only", async () => {
const stub = await stubCoolifyWithDatabases();
const f = genFixture(stub.url);
const r = await runCapture(
[...genBase(f), ...FROM, "--override", "MAILGUN_API_KEY"],
{ stdin: "staging\n", env: withKey() },
);
expect(r.code).not.toBe(0);
expect(r.output).toMatch(/refuses --override with --generated-only/);
});
it("refuses --from without --generated-only", async () => {
const stub = await stubCoolifyWithDatabases();
const f = fixture(stub.url, {
manifest: GEN_MANIFEST,
template: GEN_TEMPLATE,
});
const r = await runCapture([...base(f), ...FROM], { stdin: "staging\n" });
expect(r.code).not.toBe(0);
expect(r.output).toMatch(/refuses --from without --generated-only/);
});
// A --from for a name that is not generated would be silently ignored, and
// the operator would walk away believing they had set a value.
it("refuses a --from naming something that is not a generated secret", async () => {
const stub = await stubCoolifyWithDatabases();
const f = genFixture(stub.url);
const r = await runCapture(
[...genBase(f), "--from", "MAILGUN_API_KEY=incubator-db"],
{ stdin: "staging\n", env: withKey() },
);
expect(r.code).not.toBe(0);
expect(r.output).toMatch(/is not a generated secret/);
});
});
// --resource maps manifest names to box names for the env-reading pass, and pass
// 2 reads no env. Accepted, it would be silently ignored.
describe("cast capture --generated-only (flag hygiene)", () => {
it("refuses --resource with --generated-only", async () => {
const stub = await stubCoolifyWithDatabases();
const f = genFixture(stub.url);
const r = await runCapture(
[...genBase(f), ...FROM, "--resource", "core=Core"],
{ stdin: "staging\n", env: withKey() },
);
expect(r.code).not.toBe(0);
expect(r.output).toMatch(/refuses --resource with --generated-only/);
});
});

287
test/capture.test.ts
View file

@ -4,8 +4,14 @@ import { join } from "node:path";
import { describe, expect, it } from "vitest";
import {
GENERATED_PLACEHOLDER,
type GeneratedSource,
assertGeneratedComplete,
classify,
generatedPlanRefuses,
planGenerated,
renderCapturePlan,
renderGeneratedPlan,
resolveGeneratedSources,
} from "../src/capture.js";
import { requiredSecrets } from "../src/resolve.js";
@ -307,3 +313,284 @@ environments:
);
});
});
// ---------------------------------------------------------------------------
// Pass 2 — capture --generated-only
// ---------------------------------------------------------------------------
// The live shape this verb exists for: the incubator, mid-bootstrap. Two
// databases in the project+environment, and a third Postgres that belongs to
// umami — the row the hand-run `jq | head` had to be careful not to pick, and
// the one a name-directed lookup across `GET /databases` would eventually take.
const PG: GeneratedSource = {
resource: "incubator-db",
type: "postgresql",
url: "postgres://u:REAL-PG-PASSWORD@abc123:5432/app",
};
const REDIS: GeneratedSource = {
resource: "incubator-redis",
type: "redis",
url: "redis://default:REAL-REDIS-PASSWORD@def456:6379/0",
};
// Pass 1 left these two placeheld; every other name is real and must survive.
const STORE = {
DATABASE_URL: GENERATED_PLACEHOLDER,
REDIS_URL: GENERATED_PLACEHOLDER,
MAILGUN_API_KEY: "key-REAL",
ADMIN_EMAIL: "operator@example.com",
};
const GENERATED = ["DATABASE_URL", "REDIS_URL"];
const GCTX = {
orgRepo: "heavy-duty/incubator",
env: "prod",
instance: "default",
store: "/s/secrets/incubator.prod.env.age",
recipient: "age1abc",
project: "incubator",
environment: "production",
};
// The mapping is stated, then the plan is built from it.
const planWith = (
from: Record<string, string>,
store = STORE,
databases = [PG, REDIS],
opts?: { force: boolean },
) => {
const { mapping, unmapped } = resolveGeneratedSources(
GENERATED,
databases,
from,
);
return planGenerated(GENERATED, store, mapping, unmapped, opts);
};
describe("resolveGeneratedSources", () => {
// The whole point: the value comes off the DATABASE, not off an app.
it("maps each generated name to the database the operator named", () => {
const { mapping, unmapped } = resolveGeneratedSources(
GENERATED,
[PG, REDIS],
{ DATABASE_URL: "incubator-db", REDIS_URL: "incubator-redis" },
);
expect(unmapped).toEqual([]);
expect(mapping.DATABASE_URL).toEqual(PG);
expect(mapping.REDIS_URL).toEqual(REDIS);
});
// #29 wearing a different hat. Nothing anywhere says DATABASE_URL comes from
// the postgres one — inferring it from the NAME is exactly the silent wrong
// pick this refuses, because what it writes is a well-formed URL to somebody
// else's database.
it("refuses to guess when more than one database could be meant", () => {
const { mapping, unmapped } = resolveGeneratedSources(
GENERATED,
[PG, REDIS],
{},
);
expect(mapping).toEqual({});
expect(unmapped.map((u) => u.ref)).toEqual(["DATABASE_URL", "REDIS_URL"]);
expect(unmapped[0].why).toMatch(/will not pick by name/);
// It names the candidates rather than picking one.
expect(unmapped[0].why).toContain("incubator-db:postgresql");
expect(unmapped[0].why).toContain("incubator-redis:redis");
});
// The one inference that cannot be wrong: nothing else it could be.
it("infers the only database when there is exactly one, for one name", () => {
const { mapping, unmapped } = resolveGeneratedSources(
["DATABASE_URL"],
[PG],
{},
);
expect(unmapped).toEqual([]);
expect(mapping.DATABASE_URL).toEqual(PG);
});
// ...and still refuses two names against that one database: filling REDIS_URL
// from the Postgres would be a perfectly well-formed lie.
it("does not infer when one database must serve two generated names", () => {
const { unmapped } = resolveGeneratedSources(GENERATED, [PG], {});
expect(unmapped.map((u) => u.ref)).toEqual(["DATABASE_URL", "REDIS_URL"]);
});
it("refuses a --from naming a database that is not in this project+env", () => {
const { unmapped } = resolveGeneratedSources(GENERATED, [PG, REDIS], {
DATABASE_URL: "umami-db",
REDIS_URL: "incubator-redis",
});
expect(unmapped).toHaveLength(1);
expect(unmapped[0].why).toMatch(/no database named "umami-db" exists/);
});
it("refuses when the environment holds no database at all", () => {
const { unmapped } = resolveGeneratedSources(["DATABASE_URL"], [], {});
expect(unmapped[0].why).toMatch(/no database exists/);
});
});
describe("planGenerated", () => {
it("fills the generated names and keeps every other one byte for byte", () => {
const p = planWith({
DATABASE_URL: "incubator-db",
REDIS_URL: "incubator-redis",
});
expect(p.fills).toEqual([
{
ref: "DATABASE_URL",
from: { resource: "incubator-db", type: "postgresql" },
value: PG.url,
},
{
ref: "REDIS_URL",
from: { resource: "incubator-redis", type: "redis" },
value: REDIS.url,
},
]);
// Untouched — and NOT re-read from the box, which is what makes pass 2 safe
// to run against an environment whose other secrets were rotated by hand.
expect(p.kept).toEqual(["ADMIN_EMAIL", "MAILGUN_API_KEY"]);
expect(p.occupied).toEqual([]);
expect(p.absent).toEqual([]);
expect(p.stillPending).toEqual([]);
});
// The refusal that stops a silent credential rotation: someone already filled
// it (a previous pass 2, or by hand) and the value is live.
it("refuses to overwrite a generated name that already holds a real value", () => {
const p = planWith(
{ DATABASE_URL: "incubator-db", REDIS_URL: "incubator-redis" },
{ ...STORE, DATABASE_URL: "postgres://already:filled@live/db" },
);
expect(p.occupied).toEqual(["DATABASE_URL"]);
expect(p.fills.map((f) => f.ref)).toEqual(["REDIS_URL"]);
expect(generatedPlanRefuses(p)).toBe(true);
});
it("--force fills it anyway, deliberately", () => {
const p = planWith(
{ DATABASE_URL: "incubator-db", REDIS_URL: "incubator-redis" },
{ ...STORE, DATABASE_URL: "postgres://already:filled@live/db" },
[PG, REDIS],
{ force: true },
);
expect(p.occupied).toEqual([]);
expect(p.fills.map((f) => f.ref)).toEqual(["DATABASE_URL", "REDIS_URL"]);
expect(generatedPlanRefuses(p)).toBe(false);
});
// Pass 2 FILLS names; it does not add them. A store missing one did not come
// from pass 1, and the name-count postcondition could not hold anyway.
it("refuses a generated name the store does not carry at all", () => {
const { REDIS_URL, ...withoutRedis } = STORE;
const p = planWith(
{ DATABASE_URL: "incubator-db", REDIS_URL: "incubator-redis" },
withoutRedis,
);
expect(p.absent).toEqual(["REDIS_URL"]);
expect(generatedPlanRefuses(p)).toBe(true);
});
// A placeholder standing in a name nothing here fills: the run would
// "succeed" and the store would still be a lie.
it("refuses when a name nobody fills would be left still pending", () => {
const p = planWith(
{ DATABASE_URL: "incubator-db", REDIS_URL: "incubator-redis" },
{ ...STORE, SESSION_SECRET: GENERATED_PLACEHOLDER },
);
expect(p.stillPending).toEqual(["SESSION_SECRET"]);
expect(generatedPlanRefuses(p)).toBe(true);
});
it("a refused name is not also reported as kept", () => {
const p = planWith({});
expect(p.kept).toEqual(["ADMIN_EMAIL", "MAILGUN_API_KEY"]);
expect(p.unmapped).toHaveLength(2);
});
});
describe("renderGeneratedPlan", () => {
// capture.ts:166's rule, unchanged: names, provenance, and the resource a
// value came FROM. The only value-shaped thing printed is the placeholder
// literal being replaced.
it("prints names and never a value", () => {
const out = renderGeneratedPlan(
planWith({ DATABASE_URL: "incubator-db", REDIS_URL: "incubator-redis" }),
GCTX,
);
expect(out).not.toContain("REAL-PG-PASSWORD");
expect(out).not.toContain("REAL-REDIS-PASSWORD");
expect(out).not.toContain("key-REAL");
expect(out).toContain("DATABASE_URL");
expect(out).toContain("incubator-db (postgresql) internal_db_url");
expect(out).toContain(GENERATED_PLACEHOLDER);
// The store is not being rewritten around the names it keeps, and says so.
expect(out).toMatch(/MAILGUN_API_KEY\s+keep/);
});
it("hands back a ready-to-paste --from when it will not guess", () => {
const out = renderGeneratedPlan(planWith({}), GCTX);
expect(out).toMatch(/refusing to write the store/);
expect(out).toContain("--from DATABASE_URL=<database name>");
expect(out).toContain("--from REDIS_URL=<database name>");
});
it("says a fill would rotate a live credential", () => {
const out = renderGeneratedPlan(
planWith(
{ DATABASE_URL: "incubator-db", REDIS_URL: "incubator-redis" },
{ ...STORE, DATABASE_URL: "postgres://live" },
),
GCTX,
);
expect(out).toMatch(/OCCUPIED/);
expect(out).toMatch(/rotate a live credential/);
expect(out).not.toContain("postgres://live");
});
});
// The assertion that was a line in a human runbook ("assert 14 names / zero
// placeholders"), which is to say a step that could be skipped.
describe("assertGeneratedComplete", () => {
it("passes when every placeholder is gone and the name set is unchanged", () => {
const after = { ...STORE, DATABASE_URL: PG.url, REDIS_URL: REDIS.url };
expect(assertGeneratedComplete(STORE, after)).toEqual([]);
});
it("catches a placeholder left standing", () => {
const after = { ...STORE, DATABASE_URL: PG.url };
const v = assertGeneratedComplete(STORE, after);
expect(v).toHaveLength(1);
expect(v[0]).toMatch(
/still hold the pending-coolify-generated literal: REDIS_URL/,
);
});
// A store that LOST a name re-encrypts perfectly and reads back perfectly.
// The failure surfaces at the next apply, in an environment whose plaintext
// nobody has any more.
it("catches a name dropped on the way through", () => {
const { MAILGUN_API_KEY, ...lost } = {
...STORE,
DATABASE_URL: PG.url,
REDIS_URL: REDIS.url,
};
const v = assertGeneratedComplete(STORE, lost);
expect(v.join(" ")).toMatch(/went in with 4 name\(s\) and came out with 3/);
expect(v.join(" ")).toMatch(/names LOST: MAILGUN_API_KEY/);
});
it("catches a name that was never supposed to be added", () => {
const after = {
...STORE,
DATABASE_URL: PG.url,
REDIS_URL: REDIS.url,
SURPRISE: "x",
};
expect(assertGeneratedComplete(STORE, after).join(" ")).toMatch(
/names ADDED: SURPRISE/,
);
});
});