11 changed files with 1478 additions and 50 deletions
121
README.md
121
README.md
|
|
@ -52,6 +52,7 @@ environments.yaml # bindings: the team each env's token must belon
|
|||
# GitHub App name, smoke target, guards
|
||||
secrets/<repo>.<env>.env.age # age-encrypted values for the ${…} placeholders
|
||||
.coolify.env # COOLIFY_BASE_URL + COOLIFY_ACCESS_TOKEN (never commit)
|
||||
.coolify/<name>.env # …the same, for a NAMED instance (see below)
|
||||
```
|
||||
|
||||
Pass it with `--state <dir>`, or set `CAST_STATE`. Defaults to the cwd.
|
||||
|
|
@ -59,8 +60,9 @@ Pass it with `--state <dir>`, or set `CAST_STATE`. Defaults to the cwd.
|
|||
## Commands
|
||||
|
||||
```sh
|
||||
cast apply <org>/<repo> --env <env> [--path <dir>] [--hostname-overlay <file>]
|
||||
cast diff <org>/<repo> --env <env> [--full]
|
||||
cast apply <org>/<repo> --env <env> [--path <dir>] [--hostname-overlay <file>]
|
||||
cast diff <org>/<repo> --env <env> [--full]
|
||||
cast capture <org>/<repo> --env <env> [--generated <NAME>] [--override <NAME>]
|
||||
cast server add <name> --ip <ip> --key <file> --env <env> [--user root] [--port 22]
|
||||
cast smoke --env <env>
|
||||
cast team [--env <env>]
|
||||
|
|
@ -73,6 +75,9 @@ cast team [--env <env>]
|
|||
default branch).
|
||||
- **`diff`** — reports drift, manifest → Coolify. Structural by default; `--full`
|
||||
also compares env vars. Exits non-zero when dirty, so CI can gate on it.
|
||||
- **`capture`** — the adoption path: reads a hand-built instance's live env and
|
||||
writes the environment's age store from it. See *Adopting a hand-built
|
||||
instance* below.
|
||||
- **`server add`** — uploads a server's private key and registers it with Coolify.
|
||||
- **`smoke`** — contract test against `smoke_target`: proves Coolify's bulk env
|
||||
endpoint still *upserts* rather than replacing. Run it after every Coolify
|
||||
|
|
@ -89,6 +94,118 @@ them first asserts the token's team (below).
|
|||
`--hostname-overlay` swaps domains for a pre-flight run against temporary
|
||||
hostnames; re-applying **without** it is the cutover.
|
||||
|
||||
## Cloning: cast authenticates, and never prompts
|
||||
|
||||
`apply`, `diff` and `capture` clone the product repo (unless `--path` points at a
|
||||
local checkout — refused for prod, which always reads the default branch). For a
|
||||
private repo that needs credentials, and cast resolves them itself:
|
||||
|
||||
1. **`gh`**, borrowed as a credential helper for that one invocation — it does
|
||||
not touch your global git config.
|
||||
2. **`GITHUB_TOKEN` / `GH_TOKEN`** from the environment (the CI path).
|
||||
3. Whatever git's own credential helper does, if you have one.
|
||||
|
||||
Being logged into `gh` is enough. You do **not** need `gh auth setup-git` —
|
||||
that separate act is what wires git's helper, and not running it is exactly how
|
||||
you end up at git's interactive username/password prompt, which GitHub no longer
|
||||
accepts. cast sets `GIT_TERMINAL_PROMPT=0` on every path, so it can never hang
|
||||
there or hide a credentials failure behind an error about *the repository*. With
|
||||
no credentials at all it says so, and names the fix.
|
||||
|
||||
The token is never put in the clone URL or in `http.extraheader` — both leak it
|
||||
into `ps`, and the latter persists it into the clone's git config.
|
||||
|
||||
## Many Coolifys
|
||||
|
||||
`--instance <name>` reads `<state>/.coolify/<name>.env` instead of
|
||||
`<state>/.coolify.env`. Every verb that reaches Coolify takes it.
|
||||
|
||||
```sh
|
||||
cast diff heavy-duty/incubator --env prod --full --instance legacy
|
||||
```
|
||||
|
||||
An environment can bind one, so `--env` selects the right control plane with no
|
||||
flag at all:
|
||||
|
||||
```yaml
|
||||
environments:
|
||||
prod:
|
||||
server: prod-box
|
||||
team: { id: 1, name: heavy-duty }
|
||||
instance: prod-cp # → <state>/.coolify/prod-cp.env
|
||||
```
|
||||
|
||||
An explicit `--instance` still wins, so a one-off read against a legacy box needs
|
||||
no edit to that file either. **With no flag and no binding, nothing changes** —
|
||||
`.coolify.env` is read exactly as before.
|
||||
|
||||
Two properties, both deliberate:
|
||||
|
||||
- **An unknown `--instance` refuses**, and names the instances that do exist.
|
||||
Falling back to the default is how a diff meant for a legacy box gets run
|
||||
against production.
|
||||
- **An instance may declare `COOLIFY_READ_ONLY=true`**, and then `apply`,
|
||||
`smoke` and `server add` refuse it — *before their first call*, and even
|
||||
though the token itself would permit the writes. That turns "I pointed the
|
||||
wrong token at the wrong box" from a live incident into an exit code.
|
||||
|
||||
Every command that reaches a Coolify now says which one, next to the team
|
||||
assert. It is the most consequential input to any run, and the least visible.
|
||||
|
||||
## Adopting a hand-built instance
|
||||
|
||||
cast is otherwise scoped to the steady state: manifest → Coolify, forever.
|
||||
`capture` is the one-way-in — it bootstraps an environment's age store from an
|
||||
instance that was built by hand, before any manifest existed.
|
||||
|
||||
```sh
|
||||
CAST_CAPTURE_ADMIN_EMAIL=me@example.com \
|
||||
cast capture heavy-duty/incubator --env prod --instance legacy \
|
||||
--override ADMIN_EMAIL
|
||||
```
|
||||
|
||||
It reads the required secret **names** from the manifest's own env templates (the
|
||||
`${…}` refs — the manifest already declares exactly this set), reads the live
|
||||
values off the instance, and classifies every name:
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| **captured** | found live, value taken |
|
||||
| **generated** | the manifest's `generated_secrets` declares it provider-made → written as the literal `pending-coolify-generated`, never the live value |
|
||||
| **overridden** | supplied by you, for a value that must *not* be carried over |
|
||||
| **missing** | required by a template, absent live → **refuses** |
|
||||
|
||||
Then it prints a plan of **names and provenance — never values** — and waits for
|
||||
you to type the environment's name.
|
||||
|
||||
The mapping is not mechanical, and that is the whole design. A `DATABASE_URL`
|
||||
copied off the source box points at the *source box's* Postgres: confidently
|
||||
wrong, entirely plausible, and the target's real URL does not exist until Coolify
|
||||
creates the resource. So the manifest declares those names, and cast placeholds
|
||||
them:
|
||||
|
||||
```yaml
|
||||
environments:
|
||||
prod:
|
||||
generated_secrets: [DATABASE_URL_PROD, REDIS_URL_PROD, UMAMI_DATABASE_URL]
|
||||
```
|
||||
|
||||
It is a manifest property rather than a flag you have to remember, because the
|
||||
manifest is what knows `DATABASE_URL` comes from a database it declares. (A
|
||||
`generated_secrets` entry no template refers to is a schema error — a guard
|
||||
standing over nothing is worse than no guard, because it reads like one.
|
||||
`--generated <NAME>` covers a manifest that hasn't declared them yet.)
|
||||
|
||||
An **`--override`**'s value is read from `$CAST_CAPTURE_<NAME>`, never from the
|
||||
command line: argv is visible in `ps` to every process on the box. It exists for
|
||||
values that must not survive the copy — staging and prod sharing a Mailgun
|
||||
domain means a staging box carrying the real `ADMIN_EMAIL` can mail real users.
|
||||
|
||||
The store is encrypted to the environment's `age_recipient` (add it to
|
||||
`environments.yaml` — it's the public half, safe to commit). Plaintext goes to
|
||||
`age` on stdin: it is never a temp file, never on stdout, never in your shell
|
||||
history. An existing store is not overwritten without `--force`.
|
||||
|
||||
**[docs/semantics.md](docs/semantics.md)** is the contract behind those
|
||||
commands: what `apply` guarantees (never deletes, never recreates a database,
|
||||
fails loudly rather than recreating on un-updatable drift), the `dockercompose`
|
||||
|
|
|
|||
|
|
@ -158,6 +158,132 @@ softened by an implementation detail):
|
|||
refuses `--path` combined with `--env prod`: prod always reads the default
|
||||
branch, so a feature-branch checkout can never reach it.
|
||||
|
||||
## Instance selection
|
||||
|
||||
**The Coolify a command talks to is an explicit, named value** — not a property
|
||||
of whatever `<state>/.coolify.env` happens to contain at the moment. Resolution
|
||||
order, highest first:
|
||||
|
||||
1. `--instance <name>` → `<state>/.coolify/<name>.env`
|
||||
2. the environment's `instance:` binding in `environments.yaml`
|
||||
3. `<state>/.coolify.env` (the default; unchanged when neither of the above is
|
||||
used)
|
||||
|
||||
Two refusals, both fail-closed:
|
||||
|
||||
- **An unknown `--instance` aborts**, naming the instances that do exist. It
|
||||
does *not* fall back to the default — that fallback is how a `--full` diff
|
||||
meant for a legacy box gets run against production.
|
||||
- **`COOLIFY_READ_ONLY=true` in an instance file makes it read-only**, and
|
||||
`apply` / `smoke` / `server add` refuse it *before their first call*. The
|
||||
guard is the **declaration**, not the token's scope: an instance configured
|
||||
for inspection must not be writable even when the token it holds would permit
|
||||
the writes. `diff`, `team` and `capture` still work against it — they read.
|
||||
|
||||
Every command that reaches a live Coolify prints which one, next to the team
|
||||
assert.
|
||||
|
||||
## Adoption (`capture`)
|
||||
|
||||
`capture` is the only verb that writes *into* the state directory rather than
|
||||
into Coolify, and the only one that reads a hand-built instance as a **source**
|
||||
rather than as a target. It exists because cast is otherwise scoped to the
|
||||
steady state and has no bootstrap path for a box that predates its manifest.
|
||||
|
||||
**The required set comes from the manifest, not from the box.** The names are
|
||||
the `${…}` refs in that environment's env templates, read by the same parser
|
||||
`apply` uses to demand them (`parseTemplate`, shared by `resolveTemplate` and
|
||||
`templateRefs` — deliberately one grammar, because a drift between the two
|
||||
would mean `capture` collects a different set than `apply` will later require,
|
||||
which is the "a name silently missed" failure it exists to remove). So the store
|
||||
it writes contains **exactly** the names the manifest requires: a live var
|
||||
nobody asked for is not the store's business, and a template literal
|
||||
(`NODE_ENV=production`) is not a secret.
|
||||
|
||||
**The mapping is not mechanical, and must not be.** Some entries encode
|
||||
migration decisions rather than facts about the source box:
|
||||
|
||||
- A `DATABASE_URL` / `REDIS_URL` read off the source points at the **source
|
||||
box's** Postgres/Redis. Copying it is confidently wrong in a way that looks
|
||||
entirely plausible, and the target's real URL does not exist until Coolify
|
||||
creates the resource. These are declared `generated_secrets:` in the manifest
|
||||
environment and written as the literal `pending-coolify-generated`.
|
||||
- staging's `ADMIN_EMAIL` must be the operator, not the source's value: staging
|
||||
and prod share a Mailgun domain, so a staging box carrying the real address
|
||||
can mail real users. That is `--override`.
|
||||
|
||||
A "capture everything" verb would therefore be silently wrong in a handful of
|
||||
entries out of seventeen — worse than being wrong in all of them. So every
|
||||
required name is **forced into a disposition**, and two of the four stop the
|
||||
run:
|
||||
|
||||
| disposition | source | outcome |
|
||||
| --- | --- | --- |
|
||||
| captured | found live | value taken |
|
||||
| generated | manifest `generated_secrets` (or `--generated`) | `pending-coolify-generated` |
|
||||
| overridden | `$CAST_CAPTURE_<NAME>` | operator's value |
|
||||
| **missing** | required by a template, absent live | **refuses** |
|
||||
| **conflict** | one name, different live values on two resources | **refuses** |
|
||||
|
||||
`generated_secrets` is a **manifest** property, not a flag: the manifest is what
|
||||
knows `DATABASE_URL` comes from a database it declares. An entry naming
|
||||
something no template refs is a hard error — dead config here is not untidy but
|
||||
dangerous, because it reads like a guard standing over a name while standing
|
||||
over nothing, and the likeliest cause is a typo whose real name is then
|
||||
*captured* from the source box instead of placeheld.
|
||||
|
||||
**Secret hygiene**, all enforced by tests against real values:
|
||||
|
||||
- The plan prints **names and provenance, never values**. (The one value-shaped
|
||||
thing it prints is the `pending-coolify-generated` literal, which carries no
|
||||
information about the source.)
|
||||
- An `--override`'s value is read from `$CAST_CAPTURE_<NAME>`, **never from
|
||||
argv** — a command-line value is visible in `ps` to every process on the box.
|
||||
- Plaintext is piped to `age` on **stdin**: never a temp file, never stdout,
|
||||
never shell history. The hand-run recipe this replaces wrote
|
||||
`/dev/shm/prod.env` and relied on remembering to `shred -u` it.
|
||||
- An existing store is **not overwritten** without `--force`: it may hold the
|
||||
only copy of values the source box no longer has. Same disposition as apply's
|
||||
never-delete.
|
||||
|
||||
**`capture` takes `diff`'s position on an absent target** (see `LiveLookup`),
|
||||
and refuses one: against a project or environment that isn't there it would read
|
||||
back zero live values and report every required secret as *missing* — an
|
||||
alarming, meaningless report about the wrong box. It also inherits the team
|
||||
assert (a wrong-team token reads back `null` for everything, producing the same
|
||||
lie) and the `--path`-with-`--env prod` refusal (a feature-branch manifest must
|
||||
not decide which names land in the prod store).
|
||||
|
||||
The final gate is a **typed confirmation** — the environment's own name, after
|
||||
the plan. There is no `--yes`: a store written without someone reading the
|
||||
provenance column is the outcome the verb exists to prevent. A closed stdin
|
||||
aborts rather than hanging.
|
||||
|
||||
## Cloning a private manifest
|
||||
|
||||
`resolveCheckout` resolves git credentials **inside cast**, in a fixed order —
|
||||
`gh` borrowed as a per-invocation credential helper, then
|
||||
`GITHUB_TOKEN`/`GH_TOKEN`, then the ambient helper — rather than leaving it to
|
||||
whatever the workstation's git config happens to do.
|
||||
|
||||
It matters because `gh auth login` **does not** wire git's credential helper
|
||||
(that is `gh auth setup-git`, a separate act most people never run), so a
|
||||
perfectly logged-in operator still fell through to git's interactive
|
||||
username/password prompt — which GitHub no longer accepts — and got an error
|
||||
about *the repository* rather than about the missing credentials. There is no
|
||||
routing around it for prod: `--path` is refused there, so the clone is the only
|
||||
path and its auth is mandatory.
|
||||
|
||||
`GIT_TERMINAL_PROMPT=0` is set on every path, so cast can never hang on or fall
|
||||
into that prompt. The token is never placed in the clone URL or in
|
||||
`http.extraheader` — both leak it into `ps`, and the latter persists it into the
|
||||
clone's `.git/config`; the helper reads it from the environment at run time, so
|
||||
what lands in argv is the literal text `$CAST_GIT_TOKEN`. Note that the empty
|
||||
`credential.helper=` reset clears **URL-scoped** helpers
|
||||
(`credential.https://github.com.helper`, which is what `gh auth setup-git`
|
||||
writes) as well as generic ones, so cast's chosen credential is genuinely the
|
||||
one used — verified against a live private clone.
|
||||
|
||||
**Known limitations, not defects:**
|
||||
|
||||
- **Backup schedules are create-time only.** A manifest database's `backup`
|
||||
|
|
|
|||
|
|
@ -47,6 +47,12 @@ const BindingsSchema = z
|
|||
// An explicit --instance still wins, so a one-off read against a
|
||||
// legacy box needs no change to this file either.
|
||||
instance: z.string().optional(),
|
||||
// The age recipient (public key) this environment's secret store is
|
||||
// encrypted TO. Only `capture` needs it — decryption resolves an
|
||||
// identity per keyFileFor, and the state repo deliberately holds
|
||||
// ciphertext but never the identity that opens it. This is the
|
||||
// public half, so it is safe to commit here next to the bindings.
|
||||
age_recipient: z.string().optional(),
|
||||
s3_destination: z.string().optional(),
|
||||
// Var-name patterns this environment refuses outright (see
|
||||
// assertEnvVarPolicy). Operator-owned guard: prod typically bans
|
||||
|
|
|
|||
189
src/capture.ts
Normal file
189
src/capture.ts
Normal file
|
|
@ -0,0 +1,189 @@
|
|||
import type { RequiredSecret } from "./resolve.js";
|
||||
|
||||
// What the manifest writes for a provider-generated name. Not the source box's
|
||||
// value — that points at the SOURCE box's Postgres/Redis — and not an empty
|
||||
// string, which would boot the app misconfigured. A literal that is obviously
|
||||
// a placeholder, and that Coolify replaces when it creates the resource.
|
||||
export const GENERATED_PLACEHOLDER = "pending-coolify-generated";
|
||||
|
||||
// Live env vars, per resource: resource name -> (env key -> value).
|
||||
export type LiveEnvs = Record<string, Record<string, string>>;
|
||||
|
||||
export type Provenance = "captured" | "generated" | "overridden";
|
||||
|
||||
export type Site = { resource: string; key: string };
|
||||
|
||||
export type Disposition = {
|
||||
ref: string;
|
||||
provenance: Provenance;
|
||||
// Never rendered. Kept here so the caller can encrypt it, and nowhere else.
|
||||
value: string;
|
||||
sites: Site[];
|
||||
};
|
||||
|
||||
export type Classification = {
|
||||
plan: Disposition[];
|
||||
// Required by a template, absent from the source, and not dispositioned
|
||||
// otherwise. Refuses the run: writing an empty value substitutes to nothing
|
||||
// and the app boots misconfigured — the exact failure capture exists to
|
||||
// remove, and one that looks entirely plausible from the outside.
|
||||
missing: Array<{ ref: string; sites: Site[] }>;
|
||||
// The same ref carrying DIFFERENT live values on two resources. cast cannot
|
||||
// pick, and picking wrong is silent, so it refuses.
|
||||
conflicts: Array<{ ref: string; values: Site[] }>;
|
||||
};
|
||||
|
||||
function groupByRef(required: RequiredSecret[]): Map<string, Site[]> {
|
||||
const byRef = new Map<string, Site[]>();
|
||||
for (const { ref, resource, key } of required) {
|
||||
const sites = byRef.get(ref) ?? [];
|
||||
sites.push({ resource, key });
|
||||
byRef.set(ref, sites);
|
||||
}
|
||||
return byRef;
|
||||
}
|
||||
|
||||
// Force disposition, never guess. Every name the manifest requires lands in
|
||||
// exactly one of four buckets, and two of them stop the run.
|
||||
//
|
||||
// The mapping is deliberately NOT a mechanical dump of the source box: some
|
||||
// entries encode migration decisions rather than facts about the source. A
|
||||
// "capture everything" verb would be wrong in a handful of entries out of
|
||||
// seventeen, silently — which is worse than being wrong in all of them.
|
||||
export function classify(
|
||||
required: RequiredSecret[],
|
||||
generated: string[],
|
||||
live: LiveEnvs,
|
||||
overrides: Record<string, string>,
|
||||
): Classification {
|
||||
const generatedSet = new Set(generated);
|
||||
const plan: Disposition[] = [];
|
||||
const missing: Classification["missing"] = [];
|
||||
const conflicts: Classification["conflicts"] = [];
|
||||
|
||||
for (const [ref, sites] of groupByRef(required)) {
|
||||
// The operator's word beats both the manifest and the source box: this is
|
||||
// the escape hatch for a value that must NOT be carried over (staging's
|
||||
// ADMIN_EMAIL, where the source's value is a real founder and staging
|
||||
// shares a Mailgun domain with prod).
|
||||
if (ref in overrides) {
|
||||
plan.push({
|
||||
ref,
|
||||
provenance: "overridden",
|
||||
value: overrides[ref],
|
||||
sites,
|
||||
});
|
||||
continue;
|
||||
}
|
||||
if (generatedSet.has(ref)) {
|
||||
plan.push({
|
||||
ref,
|
||||
provenance: "generated",
|
||||
value: GENERATED_PLACEHOLDER,
|
||||
sites,
|
||||
});
|
||||
continue;
|
||||
}
|
||||
// Captured: read the live value off whichever resources declare it.
|
||||
const found = sites
|
||||
.map((s) => ({ site: s, value: live[s.resource]?.[s.key] }))
|
||||
.filter((f): f is { site: Site; value: string } => f.value !== undefined);
|
||||
if (found.length === 0) {
|
||||
missing.push({ ref, sites });
|
||||
continue;
|
||||
}
|
||||
const distinct = new Set(found.map((f) => f.value));
|
||||
if (distinct.size > 1) {
|
||||
conflicts.push({ ref, values: found.map((f) => f.site) });
|
||||
continue;
|
||||
}
|
||||
plan.push({
|
||||
ref,
|
||||
provenance: "captured",
|
||||
value: found[0].value,
|
||||
sites,
|
||||
});
|
||||
}
|
||||
return { plan, missing, conflicts };
|
||||
}
|
||||
|
||||
const site = (s: Site) => `${s.resource}.${s.key}`;
|
||||
|
||||
// Names and provenance. NEVER values.
|
||||
//
|
||||
// The one thing printed that looks like a value is GENERATED_PLACEHOLDER,
|
||||
// which is a literal constant in this file and carries no information about
|
||||
// the source box. Everything else is a name the manifest already declares in
|
||||
// plaintext, in a committed file.
|
||||
export function renderCapturePlan(
|
||||
c: Classification,
|
||||
ctx: {
|
||||
orgRepo: string;
|
||||
env: string;
|
||||
instance: string;
|
||||
store: string;
|
||||
recipient: string;
|
||||
},
|
||||
): string {
|
||||
const lines = [
|
||||
`capture plan — ${ctx.orgRepo} ${ctx.env}`,
|
||||
"",
|
||||
` source: instance ${ctx.instance} (live values read from it)`,
|
||||
` store: ${ctx.store}`,
|
||||
` recipient: ${ctx.recipient}`,
|
||||
"",
|
||||
];
|
||||
const width = Math.max(
|
||||
0,
|
||||
...[...c.plan, ...c.missing, ...c.conflicts].map((d) => d.ref.length),
|
||||
);
|
||||
for (const d of c.plan) {
|
||||
const where = d.sites.map(site).join(", ");
|
||||
const note =
|
||||
d.provenance === "generated"
|
||||
? ` → ${GENERATED_PLACEHOLDER}`
|
||||
: d.provenance === "overridden"
|
||||
? ` (from CAST_CAPTURE_${d.ref})`
|
||||
: "";
|
||||
lines.push(
|
||||
` ${d.ref.padEnd(width)} ${d.provenance.padEnd(10)} ${where}${note}`,
|
||||
);
|
||||
}
|
||||
for (const m of c.missing) {
|
||||
lines.push(
|
||||
` ${m.ref.padEnd(width)} MISSING required by ${m.sites.map(site).join(", ")}, absent from the source`,
|
||||
);
|
||||
}
|
||||
for (const c2 of c.conflicts) {
|
||||
lines.push(
|
||||
` ${c2.ref.padEnd(width)} CONFLICT differs between ${c2.values.map(site).join(" and ")}`,
|
||||
);
|
||||
}
|
||||
const counts = (["captured", "generated", "overridden"] as const)
|
||||
.map((p) => [p, c.plan.filter((d) => d.provenance === p).length] as const)
|
||||
.filter(([, n]) => n > 0)
|
||||
.map(([p, n]) => `${n} ${p}`)
|
||||
.join(", ");
|
||||
lines.push(
|
||||
"",
|
||||
`${c.plan.length} name(s) to write${counts ? `: ${counts}` : ""}`,
|
||||
);
|
||||
if (c.missing.length > 0) {
|
||||
lines.push(
|
||||
"",
|
||||
`refusing to write the store: ${c.missing.length} name(s) the manifest requires are not`,
|
||||
"present on the source. An empty value substitutes to nothing and the app boots",
|
||||
"misconfigured — plausibly, and silently. Supply each one with --override <NAME>",
|
||||
"(its value is read from CAST_CAPTURE_<NAME>, never from argv), or fix the source.",
|
||||
);
|
||||
}
|
||||
if (c.conflicts.length > 0) {
|
||||
lines.push(
|
||||
"",
|
||||
`refusing to write the store: ${c.conflicts.length} name(s) carry different values on`,
|
||||
"different resources of the source. The store holds one value per name, and cast",
|
||||
"will not pick for you. Reconcile them on the source, or pin one with --override.",
|
||||
);
|
||||
}
|
||||
return lines.join("\n");
|
||||
}
|
||||
275
src/cli.ts
275
src/cli.ts
|
|
@ -1,10 +1,12 @@
|
|||
#!/usr/bin/env node
|
||||
import { readFileSync } from "node:fs";
|
||||
import { existsSync, readFileSync } from "node:fs";
|
||||
import { join } from "node:path";
|
||||
import { createInterface } from "node:readline/promises";
|
||||
import { parseArgs } from "node:util";
|
||||
import { parse as parseYaml } from "yaml";
|
||||
import { type Executor, applyHostnameOverlay, applyPlan } from "./apply.js";
|
||||
import { githubAppNameFor, loadBindings } from "./bindings.js";
|
||||
import { type LiveEnvs, classify, renderCapturePlan } from "./capture.js";
|
||||
import {
|
||||
type CoolifyInstance,
|
||||
assertWritable,
|
||||
|
|
@ -19,14 +21,24 @@ import {
|
|||
renderDiff,
|
||||
} from "./diff.js";
|
||||
import { assertEnvVarPolicy } from "./envtemplate.js";
|
||||
import { desiredFromManifest, resolveCheckout } from "./resolve.js";
|
||||
import { decryptSecrets, keyFileFor, secretsFileFor } from "./secrets.js";
|
||||
import {
|
||||
desiredFromManifest,
|
||||
requiredSecrets,
|
||||
resolveCheckout,
|
||||
} from "./resolve.js";
|
||||
import {
|
||||
decryptSecrets,
|
||||
encryptSecrets,
|
||||
keyFileFor,
|
||||
secretsFileFor,
|
||||
} from "./secrets.js";
|
||||
import { serverAdd } from "./server.js";
|
||||
import { smoke } from "./smoke.js";
|
||||
import { assertTeam, formatTeam } from "./team.js";
|
||||
|
||||
const USAGE = `usage: cast apply <org>/<repo> --env <env> [--path <dir>] [--project <name>] [--hostname-overlay <file>]
|
||||
cast diff <org>/<repo> --env <env> [--full] [--project <name>]
|
||||
const USAGE = `usage: cast apply <org>/<repo> --env <env> [--path <dir>] [--project <name>] [--hostname-overlay <file>]
|
||||
cast diff <org>/<repo> --env <env> [--full] [--project <name>]
|
||||
cast capture <org>/<repo> --env <env> [--path <dir>] [--project <name>] [--generated <NAME>] [--override <NAME>] [--force]
|
||||
cast server add <name> --ip <ip> --key <file> --env <env> [--user root] [--port 22]
|
||||
cast smoke --env <env>
|
||||
cast team [--env <env>]
|
||||
|
|
@ -50,7 +62,16 @@ const USAGE = `usage: cast apply <org>/<repo> --env <env> [--path <dir>] [--proj
|
|||
repo (the default). A project built by hand in the UI is called
|
||||
whatever someone typed; \`diff\` refuses rather than reporting an
|
||||
absent project as an empty one, and this is how you point it at
|
||||
the real name.`;
|
||||
the real name.
|
||||
|
||||
capture (adopt a hand-built instance into the age secret store):
|
||||
--generated <NAME> force NAME to the \`pending-coolify-generated\` placeholder,
|
||||
for a manifest that has not declared generated_secrets yet.
|
||||
Repeatable.
|
||||
--override <NAME> supply NAME yourself instead of copying the source's value.
|
||||
The VALUE is read from \$CAST_CAPTURE_<NAME>, never from the
|
||||
command line — argv is visible in \`ps\`. Repeatable.
|
||||
--force overwrite an existing store (refused by default).`;
|
||||
|
||||
// cast is stateless: every instance-scoped input is read from the state
|
||||
// directory it is pointed at, never from a location the tool itself knows.
|
||||
|
|
@ -290,21 +311,26 @@ export async function fetchLive(
|
|||
// exists next to it.
|
||||
export function renderAbsentTarget(
|
||||
lookup: Extract<LiveLookup, { found: false }>,
|
||||
ctx: { orgRepo: string; overridden: boolean },
|
||||
ctx: { orgRepo: string; overridden: boolean; verb?: string },
|
||||
): string {
|
||||
// `capture` takes the same position as `diff`, and for the same reason: it
|
||||
// is only ever a claim about something that already exists. Against an
|
||||
// absent target it would read back zero live values and call every required
|
||||
// secret "missing" — an alarming-but-meaningless report about the wrong box.
|
||||
const verb = ctx.verb ?? "diff";
|
||||
const origin = ctx.overridden
|
||||
? "--project"
|
||||
: `derived from the repo slug ${ctx.orgRepo}`;
|
||||
const head =
|
||||
lookup.missing === "project"
|
||||
? [
|
||||
`refusing to diff: no project named "${lookup.project}" exists in this team`,
|
||||
`refusing to ${verb}: no project named "${lookup.project}" exists in this team`,
|
||||
"",
|
||||
` looked for: project "${lookup.project}" (${origin})`,
|
||||
` exists here: ${lookup.available.join(", ") || "(no projects at all)"}`,
|
||||
]
|
||||
: [
|
||||
`refusing to diff: project "${lookup.project}" has no environment "${lookup.environment}"`,
|
||||
`refusing to ${verb}: project "${lookup.project}" has no environment "${lookup.environment}"`,
|
||||
"",
|
||||
` looked for: environment "${lookup.environment}" in project "${lookup.project}"`,
|
||||
" note: cast names environments after --env, so a project built by",
|
||||
|
|
@ -316,7 +342,7 @@ export function renderAbsentTarget(
|
|||
"",
|
||||
"An absent target reads back exactly like an empty one, so continuing would diff",
|
||||
'it as "nothing exists — create everything": a clean-looking report that verified',
|
||||
"nothing. `apply` may create a target; `diff` may only ever describe one that is",
|
||||
`nothing. \`apply\` may create a target; \`${verb}\` may only ever describe one that is`,
|
||||
"already there.",
|
||||
"",
|
||||
lookup.missing === "project"
|
||||
|
|
@ -325,6 +351,79 @@ export function renderAbsentTarget(
|
|||
].join("\n");
|
||||
}
|
||||
|
||||
// A live resource's env vars, by key. `real_value` is the decrypted one and
|
||||
// needs a token with read:sensitive; `value` is what a lesser token sees.
|
||||
//
|
||||
// A 404 (a resource we just listed no longer having an envs endpoint — not
|
||||
// expected in practice, but consistent with treating "gone" as "no env vars")
|
||||
// collapses to {}; anything else (401, 5xx, network) must surface. Swallowing
|
||||
// it would make a live resource's env look EMPTY, which turns every one of its
|
||||
// vars into a spurious create in a diff, and into a spurious "missing" in a
|
||||
// capture.
|
||||
async function fetchEnv(
|
||||
client: CoolifyClient,
|
||||
l: Live,
|
||||
): Promise<Record<string, string>> {
|
||||
const base = l.kind === "database" ? "databases" : `${l.kind}s`;
|
||||
const envs = (await client.get(`/${base}/${l.uuid}/envs`).catch((err) => {
|
||||
if (err instanceof HttpError && err.status === 404) return [];
|
||||
throw err;
|
||||
})) as Array<{ key: string; real_value?: string; value: string }>;
|
||||
return Object.fromEntries(envs.map((e) => [e.key, e.real_value ?? e.value]));
|
||||
}
|
||||
|
||||
// The value for an --override, read from the ENVIRONMENT rather than argv.
|
||||
//
|
||||
// A secret passed as a command-line argument is visible in `ps` to every
|
||||
// process on the box, and lands in shell history — the same class of leak the
|
||||
// clone-auth fix (#13) exists to avoid. So --override names the secret and the
|
||||
// environment carries it.
|
||||
function readOverrides(names: string[]): Record<string, string> {
|
||||
const out: Record<string, string> = {};
|
||||
for (const name of names) {
|
||||
const varName = `CAST_CAPTURE_${name}`;
|
||||
const value = process.env[varName];
|
||||
if (value === undefined) {
|
||||
throw new Error(
|
||||
[
|
||||
`--override ${name}: no value supplied.`,
|
||||
"",
|
||||
`cast reads an override's value from ${varName}, never from the command`,
|
||||
"line — an argv value is visible in `ps` to every process on this box.",
|
||||
"",
|
||||
` ${varName}=… cast capture …`,
|
||||
].join("\n"),
|
||||
);
|
||||
}
|
||||
out[name] = value;
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
// Typed confirmation, and deliberately NOT a --yes flag.
|
||||
//
|
||||
// This verb writes an environment's secret store, once, off a box nobody is
|
||||
// going to rebuild. The entire reason it exists is that the hand-run version
|
||||
// was easy to get subtly wrong — so the last gate is a human who has read the
|
||||
// provenance column typing the environment's own name. Nothing shorter counts:
|
||||
// not "y", not a flag. Automating it means deliberately echoing the
|
||||
// environment name into cast, which is an explicit act rather than an absent
|
||||
// one.
|
||||
//
|
||||
// EOF (a closed or empty stdin) resolves to `null` and aborts. Without that
|
||||
// race, a `< /dev/null` run would hang forever on a question nobody can answer.
|
||||
async function confirmCapture(envName: string): Promise<boolean> {
|
||||
const rl = createInterface({ input: process.stdin, output: process.stdout });
|
||||
const answer = await new Promise<string | null>((resolve) => {
|
||||
rl.question(
|
||||
`\ntype the environment name to write this store (${envName}): `,
|
||||
).then(resolve, () => resolve(null));
|
||||
rl.once("close", () => resolve(null));
|
||||
});
|
||||
rl.close();
|
||||
return answer?.trim() === envName;
|
||||
}
|
||||
|
||||
async function main(): Promise<number> {
|
||||
const [command, ...rest] = process.argv.slice(2);
|
||||
if (command === "-h" || command === "--help" || command === "help") {
|
||||
|
|
@ -419,28 +518,7 @@ async function main(): Promise<number> {
|
|||
const live = lookup.found ? lookup.live : [];
|
||||
if (mode === "full") {
|
||||
for (const l of live) {
|
||||
const envs = (await client
|
||||
.get(
|
||||
`/${l.kind === "database" ? "databases" : `${l.kind}s`}/${l.uuid}/envs`,
|
||||
)
|
||||
.catch((err) => {
|
||||
// Same policy as fetchLive's environment fetch: a 404 (a
|
||||
// resource we just listed no longer having an envs endpoint —
|
||||
// not expected in practice, but consistent with treating
|
||||
// "gone" as "no env vars") collapses to []; anything else
|
||||
// (401, 5xx, network) must surface. Swallowing it here would
|
||||
// make a live resource's env look empty and turn every one of
|
||||
// its vars into a spurious create in the diff.
|
||||
if (err instanceof HttpError && err.status === 404) return [];
|
||||
throw err;
|
||||
})) as Array<{
|
||||
key: string;
|
||||
real_value?: string;
|
||||
value: string;
|
||||
}>;
|
||||
l.env = Object.fromEntries(
|
||||
envs.map((e) => [e.key, e.real_value ?? e.value]),
|
||||
);
|
||||
l.env = await fetchEnv(client, l);
|
||||
}
|
||||
}
|
||||
const report = computeDiff(desired, live, mode);
|
||||
|
|
@ -466,6 +544,139 @@ async function main(): Promise<number> {
|
|||
);
|
||||
return 0;
|
||||
}
|
||||
if (command === "capture") {
|
||||
const { values, positionals } = parseArgs({
|
||||
args: rest,
|
||||
allowPositionals: true,
|
||||
options: {
|
||||
env: { type: "string" },
|
||||
state: { type: "string" },
|
||||
path: { type: "string" },
|
||||
project: { type: "string" },
|
||||
instance: { type: "string" },
|
||||
generated: { type: "string", multiple: true },
|
||||
override: { type: "string", multiple: true },
|
||||
force: { type: "boolean", default: false },
|
||||
},
|
||||
});
|
||||
const orgRepo = positionals[0];
|
||||
const envName = values.env;
|
||||
if (!orgRepo || !envName) {
|
||||
console.error(USAGE);
|
||||
return 2;
|
||||
}
|
||||
const stateDir = stateDirFrom(values.state);
|
||||
const repoShort = orgRepo.split("/")[1];
|
||||
const projectName = values.project ?? repoShort;
|
||||
const store = secretsFileFor(stateDir, repoShort, envName);
|
||||
// Never overwrite a store by accident. `apply` never deletes; the verb
|
||||
// that WRITES the store gets the same disposition, because the thing it
|
||||
// would destroy is the only copy of values that may not exist anywhere
|
||||
// else any more.
|
||||
if (existsSync(store) && !values.force) {
|
||||
console.error(
|
||||
[
|
||||
`refusing to capture: ${store} already exists`,
|
||||
"",
|
||||
"That store may hold the only copy of values the source box no longer has.",
|
||||
"Pass --force to overwrite it deliberately, or move it aside first.",
|
||||
].join("\n"),
|
||||
);
|
||||
return 2;
|
||||
}
|
||||
const bindings = loadBindings(join(stateDir, "environments.yaml"));
|
||||
const binding = bindings.environments[envName];
|
||||
if (!binding) {
|
||||
console.error(`environment ${envName} not in environments.yaml`);
|
||||
return 2;
|
||||
}
|
||||
const recipient = binding.age_recipient;
|
||||
if (!recipient) {
|
||||
console.error(
|
||||
[
|
||||
`environment ${envName} has no age_recipient in environments.yaml`,
|
||||
"",
|
||||
"capture encrypts the store TO that recipient (the public half of the",
|
||||
"environment's age key — safe to commit next to the bindings). Add it:",
|
||||
"",
|
||||
" environments:",
|
||||
` ${envName}:`,
|
||||
" age_recipient: age1…",
|
||||
].join("\n"),
|
||||
);
|
||||
return 2;
|
||||
}
|
||||
// Same rule as apply (resolveCheckout enforces it): prod always reads the
|
||||
// default branch. A feature-branch manifest must not be able to decide
|
||||
// which names land in the prod store.
|
||||
const checkout = resolveCheckout(orgRepo, {
|
||||
env: envName,
|
||||
path: values.path,
|
||||
});
|
||||
const { required, generated } = requiredSecrets(checkout, envName);
|
||||
const overrides = readOverrides(values.override ?? []);
|
||||
const { client } = openCoolify(stateDir, values.instance, binding);
|
||||
// capture READS Coolify and writes only to the local store, so it is
|
||||
// allowed against a read-only instance — inspecting a legacy box is
|
||||
// precisely what such an instance is for. It still takes the team assert:
|
||||
// a wrong-team token reads back nothing, and "nothing" here would render
|
||||
// as "every secret is missing" against a box that is fine.
|
||||
const team = await assertTeam(client, binding.team, envName);
|
||||
console.log(`team ${formatTeam(team)} ✓`);
|
||||
const lookup = await fetchLive(client, projectName, envName);
|
||||
if (!lookup.found) {
|
||||
console.error(
|
||||
renderAbsentTarget(lookup, {
|
||||
orgRepo,
|
||||
overridden: values.project !== undefined,
|
||||
verb: "capture",
|
||||
}),
|
||||
);
|
||||
return 2;
|
||||
}
|
||||
const liveEnvs: LiveEnvs = {};
|
||||
for (const l of lookup.live) {
|
||||
// Databases hold no manifest-templated env of their own — their URL is
|
||||
// what the APPS reference, and that name is generated, not captured.
|
||||
if (l.kind === "database") continue;
|
||||
liveEnvs[l.name] = await fetchEnv(client, l);
|
||||
}
|
||||
const classification = classify(
|
||||
required,
|
||||
[...generated, ...(values.generated ?? [])],
|
||||
liveEnvs,
|
||||
overrides,
|
||||
);
|
||||
console.log(
|
||||
renderCapturePlan(classification, {
|
||||
orgRepo,
|
||||
env: envName,
|
||||
instance: values.instance ?? binding.instance ?? "default",
|
||||
store,
|
||||
recipient,
|
||||
}),
|
||||
);
|
||||
// Refuse, don't write a wrong store. Both of these are stop conditions,
|
||||
// and the plan above has already named every offending entry.
|
||||
if (
|
||||
classification.missing.length > 0 ||
|
||||
classification.conflicts.length > 0
|
||||
)
|
||||
return 2;
|
||||
if (!(await confirmCapture(envName))) {
|
||||
console.error("aborted — nothing written");
|
||||
return 2;
|
||||
}
|
||||
encryptSecrets(
|
||||
recipient,
|
||||
store,
|
||||
Object.fromEntries(classification.plan.map((d) => [d.ref, d.value])),
|
||||
);
|
||||
console.log(
|
||||
`wrote ${store} — ${classification.plan.length} name(s), encrypted to ${recipient}`,
|
||||
);
|
||||
return 0;
|
||||
}
|
||||
if (command === "server" && rest[0] === "add") {
|
||||
const { values, positionals } = parseArgs({
|
||||
args: rest.slice(1),
|
||||
|
|
|
|||
|
|
@ -2,11 +2,18 @@ export type ResolvedEnv = {
|
|||
vars: Record<string, { value: string; secret: boolean }>;
|
||||
};
|
||||
|
||||
export function resolveTemplate(
|
||||
text: string,
|
||||
secrets: Record<string, string>,
|
||||
): ResolvedEnv {
|
||||
const vars: ResolvedEnv["vars"] = {};
|
||||
// A template line, parsed but not resolved: `ref` is set when the whole RHS is
|
||||
// a single ${NAME} placeholder.
|
||||
export type TemplateVar = { key: string; rhs: string; ref?: string };
|
||||
|
||||
// ONE grammar, shared by both readers of a template — resolveTemplate (which
|
||||
// needs the values) and templateRefs (which needs only the names). Keeping
|
||||
// them on separate parsers would let the two drift, and a drift here is not
|
||||
// cosmetic: `capture` would collect a different set of names than `apply` will
|
||||
// later demand, which is precisely the "a name silently missed" failure the
|
||||
// capture verb exists to remove.
|
||||
function parseTemplate(text: string): TemplateVar[] {
|
||||
const vars: TemplateVar[] = [];
|
||||
const lines = text.split("\n");
|
||||
for (let i = 0; i < lines.length; i++) {
|
||||
const line = lines[i].trim();
|
||||
|
|
@ -18,21 +25,42 @@ export function resolveTemplate(
|
|||
);
|
||||
const [, key, rhs] = m;
|
||||
const placeholder = rhs.match(/^\$\{([A-Z][A-Z0-9_]*)\}$/);
|
||||
if (placeholder) {
|
||||
const value = secrets[placeholder[1]];
|
||||
if (value === undefined) {
|
||||
throw new Error(
|
||||
`secret ${placeholder[1]} (for ${key}) missing from the age store`,
|
||||
);
|
||||
}
|
||||
vars[key] = { value, secret: true };
|
||||
} else {
|
||||
vars.push({ key, rhs, ...(placeholder ? { ref: placeholder[1] } : {}) });
|
||||
}
|
||||
return vars;
|
||||
}
|
||||
|
||||
export function resolveTemplate(
|
||||
text: string,
|
||||
secrets: Record<string, string>,
|
||||
): ResolvedEnv {
|
||||
const vars: ResolvedEnv["vars"] = {};
|
||||
for (const { key, rhs, ref } of parseTemplate(text)) {
|
||||
if (ref === undefined) {
|
||||
vars[key] = { value: rhs, secret: false };
|
||||
continue;
|
||||
}
|
||||
const value = secrets[ref];
|
||||
if (value === undefined) {
|
||||
throw new Error(`secret ${ref} (for ${key}) missing from the age store`);
|
||||
}
|
||||
vars[key] = { value, secret: true };
|
||||
}
|
||||
return { vars };
|
||||
}
|
||||
|
||||
// The ${NAME} refs a template declares: the secret names the manifest requires,
|
||||
// paired with the env var each one lands on. `capture` reads these to learn
|
||||
// what to go and fetch — at capture time there is no store to resolve against
|
||||
// yet, which is the whole point of the verb.
|
||||
export function templateRefs(
|
||||
text: string,
|
||||
): Array<{ key: string; ref: string }> {
|
||||
return parseTemplate(text).flatMap(({ key, ref }) =>
|
||||
ref === undefined ? [] : [{ key, ref }],
|
||||
);
|
||||
}
|
||||
|
||||
// An environment may forbid variables by name pattern, declared as
|
||||
// `environments.<env>.forbidden_var_patterns` in the state repo. The rule is
|
||||
// PRESENCE, not value: a forbidden var set to "false" still refuses the apply,
|
||||
|
|
|
|||
|
|
@ -82,6 +82,19 @@ const EnvironmentSpecSchema = z
|
|||
applications: z.record(AppSpecSchema),
|
||||
databases: z.record(DatabaseSpecSchema).optional(),
|
||||
services: z.record(ServiceSpecSchema).optional(),
|
||||
// Secret names whose values the PROVIDER generates — a Coolify-created
|
||||
// Postgres/Redis URL, a service's own generated credentials. `capture`
|
||||
// writes these as the literal `pending-coolify-generated` and never copies
|
||||
// the source box's live value: that value points at the SOURCE box's
|
||||
// database, so carrying it over would be confidently wrong in a way that
|
||||
// looks entirely plausible, and the target's real URL does not exist until
|
||||
// Coolify creates the resource.
|
||||
//
|
||||
// It is a manifest property rather than a flag the operator has to
|
||||
// remember, because the manifest is what knows DATABASE_URL comes from a
|
||||
// database it declares. Optional: a manifest that names none simply has no
|
||||
// generated secrets, and `capture` will say so in its plan.
|
||||
generated_secrets: z.array(z.string()).optional(),
|
||||
})
|
||||
.strict();
|
||||
|
||||
|
|
|
|||
|
|
@ -3,7 +3,11 @@ import { existsSync, mkdtempSync, readFileSync } from "node:fs";
|
|||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import type { Desired } from "./diff.js";
|
||||
import { type ResolvedEnv, resolveTemplate } from "./envtemplate.js";
|
||||
import {
|
||||
type ResolvedEnv,
|
||||
resolveTemplate,
|
||||
templateRefs,
|
||||
} from "./envtemplate.js";
|
||||
import { loadManifest } from "./manifest.js";
|
||||
|
||||
// How cast authenticated (or failed to authenticate) a clone.
|
||||
|
|
@ -132,8 +136,11 @@ export function resolveCheckout(
|
|||
opts: { env: string; path?: string },
|
||||
): string {
|
||||
if (opts.path && opts.env === "prod") {
|
||||
// Holds for every verb that reads a manifest (apply, and now capture): a
|
||||
// feature-branch checkout must not be able to decide what prod runs, nor
|
||||
// which secret names land in prod's store.
|
||||
throw new Error(
|
||||
"apply refuses --path with --env prod: prod always reads the default branch",
|
||||
"refuses --path with --env prod: prod always reads the default branch",
|
||||
);
|
||||
}
|
||||
if (opts.path) return opts.path;
|
||||
|
|
@ -171,6 +178,73 @@ export function resolveCheckout(
|
|||
return dir;
|
||||
}
|
||||
|
||||
// One secret the manifest requires: the ${REF} a template names (the key it
|
||||
// gets in the age store), the resource that needs it, and the env var it lands
|
||||
// on there. That last pair is what `capture` reads the live value from — the
|
||||
// store is keyed by REF, but the live box knows it as `resource.key`.
|
||||
export type RequiredSecret = { ref: string; resource: string; key: string };
|
||||
|
||||
// Exactly the set of secret names an environment's manifest demands — the same
|
||||
// set `apply` will later insist on, read from the same templates by the same
|
||||
// parser. `capture` uses this to know what to go and fetch; nothing else has to
|
||||
// be told, and nothing can be silently missed.
|
||||
//
|
||||
// Deliberately does NOT take a secrets map: at capture time the store does not
|
||||
// exist yet. That is the whole point of the verb.
|
||||
export function requiredSecrets(
|
||||
checkoutDir: string,
|
||||
envName: string,
|
||||
): { required: RequiredSecret[]; generated: string[] } {
|
||||
const manifest = loadManifest(join(checkoutDir, ".infra", "manifest.yaml"));
|
||||
const envSpec = manifest.environments[envName];
|
||||
if (!envSpec) {
|
||||
throw new Error(
|
||||
`environment ${envName} not in manifest (has: ${Object.keys(manifest.environments).join(", ") || "none"})`,
|
||||
);
|
||||
}
|
||||
const required: RequiredSecret[] = [];
|
||||
const collect = (resource: string, template?: string) => {
|
||||
if (!template) return;
|
||||
const file = join(checkoutDir, ".infra", "env", template);
|
||||
if (!existsSync(file))
|
||||
throw new Error(
|
||||
`env template missing: ${file} (referenced by ${resource})`,
|
||||
);
|
||||
for (const { key, ref } of templateRefs(readFileSync(file, "utf8"))) {
|
||||
required.push({ ref, resource, key });
|
||||
}
|
||||
};
|
||||
for (const [name, app] of Object.entries(envSpec.applications)) {
|
||||
collect(name, app.env_template);
|
||||
}
|
||||
for (const [name, svc] of Object.entries(envSpec.services ?? {})) {
|
||||
collect(name, svc.env_template);
|
||||
}
|
||||
const generated = envSpec.generated_secrets ?? [];
|
||||
// A generated_secrets entry naming something no template refs is dead
|
||||
// config — and dead config in THIS list is not merely untidy, it is
|
||||
// dangerous: it reads like a guard standing over a name while standing over
|
||||
// nothing. The likeliest cause is a typo, and the consequence of the typo is
|
||||
// that the real name gets CAPTURED from the source box instead of placeheld.
|
||||
const refs = new Set(required.map((r) => r.ref));
|
||||
const dead = generated.filter((g) => !refs.has(g));
|
||||
if (dead.length > 0) {
|
||||
throw new Error(
|
||||
[
|
||||
`manifest environment ${envName}: generated_secrets names ${dead.join(", ")}, which no env template refers to`,
|
||||
"",
|
||||
` declared: ${generated.join(", ")}`,
|
||||
` templates: ${[...refs].sort().join(", ") || "(no ${...} refs at all)"}`,
|
||||
"",
|
||||
"A generated name that matches nothing guards nothing — and if this is a",
|
||||
"typo, the name it was meant to guard is being captured from the source",
|
||||
"box instead of placeheld. Fix the spelling, or drop the entry.",
|
||||
].join("\n"),
|
||||
);
|
||||
}
|
||||
return { required, generated };
|
||||
}
|
||||
|
||||
export function desiredFromManifest(
|
||||
checkoutDir: string,
|
||||
envName: string,
|
||||
|
|
|
|||
|
|
@ -22,6 +22,27 @@ export function decryptSecrets(
|
|||
return secrets;
|
||||
}
|
||||
|
||||
// Write an environment's secret store, encrypted to its recipient.
|
||||
//
|
||||
// The plaintext goes to age on STDIN and the ciphertext straight to `file`: it
|
||||
// is never a temp file, never reaches the terminal, and never lands in shell
|
||||
// history. The hand-run recipe this replaces assembled /dev/shm/prod.env and
|
||||
// relied on remembering to `shred -u` it afterwards — a step that is invisible
|
||||
// when it is skipped.
|
||||
export function encryptSecrets(
|
||||
recipient: string,
|
||||
file: string,
|
||||
vars: Record<string, string>,
|
||||
): void {
|
||||
const plaintext = `${Object.entries(vars)
|
||||
.map(([k, v]) => `${k}=${v}`)
|
||||
.join("\n")}\n`;
|
||||
execFileSync("age", ["-r", recipient, "-o", file], {
|
||||
input: plaintext,
|
||||
stdio: ["pipe", "pipe", "pipe"],
|
||||
});
|
||||
}
|
||||
|
||||
// The age identity for an environment, resolved without cast knowing anything
|
||||
// about your environment names:
|
||||
//
|
||||
|
|
|
|||
334
test/capture-cli.test.ts
Normal file
334
test/capture-cli.test.ts
Normal file
|
|
@ -0,0 +1,334 @@
|
|||
import { execFileSync, spawn } from "node:child_process";
|
||||
import {
|
||||
existsSync,
|
||||
mkdirSync,
|
||||
mkdtempSync,
|
||||
readFileSync,
|
||||
writeFileSync,
|
||||
} from "node:fs";
|
||||
import { createServer } from "node:http";
|
||||
import type { AddressInfo } from "node:net";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { afterEach, beforeAll, describe, expect, it } from "vitest";
|
||||
import { decryptSecrets } from "../src/secrets.js";
|
||||
|
||||
// End-to-end: the real CLI, a real age identity, a stub Coolify holding real
|
||||
// live values. The point is the store that comes out the other side — it is
|
||||
// decrypted and asserted on, so "exactly the names the manifest requires, no
|
||||
// more and no fewer" is checked against the actual ciphertext rather than
|
||||
// against cast's own console output.
|
||||
|
||||
const SECRETS = {
|
||||
// Points at the SOURCE box. Must NOT be carried over.
|
||||
DATABASE_URL: "postgres://user:pw@SOURCE-BOX-postgres:5432/app",
|
||||
MAILGUN_API_KEY: "key-REAL-MAILGUN-SECRET",
|
||||
OPENROUTER_API_KEY: "sk-or-REAL-OPENROUTER-SECRET",
|
||||
// A real founder. Must NOT be carried over to staging.
|
||||
ADMIN_EMAIL: "founder@real-company.com",
|
||||
// Live on the box, but the manifest never asks for it.
|
||||
UNRELATED_LIVE_VAR: "nobody-asked-for-this",
|
||||
};
|
||||
|
||||
let keyFile: string;
|
||||
let recipient: string;
|
||||
|
||||
beforeAll(() => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "cast-age-"));
|
||||
keyFile = join(dir, "age-staging.key");
|
||||
execFileSync("age-keygen", ["-o", keyFile], { stdio: "pipe" });
|
||||
const pub = execFileSync("age-keygen", ["-y", keyFile], { encoding: "utf8" });
|
||||
recipient = pub.trim();
|
||||
});
|
||||
|
||||
type Stub = { url: string; close: () => Promise<void> };
|
||||
const stubs: Stub[] = [];
|
||||
|
||||
// A Coolify with one project, one environment, one application carrying the
|
||||
// live env above.
|
||||
async function stubCoolify(): Promise<Stub> {
|
||||
const server = createServer((req, res) => {
|
||||
const path = (req.url ?? "").replace("/api/v1", "");
|
||||
const json = (body: unknown) => {
|
||||
res.writeHead(200, { "content-type": "application/json" });
|
||||
res.end(JSON.stringify(body));
|
||||
};
|
||||
if (path === "/teams/current") return json({ id: 0, name: "Root Team" });
|
||||
if (path === "/projects") return json([{ uuid: "p1", name: "incubator" }]);
|
||||
if (path === "/projects/p1/staging")
|
||||
return json({ applications: [{ name: "core", uuid: "a1" }] });
|
||||
if (path === "/applications/a1/envs")
|
||||
return json(
|
||||
Object.entries(SECRETS).map(([key, real_value]) => ({
|
||||
key,
|
||||
real_value,
|
||||
value: "REDACTED",
|
||||
})),
|
||||
);
|
||||
res.writeHead(404);
|
||||
res.end("{}");
|
||||
});
|
||||
await new Promise<void>((r) => {
|
||||
server.listen(0, "127.0.0.1", r);
|
||||
});
|
||||
const stub: Stub = {
|
||||
url: `http://127.0.0.1:${(server.address() as AddressInfo).port}`,
|
||||
close: () =>
|
||||
new Promise<void>((r) => {
|
||||
server.close(() => r());
|
||||
}),
|
||||
};
|
||||
stubs.push(stub);
|
||||
return stub;
|
||||
}
|
||||
|
||||
afterEach(async () => {
|
||||
await Promise.all(stubs.splice(0).map((s) => s.close()));
|
||||
});
|
||||
|
||||
const MANIFEST = `project: incubator
|
||||
environments:
|
||||
staging:
|
||||
generated_secrets: [DATABASE_URL_STAGING]
|
||||
applications:
|
||||
core:
|
||||
source: { repo: heavy-duty/incubator, branch: main }
|
||||
build: { pack: nixpacks, base_directory: / }
|
||||
domains: ["http://core.example.com"]
|
||||
env_template: core.staging.env.template
|
||||
`;
|
||||
|
||||
// NODE_ENV is a literal, not a secret — it must not reach the store.
|
||||
const TEMPLATE = `NODE_ENV=production
|
||||
DATABASE_URL=\${DATABASE_URL_STAGING}
|
||||
MAILGUN_API_KEY=\${MAILGUN_API_KEY}
|
||||
OPENROUTER_API_KEY=\${OPENROUTER_API_KEY}
|
||||
ADMIN_EMAIL=\${ADMIN_EMAIL}
|
||||
`;
|
||||
|
||||
function fixture(url: string, opts: { template?: string } = {}) {
|
||||
const checkout = mkdtempSync(join(tmpdir(), "cast-co-"));
|
||||
mkdirSync(join(checkout, ".infra", "env"), { recursive: true });
|
||||
writeFileSync(join(checkout, ".infra", "manifest.yaml"), MANIFEST);
|
||||
writeFileSync(
|
||||
join(checkout, ".infra", "env", "core.staging.env.template"),
|
||||
opts.template ?? TEMPLATE,
|
||||
);
|
||||
|
||||
const state = mkdtempSync(join(tmpdir(), "cast-state-"));
|
||||
mkdirSync(join(state, "secrets"));
|
||||
writeFileSync(
|
||||
join(state, ".coolify.env"),
|
||||
`COOLIFY_BASE_URL="${url}"\nCOOLIFY_ACCESS_TOKEN="t"\n`,
|
||||
);
|
||||
writeFileSync(
|
||||
join(state, "environments.yaml"),
|
||||
[
|
||||
"environments:",
|
||||
" staging:",
|
||||
" server: staging-box",
|
||||
" team: { id: 0, name: Root Team }",
|
||||
` age_recipient: ${recipient}`,
|
||||
"github_apps:",
|
||||
" incubator: hdb-coolify",
|
||||
"",
|
||||
].join("\n"),
|
||||
);
|
||||
return {
|
||||
checkout,
|
||||
state,
|
||||
store: join(state, "secrets", "incubator.staging.env.age"),
|
||||
};
|
||||
}
|
||||
|
||||
function runCapture(
|
||||
args: string[],
|
||||
opts: { stdin?: string; env?: Record<string, string> } = {},
|
||||
): Promise<{ code: number; output: string }> {
|
||||
return new Promise((resolve) => {
|
||||
const child = spawn("node", ["dist/cli.js", "capture", ...args], {
|
||||
stdio: ["pipe", "pipe", "pipe"],
|
||||
env: { ...process.env, ...opts.env },
|
||||
});
|
||||
let output = "";
|
||||
child.stdout.on("data", (d) => {
|
||||
output += String(d);
|
||||
});
|
||||
child.stderr.on("data", (d) => {
|
||||
output += String(d);
|
||||
});
|
||||
child.stdin.end(opts.stdin ?? "");
|
||||
child.on("close", (code) => resolve({ code: code ?? 0, output }));
|
||||
});
|
||||
}
|
||||
|
||||
const base = (f: ReturnType<typeof fixture>) => [
|
||||
"heavy-duty/incubator",
|
||||
"--env",
|
||||
"staging",
|
||||
"--state",
|
||||
f.state,
|
||||
"--path",
|
||||
f.checkout,
|
||||
];
|
||||
|
||||
describe("cast capture (end to end)", () => {
|
||||
it("writes a store with exactly the manifest's names, and the right provenance", async () => {
|
||||
const f = fixture((await stubCoolify()).url);
|
||||
const r = await runCapture([...base(f), "--override", "ADMIN_EMAIL"], {
|
||||
stdin: "staging\n",
|
||||
env: { CAST_CAPTURE_ADMIN_EMAIL: "operator@example.com" },
|
||||
});
|
||||
expect(r.code).toBe(0);
|
||||
expect(existsSync(f.store)).toBe(true);
|
||||
|
||||
const store = decryptSecrets(f.store, keyFile);
|
||||
// No more, no fewer: the four ${...} refs. NODE_ENV is a literal, and
|
||||
// UNRELATED_LIVE_VAR is live but unasked-for — neither belongs here.
|
||||
expect(Object.keys(store).sort()).toEqual([
|
||||
"ADMIN_EMAIL",
|
||||
"DATABASE_URL_STAGING",
|
||||
"MAILGUN_API_KEY",
|
||||
"OPENROUTER_API_KEY",
|
||||
]);
|
||||
// Captured verbatim.
|
||||
expect(store.MAILGUN_API_KEY).toBe(SECRETS.MAILGUN_API_KEY);
|
||||
expect(store.OPENROUTER_API_KEY).toBe(SECRETS.OPENROUTER_API_KEY);
|
||||
// Generated: the placeholder, NEVER the source box's own database URL.
|
||||
expect(store.DATABASE_URL_STAGING).toBe("pending-coolify-generated");
|
||||
expect(store.DATABASE_URL_STAGING).not.toContain("SOURCE-BOX");
|
||||
// Overridden: the operator's value, not the real founder's address.
|
||||
expect(store.ADMIN_EMAIL).toBe("operator@example.com");
|
||||
expect(store.ADMIN_EMAIL).not.toBe(SECRETS.ADMIN_EMAIL);
|
||||
});
|
||||
|
||||
// "No secret value is ever written to stdout" — checked against the real
|
||||
// values the stub served, on the real console output of a real run.
|
||||
it("never prints a secret value to the console", async () => {
|
||||
const f = fixture((await stubCoolify()).url);
|
||||
const r = await runCapture([...base(f), "--override", "ADMIN_EMAIL"], {
|
||||
stdin: "staging\n",
|
||||
env: { CAST_CAPTURE_ADMIN_EMAIL: "operator@example.com" },
|
||||
});
|
||||
expect(r.code).toBe(0);
|
||||
for (const value of Object.values(SECRETS)) {
|
||||
expect(r.output).not.toContain(value);
|
||||
}
|
||||
expect(r.output).not.toContain("operator@example.com");
|
||||
// It did print the NAMES, though — that is the plan.
|
||||
expect(r.output).toContain("MAILGUN_API_KEY");
|
||||
expect(r.output).toContain("captured");
|
||||
expect(r.output).toContain("generated");
|
||||
expect(r.output).toContain("overridden");
|
||||
});
|
||||
|
||||
// The plaintext exists only in memory and on age's stdin.
|
||||
it("leaves no plaintext behind — the store is real ciphertext", async () => {
|
||||
const f = fixture((await stubCoolify()).url);
|
||||
await runCapture([...base(f), "--override", "ADMIN_EMAIL"], {
|
||||
stdin: "staging\n",
|
||||
env: { CAST_CAPTURE_ADMIN_EMAIL: "operator@example.com" },
|
||||
});
|
||||
const raw = readFileSync(f.store, "utf8");
|
||||
expect(raw).toContain("age-encryption.org");
|
||||
for (const value of Object.values(SECRETS)) {
|
||||
expect(raw).not.toContain(value);
|
||||
}
|
||||
});
|
||||
|
||||
// A name required by the template but absent from the source refuses the run
|
||||
// — writing an empty would boot the app misconfigured, plausibly.
|
||||
it("refuses when a required name is absent from the source", async () => {
|
||||
const f = fixture((await stubCoolify()).url, {
|
||||
template: `${TEMPLATE}TURNSTILE_SECRET=\${TURNSTILE_SECRET}\n`,
|
||||
});
|
||||
const r = await runCapture([...base(f), "--override", "ADMIN_EMAIL"], {
|
||||
stdin: "staging\n",
|
||||
env: { CAST_CAPTURE_ADMIN_EMAIL: "operator@example.com" },
|
||||
});
|
||||
expect(r.code).not.toBe(0);
|
||||
expect(r.output).toMatch(/TURNSTILE_SECRET\s+MISSING/);
|
||||
expect(r.output).toMatch(/refusing to write the store/);
|
||||
expect(existsSync(f.store)).toBe(false);
|
||||
});
|
||||
|
||||
// The confirmation is the last gate, and it is not "y".
|
||||
it("aborts, writing nothing, when the confirmation does not name the env", async () => {
|
||||
const f = fixture((await stubCoolify()).url);
|
||||
const r = await runCapture([...base(f), "--override", "ADMIN_EMAIL"], {
|
||||
stdin: "y\n",
|
||||
env: { CAST_CAPTURE_ADMIN_EMAIL: "operator@example.com" },
|
||||
});
|
||||
expect(r.code).not.toBe(0);
|
||||
expect(r.output).toMatch(/aborted/);
|
||||
expect(existsSync(f.store)).toBe(false);
|
||||
});
|
||||
|
||||
it("aborts on a closed stdin rather than hanging", async () => {
|
||||
const f = fixture((await stubCoolify()).url);
|
||||
const r = await runCapture([...base(f), "--override", "ADMIN_EMAIL"], {
|
||||
stdin: "",
|
||||
env: { CAST_CAPTURE_ADMIN_EMAIL: "operator@example.com" },
|
||||
});
|
||||
expect(r.code).not.toBe(0);
|
||||
expect(r.output).toMatch(/aborted/);
|
||||
});
|
||||
|
||||
// An override's VALUE never comes from argv — argv is visible in `ps`.
|
||||
it("refuses an --override whose CAST_CAPTURE_<NAME> is unset", async () => {
|
||||
const f = fixture((await stubCoolify()).url);
|
||||
const r = await runCapture([...base(f), "--override", "ADMIN_EMAIL"], {
|
||||
stdin: "staging\n",
|
||||
});
|
||||
expect(r.code).not.toBe(0);
|
||||
expect(r.output).toMatch(/CAST_CAPTURE_ADMIN_EMAIL/);
|
||||
expect(r.output).toMatch(/never from the command/);
|
||||
});
|
||||
|
||||
// The store may hold the only copy of values the source box no longer has.
|
||||
it("refuses to overwrite an existing store without --force", async () => {
|
||||
const f = fixture((await stubCoolify()).url);
|
||||
writeFileSync(f.store, "PRE-EXISTING");
|
||||
const r = await runCapture([...base(f), "--override", "ADMIN_EMAIL"], {
|
||||
stdin: "staging\n",
|
||||
env: { CAST_CAPTURE_ADMIN_EMAIL: "operator@example.com" },
|
||||
});
|
||||
expect(r.code).not.toBe(0);
|
||||
expect(r.output).toMatch(/already exists/);
|
||||
expect(r.output).toMatch(/--force/);
|
||||
expect(readFileSync(f.store, "utf8")).toBe("PRE-EXISTING");
|
||||
});
|
||||
|
||||
it("refuses an environment with no age_recipient", async () => {
|
||||
const f = fixture((await stubCoolify()).url);
|
||||
writeFileSync(
|
||||
join(f.state, "environments.yaml"),
|
||||
[
|
||||
"environments:",
|
||||
" staging:",
|
||||
" server: staging-box",
|
||||
" team: { id: 0, name: Root Team }",
|
||||
"github_apps:",
|
||||
" incubator: hdb-coolify",
|
||||
"",
|
||||
].join("\n"),
|
||||
);
|
||||
const r = await runCapture(base(f), { stdin: "staging\n" });
|
||||
expect(r.code).not.toBe(0);
|
||||
expect(r.output).toMatch(/age_recipient/);
|
||||
});
|
||||
|
||||
// Same position as diff: capture is only ever a claim about something that
|
||||
// already exists. Against an absent target it would call every secret
|
||||
// "missing" — an alarming report about the wrong box.
|
||||
it("refuses an absent target rather than reporting every secret missing", async () => {
|
||||
const f = fixture((await stubCoolify()).url);
|
||||
const r = await runCapture([...base(f), "--project", "typo"], {
|
||||
stdin: "staging\n",
|
||||
});
|
||||
expect(r.code).not.toBe(0);
|
||||
expect(r.output).toMatch(/refusing to capture/);
|
||||
expect(r.output).toMatch(/no project named "typo"/);
|
||||
expect(r.output).not.toMatch(/MISSING/);
|
||||
});
|
||||
});
|
||||
309
test/capture.test.ts
Normal file
309
test/capture.test.ts
Normal file
|
|
@ -0,0 +1,309 @@
|
|||
import { mkdirSync, mkdtempSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import {
|
||||
GENERATED_PLACEHOLDER,
|
||||
classify,
|
||||
renderCapturePlan,
|
||||
} from "../src/capture.js";
|
||||
import { requiredSecrets } from "../src/resolve.js";
|
||||
|
||||
const CTX = {
|
||||
orgRepo: "heavy-duty/incubator",
|
||||
env: "prod",
|
||||
instance: "legacy",
|
||||
store: "/s/secrets/incubator.prod.env.age",
|
||||
recipient: "age1abc",
|
||||
};
|
||||
|
||||
// The live case, shrunk: an app whose env template needs a generated database
|
||||
// URL, a carried-over API key, and an address that must NOT be carried over.
|
||||
const REQUIRED = [
|
||||
{ ref: "DATABASE_URL_PROD", resource: "core", key: "DATABASE_URL" },
|
||||
{ ref: "MAILGUN_API_KEY", resource: "core", key: "MAILGUN_API_KEY" },
|
||||
{ ref: "ADMIN_EMAIL", resource: "core", key: "ADMIN_EMAIL" },
|
||||
];
|
||||
const LIVE = {
|
||||
core: {
|
||||
DATABASE_URL: "postgres://SOURCE-BOX-INTERNAL/db",
|
||||
MAILGUN_API_KEY: "key-abc123-REAL-SECRET",
|
||||
ADMIN_EMAIL: "founder@real-company.com",
|
||||
},
|
||||
};
|
||||
|
||||
describe("classify", () => {
|
||||
it("captures a live value, and records where it came from", () => {
|
||||
const c = classify([REQUIRED[1]], [], LIVE, {});
|
||||
expect(c.plan).toEqual([
|
||||
{
|
||||
ref: "MAILGUN_API_KEY",
|
||||
provenance: "captured",
|
||||
value: "key-abc123-REAL-SECRET",
|
||||
sites: [{ resource: "core", key: "MAILGUN_API_KEY" }],
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
// The failure this verb exists to prevent: the source box's DATABASE_URL
|
||||
// points at the SOURCE box's Postgres. Copying it over is confidently wrong
|
||||
// in a way that looks entirely plausible.
|
||||
it("placeholds a generated name, never copying the source's value", () => {
|
||||
const c = classify([REQUIRED[0]], ["DATABASE_URL_PROD"], LIVE, {});
|
||||
expect(c.plan[0]).toMatchObject({
|
||||
ref: "DATABASE_URL_PROD",
|
||||
provenance: "generated",
|
||||
value: GENERATED_PLACEHOLDER,
|
||||
});
|
||||
expect(c.plan[0].value).not.toContain("SOURCE-BOX");
|
||||
});
|
||||
|
||||
// staging and prod share a Mailgun domain, so a staging box carrying the
|
||||
// real ADMIN_EMAIL can mail real users.
|
||||
it("takes an override from the operator, over the source's value", () => {
|
||||
const c = classify([REQUIRED[2]], [], LIVE, {
|
||||
ADMIN_EMAIL: "operator@example.com",
|
||||
});
|
||||
expect(c.plan[0]).toMatchObject({
|
||||
provenance: "overridden",
|
||||
value: "operator@example.com",
|
||||
});
|
||||
});
|
||||
|
||||
it("an override beats a generated declaration too", () => {
|
||||
const c = classify([REQUIRED[0]], ["DATABASE_URL_PROD"], LIVE, {
|
||||
DATABASE_URL_PROD: "postgres://explicit",
|
||||
});
|
||||
expect(c.plan[0]).toMatchObject({
|
||||
provenance: "overridden",
|
||||
value: "postgres://explicit",
|
||||
});
|
||||
});
|
||||
|
||||
// Required by the template, absent from the source: refuse rather than write
|
||||
// an empty. An empty substitutes to nothing and the app boots misconfigured.
|
||||
it("refuses on a name required by the template but absent from the source", () => {
|
||||
const c = classify(
|
||||
[{ ref: "TURNSTILE_SECRET", resource: "core", key: "TURNSTILE_SECRET" }],
|
||||
[],
|
||||
LIVE,
|
||||
{},
|
||||
);
|
||||
expect(c.plan).toEqual([]);
|
||||
expect(c.missing).toEqual([
|
||||
{
|
||||
ref: "TURNSTILE_SECRET",
|
||||
sites: [{ resource: "core", key: "TURNSTILE_SECRET" }],
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("a missing name can be rescued by an override", () => {
|
||||
const c = classify(
|
||||
[{ ref: "TURNSTILE_SECRET", resource: "core", key: "TURNSTILE_SECRET" }],
|
||||
[],
|
||||
LIVE,
|
||||
{ TURNSTILE_SECRET: "supplied" },
|
||||
);
|
||||
expect(c.missing).toEqual([]);
|
||||
expect(c.plan[0].provenance).toBe("overridden");
|
||||
});
|
||||
|
||||
// One name, two resources, two different live values. The store holds one
|
||||
// value per name; picking wrong would be silent.
|
||||
it("refuses when one name carries different values on two resources", () => {
|
||||
const c = classify(
|
||||
[
|
||||
{ ref: "SHARED", resource: "core", key: "SHARED" },
|
||||
{ ref: "SHARED", resource: "worker", key: "SHARED" },
|
||||
],
|
||||
[],
|
||||
{ core: { SHARED: "a" }, worker: { SHARED: "b" } },
|
||||
{},
|
||||
);
|
||||
expect(c.plan).toEqual([]);
|
||||
expect(c.conflicts).toEqual([
|
||||
{
|
||||
ref: "SHARED",
|
||||
values: [
|
||||
{ resource: "core", key: "SHARED" },
|
||||
{ resource: "worker", key: "SHARED" },
|
||||
],
|
||||
},
|
||||
]);
|
||||
});
|
||||
|
||||
it("is fine when one name carries the SAME value on two resources", () => {
|
||||
const c = classify(
|
||||
[
|
||||
{ ref: "SHARED", resource: "core", key: "SHARED" },
|
||||
{ ref: "SHARED", resource: "worker", key: "SHARED" },
|
||||
],
|
||||
[],
|
||||
{ core: { SHARED: "same" }, worker: { SHARED: "same" } },
|
||||
{},
|
||||
);
|
||||
expect(c.conflicts).toEqual([]);
|
||||
expect(c.plan[0].value).toBe("same");
|
||||
});
|
||||
|
||||
// The acceptance criterion: exactly the names the manifest requires, no more
|
||||
// and no fewer. A live var the manifest does not ask for is not the store's
|
||||
// business.
|
||||
it("writes exactly the required names — ignoring live vars nobody asked for", () => {
|
||||
const c = classify(
|
||||
REQUIRED,
|
||||
["DATABASE_URL_PROD"],
|
||||
{
|
||||
core: { ...LIVE.core, SOME_OTHER_LIVE_VAR: "not in the manifest" },
|
||||
},
|
||||
{},
|
||||
);
|
||||
expect(c.plan.map((d) => d.ref).sort()).toEqual([
|
||||
"ADMIN_EMAIL",
|
||||
"DATABASE_URL_PROD",
|
||||
"MAILGUN_API_KEY",
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("renderCapturePlan", () => {
|
||||
// THE invariant. "No secret value is ever written to stdout" — so the plan
|
||||
// is names and provenance, and the test asserts on the actual live values
|
||||
// rather than on a pattern that could drift away from them.
|
||||
it("never prints a secret value", () => {
|
||||
const c = classify(REQUIRED, ["DATABASE_URL_PROD"], LIVE, {
|
||||
ADMIN_EMAIL: "operator@example.com",
|
||||
});
|
||||
const out = renderCapturePlan(c, CTX);
|
||||
for (const secret of [
|
||||
"postgres://SOURCE-BOX-INTERNAL/db",
|
||||
"key-abc123-REAL-SECRET",
|
||||
"founder@real-company.com",
|
||||
"operator@example.com",
|
||||
]) {
|
||||
expect(out).not.toContain(secret);
|
||||
}
|
||||
});
|
||||
|
||||
it("shows every name with its provenance and where it lands", () => {
|
||||
const c = classify(REQUIRED, ["DATABASE_URL_PROD"], LIVE, {
|
||||
ADMIN_EMAIL: "operator@example.com",
|
||||
});
|
||||
const out = renderCapturePlan(c, CTX);
|
||||
expect(out).toMatch(/MAILGUN_API_KEY\s+captured\s+core\.MAILGUN_API_KEY/);
|
||||
expect(out).toMatch(/DATABASE_URL_PROD\s+generated/);
|
||||
expect(out).toContain(GENERATED_PLACEHOLDER);
|
||||
expect(out).toMatch(/ADMIN_EMAIL\s+overridden/);
|
||||
expect(out).toContain("CAST_CAPTURE_ADMIN_EMAIL");
|
||||
expect(out).toMatch(/3 name\(s\) to write/);
|
||||
expect(out).toContain("/s/secrets/incubator.prod.env.age");
|
||||
expect(out).toContain("age1abc");
|
||||
});
|
||||
|
||||
it("names what is missing, and says why an empty would be worse", () => {
|
||||
const c = classify(
|
||||
[{ ref: "TURNSTILE_SECRET", resource: "core", key: "TURNSTILE_SECRET" }],
|
||||
[],
|
||||
LIVE,
|
||||
{},
|
||||
);
|
||||
const out = renderCapturePlan(c, CTX);
|
||||
expect(out).toMatch(/TURNSTILE_SECRET\s+MISSING/);
|
||||
expect(out).toMatch(/refusing to write the store/);
|
||||
expect(out).toMatch(/--override/);
|
||||
});
|
||||
|
||||
it("names a conflict rather than picking a side", () => {
|
||||
const c = classify(
|
||||
[
|
||||
{ ref: "SHARED", resource: "core", key: "SHARED" },
|
||||
{ ref: "SHARED", resource: "worker", key: "SHARED" },
|
||||
],
|
||||
[],
|
||||
{ core: { SHARED: "a" }, worker: { SHARED: "b" } },
|
||||
{},
|
||||
);
|
||||
const out = renderCapturePlan(c, CTX);
|
||||
expect(out).toMatch(/SHARED\s+CONFLICT/);
|
||||
expect(out).toMatch(/core\.SHARED and worker\.SHARED/);
|
||||
expect(out).not.toMatch(/\ba\b.*\bb\b/);
|
||||
});
|
||||
});
|
||||
|
||||
// requiredSecrets is what makes "no more, no fewer" true: the set comes from
|
||||
// the manifest's own templates, read by the same parser apply uses.
|
||||
describe("requiredSecrets", () => {
|
||||
function checkout(manifest: string, templates: Record<string, string>) {
|
||||
const dir = mkdtempSync(join(tmpdir(), "cast-cap-"));
|
||||
mkdirSync(join(dir, ".infra", "env"), { recursive: true });
|
||||
writeFileSync(join(dir, ".infra", "manifest.yaml"), manifest);
|
||||
for (const [name, body] of Object.entries(templates)) {
|
||||
writeFileSync(join(dir, ".infra", "env", name), body);
|
||||
}
|
||||
return dir;
|
||||
}
|
||||
|
||||
const MANIFEST = `project: incubator
|
||||
environments:
|
||||
prod:
|
||||
generated_secrets: [DATABASE_URL_PROD]
|
||||
applications:
|
||||
core:
|
||||
source: { repo: heavy-duty/incubator, branch: main }
|
||||
build: { pack: dockercompose, base_directory: /, compose_file: docker-compose.yaml }
|
||||
service_domains:
|
||||
api: ["https://api.example.com"]
|
||||
env_template: core.prod.env.template
|
||||
services:
|
||||
umami:
|
||||
type: umami
|
||||
env_template: umami.prod.env.template
|
||||
`;
|
||||
|
||||
it("collects the ${...} refs from every app and service template", () => {
|
||||
const dir = checkout(MANIFEST, {
|
||||
"core.prod.env.template":
|
||||
"NODE_ENV=production\nDATABASE_URL=${DATABASE_URL_PROD}\nMAILGUN_API_KEY=${MAILGUN_API_KEY}\n",
|
||||
"umami.prod.env.template": "APP_SECRET=${UMAMI_APP_SECRET}\n",
|
||||
});
|
||||
const { required, generated } = requiredSecrets(dir, "prod");
|
||||
expect(required).toEqual([
|
||||
{ ref: "DATABASE_URL_PROD", resource: "core", key: "DATABASE_URL" },
|
||||
{ ref: "MAILGUN_API_KEY", resource: "core", key: "MAILGUN_API_KEY" },
|
||||
{ ref: "UMAMI_APP_SECRET", resource: "umami", key: "APP_SECRET" },
|
||||
]);
|
||||
expect(generated).toEqual(["DATABASE_URL_PROD"]);
|
||||
});
|
||||
|
||||
// A non-placeholder line (NODE_ENV=production) is not a secret and must not
|
||||
// land in the store — the store holds the ${...} refs, nothing else.
|
||||
it("ignores literal template values — only ${...} refs are secrets", () => {
|
||||
const dir = checkout(MANIFEST, {
|
||||
"core.prod.env.template":
|
||||
"NODE_ENV=production\nREPORTING_ENABLED=false\nDATABASE_URL=${DATABASE_URL_PROD}\n",
|
||||
"umami.prod.env.template": "",
|
||||
});
|
||||
const { required } = requiredSecrets(dir, "prod");
|
||||
expect(required.map((r) => r.ref)).toEqual(["DATABASE_URL_PROD"]);
|
||||
});
|
||||
|
||||
// Dead config in THIS list is dangerous, not merely untidy: it reads like a
|
||||
// guard standing over a name while standing over nothing, and the likeliest
|
||||
// cause is a typo whose real name is then CAPTURED from the source box.
|
||||
it("refuses a generated_secrets entry that no template refers to", () => {
|
||||
const dir = checkout(
|
||||
MANIFEST.replace(
|
||||
"generated_secrets: [DATABASE_URL_PROD]",
|
||||
"generated_secrets: [DATABASE_URL_TYPO]",
|
||||
),
|
||||
{
|
||||
"core.prod.env.template": "DATABASE_URL=${DATABASE_URL_PROD}\n",
|
||||
"umami.prod.env.template": "",
|
||||
},
|
||||
);
|
||||
expect(() => requiredSecrets(dir, "prod")).toThrow(
|
||||
/generated_secrets names DATABASE_URL_TYPO/,
|
||||
);
|
||||
});
|
||||
});
|
||||
Loading…
Reference in a new issue