Merge pull request #33 from claude-hdb/feat/inventory-emit-draft
inventory --emit-draft: a reviewable blueprint of a live instance (#27)
This commit is contained in:
commit
7d65524054
6 changed files with 2632 additions and 3 deletions
127
README.md
127
README.md
|
|
@ -69,6 +69,7 @@ cast diff <org>/<repo> --env <env> [--full]
|
|||
cast diff --env <env> --all [--full] # no repo: EVERY registered project
|
||||
cast capture <org>/<repo> --env <env> [--generated <NAME>] [--override <NAME>]
|
||||
cast inventory <org>/<repo> --env <env>
|
||||
cast inventory --env <env> [--emit-draft <dir> [--recipient age1…] [--no-secrets]]
|
||||
cast server add <name> --ip <ip> --key <file> --env <env> [--user root] [--port 22]
|
||||
cast smoke <org>/<repo> --env <env> [--project <name>] [--environment <name>]
|
||||
cast team [--env <env>]
|
||||
|
|
@ -91,6 +92,12 @@ cast team [--env <env>]
|
|||
no age key, and no recipient — it runs *before* adoption, which is the point of
|
||||
it. A document, read by a person; nothing here is consumed by `apply`. See
|
||||
*Adopting a hand-built instance*.
|
||||
- **`inventory --emit-draft <dir>`** — the sweep, written down as a **draft of
|
||||
cast's own inputs**: a manifest per project, env templates, an
|
||||
`environments.yaml` with the registry, an age store, and `UNCAPTURED.md`. A
|
||||
**proposal**, never desired state — `apply` does not read it. It is how a
|
||||
project that has *no* manifest gets its first one, and how you take a
|
||||
point-in-time blueprint of a box. See *Drafting a box that was never declared*.
|
||||
- **`capture`** — the adoption path: reads a hand-built instance's live env and
|
||||
writes the environment's age store from it. See *Adopting a hand-built
|
||||
instance* below.
|
||||
|
|
@ -323,6 +330,126 @@ build pack, the hostname-overlay shapes, and the places Coolify 4.1.2 does not
|
|||
cooperate — each citation verified against `coollabsio/coolify` v4.1.2 and the
|
||||
vendored OpenAPI in `reference/`. Read it before changing `apply`.
|
||||
|
||||
## Drafting a box that was never declared
|
||||
|
||||
`inventory` can see a whole instance. `--emit-draft` makes it **write down what
|
||||
it sees**, in the shape of cast's own inputs:
|
||||
|
||||
```sh
|
||||
cast inventory --env prod --instance box-b --emit-draft ./draft --recipient age1…
|
||||
```
|
||||
|
||||
```
|
||||
draft/
|
||||
environments.yaml # bindings as far as they can be read — with the projects: registry
|
||||
incubator/.infra/manifest.yaml # one per project
|
||||
incubator/.infra/env/*.env.template
|
||||
la-familia/.infra/manifest.yaml # …including the client sites nobody ever declared
|
||||
secrets/<project>.<env>.env.age # encrypted to a recipient you name
|
||||
UNCAPTURED.md # ← the important file
|
||||
```
|
||||
|
||||
Two uses: **bootstrapping a project that has no manifest** (the third-party sites
|
||||
on the box being drained were never declared, and never will be unless something
|
||||
writes the first draft — hand-transcribing them from a UI is exactly the work
|
||||
cast exists to eliminate), and **a point-in-time blueprint** you could rebuild an
|
||||
instance from.
|
||||
|
||||
### A draft is a PROPOSAL
|
||||
|
||||
**It is never desired state, and `apply` never reads it.** It is emitted,
|
||||
reviewed by a human, and lands in a repo as a PR — the same shape as
|
||||
`terraform import` → HCL:
|
||||
|
||||
**sweep → emit draft → you read it → manifest PR → `capture` → `apply`**
|
||||
|
||||
That boundary is the only reason the verb is allowed to exist, and it is
|
||||
enforced, not merely documented:
|
||||
|
||||
- It **never emits into a repo that already has a manifest.** For a declared
|
||||
project the manifest *is* the truth, and one regenerated from a live box would
|
||||
let that box's accumulated cruft overwrite the reviewed spec — in the one
|
||||
direction nobody reviews. Adoption is one-way. A non-empty target directory is
|
||||
refused, and so is writing a manifest over an existing one.
|
||||
- `--emit-draft` is **sweep-mode only**. With a repo, `inventory` is reconciling
|
||||
against a manifest that already exists, which is exactly the case where a draft
|
||||
must not be written. Refused.
|
||||
|
||||
### Two things would make a draft actively dangerous
|
||||
|
||||
**1. Copied provider-generated values.** A `DATABASE_URL` read off the source
|
||||
points at the *source box's* Postgres. Emit it, rebuild elsewhere, and the new
|
||||
box comes up **working** — reading and writing the old box's database. You find
|
||||
out the day the old box is deleted. Same for `REDIS_URL`, and for Coolify's own
|
||||
magic vars (`SERVICE_FQDN_*`, `SERVICE_URL_*`, `SERVICE_PASSWORD_*`), which are
|
||||
generated per-instance and mean nothing anywhere else.
|
||||
|
||||
So the draft applies **`capture`'s discipline**: a provider-generated name is
|
||||
**placeheld** with the same `pending-coolify-generated` literal, its live value
|
||||
is not written into any artifact, and it is listed for disposition. The emitted
|
||||
manifest declares it under `generated_secrets:`, so a later `capture` placeholds
|
||||
it again with no flag to remember. A draft that is confidently wrong in four
|
||||
entries out of seventeen is worse than one that is obviously incomplete.
|
||||
|
||||
The rule is **by name** — two families: Coolify's `SERVICE_*` magic vars, and any
|
||||
name carrying a *datastore* word (`DATABASE`, `DB`, `POSTGRES`, `REDIS`, …) and a
|
||||
*connection* word (`URL`, `HOST`, `PASSWORD`, …) as segments. It errs **wide** on
|
||||
purpose, because the two errors are not symmetric: over-matching a real secret
|
||||
placeholds it loudly and you put it back, while under-matching a generated one
|
||||
copies it silently and rebuilds a box that quietly uses a dead machine's
|
||||
database. Every value cast read is printed with its disposition — names and
|
||||
provenance, never values — and a var that points at the source box under a name
|
||||
cast does not recognize **will** have been copied. Read the table.
|
||||
|
||||
Every other live var becomes a `${REF}`, with its value in the **age store** —
|
||||
never a literal in a committed file. cast cannot know which of a box's vars are
|
||||
secret (nobody wrote it down; that is why this verb exists), and a live API key
|
||||
written as a literal is a key in a git repo. Move the plainly-not-secret ones
|
||||
back to literals yourself, in review.
|
||||
|
||||
The store is encrypted to a recipient you **name** — `--recipient age1…`, or the
|
||||
environment's `age_recipient` binding. With neither, cast **refuses**: a draft
|
||||
whose secrets were silently skipped looks complete and holds not one value.
|
||||
`--no-secrets` says so deliberately.
|
||||
|
||||
**2. Silent losses.** cast cannot express everything a Coolify holds:
|
||||
destinations (which Docker network a resource sits on — no API at all in 4.1.2),
|
||||
service hostnames (they live per-container on `service.applications[].fqdn`),
|
||||
Basic Auth and custom Traefik labels, the *Include Source Commit in Build*
|
||||
toggle, whole database kinds (a MySQL is invisible to cast's manifest), backup
|
||||
schedules, and anything else configured in the UI with no manifest field.
|
||||
|
||||
A blueprint that omits these **without saying so** is worse than no blueprint,
|
||||
because in a disaster you would trust it and rebuild a *different box*. So
|
||||
**`UNCAPTURED.md` is a first-class output**, listing per resource every live
|
||||
setting cast saw and could not express — and it is **written on every run**, even
|
||||
when it has little to say.
|
||||
|
||||
### What a blueprint still cannot restore
|
||||
|
||||
Worth stating plainly, because "rebuild from the repo" is routinely over-claimed:
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| control plane | `rig coolify install` ✅ |
|
||||
| structure | draft → manifest PR → `apply` ✅ |
|
||||
| secret **values** | the age store + your key ✅ |
|
||||
| **data** | Coolify's DB backups → S3 ✅ (a separate path) |
|
||||
| **the GitHub App private key** | ❌ re-create by hand |
|
||||
| **S3 access keys** | ❌ re-mint by hand |
|
||||
|
||||
The last two are **not in the repo** — correctly; it holds no live credentials —
|
||||
and cannot be regenerated from it. A DR runbook has to say so. The same table is
|
||||
emitted into every `UNCAPTURED.md`, because that is the file someone will be
|
||||
reading at the worst possible moment.
|
||||
|
||||
One project per Coolify environment: a project with resources in **two**
|
||||
populated environments is a tie cast will not break (picking would emit a
|
||||
blueprint of half a box), so it refuses and `--environment <name>` says which. It
|
||||
is a tiebreak, not a filter — a project with only one populated environment is
|
||||
drafted from it either way, which is what keeps the client sites (each alone in
|
||||
Coolify's default `production`) in a blueprint that claims to describe the box.
|
||||
|
||||
## Secrets, and attended applies
|
||||
|
||||
An environment's age identity is resolved in exactly two ways:
|
||||
|
|
|
|||
|
|
@ -424,6 +424,111 @@ the plan. There is no `--yes`: a store written without someone reading the
|
|||
provenance column is the outcome the verb exists to prevent. A closed stdin
|
||||
aborts rather than hanging.
|
||||
|
||||
## Drafts (`inventory --emit-draft`)
|
||||
|
||||
`inventory` with no repo sweeps an instance. `--emit-draft <dir>` writes that
|
||||
sweep down as a draft of cast's **own inputs** — a manifest per project, env
|
||||
templates, an `environments.yaml` carrying the `projects:` registry, an age store
|
||||
per project, and `UNCAPTURED.md`.
|
||||
|
||||
**A draft is a PROPOSAL. It is never desired state, and `apply` never reads it.**
|
||||
|
||||
sweep → emit draft → a human reads it → manifest PR → capture → apply
|
||||
|
||||
Same shape as `terraform import` → HCL. Every other rule in this section follows
|
||||
from that one, and each is enforced rather than merely stated:
|
||||
|
||||
| refusal | why |
|
||||
| --- | --- |
|
||||
| a **non-empty** target directory | emitted over a repo that has a manifest, a draft would overwrite a reviewed spec with a live box's accumulated cruft — the one direction nobody reviews. **Adoption is one-way.** |
|
||||
| an **existing manifest** at the path it would write | the same invariant, once more at the file (`assertNoExistingManifest`). For a declared project the manifest *is* the truth; `cast inventory <org>/<repo>` reconciles it instead. |
|
||||
| `--emit-draft` with a **repo positional** | with a repo, inventory reconciles against a manifest that already exists — exactly the case where a draft must not be written. |
|
||||
| **no age recipient** (and no `--no-secrets`) | a draft whose store was silently skipped *looks complete*: a manifest, templates full of `${REF}`s, and not one value anywhere. You would find out when `apply` refused, some time after the box those values were on stopped existing. |
|
||||
| a project with **two populated environments** | a draft carries one environment per project. Picking would emit a blueprint of *half a box* that says nothing about the other half. `--environment` breaks the tie — as a **tiebreak, not a filter**: a project with one populated environment is drafted from it either way, or filtering by name would drop whole projects (each client site sits alone in Coolify's default `production`) out of a blueprint that claims to describe the box. |
|
||||
|
||||
**Provider-generated names are placeheld, never copied.** This is `capture`'s
|
||||
discipline (see above), applied to a verb that has no manifest to tell it which
|
||||
names are generated — so it decides **by name**, in two families:
|
||||
|
||||
1. Coolify's per-instance magic vars — `SERVICE_FQDN_*`, `SERVICE_URL_*`,
|
||||
`SERVICE_PASSWORD_*`, `SERVICE_USER_*`, `SERVICE_BASE64_*`.
|
||||
2. Any name carrying a **datastore** word (`DATABASE`, `DB`, `POSTGRES`, `PG`,
|
||||
`REDIS`, `MONGO`, …) *and* a **connection** word (`URL`, `URI`, `DSN`, `HOST`,
|
||||
`PORT`, `PASSWORD`, `USER`, …) as underscore-delimited segments —
|
||||
`DATABASE_URL`, `UMAMI_DATABASE_URL`, `REDIS_URL_PROD`, `DB_HOST`.
|
||||
|
||||
Each such name is written as the literal `pending-coolify-generated`, listed in
|
||||
the run's disposition table, and declared under the emitted manifest's
|
||||
`generated_secrets:` — so a later `capture` placeholds it again with no flag to
|
||||
remember. **Its live value is not written into any artifact.**
|
||||
|
||||
The rule errs **wide**, deliberately, because the two errors are not symmetric:
|
||||
|
||||
- over-match a real secret → it is placeheld, reported, and you put the value
|
||||
back. Noisy, recoverable, **loud**.
|
||||
- under-match a generated one → it is copied, and a box rebuilt from the draft
|
||||
comes up **working**, reading and writing the *source box's* database, until
|
||||
the day that box is deleted. Silent, unrecoverable, **quiet**.
|
||||
|
||||
It is a name-pattern rule, not a promise: a var that points at the source box
|
||||
under a name cast does not recognize **will** be copied. The disposition table
|
||||
(names and provenance, **never values** — same contract as the capture plan) is
|
||||
what a reviewer reads to catch it.
|
||||
|
||||
**Every other live var becomes a `${REF}`**, with its value in the age store —
|
||||
never a template literal. cast cannot know which of a box's vars are secret
|
||||
(nobody wrote it down, which is why the verb exists), and the two mistakes are
|
||||
again asymmetric: a non-secret in an encrypted store is untidy, a live API key
|
||||
written as a literal into a manifest is a key in a git repo. One name carrying
|
||||
**different values** on two resources is not a conflict cast resolves (one store
|
||||
holds one value per name — see `capture`'s CONFLICT refusal): both are kept,
|
||||
under `<RESOURCE>_<KEY>` refs, and the split is reported.
|
||||
|
||||
**`UNCAPTURED.md` is a first-class output, emitted on every run.** cast cannot
|
||||
express everything a Coolify holds, and a blueprint that omits those things
|
||||
without saying so is worse than no blueprint — in a disaster you would trust it
|
||||
and rebuild a *different box*. Per resource, it names what was seen and could not
|
||||
be written: `destination_id` (which Docker network — no destinations API in 4.1.2
|
||||
to resolve it to the UUID `destination_uuid:` wants, #21), service hostnames (no
|
||||
flat `domains` on a Coolify 4.1.2 service), Basic Auth / custom Traefik labels,
|
||||
build and deploy command overrides, backup schedules (not exposed on a database's
|
||||
GET — **a rebuild has no backups until you declare them**), database kinds cast
|
||||
does not model (MySQL, MariaDB, MongoDB, KeyDB, Dragonfly, ClickHouse — named,
|
||||
never silently dropped), env var names a cast template cannot express, and
|
||||
applications whose build pack the manifest has no vocabulary for (left *out* of
|
||||
the manifest rather than fabricated into the nearest pack).
|
||||
|
||||
It also carries the table below, because that is the file someone will be reading
|
||||
at the worst possible moment.
|
||||
|
||||
### What a blueprint still cannot restore
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| control plane | `rig coolify install` ✅ |
|
||||
| structure | draft → manifest PR → `apply` ✅ |
|
||||
| secret **values** | the age store + your key ✅ |
|
||||
| **data** | Coolify's DB backups → S3 ✅ (a separate path) |
|
||||
| **the GitHub App private key** | ❌ re-create by hand |
|
||||
| **S3 access keys** | ❌ re-mint by hand |
|
||||
|
||||
The last two are not in the state repo — correctly, it holds no live credentials
|
||||
— and cannot be regenerated from it. A DR runbook that does not say so is not a
|
||||
runbook.
|
||||
|
||||
**What the box cannot tell you, and cast therefore does not invent:** the
|
||||
`<org>/<repo>` slug comes from an application's git remote (the only place a live
|
||||
box knows it), so a project with **no application** — a lone service — has no repo
|
||||
on the box at all. cast writes the bare project name as the registry key, and the
|
||||
registry's own parse-time refusal (*"a registry key has no meaning without its
|
||||
org"*) then stops the file being used until a human supplies it. That refusal is
|
||||
the design: the alternatives are inventing an org, or leaving the project out of
|
||||
the registry — and a project missing from the registry is one every fleet run
|
||||
skips **in silence**. Likewise `github_apps`: nothing Coolify returns about an
|
||||
application says which App cloned it, so cast binds every repo to the instance's
|
||||
only GitHub App when there is exactly one (there is no other it could be), and
|
||||
writes a `REVIEW-…` marker when there is not.
|
||||
|
||||
## Cloning a private manifest
|
||||
|
||||
`resolveCheckout` resolves git credentials **inside cast**, in a fixed order —
|
||||
|
|
|
|||
191
src/cli.ts
191
src/cli.ts
|
|
@ -34,6 +34,17 @@ import {
|
|||
computeDiff,
|
||||
renderDiff,
|
||||
} from "./diff.js";
|
||||
import {
|
||||
type DraftProject,
|
||||
assertEmptyTarget,
|
||||
draftResourcesFrom,
|
||||
emitDraft,
|
||||
planDraft,
|
||||
renderAmbiguousEnvironments,
|
||||
renderDraftPlan,
|
||||
renderNoRecipient,
|
||||
renderRepoWithDraft,
|
||||
} from "./draft.js";
|
||||
import { assertEnvVarPolicy } from "./envtemplate.js";
|
||||
import {
|
||||
type ProjectOutcome,
|
||||
|
|
@ -78,6 +89,7 @@ const USAGE = `usage: cast apply <org>/<repo> --env <env> [--path <dir>] [--
|
|||
cast capture <org>/<repo> --env <env> [--path <dir>] [--project <name>] [--environment <name>] [--generated <NAME>] [--override <NAME>] [--force]
|
||||
cast inventory <org>/<repo> --env <env> [--path <dir>] [--project <name>] [--environment <name>] [--resource <m>=<l>]
|
||||
cast inventory --env <env> [--instance <name>] # no repo: SWEEP the whole instance
|
||||
cast inventory --env <env> --emit-draft <dir> [--recipient age1…] [--no-secrets]
|
||||
cast server add <name> --ip <ip> --key <file> --env <env> [--user root] [--port 22]
|
||||
cast smoke <org>/<repo> --env <env> [--project <name>] [--environment <name>]
|
||||
cast team [--env <env>]
|
||||
|
|
@ -137,7 +149,27 @@ capture (adopt a hand-built instance into the age secret store):
|
|||
--override <NAME> supply NAME yourself instead of copying the source's value.
|
||||
The VALUE is read from \$CAST_CAPTURE_<NAME>, never from the
|
||||
command line — argv is visible in \`ps\`. Repeatable.
|
||||
--force overwrite an existing store (refused by default).`;
|
||||
--force overwrite an existing store (refused by default).
|
||||
|
||||
inventory --emit-draft (write down what a box has, as a PROPOSAL):
|
||||
--emit-draft <dir> emit what the sweep saw as a draft of cast's own inputs — a
|
||||
manifest per project, env templates, an environments.yaml
|
||||
carrying the \`projects:\` registry, an age store per project,
|
||||
and UNCAPTURED.md. Into a NEW directory, always: a draft is a
|
||||
proposal, reviewed by a human and landed as a PR, and \`apply\`
|
||||
never reads one. SWEEP MODE ONLY — with a repo there is already
|
||||
a manifest, and a manifest regenerated from a live box would
|
||||
overwrite a reviewed spec with that box's accumulated cruft.
|
||||
--recipient age1… the age recipient the draft's stores are encrypted to. Defaults
|
||||
to the environment's \`age_recipient\` binding.
|
||||
--no-secrets emit no stores. Required when no recipient is available: cast
|
||||
will not silently drop the values it read off the box.
|
||||
--environment <name> a TIEBREAK, not a filter: which environment to draft for a
|
||||
project that has resources in more than one (cast refuses to
|
||||
pick). A project with only one populated environment is drafted
|
||||
from it either way — filtering the instance by an environment
|
||||
name would drop whole projects out of a blueprint that claims to
|
||||
describe the box.`;
|
||||
|
||||
// cast is stateless: every instance-scoped input is read from the state
|
||||
// directory it is pointed at, never from a location the tool itself knows.
|
||||
|
|
@ -568,7 +600,7 @@ export function aliasLive<T extends { name: string }>(
|
|||
// capture.
|
||||
async function fetchEnv(
|
||||
client: CoolifyClient,
|
||||
l: Live,
|
||||
l: Pick<Live, "kind" | "uuid">,
|
||||
): Promise<Record<string, string>> {
|
||||
const base = l.kind === "database" ? "databases" : `${l.kind}s`;
|
||||
const envs = (await client.get(`/${base}/${l.uuid}/envs`).catch((err) => {
|
||||
|
|
@ -1155,6 +1187,9 @@ async function main(): Promise<number> {
|
|||
environment: { type: "string" },
|
||||
resource: { type: "string", multiple: true },
|
||||
instance: { type: "string" },
|
||||
"emit-draft": { type: "string" },
|
||||
recipient: { type: "string" },
|
||||
"no-secrets": { type: "boolean", default: false },
|
||||
},
|
||||
});
|
||||
const orgRepo = positionals[0];
|
||||
|
|
@ -1163,6 +1198,17 @@ async function main(): Promise<number> {
|
|||
console.error(USAGE);
|
||||
return 2;
|
||||
}
|
||||
const draftDir = values["emit-draft"];
|
||||
// A draft is emitted from the SWEEP, and only from the sweep. With a repo,
|
||||
// inventory is reconciling against a manifest that already exists — which is
|
||||
// exactly the case where a draft must not be written: for a declared project
|
||||
// the manifest IS the truth, and one regenerated from a live box would let
|
||||
// that box's accumulated cruft overwrite the reviewed spec. Adoption is
|
||||
// one-way, so the two flags cannot be combined at all.
|
||||
if (draftDir && orgRepo) {
|
||||
console.error(renderRepoWithDraft(orgRepo, draftDir));
|
||||
return 2;
|
||||
}
|
||||
const stateDir = stateDirFrom(values.state);
|
||||
const sweepBindings = loadBindings(join(stateDir, "environments.yaml"));
|
||||
const sweepBinding = sweepBindings.environments[envName];
|
||||
|
|
@ -1176,6 +1222,28 @@ async function main(): Promise<number> {
|
|||
// made inventory a discovery verb that needed you to have already
|
||||
// discovered.
|
||||
if (!orgRepo) {
|
||||
// Both refusals BEFORE the first live call. A draft that is going to be
|
||||
// refused should be refused before an operator watches a whole instance be
|
||||
// swept for it — and, more to the point, before cast reads every env var on
|
||||
// a box it is then not going to write down.
|
||||
const recipient = values.recipient ?? sweepBinding.age_recipient;
|
||||
if (draftDir) {
|
||||
try {
|
||||
assertEmptyTarget(draftDir);
|
||||
} catch (err) {
|
||||
console.error(err instanceof Error ? err.message : String(err));
|
||||
return 2;
|
||||
}
|
||||
// No recipient, no store — and cast will not make that decision quietly.
|
||||
// Silently skipping the secrets would emit a draft that LOOKS complete: a
|
||||
// manifest, templates full of ${REF}s, and nothing anywhere holding a
|
||||
// single value. You would find out when `apply` refused, having already
|
||||
// deleted the box the values were on.
|
||||
if (!recipient && !values["no-secrets"]) {
|
||||
console.error(renderNoRecipient(envName));
|
||||
return 2;
|
||||
}
|
||||
}
|
||||
const { instance, client } = openCoolify(
|
||||
stateDir,
|
||||
values.instance,
|
||||
|
|
@ -1186,8 +1254,9 @@ async function main(): Promise<number> {
|
|||
// truthfully reports that it is empty.
|
||||
const team = await assertTeam(client, sweepBinding.team, envName);
|
||||
console.log(`team ${formatTeam(team)} ✓`);
|
||||
const live = await client.projects();
|
||||
const projects: SweepProject[] = [];
|
||||
for (const p of await client.projects()) {
|
||||
for (const p of live) {
|
||||
const environments: SweepEnvironment[] = [];
|
||||
for (const name of await client.environments(p.uuid)) {
|
||||
const found = await fetchLive(client, p.name, name);
|
||||
|
|
@ -1206,6 +1275,122 @@ async function main(): Promise<number> {
|
|||
baseUrl: instance.baseUrl,
|
||||
}),
|
||||
);
|
||||
if (!draftDir) return 0;
|
||||
|
||||
// --- The draft (#27) ---
|
||||
//
|
||||
// The sweep above is a DOCUMENT. This is the same reading, written into the
|
||||
// shape of cast's own inputs — and it is still a proposal, not desired
|
||||
// state. See draft.ts for the boundary that lets this verb exist at all.
|
||||
//
|
||||
// A project with resources in TWO environments cannot be drafted without
|
||||
// picking one, and cast does not pick: a blueprint of half a box, silently
|
||||
// chosen, is the failure mode this whole issue is about. --environment says
|
||||
// which.
|
||||
//
|
||||
// --environment is a TIEBREAK here, not a filter. A project with resources
|
||||
// in exactly one environment has no tie to break, and is drafted from it
|
||||
// whatever the flag says — filtering the instance by an environment NAME
|
||||
// would drop the projects that most need drafting (the third-party sites,
|
||||
// each sitting in its own Coolify-default `production`) out of a blueprint
|
||||
// that claims to describe the box.
|
||||
const populatedIn = (p: SweepProject) =>
|
||||
p.environments.filter((e) => e.resources.length > 0);
|
||||
const pick = (p: SweepProject) => {
|
||||
const populated = populatedIn(p);
|
||||
return populated.length === 1
|
||||
? populated[0]
|
||||
: populated.find((e) => e.name === values.environment);
|
||||
};
|
||||
const ambiguous = projects.filter(
|
||||
(p) => populatedIn(p).length > 1 && !pick(p),
|
||||
);
|
||||
if (ambiguous.length > 0) {
|
||||
console.error(
|
||||
renderAmbiguousEnvironments(
|
||||
ambiguous.map((p) => ({
|
||||
name: p.name,
|
||||
environments: populatedIn(p).map(
|
||||
(e) => `${e.name} (${e.resources.length})`,
|
||||
),
|
||||
})),
|
||||
envName,
|
||||
),
|
||||
);
|
||||
return 2;
|
||||
}
|
||||
const draftProjects: DraftProject[] = [];
|
||||
for (const p of live) {
|
||||
const swept = projects.find((s) => s.name === p.name);
|
||||
const populated = swept ? populatedIn(swept) : [];
|
||||
const chosen = swept ? pick(swept) : undefined;
|
||||
const others = populated
|
||||
.filter((e) => e.name !== chosen?.name)
|
||||
.map((e) => ({ name: e.name, resources: e.resources.length }));
|
||||
if (!chosen) {
|
||||
// Not drafted — and SAID, in UNCAPTURED.md, rather than left out of a
|
||||
// blueprint that a reader would take for the whole box.
|
||||
draftProjects.push({
|
||||
name: p.name,
|
||||
coolifyEnv: "(none)",
|
||||
resources: [],
|
||||
unreadable: [],
|
||||
otherEnvironments: others,
|
||||
skipReason: "every environment on it is empty",
|
||||
});
|
||||
continue;
|
||||
}
|
||||
// The RAW environment document, not fetchLive's projection: the uncaptured
|
||||
// pass's whole job is to notice fields cast has no home for, and it cannot
|
||||
// notice what a projection has already thrown away.
|
||||
const raw = (await client.get(
|
||||
`/projects/${p.uuid}/${chosen.name}`,
|
||||
)) as Record<string, unknown> | null;
|
||||
const { resources, unreadable } = draftResourcesFrom(raw ?? {});
|
||||
for (const r of resources) {
|
||||
// Databases hold no manifest-templated env of their own — their URL is
|
||||
// what the APPS reference, and that name is generated, not captured.
|
||||
if (r.kind === "database") continue;
|
||||
r.env = await fetchEnv(client, { kind: r.kind, uuid: r.uuid });
|
||||
}
|
||||
draftProjects.push({
|
||||
name: p.name,
|
||||
coolifyEnv: chosen.name,
|
||||
resources,
|
||||
unreadable,
|
||||
otherEnvironments: others,
|
||||
});
|
||||
}
|
||||
// Which GitHub App clones a repo is NOT a property of any resource — no
|
||||
// field Coolify returns about an application says so. What the instance can
|
||||
// answer is which Apps exist; with exactly one, there is no other it could
|
||||
// be. Best-effort: an instance that will not list them still gets a draft,
|
||||
// with a REVIEW marker where the binding goes.
|
||||
const githubApps = (await client.get("/github-apps").catch(() => [])) as
|
||||
| Array<{ name?: unknown }>
|
||||
| undefined;
|
||||
const draftCtx = {
|
||||
env: envName,
|
||||
instance: instance.name,
|
||||
baseUrl: instance.baseUrl,
|
||||
team,
|
||||
server: sweepBinding.server,
|
||||
githubApps: (Array.isArray(githubApps) ? githubApps : [])
|
||||
.map((a) => a?.name)
|
||||
.filter((n): n is string => typeof n === "string"),
|
||||
recipient,
|
||||
generatedAt: new Date().toISOString(),
|
||||
};
|
||||
const storeRecipient = values["no-secrets"] ? undefined : recipient;
|
||||
const plan = planDraft(draftProjects, draftCtx);
|
||||
const written = emitDraft(draftDir, plan, { recipient: storeRecipient });
|
||||
console.log(
|
||||
renderDraftPlan(plan, draftCtx, {
|
||||
dir: draftDir,
|
||||
recipient: storeRecipient,
|
||||
written,
|
||||
}),
|
||||
);
|
||||
return 0;
|
||||
}
|
||||
const repoShort = orgRepo.split("/")[1];
|
||||
|
|
|
|||
1321
src/draft.ts
Normal file
1321
src/draft.ts
Normal file
File diff suppressed because it is too large
Load diff
574
test/draft-cli.test.ts
Normal file
574
test/draft-cli.test.ts
Normal file
|
|
@ -0,0 +1,574 @@
|
|||
import { execFileSync, spawn } from "node:child_process";
|
||||
import {
|
||||
existsSync,
|
||||
mkdirSync,
|
||||
mkdtempSync,
|
||||
readFileSync,
|
||||
readdirSync,
|
||||
writeFileSync,
|
||||
} from "node:fs";
|
||||
import { createServer } from "node:http";
|
||||
import type { AddressInfo } from "node:net";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { afterEach, beforeAll, describe, expect, it } from "vitest";
|
||||
import { loadBindings } from "../src/bindings.js";
|
||||
import { GENERATED_PLACEHOLDER } from "../src/capture.js";
|
||||
import { decryptSecrets } from "../src/secrets.js";
|
||||
|
||||
// `cast inventory --emit-draft` against a stub shaped like the box that made it
|
||||
// necessary: a Coolify nobody declared, holding our stack under names someone
|
||||
// typed, two third-party client sites, and a database cast cannot even model.
|
||||
//
|
||||
// The single most important assertion in this file is that POISON — the source
|
||||
// box's real DATABASE_URL — appears in NO emitted artifact. A draft that carried
|
||||
// it would rebuild a box that comes up WORKING, reading and writing the old
|
||||
// box's database, and nobody would find out until the old box was deleted.
|
||||
|
||||
// The live values. If any of these reaches an emitted file, the test fails.
|
||||
const POISON =
|
||||
"postgres://postgres:s3cr3t@incubator-database-v2.box-b.internal:5432/app";
|
||||
const POISON_REDIS = "redis://:r3d1s@incubator-redis.box-b.internal:6379/0";
|
||||
const POISON_SERVICE_PW = "umami-generated-9f3a1c";
|
||||
// …and one that MUST survive: an ordinary secret is captured, not placeheld.
|
||||
const MAILGUN = "key-1a2b3c-real-mailgun";
|
||||
|
||||
type Stub = { url: string; close: () => Promise<void> };
|
||||
const stubs: Stub[] = [];
|
||||
|
||||
const ENVS: Record<string, Array<{ key: string; value: string }>> = {
|
||||
a1: [
|
||||
{ key: "DATABASE_URL", value: POISON },
|
||||
{ key: "REDIS_URL", value: POISON_REDIS },
|
||||
{ key: "MAILGUN_KEY", value: MAILGUN },
|
||||
{ key: "NODE_ENV", value: "production" },
|
||||
// Not a name a cast env template can hold. Reported, never dropped in silence.
|
||||
{ key: "legacy.flag", value: "on" },
|
||||
],
|
||||
s1: [
|
||||
{ key: "SERVICE_PASSWORD_UMAMI", value: POISON_SERVICE_PW },
|
||||
{ key: "SERVICE_FQDN_UMAMI", value: "https://umami.box-b.example.com" },
|
||||
{ key: "UMAMI_APP_SECRET", value: "umami-app-secret-xyz" },
|
||||
],
|
||||
a9: [{ key: "WP_HOME", value: "https://lafamilia.example.com" }],
|
||||
};
|
||||
|
||||
// One project with resources in TWO environments — cast must refuse to pick.
|
||||
async function stubCoolify(opts: { ambiguous?: boolean } = {}): Promise<Stub> {
|
||||
const server = createServer((req, res) => {
|
||||
const path = (req.url ?? "").replace("/api/v1", "");
|
||||
const json = (body: unknown) => {
|
||||
res.writeHead(200, { "content-type": "application/json" });
|
||||
res.end(JSON.stringify(body));
|
||||
};
|
||||
if (path === "/teams/current") return json({ id: 0, name: "Root Team" });
|
||||
// Exactly one App: there is no other one an application could have been
|
||||
// cloned by, so cast binds every repo to it rather than leave a marker.
|
||||
if (path === "/github-apps")
|
||||
return json([{ uuid: "g1", name: "hdb-coolify" }]);
|
||||
if (path === "/projects")
|
||||
return json([
|
||||
{ uuid: "p1", name: "Incubator" },
|
||||
{ uuid: "p2", name: "La Familia Site" },
|
||||
{ uuid: "p3", name: "Martin Reyes Barber Shop" },
|
||||
]);
|
||||
if (path === "/projects/p1/environments")
|
||||
return json([{ name: "production" }, { name: "staging" }]);
|
||||
if (path === "/projects/p2/environments")
|
||||
return json([{ name: "production" }]);
|
||||
if (path === "/projects/p3/environments")
|
||||
return json([{ name: "production" }]);
|
||||
|
||||
// Coolify auto-creates `production`. It is empty (unless the ambiguous
|
||||
// variant puts a resource in it too). Everything real lives in `staging`.
|
||||
if (path === "/projects/p1/production")
|
||||
return json(
|
||||
opts.ambiguous
|
||||
? { applications: [{ name: "old-core", uuid: "a7", fqdn: "" }] }
|
||||
: {},
|
||||
);
|
||||
if (path === "/projects/p1/staging")
|
||||
return json({
|
||||
applications: [
|
||||
{
|
||||
name: "Incubator Stack v2",
|
||||
uuid: "a1",
|
||||
git_repository: "heavy-duty/incubator",
|
||||
git_branch: "main",
|
||||
build_pack: "dockercompose",
|
||||
base_directory: "/",
|
||||
docker_compose_location: "/docker-compose.yaml",
|
||||
docker_compose_domains: JSON.stringify([
|
||||
{ name: "core", domain: "https://app.example.com" },
|
||||
]),
|
||||
destination_id: 3,
|
||||
// Basic Auth lives here, and cast's manifest has no field for it.
|
||||
custom_labels:
|
||||
"traefik.http.middlewares.auth.basicauth.users=admin:$2y$05$x",
|
||||
},
|
||||
],
|
||||
postgresqls: [
|
||||
{
|
||||
name: "Incubator Database v2",
|
||||
uuid: "d1",
|
||||
database_type: "standalone-postgresql",
|
||||
image: "postgres:16-alpine",
|
||||
destination_id: 3,
|
||||
},
|
||||
],
|
||||
services: [
|
||||
{
|
||||
name: "Incubator Umami",
|
||||
uuid: "s1",
|
||||
service_type: "umami",
|
||||
destination_id: 3,
|
||||
},
|
||||
],
|
||||
// A database cast's manifest cannot express at all.
|
||||
mysqls: [{ name: "legacy-analytics", uuid: "m1" }],
|
||||
});
|
||||
if (path === "/projects/p2/production")
|
||||
return json({
|
||||
applications: [
|
||||
{
|
||||
name: "lafamilia-web",
|
||||
uuid: "a9",
|
||||
git_repository: "https://github.com/third-party/la-familia.git",
|
||||
git_branch: "main",
|
||||
build_pack: "static",
|
||||
base_directory: "/",
|
||||
publish_directory: "/dist",
|
||||
fqdn: "https://lafamilia.example.com",
|
||||
destination_id: 4,
|
||||
},
|
||||
],
|
||||
});
|
||||
// A project with no application at all: nothing on the box knows its repo.
|
||||
if (path === "/projects/p3/production")
|
||||
return json({
|
||||
services: [
|
||||
{ name: "barber-site", uuid: "s9", service_type: "wordpress" },
|
||||
],
|
||||
});
|
||||
|
||||
const envs = path.match(/^\/[a-z]+\/([a-z0-9]+)\/envs$/);
|
||||
if (envs) return json(ENVS[envs[1]] ?? []);
|
||||
res.writeHead(404);
|
||||
res.end("{}");
|
||||
});
|
||||
await new Promise<void>((r) => {
|
||||
server.listen(0, "127.0.0.1", r);
|
||||
});
|
||||
const stub: Stub = {
|
||||
url: `http://127.0.0.1:${(server.address() as AddressInfo).port}`,
|
||||
close: () =>
|
||||
new Promise<void>((r) => {
|
||||
server.close(() => r());
|
||||
}),
|
||||
};
|
||||
stubs.push(stub);
|
||||
return stub;
|
||||
}
|
||||
|
||||
afterEach(async () => {
|
||||
await Promise.all(stubs.splice(0).map((s) => s.close()));
|
||||
});
|
||||
|
||||
let KEY_FILE = "";
|
||||
let RECIPIENT = "";
|
||||
|
||||
beforeAll(() => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "cast-age-"));
|
||||
KEY_FILE = join(dir, "key.txt");
|
||||
execFileSync("age-keygen", ["-o", KEY_FILE], { stdio: "ignore" });
|
||||
RECIPIENT = execFileSync("age-keygen", ["-y", KEY_FILE], {
|
||||
encoding: "utf8",
|
||||
}).trim();
|
||||
});
|
||||
|
||||
function fixture(url: string, opts: { recipient?: string } = {}) {
|
||||
const state = mkdtempSync(join(tmpdir(), "cast-state-"));
|
||||
writeFileSync(
|
||||
join(state, ".coolify.env"),
|
||||
`COOLIFY_BASE_URL="${url}"\nCOOLIFY_ACCESS_TOKEN="t"\n`,
|
||||
);
|
||||
writeFileSync(
|
||||
join(state, "environments.yaml"),
|
||||
[
|
||||
"environments:",
|
||||
" prod:",
|
||||
" server: box-b",
|
||||
" team: { id: 0, name: Root Team }",
|
||||
...(opts.recipient ? [` age_recipient: ${opts.recipient}`] : []),
|
||||
"github_apps:",
|
||||
" heavy-duty/incubator: hdb-coolify",
|
||||
"",
|
||||
].join("\n"),
|
||||
);
|
||||
const out = join(mkdtempSync(join(tmpdir(), "cast-out-")), "draft");
|
||||
return { state, out };
|
||||
}
|
||||
|
||||
function run(args: string[]): Promise<{ code: number; output: string }> {
|
||||
return new Promise((resolve) => {
|
||||
const child = spawn("node", ["dist/cli.js", "inventory", ...args], {
|
||||
stdio: ["pipe", "pipe", "pipe"],
|
||||
});
|
||||
let output = "";
|
||||
child.stdout.on("data", (d) => {
|
||||
output += String(d);
|
||||
});
|
||||
child.stderr.on("data", (d) => {
|
||||
output += String(d);
|
||||
});
|
||||
child.stdin.end();
|
||||
child.on("close", (code) => resolve({ code: code ?? 0, output }));
|
||||
});
|
||||
}
|
||||
|
||||
// Every file in the emitted tree, path -> bytes as text.
|
||||
function tree(dir: string, prefix = ""): Record<string, string> {
|
||||
const out: Record<string, string> = {};
|
||||
for (const entry of readdirSync(dir, { withFileTypes: true })) {
|
||||
const path = join(dir, entry.name);
|
||||
const rel = prefix ? `${prefix}/${entry.name}` : entry.name;
|
||||
if (entry.isDirectory()) Object.assign(out, tree(path, rel));
|
||||
else out[rel] = readFileSync(path, "utf8");
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
describe("cast inventory --emit-draft (#27)", () => {
|
||||
it("emits the tree: bindings + registry, a manifest per project, templates, stores, UNCAPTURED", async () => {
|
||||
const f = fixture((await stubCoolify()).url);
|
||||
const r = await run([
|
||||
"--env",
|
||||
"prod",
|
||||
"--state",
|
||||
f.state,
|
||||
"--emit-draft",
|
||||
f.out,
|
||||
"--recipient",
|
||||
RECIPIENT,
|
||||
]);
|
||||
expect(r.code).toBe(0);
|
||||
expect(Object.keys(tree(f.out)).sort()).toEqual([
|
||||
"UNCAPTURED.md",
|
||||
"environments.yaml",
|
||||
// One per project — INCLUDING the two third-party client sites nobody ever
|
||||
// declared. They each sit alone in a Coolify-default `production`, and a
|
||||
// draft that filtered the instance by one environment name would drop
|
||||
// exactly them: the projects the verb exists to bootstrap.
|
||||
"incubator/.infra/env/incubator-stack-v2.prod.env.template",
|
||||
"incubator/.infra/env/incubator-umami.prod.env.template",
|
||||
"incubator/.infra/manifest.yaml",
|
||||
"la-familia/.infra/env/lafamilia-web.prod.env.template",
|
||||
"la-familia/.infra/manifest.yaml",
|
||||
"martin-reyes-barber-shop/.infra/manifest.yaml",
|
||||
"secrets/incubator.prod.env.age",
|
||||
"secrets/la-familia.prod.env.age",
|
||||
]);
|
||||
// Every emitted file says what it is, in its own body.
|
||||
for (const [path, body] of Object.entries(tree(f.out))) {
|
||||
if (path.endsWith(".age")) continue;
|
||||
expect(body, path).toContain("PROPOSAL");
|
||||
expect(body, path).toContain("apply` does not read");
|
||||
}
|
||||
});
|
||||
|
||||
it("NEVER copies a provider-generated value — not into any artifact", async () => {
|
||||
const f = fixture((await stubCoolify()).url);
|
||||
const r = await run([
|
||||
"--env",
|
||||
"prod",
|
||||
"--state",
|
||||
f.state,
|
||||
"--emit-draft",
|
||||
f.out,
|
||||
"--recipient",
|
||||
RECIPIENT,
|
||||
]);
|
||||
expect(r.code).toBe(0);
|
||||
|
||||
// 1. Not in any emitted FILE.
|
||||
for (const [path, body] of Object.entries(tree(f.out))) {
|
||||
expect(body, path).not.toContain(POISON);
|
||||
expect(body, path).not.toContain(POISON_REDIS);
|
||||
expect(body, path).not.toContain(POISON_SERVICE_PW);
|
||||
}
|
||||
// 2. Not in the age store either, once decrypted — the store is the one place
|
||||
// a copied value would actually be, and the one place you cannot grep.
|
||||
const store = decryptSecrets(
|
||||
join(f.out, "secrets", "incubator.prod.env.age"),
|
||||
KEY_FILE,
|
||||
);
|
||||
expect(store.DATABASE_URL).toBe(GENERATED_PLACEHOLDER);
|
||||
expect(store.REDIS_URL).toBe(GENERATED_PLACEHOLDER);
|
||||
expect(store.SERVICE_PASSWORD_UMAMI).toBe(GENERATED_PLACEHOLDER);
|
||||
expect(store.SERVICE_FQDN_UMAMI).toBe(GENERATED_PLACEHOLDER);
|
||||
expect(Object.values(store)).not.toContain(POISON);
|
||||
// 3. Not on stdout.
|
||||
expect(r.output).not.toContain(POISON);
|
||||
|
||||
// …and an ORDINARY secret IS carried. The discipline is placeholding the
|
||||
// provider's names, not refusing to capture anything.
|
||||
expect(store.MAILGUN_KEY).toBe(MAILGUN);
|
||||
expect(store.UMAMI_APP_SECRET).toBe("umami-app-secret-xyz");
|
||||
|
||||
// The manifest declares the placeheld names, so a later `capture` placeholds
|
||||
// them again with no flag to remember.
|
||||
const manifest = readFileSync(
|
||||
join(f.out, "incubator", ".infra", "manifest.yaml"),
|
||||
"utf8",
|
||||
);
|
||||
expect(manifest).toContain("generated_secrets:");
|
||||
expect(manifest).toContain("- DATABASE_URL");
|
||||
expect(manifest).toContain("- REDIS_URL");
|
||||
// The plan says so out loud, in names and provenance — never values.
|
||||
expect(r.output).toContain("generated");
|
||||
expect(r.output).toContain(GENERATED_PLACEHOLDER);
|
||||
expect(r.output).toContain("captured");
|
||||
});
|
||||
|
||||
it("UNCAPTURED.md lists every known-inexpressible setting it SAW", async () => {
|
||||
const f = fixture((await stubCoolify()).url);
|
||||
const r = await run([
|
||||
"--env",
|
||||
"prod",
|
||||
"--state",
|
||||
f.state,
|
||||
"--emit-draft",
|
||||
f.out,
|
||||
"--recipient",
|
||||
RECIPIENT,
|
||||
]);
|
||||
expect(r.code).toBe(0);
|
||||
const md = readFileSync(join(f.out, "UNCAPTURED.md"), "utf8");
|
||||
|
||||
// Seen on the box, and inexpressible:
|
||||
expect(md).toContain("Incubator Umami"); // service hostnames (#21 / 4.1.2)
|
||||
expect(md).toContain("domains (hostnames)");
|
||||
expect(md).toContain("destination"); // which Docker network (#21)
|
||||
expect(md).toContain("destination_id 3");
|
||||
expect(md).toContain("legacy-analytics"); // a MySQL cast cannot model
|
||||
expect(md).toContain("custom_labels"); // Basic Auth / Traefik labels
|
||||
expect(md).toContain("backup"); // not exposed by the API
|
||||
expect(md).toContain("legacy.flag"); // not a name a template can hold
|
||||
|
||||
// And the standing sections, emitted on every run whatever was found:
|
||||
expect(md).toContain("no API coverage in Coolify 4.1.2");
|
||||
expect(md).toContain("Include Source Commit in Build");
|
||||
expect(md).toContain("What a rebuild from this draft still cannot restore");
|
||||
expect(md).toContain("the GitHub App private key");
|
||||
expect(md).toContain("S3 access keys");
|
||||
|
||||
// The run points at it rather than leaving it to be found.
|
||||
expect(r.output).toContain("UNCAPTURED.md");
|
||||
});
|
||||
|
||||
it("writes the projects: registry — the list of what exists", async () => {
|
||||
const f = fixture((await stubCoolify()).url);
|
||||
await run([
|
||||
"--env",
|
||||
"prod",
|
||||
"--state",
|
||||
f.state,
|
||||
"--emit-draft",
|
||||
f.out,
|
||||
"--recipient",
|
||||
RECIPIENT,
|
||||
]);
|
||||
const yaml = readFileSync(join(f.out, "environments.yaml"), "utf8");
|
||||
expect(yaml).toContain("projects:");
|
||||
// Read off the application's git remote — the only place a box knows its repo.
|
||||
expect(yaml).toContain("third-party/la-familia:");
|
||||
expect(yaml).toContain("environments:\n - prod");
|
||||
// The bindings the sweep could actually read.
|
||||
expect(yaml).toContain("server: box-b");
|
||||
expect(yaml).toContain("name: Root Team");
|
||||
// Which GitHub App clones a repo is not on any resource — but this instance
|
||||
// has exactly one, and there is no other it could be.
|
||||
expect(yaml).toContain("github_apps:");
|
||||
expect(yaml).toContain("third-party/la-familia: hdb-coolify");
|
||||
|
||||
// The barber shop has no application, so the box knows no repo for it. cast
|
||||
// writes the bare project name — and the registry's own parse-time refusal
|
||||
// (#25: "a registry key has no meaning without its org") then stops the file
|
||||
// being used until a human supplies the org. That refusal IS the design: the
|
||||
// alternatives are inventing an org, or dropping a project from the list —
|
||||
// and a project missing from the registry is one every fleet run skips in
|
||||
// silence.
|
||||
expect(yaml).toContain("martin-reyes-barber-shop:");
|
||||
const path = join(f.out, "environments.yaml");
|
||||
expect(() => loadBindings(path)).toThrow(/has no meaning without its org/);
|
||||
});
|
||||
|
||||
it("refuses --emit-draft together with a repo — that is the reconcile path", async () => {
|
||||
const f = fixture((await stubCoolify()).url);
|
||||
const r = await run([
|
||||
"heavy-duty/incubator",
|
||||
"--env",
|
||||
"prod",
|
||||
"--state",
|
||||
f.state,
|
||||
"--emit-draft",
|
||||
f.out,
|
||||
]);
|
||||
expect(r.code).toBe(2);
|
||||
expect(r.output).toContain("SWEEP-mode flag");
|
||||
expect(r.output).toContain("Adoption is one-way");
|
||||
expect(existsSync(f.out)).toBe(false);
|
||||
});
|
||||
|
||||
it("refuses a target directory that already holds a repo", async () => {
|
||||
const f = fixture((await stubCoolify()).url);
|
||||
mkdirSync(join(f.out, ".infra"), { recursive: true });
|
||||
writeFileSync(
|
||||
join(f.out, ".infra", "manifest.yaml"),
|
||||
"project: incubator\nenvironments: {}\n",
|
||||
);
|
||||
const r = await run([
|
||||
"--env",
|
||||
"prod",
|
||||
"--state",
|
||||
f.state,
|
||||
"--emit-draft",
|
||||
f.out,
|
||||
"--recipient",
|
||||
RECIPIENT,
|
||||
]);
|
||||
expect(r.code).toBe(2);
|
||||
expect(r.output).toContain("is not empty");
|
||||
// For a declared project the manifest is the truth — a draft must not be able
|
||||
// to overwrite a reviewed spec with a live box's accumulated cruft.
|
||||
expect(r.output).toContain("Adoption is one-way");
|
||||
expect(readFileSync(join(f.out, ".infra", "manifest.yaml"), "utf8")).toBe(
|
||||
"project: incubator\nenvironments: {}\n",
|
||||
);
|
||||
});
|
||||
|
||||
it("refuses to emit secrets nobody can decrypt — and takes an explicit opt-out", async () => {
|
||||
const f = fixture((await stubCoolify()).url);
|
||||
const refused = await run([
|
||||
"--env",
|
||||
"prod",
|
||||
"--state",
|
||||
f.state,
|
||||
"--emit-draft",
|
||||
f.out,
|
||||
]);
|
||||
// No --recipient, no age_recipient binding. Silently skipping the store would
|
||||
// emit a draft that LOOKS complete and holds not one value.
|
||||
expect(refused.code).toBe(2);
|
||||
expect(refused.output).toContain("no age recipient");
|
||||
expect(refused.output).toContain("--no-secrets");
|
||||
expect(existsSync(f.out)).toBe(false);
|
||||
|
||||
const optOut = await run([
|
||||
"--env",
|
||||
"prod",
|
||||
"--state",
|
||||
f.state,
|
||||
"--emit-draft",
|
||||
f.out,
|
||||
"--no-secrets",
|
||||
]);
|
||||
expect(optOut.code).toBe(0);
|
||||
expect(existsSync(join(f.out, "secrets"))).toBe(false);
|
||||
expect(
|
||||
existsSync(join(f.out, "incubator", ".infra", "manifest.yaml")),
|
||||
).toBe(true);
|
||||
expect(optOut.output).toContain("NOT WRITTEN");
|
||||
});
|
||||
|
||||
it("takes the recipient from the environment's binding when no flag is given", async () => {
|
||||
const f = fixture((await stubCoolify()).url, { recipient: RECIPIENT });
|
||||
const r = await run([
|
||||
"--env",
|
||||
"prod",
|
||||
"--state",
|
||||
f.state,
|
||||
"--emit-draft",
|
||||
f.out,
|
||||
]);
|
||||
expect(r.code).toBe(0);
|
||||
const store = decryptSecrets(
|
||||
join(f.out, "secrets", "incubator.prod.env.age"),
|
||||
KEY_FILE,
|
||||
);
|
||||
expect(store.MAILGUN_KEY).toBe(MAILGUN);
|
||||
});
|
||||
|
||||
it("refuses to pick between two environments that both have resources", async () => {
|
||||
const f = fixture((await stubCoolify({ ambiguous: true })).url);
|
||||
const r = await run([
|
||||
"--env",
|
||||
"prod",
|
||||
"--state",
|
||||
f.state,
|
||||
"--emit-draft",
|
||||
f.out,
|
||||
"--recipient",
|
||||
RECIPIENT,
|
||||
]);
|
||||
// Picking would emit a blueprint of half a box that says nothing about the
|
||||
// other half — and this box is where that already happened once (#22).
|
||||
expect(r.code).toBe(2);
|
||||
expect(r.output).toContain("MORE THAN ONE environment");
|
||||
expect(r.output).toContain("Incubator");
|
||||
expect(r.output).toContain("--environment");
|
||||
expect(existsSync(f.out)).toBe(false);
|
||||
|
||||
// …and --environment breaks the tie. For the projects that HAVE no tie (the
|
||||
// client sites, each alone in its own `production`), it changes nothing: they
|
||||
// are drafted either way.
|
||||
const tied = await run([
|
||||
"--env",
|
||||
"prod",
|
||||
"--state",
|
||||
f.state,
|
||||
"--emit-draft",
|
||||
f.out,
|
||||
"--recipient",
|
||||
RECIPIENT,
|
||||
"--environment",
|
||||
"staging",
|
||||
]);
|
||||
expect(tied.code).toBe(0);
|
||||
const files = Object.keys(tree(f.out));
|
||||
expect(files).toContain("incubator/.infra/manifest.yaml");
|
||||
expect(files).toContain("la-familia/.infra/manifest.yaml");
|
||||
// The environment it did NOT draft is named, not dropped in silence.
|
||||
expect(readFileSync(join(f.out, "UNCAPTURED.md"), "utf8")).toContain(
|
||||
"other environments",
|
||||
);
|
||||
});
|
||||
|
||||
it("still sweeps, and still asserts the team, before it writes anything", async () => {
|
||||
const f = fixture((await stubCoolify()).url);
|
||||
writeFileSync(
|
||||
join(f.state, "environments.yaml"),
|
||||
[
|
||||
"environments:",
|
||||
" prod:",
|
||||
" server: box-b",
|
||||
" team: { id: 9, name: Some Other Team }",
|
||||
"",
|
||||
].join("\n"),
|
||||
);
|
||||
const r = await run([
|
||||
"--env",
|
||||
"prod",
|
||||
"--state",
|
||||
f.state,
|
||||
"--emit-draft",
|
||||
f.out,
|
||||
"--recipient",
|
||||
RECIPIENT,
|
||||
]);
|
||||
// A wrong-team token reads back an empty instance — and would draft a
|
||||
// blueprint of nothing at all, confidently.
|
||||
expect(r.code).not.toBe(0);
|
||||
expect(existsSync(f.out)).toBe(false);
|
||||
});
|
||||
});
|
||||
317
test/draft.test.ts
Normal file
317
test/draft.test.ts
Normal file
|
|
@ -0,0 +1,317 @@
|
|||
import { mkdirSync, mkdtempSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { GENERATED_PLACEHOLDER } from "../src/capture.js";
|
||||
import {
|
||||
type DraftProject,
|
||||
assertEmptyTarget,
|
||||
assertNoExistingManifest,
|
||||
draftResourcesFrom,
|
||||
isProviderGenerated,
|
||||
planDraft,
|
||||
repoFromGitUrl,
|
||||
} from "../src/draft.js";
|
||||
import { templateKeys, templateRefs } from "../src/envtemplate.js";
|
||||
import { loadManifest } from "../src/manifest.js";
|
||||
|
||||
const ctx = {
|
||||
env: "prod",
|
||||
instance: "box-b",
|
||||
baseUrl: "https://coolify.example.com",
|
||||
team: { id: 0, name: "Root Team" },
|
||||
server: "box-b",
|
||||
recipient: "age1example",
|
||||
generatedAt: "2026-07-13T00:00:00.000Z",
|
||||
};
|
||||
|
||||
// The value that must never leave the box it was read from.
|
||||
const POISON = "postgres://postgres:pw@incubator-db-v2.box-b.internal:5432/app";
|
||||
|
||||
const project = (over: Partial<DraftProject> = {}): DraftProject => ({
|
||||
name: "Incubator",
|
||||
coolifyEnv: "staging",
|
||||
resources: [
|
||||
{
|
||||
kind: "application",
|
||||
name: "Incubator Stack v2",
|
||||
uuid: "a1",
|
||||
raw: {
|
||||
git_repository: "https://github.com/heavy-duty/incubator",
|
||||
git_branch: "main",
|
||||
build_pack: "nixpacks",
|
||||
base_directory: "/",
|
||||
ports_exposes: "3000",
|
||||
fqdn: "https://app.example.com",
|
||||
destination_id: 3,
|
||||
},
|
||||
env: {
|
||||
DATABASE_URL: POISON,
|
||||
MAILGUN_KEY: "key-abc123",
|
||||
NODE_ENV: "production",
|
||||
},
|
||||
},
|
||||
],
|
||||
unreadable: [],
|
||||
otherEnvironments: [],
|
||||
...over,
|
||||
});
|
||||
|
||||
describe("isProviderGenerated — the one judgment that must not be wrong", () => {
|
||||
it("recognizes the datastore families whose value points at the SOURCE box", () => {
|
||||
for (const key of [
|
||||
"DATABASE_URL",
|
||||
"DATABASE_URL_PROD",
|
||||
"UMAMI_DATABASE_URL",
|
||||
"REDIS_URL",
|
||||
"POSTGRES_PASSWORD",
|
||||
"DB_HOST",
|
||||
"MONGO_URI",
|
||||
]) {
|
||||
expect(isProviderGenerated(key), key).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it("recognizes Coolify's own per-instance magic vars", () => {
|
||||
for (const key of [
|
||||
"SERVICE_FQDN_UMAMI",
|
||||
"SERVICE_URL_UMAMI",
|
||||
"SERVICE_PASSWORD_POSTGRES",
|
||||
"SERVICE_USER_UMAMI",
|
||||
"SERVICE_BASE64_KEY",
|
||||
]) {
|
||||
expect(isProviderGenerated(key), key).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it("leaves an ordinary secret alone — it is captured, not placeheld", () => {
|
||||
for (const key of [
|
||||
"MAILGUN_KEY",
|
||||
"OPENROUTER_KEY",
|
||||
"ADMIN_EMAIL",
|
||||
"NODE_ENV",
|
||||
"SERVICE_NAME",
|
||||
"PORT",
|
||||
]) {
|
||||
expect(isProviderGenerated(key), key).toBe(false);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("repoFromGitUrl — the only place a box knows which repo it is", () => {
|
||||
it("reads a slug out of every remote shape Coolify stores", () => {
|
||||
expect(repoFromGitUrl("https://github.com/heavy-duty/incubator")).toBe(
|
||||
"heavy-duty/incubator",
|
||||
);
|
||||
expect(repoFromGitUrl("https://github.com/heavy-duty/incubator.git")).toBe(
|
||||
"heavy-duty/incubator",
|
||||
);
|
||||
expect(repoFromGitUrl("git@github.com:heavy-duty/incubator.git")).toBe(
|
||||
"heavy-duty/incubator",
|
||||
);
|
||||
expect(repoFromGitUrl("heavy-duty/incubator")).toBe("heavy-duty/incubator");
|
||||
});
|
||||
|
||||
it("answers undefined rather than guessing", () => {
|
||||
expect(repoFromGitUrl("")).toBeUndefined();
|
||||
expect(repoFromGitUrl(undefined)).toBeUndefined();
|
||||
expect(repoFromGitUrl("not-a-remote")).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe("draftResourcesFrom — including what cast cannot model", () => {
|
||||
it("names a MySQL rather than silently omitting it", () => {
|
||||
const { resources, unreadable } = draftResourcesFrom({
|
||||
applications: [{ name: "web", uuid: "a1" }],
|
||||
postgresqls: [{ name: "db", uuid: "d1" }],
|
||||
redis: [{ name: "cache", uuid: "d2" }],
|
||||
services: [{ name: "umami", uuid: "s1" }],
|
||||
mysqls: [{ name: "legacy-mysql", uuid: "m1" }],
|
||||
mongodbs: [{ name: "old-mongo", uuid: "m2" }],
|
||||
});
|
||||
expect(resources.map((r) => [r.kind, r.name])).toEqual([
|
||||
["application", "web"],
|
||||
["database", "db"],
|
||||
["database", "cache"],
|
||||
["service", "umami"],
|
||||
]);
|
||||
expect(unreadable).toEqual([
|
||||
{ kind: "mysql", name: "legacy-mysql" },
|
||||
{ kind: "mongodb", name: "old-mongo" },
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("planDraft — the emitted shape", () => {
|
||||
it("writes a manifest cast itself can load, keyed by --env", () => {
|
||||
const plan = planDraft([project()], ctx);
|
||||
const manifest = plan.files.find((f) => f.path.endsWith("manifest.yaml"));
|
||||
expect(manifest?.path).toBe("incubator/.infra/manifest.yaml");
|
||||
// A file that says what it is. It leaves this process and is read by someone
|
||||
// deciding whether to trust it.
|
||||
expect(manifest?.content).toContain("PROPOSAL");
|
||||
expect(manifest?.content).toContain("`apply` does not read this file");
|
||||
expect(manifest?.content).toContain("box-b");
|
||||
|
||||
const dir = mkdtempSync(join(tmpdir(), "cast-draft-"));
|
||||
const path = join(dir, "manifest.yaml");
|
||||
writeFileSync(path, manifest?.content ?? "");
|
||||
const loaded = loadManifest(path);
|
||||
expect(loaded.project).toBe("Incubator");
|
||||
const env = loaded.environments.prod;
|
||||
// The resource keeps the BOX's name — renaming it here would make the file
|
||||
// unusable against the UI it was read from.
|
||||
expect(Object.keys(env.applications)).toEqual(["Incubator Stack v2"]);
|
||||
expect(env.applications["Incubator Stack v2"].source).toEqual({
|
||||
repo: "heavy-duty/incubator",
|
||||
branch: "main",
|
||||
});
|
||||
// And the manifest DECLARES the placeheld name, so a later `capture` does the
|
||||
// same placeholding with no flag to remember.
|
||||
expect(env.generated_secrets).toEqual(["DATABASE_URL"]);
|
||||
});
|
||||
|
||||
it("emits an env template every cast reader can parse", () => {
|
||||
const plan = planDraft([project()], ctx);
|
||||
const tpl = plan.files.find((f) => f.path.endsWith(".env.template"));
|
||||
expect(tpl?.path).toBe(
|
||||
"incubator/.infra/env/incubator-stack-v2.prod.env.template",
|
||||
);
|
||||
const body = tpl?.content ?? "";
|
||||
expect(templateKeys(body).sort()).toEqual([
|
||||
"DATABASE_URL",
|
||||
"MAILGUN_KEY",
|
||||
"NODE_ENV",
|
||||
]);
|
||||
// EVERY var is a ${REF}: values live in the store, never as a literal in a
|
||||
// file that is about to be committed to a product repo.
|
||||
expect(
|
||||
templateRefs(body)
|
||||
.map((r) => r.ref)
|
||||
.sort(),
|
||||
).toEqual(["DATABASE_URL", "MAILGUN_KEY", "NODE_ENV"]);
|
||||
expect(body).not.toContain(POISON);
|
||||
expect(body).not.toContain("key-abc123");
|
||||
});
|
||||
|
||||
it("PLACEHOLDS a provider-generated value and captures an ordinary one", () => {
|
||||
const plan = planDraft([project()], ctx);
|
||||
const byRef = Object.fromEntries(plan.dispositions.map((d) => [d.ref, d]));
|
||||
expect(byRef.DATABASE_URL.provenance).toBe("generated");
|
||||
expect(byRef.DATABASE_URL.value).toBe(GENERATED_PLACEHOLDER);
|
||||
expect(byRef.MAILGUN_KEY.provenance).toBe("captured");
|
||||
expect(byRef.MAILGUN_KEY.value).toBe("key-abc123");
|
||||
|
||||
// The store carries the placeholder, NOT the source box's Postgres.
|
||||
const store = plan.stores[0];
|
||||
expect(store.path).toBe("secrets/incubator.prod.env.age");
|
||||
expect(store.vars.DATABASE_URL).toBe(GENERATED_PLACEHOLDER);
|
||||
expect(JSON.stringify(plan.files)).not.toContain(POISON);
|
||||
});
|
||||
|
||||
it("splits one name carrying two values rather than picking", () => {
|
||||
const p = project();
|
||||
p.resources.push({
|
||||
kind: "application",
|
||||
name: "Landing",
|
||||
uuid: "a2",
|
||||
raw: {
|
||||
git_repository: "https://github.com/heavy-duty/incubator",
|
||||
git_branch: "main",
|
||||
build_pack: "static",
|
||||
base_directory: "/",
|
||||
fqdn: "https://www.example.com",
|
||||
},
|
||||
env: { MAILGUN_KEY: "key-DIFFERENT" },
|
||||
});
|
||||
const plan = planDraft([p], ctx);
|
||||
const refs = plan.dispositions.map((d) => d.ref);
|
||||
// One store holds one value per name (capture refuses a CONFLICT for exactly
|
||||
// this reason). cast will not pick, so both survive under distinct names.
|
||||
expect(refs).toContain("INCUBATOR_STACK_V2_MAILGUN_KEY");
|
||||
expect(refs).toContain("LANDING_MAILGUN_KEY");
|
||||
expect(refs).not.toContain("MAILGUN_KEY");
|
||||
expect(
|
||||
plan.uncaptured.some((u) => u.detail.includes("DIFFERENT values")),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("always emits UNCAPTURED.md — even with little to say", () => {
|
||||
const bare = project({
|
||||
resources: [
|
||||
{
|
||||
kind: "application",
|
||||
name: "web",
|
||||
uuid: "a1",
|
||||
raw: {
|
||||
git_repository: "git@github.com:acme/web.git",
|
||||
git_branch: "main",
|
||||
build_pack: "nixpacks",
|
||||
base_directory: "/",
|
||||
fqdn: "https://web.example.com",
|
||||
},
|
||||
env: {},
|
||||
},
|
||||
],
|
||||
});
|
||||
const md = planDraft([bare], ctx).files.find(
|
||||
(f) => f.path === "UNCAPTURED.md",
|
||||
);
|
||||
expect(md).toBeDefined();
|
||||
// The standing sections are unconditional: what cast CANNOT SEE does not
|
||||
// depend on what it happened to find.
|
||||
expect(md?.content).toContain("no API coverage in Coolify 4.1.2");
|
||||
expect(md?.content).toContain("Include Source Commit in Build");
|
||||
expect(md?.content).toContain("the GitHub App private key");
|
||||
expect(md?.content).toContain("S3 access keys");
|
||||
});
|
||||
|
||||
it("registers what it drafted, and only what it drafted", () => {
|
||||
const empty = project({
|
||||
name: "Empty Project",
|
||||
resources: [],
|
||||
skipReason: "every environment on it is empty",
|
||||
});
|
||||
const bindings = planDraft([project(), empty], ctx).files.find(
|
||||
(f) => f.path === "environments.yaml",
|
||||
);
|
||||
expect(bindings?.content).toContain("heavy-duty/incubator");
|
||||
expect(bindings?.content).toContain("environments:\n - prod");
|
||||
// Not registered — a registry entry for a project with no manifest sends
|
||||
// every future fleet run at nothing.
|
||||
expect(bindings?.content).not.toContain("Empty Project");
|
||||
// …but it is not lost either.
|
||||
const md = planDraft([project(), empty], ctx).files.find(
|
||||
(f) => f.path === "UNCAPTURED.md",
|
||||
);
|
||||
expect(md?.content).toContain("Empty Project");
|
||||
});
|
||||
});
|
||||
|
||||
describe("the emit refusals — adoption is one-way", () => {
|
||||
it("refuses a target directory that is not empty", () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "cast-draft-"));
|
||||
writeFileSync(join(dir, "README.md"), "a repo lives here\n");
|
||||
expect(() => assertEmptyTarget(dir)).toThrow(/is not empty/);
|
||||
expect(() => assertEmptyTarget(dir)).toThrow(/Adoption is one-way/);
|
||||
});
|
||||
|
||||
it("allows a directory that does not exist yet, and an empty one", () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "cast-draft-"));
|
||||
expect(() => assertEmptyTarget(dir)).not.toThrow();
|
||||
expect(() => assertEmptyTarget(join(dir, "new"))).not.toThrow();
|
||||
});
|
||||
|
||||
it("refuses to write a manifest over one that already exists", () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "cast-draft-"));
|
||||
mkdirSync(join(dir, ".infra"), { recursive: true });
|
||||
const path = join(dir, ".infra", "manifest.yaml");
|
||||
writeFileSync(path, "project: incubator\n");
|
||||
// For a declared project the manifest IS the truth: regenerating it from a
|
||||
// live box would let that box's cruft overwrite a reviewed spec.
|
||||
expect(() => assertNoExistingManifest(path)).toThrow(/already exists/);
|
||||
expect(() => assertNoExistingManifest(path)).toThrow(
|
||||
/the manifest IS the truth/,
|
||||
);
|
||||
});
|
||||
});
|
||||
Loading…
Reference in a new issue