inventory --emit-draft: a reviewable blueprint of a live instance (#27) #33
8 changed files with 3165 additions and 5 deletions
177
README.md
177
README.md
|
|
@ -51,6 +51,8 @@ environments.yaml # bindings: the team each env's token must belon
|
|||
# which server it deploys onto, the S3 destination,
|
||||
# GitHub App name, guards — and, per project,
|
||||
# the destination it deploys onto + its smoke target
|
||||
# …plus `projects:`, the registry: which projects
|
||||
# exist, and in which environments
|
||||
secrets/<repo>.<env>.env.age # age-encrypted values for the ${…} placeholders
|
||||
.coolify.env # COOLIFY_BASE_URL + COOLIFY_ACCESS_TOKEN (never commit)
|
||||
.coolify/<name>.env # …the same, for a NAMED instance (see below)
|
||||
|
|
@ -65,6 +67,7 @@ cast apply <org>/<repo> --env <env> [--path <dir>] [--hostname-overlay <file
|
|||
cast diff <org>/<repo> --env <env> [--full]
|
||||
cast capture <org>/<repo> --env <env> [--generated <NAME>] [--override <NAME>]
|
||||
cast inventory <org>/<repo> --env <env>
|
||||
cast inventory --env <env> [--emit-draft <dir> [--recipient age1…] [--no-secrets]]
|
||||
cast server add <name> --ip <ip> --key <file> --env <env> [--user root] [--port 22]
|
||||
cast smoke [<org>/<repo>] --env <env>
|
||||
cast team [--env <env>]
|
||||
|
|
@ -84,6 +87,12 @@ cast team [--env <env>]
|
|||
no age key, and no recipient — it runs *before* adoption, which is the point of
|
||||
it. A document, read by a person; nothing here is consumed by `apply`. See
|
||||
*Adopting a hand-built instance*.
|
||||
- **`inventory --emit-draft <dir>`** — the sweep, written down as a **draft of
|
||||
cast's own inputs**: a manifest per project, env templates, an
|
||||
`environments.yaml` with the registry, an age store, and `UNCAPTURED.md`. A
|
||||
**proposal**, never desired state — `apply` does not read it. It is how a
|
||||
project that has *no* manifest gets its first one, and how you take a
|
||||
point-in-time blueprint of a box. See *Drafting a box that was never declared*.
|
||||
- **`capture`** — the adoption path: reads a hand-built instance's live env and
|
||||
writes the environment's age store from it. See *Adopting a hand-built
|
||||
instance* below.
|
||||
|
|
@ -307,6 +316,126 @@ build pack, the hostname-overlay shapes, and the places Coolify 4.1.2 does not
|
|||
cooperate — each citation verified against `coollabsio/coolify` v4.1.2 and the
|
||||
vendored OpenAPI in `reference/`. Read it before changing `apply`.
|
||||
|
||||
## Drafting a box that was never declared
|
||||
|
||||
`inventory` can see a whole instance. `--emit-draft` makes it **write down what
|
||||
it sees**, in the shape of cast's own inputs:
|
||||
|
||||
```sh
|
||||
cast inventory --env prod --instance box-b --emit-draft ./draft --recipient age1…
|
||||
```
|
||||
|
||||
```
|
||||
draft/
|
||||
environments.yaml # bindings as far as they can be read — with the projects: registry
|
||||
incubator/.infra/manifest.yaml # one per project
|
||||
incubator/.infra/env/*.env.template
|
||||
la-familia/.infra/manifest.yaml # …including the client sites nobody ever declared
|
||||
secrets/<project>.<env>.env.age # encrypted to a recipient you name
|
||||
UNCAPTURED.md # ← the important file
|
||||
```
|
||||
|
||||
Two uses: **bootstrapping a project that has no manifest** (the third-party sites
|
||||
on the box being drained were never declared, and never will be unless something
|
||||
writes the first draft — hand-transcribing them from a UI is exactly the work
|
||||
cast exists to eliminate), and **a point-in-time blueprint** you could rebuild an
|
||||
instance from.
|
||||
|
||||
### A draft is a PROPOSAL
|
||||
|
||||
**It is never desired state, and `apply` never reads it.** It is emitted,
|
||||
reviewed by a human, and lands in a repo as a PR — the same shape as
|
||||
`terraform import` → HCL:
|
||||
|
||||
**sweep → emit draft → you read it → manifest PR → `capture` → `apply`**
|
||||
|
||||
That boundary is the only reason the verb is allowed to exist, and it is
|
||||
enforced, not merely documented:
|
||||
|
||||
- It **never emits into a repo that already has a manifest.** For a declared
|
||||
project the manifest *is* the truth, and one regenerated from a live box would
|
||||
let that box's accumulated cruft overwrite the reviewed spec — in the one
|
||||
direction nobody reviews. Adoption is one-way. A non-empty target directory is
|
||||
refused, and so is writing a manifest over an existing one.
|
||||
- `--emit-draft` is **sweep-mode only**. With a repo, `inventory` is reconciling
|
||||
against a manifest that already exists, which is exactly the case where a draft
|
||||
must not be written. Refused.
|
||||
|
||||
### Two things would make a draft actively dangerous
|
||||
|
||||
**1. Copied provider-generated values.** A `DATABASE_URL` read off the source
|
||||
points at the *source box's* Postgres. Emit it, rebuild elsewhere, and the new
|
||||
box comes up **working** — reading and writing the old box's database. You find
|
||||
out the day the old box is deleted. Same for `REDIS_URL`, and for Coolify's own
|
||||
magic vars (`SERVICE_FQDN_*`, `SERVICE_URL_*`, `SERVICE_PASSWORD_*`), which are
|
||||
generated per-instance and mean nothing anywhere else.
|
||||
|
||||
So the draft applies **`capture`'s discipline**: a provider-generated name is
|
||||
**placeheld** with the same `pending-coolify-generated` literal, its live value
|
||||
is not written into any artifact, and it is listed for disposition. The emitted
|
||||
manifest declares it under `generated_secrets:`, so a later `capture` placeholds
|
||||
it again with no flag to remember. A draft that is confidently wrong in four
|
||||
entries out of seventeen is worse than one that is obviously incomplete.
|
||||
|
||||
The rule is **by name** — two families: Coolify's `SERVICE_*` magic vars, and any
|
||||
name carrying a *datastore* word (`DATABASE`, `DB`, `POSTGRES`, `REDIS`, …) and a
|
||||
*connection* word (`URL`, `HOST`, `PASSWORD`, …) as segments. It errs **wide** on
|
||||
purpose, because the two errors are not symmetric: over-matching a real secret
|
||||
placeholds it loudly and you put it back, while under-matching a generated one
|
||||
copies it silently and rebuilds a box that quietly uses a dead machine's
|
||||
database. Every value cast read is printed with its disposition — names and
|
||||
provenance, never values — and a var that points at the source box under a name
|
||||
cast does not recognize **will** have been copied. Read the table.
|
||||
|
||||
Every other live var becomes a `${REF}`, with its value in the **age store** —
|
||||
never a literal in a committed file. cast cannot know which of a box's vars are
|
||||
secret (nobody wrote it down; that is why this verb exists), and a live API key
|
||||
written as a literal is a key in a git repo. Move the plainly-not-secret ones
|
||||
back to literals yourself, in review.
|
||||
|
||||
The store is encrypted to a recipient you **name** — `--recipient age1…`, or the
|
||||
environment's `age_recipient` binding. With neither, cast **refuses**: a draft
|
||||
whose secrets were silently skipped looks complete and holds not one value.
|
||||
`--no-secrets` says so deliberately.
|
||||
|
||||
**2. Silent losses.** cast cannot express everything a Coolify holds:
|
||||
destinations (which Docker network a resource sits on — no API at all in 4.1.2),
|
||||
service hostnames (they live per-container on `service.applications[].fqdn`),
|
||||
Basic Auth and custom Traefik labels, the *Include Source Commit in Build*
|
||||
toggle, whole database kinds (a MySQL is invisible to cast's manifest), backup
|
||||
schedules, and anything else configured in the UI with no manifest field.
|
||||
|
||||
A blueprint that omits these **without saying so** is worse than no blueprint,
|
||||
because in a disaster you would trust it and rebuild a *different box*. So
|
||||
**`UNCAPTURED.md` is a first-class output**, listing per resource every live
|
||||
setting cast saw and could not express — and it is **written on every run**, even
|
||||
when it has little to say.
|
||||
|
||||
### What a blueprint still cannot restore
|
||||
|
||||
Worth stating plainly, because "rebuild from the repo" is routinely over-claimed:
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| control plane | `rig coolify install` ✅ |
|
||||
| structure | draft → manifest PR → `apply` ✅ |
|
||||
| secret **values** | the age store + your key ✅ |
|
||||
| **data** | Coolify's DB backups → S3 ✅ (a separate path) |
|
||||
| **the GitHub App private key** | ❌ re-create by hand |
|
||||
| **S3 access keys** | ❌ re-mint by hand |
|
||||
|
||||
The last two are **not in the repo** — correctly; it holds no live credentials —
|
||||
and cannot be regenerated from it. A DR runbook has to say so. The same table is
|
||||
emitted into every `UNCAPTURED.md`, because that is the file someone will be
|
||||
reading at the worst possible moment.
|
||||
|
||||
One project per Coolify environment: a project with resources in **two**
|
||||
populated environments is a tie cast will not break (picking would emit a
|
||||
blueprint of half a box), so it refuses and `--environment <name>` says which. It
|
||||
is a tiebreak, not a filter — a project with only one populated environment is
|
||||
drafted from it either way, which is what keeps the client sites (each alone in
|
||||
Coolify's default `production`) in a blueprint that claims to describe the box.
|
||||
|
||||
## Secrets, and attended applies
|
||||
|
||||
An environment's age identity is resolved in exactly two ways:
|
||||
|
|
@ -357,6 +486,54 @@ own (it hangs off a project) and no API path scopes by one: **Coolify
|
|||
environments are an organizational construct, not an auth boundary.** The team
|
||||
is the only boundary there is, so it is the one cast asserts.
|
||||
|
||||
## The registry: which projects exist
|
||||
|
||||
`environments.yaml` says where things deploy *to*, and how a project you have
|
||||
already named is placed once it is there. Until the `projects:` block, nothing in
|
||||
it said **which projects exist at all** — "every project" was a thing the
|
||||
operator remembered:
|
||||
|
||||
```yaml
|
||||
projects:
|
||||
heavy-duty/incubator:
|
||||
environments: [prod, staging]
|
||||
acme/client-site:
|
||||
environments: [prod]
|
||||
```
|
||||
|
||||
Keyed by the **full `<org>/<repo>` slug**, and the key *is* the repo — there is
|
||||
no `repo:` field inside, because a second place to write the same string is a
|
||||
second place for it to be wrong. Unlike `github_apps`, a bare `<repo>` key is
|
||||
**refused** rather than resolved: this block is new, so it has no state files in
|
||||
the wild to keep working, and a bare `<repo>` is unique only *within* an org —
|
||||
which is exactly why it is not a key. `environments:` lists **our** environment
|
||||
names (the values `--env` takes), never Coolify's.
|
||||
|
||||
The block is optional; a state file written before it loads unchanged.
|
||||
|
||||
**It has to be true, so cast checks that it is — at parse time, for every verb.**
|
||||
Two ways it could quietly stop being true, both refused:
|
||||
|
||||
- an environment name that no `environments:` block defines (a typo). The project
|
||||
is real and its environment imaginary, so a fleet run visits nothing for it,
|
||||
reports nothing, and exits clean.
|
||||
- an `environments.<env>.projects.<repo>` binding — a destination, a smoke target
|
||||
— in an environment the registry does not register that project for. The two
|
||||
blocks then describe two different fleets: state real enough for a direct
|
||||
`cast apply` to use, invisible to every fleet run. (Checked only when
|
||||
`projects:` is present.)
|
||||
|
||||
Both refusals defend one failure: **a silently skipped project reads exactly like
|
||||
a clean one.** Silence is the one report that must never be ambiguous.
|
||||
|
||||
What it unlocks, neither of which was possible without a list to iterate:
|
||||
|
||||
- **fleet operations** — `cast diff --all` / `apply --all` over every project in
|
||||
an environment.
|
||||
- **rebuild-from-state** — "restore this Coolify from the state repo" cannot even
|
||||
be *attempted* without knowing what was on it. The registry is the difference
|
||||
between a documented recovery and an archaeology exercise.
|
||||
|
||||
## Two projects, one box: destinations
|
||||
|
||||
A **destination** is the Docker network a resource is created on. A server has a
|
||||
|
|
|
|||
|
|
@ -158,6 +158,51 @@ softened by an implementation detail):
|
|||
refuses `--path` combined with `--env prod`: prod always reads the default
|
||||
branch, so a feature-branch checkout can never reach it.
|
||||
|
||||
## The registry (`projects:`)
|
||||
|
||||
The top-level `projects:` block is the list of **which projects exist**, and in
|
||||
which environments. It is the only place that says so: `environments:` says where
|
||||
things deploy to, `environments.<env>.projects.<repo>` says how an already-named
|
||||
project is placed, `github_apps` says how to clone one you have already named.
|
||||
|
||||
```yaml
|
||||
projects:
|
||||
heavy-duty/incubator:
|
||||
environments: [prod, staging]
|
||||
```
|
||||
|
||||
- **Keyed by the full `<org>/<repo>` slug, with no bare-`<repo>` fallback.** The
|
||||
key is the repo; there is no `repo:` field. `github_apps` and
|
||||
`environments.<env>.projects` accept a bare key because state files in the wild
|
||||
are written that way; this block is new and has none, so it requires the slug —
|
||||
a bare `<repo>` is unique only *within* an org.
|
||||
- **`environments:` names OUR environments** — the keys of the `environments:`
|
||||
block, the values `--env` takes — never Coolify's. Non-empty.
|
||||
- **Optional.** A state file with no `projects:` block loads unchanged, and
|
||||
`projectsIn` reports `[]` for every environment.
|
||||
|
||||
**Validated at parse time, so every verb refuses a registry that lies.** Two
|
||||
refusals, both defending the same failure — *a silently skipped project reads
|
||||
exactly like a clean one*, which makes silence, the most common report there is,
|
||||
ambiguous:
|
||||
|
||||
1. **An environment that does not exist** (a typo in `projects.<slug>.environments`)
|
||||
is an error naming the unknown environment and listing the known ones. Left
|
||||
alone, the project would be registered into an environment no command can
|
||||
visit: a fleet run skips it, reports nothing, exits clean.
|
||||
2. **A binding the registry does not register.** Every
|
||||
`environments.<env>.projects.<slug>` key must be a project the registry
|
||||
registers *for that environment*. Otherwise the two blocks describe two
|
||||
different fleets: a `destination_uuid` or `smoke_target` real enough for a
|
||||
direct `cast apply <repo> --env <env>` to act on, and invisible to every
|
||||
fleet run over that environment. Enforced **only when `projects:` is
|
||||
present**, so pre-registry state files keep loading.
|
||||
|
||||
The registry is what makes two things possible, neither of which can be attempted
|
||||
without a list to iterate: **fleet operations** (`--all`), and
|
||||
**rebuild-from-state** — restoring a Coolify from the state repo, which is
|
||||
otherwise an assumption, since you cannot restore what you cannot enumerate.
|
||||
|
||||
## Placement (destinations)
|
||||
|
||||
A **destination** is the Docker network a resource is created on. It is declared
|
||||
|
|
@ -299,6 +344,111 @@ the plan. There is no `--yes`: a store written without someone reading the
|
|||
provenance column is the outcome the verb exists to prevent. A closed stdin
|
||||
aborts rather than hanging.
|
||||
|
||||
## Drafts (`inventory --emit-draft`)
|
||||
|
||||
`inventory` with no repo sweeps an instance. `--emit-draft <dir>` writes that
|
||||
sweep down as a draft of cast's **own inputs** — a manifest per project, env
|
||||
templates, an `environments.yaml` carrying the `projects:` registry, an age store
|
||||
per project, and `UNCAPTURED.md`.
|
||||
|
||||
**A draft is a PROPOSAL. It is never desired state, and `apply` never reads it.**
|
||||
|
||||
sweep → emit draft → a human reads it → manifest PR → capture → apply
|
||||
|
||||
Same shape as `terraform import` → HCL. Every other rule in this section follows
|
||||
from that one, and each is enforced rather than merely stated:
|
||||
|
||||
| refusal | why |
|
||||
| --- | --- |
|
||||
| a **non-empty** target directory | emitted over a repo that has a manifest, a draft would overwrite a reviewed spec with a live box's accumulated cruft — the one direction nobody reviews. **Adoption is one-way.** |
|
||||
| an **existing manifest** at the path it would write | the same invariant, once more at the file (`assertNoExistingManifest`). For a declared project the manifest *is* the truth; `cast inventory <org>/<repo>` reconciles it instead. |
|
||||
| `--emit-draft` with a **repo positional** | with a repo, inventory reconciles against a manifest that already exists — exactly the case where a draft must not be written. |
|
||||
| **no age recipient** (and no `--no-secrets`) | a draft whose store was silently skipped *looks complete*: a manifest, templates full of `${REF}`s, and not one value anywhere. You would find out when `apply` refused, some time after the box those values were on stopped existing. |
|
||||
| a project with **two populated environments** | a draft carries one environment per project. Picking would emit a blueprint of *half a box* that says nothing about the other half. `--environment` breaks the tie — as a **tiebreak, not a filter**: a project with one populated environment is drafted from it either way, or filtering by name would drop whole projects (each client site sits alone in Coolify's default `production`) out of a blueprint that claims to describe the box. |
|
||||
|
||||
**Provider-generated names are placeheld, never copied.** This is `capture`'s
|
||||
discipline (see above), applied to a verb that has no manifest to tell it which
|
||||
names are generated — so it decides **by name**, in two families:
|
||||
|
||||
1. Coolify's per-instance magic vars — `SERVICE_FQDN_*`, `SERVICE_URL_*`,
|
||||
`SERVICE_PASSWORD_*`, `SERVICE_USER_*`, `SERVICE_BASE64_*`.
|
||||
2. Any name carrying a **datastore** word (`DATABASE`, `DB`, `POSTGRES`, `PG`,
|
||||
`REDIS`, `MONGO`, …) *and* a **connection** word (`URL`, `URI`, `DSN`, `HOST`,
|
||||
`PORT`, `PASSWORD`, `USER`, …) as underscore-delimited segments —
|
||||
`DATABASE_URL`, `UMAMI_DATABASE_URL`, `REDIS_URL_PROD`, `DB_HOST`.
|
||||
|
||||
Each such name is written as the literal `pending-coolify-generated`, listed in
|
||||
the run's disposition table, and declared under the emitted manifest's
|
||||
`generated_secrets:` — so a later `capture` placeholds it again with no flag to
|
||||
remember. **Its live value is not written into any artifact.**
|
||||
|
||||
The rule errs **wide**, deliberately, because the two errors are not symmetric:
|
||||
|
||||
- over-match a real secret → it is placeheld, reported, and you put the value
|
||||
back. Noisy, recoverable, **loud**.
|
||||
- under-match a generated one → it is copied, and a box rebuilt from the draft
|
||||
comes up **working**, reading and writing the *source box's* database, until
|
||||
the day that box is deleted. Silent, unrecoverable, **quiet**.
|
||||
|
||||
It is a name-pattern rule, not a promise: a var that points at the source box
|
||||
under a name cast does not recognize **will** be copied. The disposition table
|
||||
(names and provenance, **never values** — same contract as the capture plan) is
|
||||
what a reviewer reads to catch it.
|
||||
|
||||
**Every other live var becomes a `${REF}`**, with its value in the age store —
|
||||
never a template literal. cast cannot know which of a box's vars are secret
|
||||
(nobody wrote it down, which is why the verb exists), and the two mistakes are
|
||||
again asymmetric: a non-secret in an encrypted store is untidy, a live API key
|
||||
written as a literal into a manifest is a key in a git repo. One name carrying
|
||||
**different values** on two resources is not a conflict cast resolves (one store
|
||||
holds one value per name — see `capture`'s CONFLICT refusal): both are kept,
|
||||
under `<RESOURCE>_<KEY>` refs, and the split is reported.
|
||||
|
||||
**`UNCAPTURED.md` is a first-class output, emitted on every run.** cast cannot
|
||||
express everything a Coolify holds, and a blueprint that omits those things
|
||||
without saying so is worse than no blueprint — in a disaster you would trust it
|
||||
and rebuild a *different box*. Per resource, it names what was seen and could not
|
||||
be written: `destination_id` (which Docker network — no destinations API in 4.1.2
|
||||
to resolve it to the UUID `destination_uuid:` wants, #21), service hostnames (no
|
||||
flat `domains` on a Coolify 4.1.2 service), Basic Auth / custom Traefik labels,
|
||||
build and deploy command overrides, backup schedules (not exposed on a database's
|
||||
GET — **a rebuild has no backups until you declare them**), database kinds cast
|
||||
does not model (MySQL, MariaDB, MongoDB, KeyDB, Dragonfly, ClickHouse — named,
|
||||
never silently dropped), env var names a cast template cannot express, and
|
||||
applications whose build pack the manifest has no vocabulary for (left *out* of
|
||||
the manifest rather than fabricated into the nearest pack).
|
||||
|
||||
It also carries the table below, because that is the file someone will be reading
|
||||
at the worst possible moment.
|
||||
|
||||
### What a blueprint still cannot restore
|
||||
|
||||
| | |
|
||||
| --- | --- |
|
||||
| control plane | `rig coolify install` ✅ |
|
||||
| structure | draft → manifest PR → `apply` ✅ |
|
||||
| secret **values** | the age store + your key ✅ |
|
||||
| **data** | Coolify's DB backups → S3 ✅ (a separate path) |
|
||||
| **the GitHub App private key** | ❌ re-create by hand |
|
||||
| **S3 access keys** | ❌ re-mint by hand |
|
||||
|
||||
The last two are not in the state repo — correctly, it holds no live credentials
|
||||
— and cannot be regenerated from it. A DR runbook that does not say so is not a
|
||||
runbook.
|
||||
|
||||
**What the box cannot tell you, and cast therefore does not invent:** the
|
||||
`<org>/<repo>` slug comes from an application's git remote (the only place a live
|
||||
box knows it), so a project with **no application** — a lone service — has no repo
|
||||
on the box at all. cast writes the bare project name as the registry key, and the
|
||||
registry's own parse-time refusal (*"a registry key has no meaning without its
|
||||
org"*) then stops the file being used until a human supplies it. That refusal is
|
||||
the design: the alternatives are inventing an org, or leaving the project out of
|
||||
the registry — and a project missing from the registry is one every fleet run
|
||||
skips **in silence**. Likewise `github_apps`: nothing Coolify returns about an
|
||||
application says which App cloned it, so cast binds every repo to the instance's
|
||||
only GitHub App when there is exactly one (there is no other it could be), and
|
||||
writes a `REVIEW-…` marker when there is not.
|
||||
|
||||
## Cloning a private manifest
|
||||
|
||||
`resolveCheckout` resolves git credentials **inside cast**, in a fixed order —
|
||||
|
|
|
|||
209
src/bindings.ts
209
src/bindings.ts
|
|
@ -58,6 +58,32 @@ const ProjectBindingSchema = z
|
|||
|
||||
export type ProjectBinding = z.infer<typeof ProjectBindingSchema>;
|
||||
|
||||
// One project in the registry — the top-level `projects:` block, which is the
|
||||
// list of what EXISTS. Nothing else in this file says that: `environments:`
|
||||
// says where things are deployed to, `environments.<env>.projects.<repo>` says
|
||||
// how one project is placed once you already know it is there, and
|
||||
// `github_apps` says how to clone one you have already named. "Every project"
|
||||
// was, until this block, a thing the operator remembered.
|
||||
//
|
||||
// Two things need the list, and neither can be built without it: fleet
|
||||
// operations (`cast diff --all`, #26 — iterating "every project in this
|
||||
// environment") and rebuild-from-state (#27 — a Coolify restored from the state
|
||||
// repo, which cannot even be attempted without knowing what was on it).
|
||||
const RegisteredProjectSchema = z
|
||||
.object({
|
||||
// OUR environment names — the values `--env` takes, the keys of the
|
||||
// `environments:` block above — never Coolify's. The distinction is the same
|
||||
// one `--env` vs `--environment` draws everywhere else in cast.
|
||||
//
|
||||
// Non-empty: a project registered into no environment is not a registration,
|
||||
// it is a line of YAML that reads like one. It would be skipped by every
|
||||
// fleet run silently.
|
||||
environments: z.array(z.string().min(1)).nonempty(),
|
||||
})
|
||||
.strict();
|
||||
|
||||
export type RegisteredProject = z.infer<typeof RegisteredProjectSchema>;
|
||||
|
||||
const BindingsSchema = z
|
||||
.object({
|
||||
environments: z.record(
|
||||
|
|
@ -100,6 +126,22 @@ const BindingsSchema = z
|
|||
})
|
||||
.strict(),
|
||||
),
|
||||
// THE REGISTRY: which projects exist at all, keyed by the full `<org>/<repo>`
|
||||
// slug. The key IS the repo — there is no `repo:` field inside, because a
|
||||
// second place to write the same string is a second place for it to be
|
||||
// wrong.
|
||||
//
|
||||
// Full slug REQUIRED, with no bare-`<repo>` fallback — the one place in this
|
||||
// file where that fallback does not exist. `github_apps` and
|
||||
// `environments.<env>.projects` carry one because they predate the lesson
|
||||
// (#12) and there are state files in the wild keyed the old way; this block
|
||||
// is new, has no such files, and so gets to be right from the start. A bare
|
||||
// `<repo>` is unique only *within* an org, which is precisely why it is not
|
||||
// a key.
|
||||
//
|
||||
// Optional: a state file written before the registry existed keeps loading
|
||||
// untouched, and `projectsIn` answers `[]` for it.
|
||||
projects: z.record(RegisteredProjectSchema).optional(),
|
||||
// Keyed by the repo the App clones for. Prefer the FULL `<org>/<repo>`
|
||||
// slug; a bare `<repo>` key still resolves (see githubAppNameFor) so
|
||||
// existing state files keep working.
|
||||
|
|
@ -112,7 +154,136 @@ const BindingsSchema = z
|
|||
// cannot distinguish prod's app from staging's either.
|
||||
smoke_target: z.string().optional(),
|
||||
})
|
||||
.strict();
|
||||
.strict()
|
||||
// The registry only earns its keep if it is TRUE. Every check here defends the
|
||||
// same failure: a project that a fleet run never visits, because a fleet run
|
||||
// that skips a project prints exactly what a fleet run over a clean project
|
||||
// prints — nothing. Silence is the one report that must never be ambiguous, so
|
||||
// these are parse-time errors (every verb loads bindings, so every verb refuses
|
||||
// a registry that lies) rather than warnings some command might print.
|
||||
.superRefine((bindings, ctx) => {
|
||||
const registry = bindings.projects;
|
||||
if (!registry) return;
|
||||
|
||||
const knownEnvs = Object.keys(bindings.environments);
|
||||
const knownEnvList = knownEnvs.join(", ") || "(none)";
|
||||
|
||||
for (const [slug, project] of Object.entries(registry)) {
|
||||
// A key with no `/` is not a repo. See the schema note above: no fallback.
|
||||
if (!slug.includes("/")) {
|
||||
ctx.addIssue({
|
||||
code: z.ZodIssueCode.custom,
|
||||
path: ["projects", slug],
|
||||
message: [
|
||||
`projects["${slug}"] is not a repo — a registry key has no meaning without its org`,
|
||||
"",
|
||||
` found: projects["${slug}"]`,
|
||||
" wanted: a full <org>/<repo> slug",
|
||||
"",
|
||||
"A bare <repo> is unique only *within* an org: heavy-duty/incubator and",
|
||||
"acme/incubator collapse onto one entry, and the registry then claims one",
|
||||
"project where there are two. Unlike github_apps, this block is new and has",
|
||||
"no legacy state files to support, so there is no bare-<repo> fallback.",
|
||||
"",
|
||||
" projects:",
|
||||
` <org>/${slug}:`,
|
||||
` environments: [${project.environments.join(", ")}]`,
|
||||
].join("\n"),
|
||||
});
|
||||
}
|
||||
|
||||
// Every environment named here must be one that actually exists. A typo
|
||||
// makes the project real but its environment imaginary — so `--all` visits
|
||||
// nothing for it, reports nothing about it, and exits clean.
|
||||
for (const envName of project.environments) {
|
||||
if (!(envName in bindings.environments)) {
|
||||
ctx.addIssue({
|
||||
code: z.ZodIssueCode.custom,
|
||||
path: ["projects", slug, "environments"],
|
||||
message: [
|
||||
`projects["${slug}"] is registered in environment "${envName}", which does not exist`,
|
||||
"",
|
||||
` registered: projects["${slug}"].environments → ${project.environments.join(", ")}`,
|
||||
` known envs: ${knownEnvList}`,
|
||||
"",
|
||||
"An environment nothing defines is one that no command can visit: a fleet",
|
||||
"run would skip this project, and a silently skipped project reads exactly",
|
||||
"like a clean one. Fix the name, or declare the environment:",
|
||||
"",
|
||||
" environments:",
|
||||
` ${envName}:`,
|
||||
" server: <server>",
|
||||
" team: { id: <id>, name: <name> }",
|
||||
].join("\n"),
|
||||
});
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
// The other direction, and the one that rots quietly: a project carrying
|
||||
// per-environment state (destination_uuid, smoke_target — #21) in an
|
||||
// environment the registry does not register it for. That state is then real
|
||||
// enough to be used by a direct `cast apply <repo> --env <env>` and invisible
|
||||
// to every fleet run — the two blocks describing two different fleets.
|
||||
//
|
||||
// Only checked when `projects:` is present, so state files written before the
|
||||
// registry keep loading exactly as they did.
|
||||
for (const [envName, env] of Object.entries(bindings.environments)) {
|
||||
for (const key of Object.keys(env.projects ?? {})) {
|
||||
const entry = registry[key];
|
||||
if (entry?.environments.includes(envName)) continue;
|
||||
|
||||
// The likeliest cause, worth saying out loud: a legacy bare-<repo> key
|
||||
// (which projectBindingFor still resolves) under a registry that is
|
||||
// correctly keyed by slug. The fix is a rename, not a registration.
|
||||
const slugFor = key.includes("/")
|
||||
? undefined
|
||||
: Object.keys(registry).find((s) => s.endsWith(`/${key}`));
|
||||
|
||||
const cause = slugFor
|
||||
? [
|
||||
` registry has: projects["${slugFor}"]`,
|
||||
"",
|
||||
"The binding uses the legacy bare-<repo> key. The registry is keyed by the",
|
||||
`full slug, so rename it to match — environments.${envName}.projects["${slugFor}"].`,
|
||||
]
|
||||
: entry
|
||||
? [
|
||||
` registered for: ${entry.environments.join(", ")}`,
|
||||
"",
|
||||
"Register the project in this environment, or drop the binding — state that",
|
||||
"no fleet run will ever visit is state that stops being true without anyone",
|
||||
"finding out:",
|
||||
"",
|
||||
" projects:",
|
||||
` ${key}:`,
|
||||
` environments: [${[...entry.environments, envName].join(", ")}]`,
|
||||
]
|
||||
: [
|
||||
` registry has: ${Object.keys(registry).join(", ") || "(nothing)"}`,
|
||||
"",
|
||||
"The project is not in the registry at all, so no fleet run will ever visit",
|
||||
"it — while this binding says it is deployed here. Register it, or drop the",
|
||||
"binding:",
|
||||
"",
|
||||
" projects:",
|
||||
` ${key}:`,
|
||||
` environments: [${envName}]`,
|
||||
];
|
||||
|
||||
ctx.addIssue({
|
||||
code: z.ZodIssueCode.custom,
|
||||
path: ["environments", envName, "projects", key],
|
||||
message: [
|
||||
`environments.${envName}.projects["${key}"] is bound in an environment the registry does not register it for`,
|
||||
"",
|
||||
` bound at: environments.${envName}.projects["${key}"]`,
|
||||
...cause,
|
||||
].join("\n"),
|
||||
});
|
||||
}
|
||||
}
|
||||
});
|
||||
|
||||
export type Bindings = z.infer<typeof BindingsSchema>;
|
||||
|
||||
|
|
@ -188,6 +359,27 @@ export function smokeTargetFor(
|
|||
return undefined;
|
||||
}
|
||||
|
||||
// The `<org>/<repo>` slugs registered for one environment — the list a fleet
|
||||
// operation iterates (`cast diff --all`, #26) and a rebuild reads (#27).
|
||||
//
|
||||
// SORTED, deliberately: the order of keys in a YAML file is an accident of who
|
||||
// typed what when, and a fleet run's output — which a human reads top to bottom,
|
||||
// and CI diffs — must not reshuffle because someone appended a project. The
|
||||
// registry is a set; this returns it as one.
|
||||
//
|
||||
// `[]` when there is no registry, which is every state file written before this
|
||||
// block existed. That is the honest answer for "which projects are registered
|
||||
// here" when nothing is registered anywhere — and it makes a fleet verb over an
|
||||
// unmigrated state file a clean no-op rather than a crash.
|
||||
export function projectsIn(bindings: Bindings, envName: string): string[] {
|
||||
const registry = bindings.projects;
|
||||
if (!registry) return [];
|
||||
return Object.entries(registry)
|
||||
.filter(([, project]) => project.environments.includes(envName))
|
||||
.map(([slug]) => slug)
|
||||
.sort();
|
||||
}
|
||||
|
||||
export function loadBindings(
|
||||
path: string,
|
||||
opts: { overrideText?: string } = {},
|
||||
|
|
@ -195,7 +387,20 @@ export function loadBindings(
|
|||
const text = opts.overrideText ?? readFileSync(path, "utf8");
|
||||
const result = BindingsSchema.safeParse(parse(text));
|
||||
if (!result.success) {
|
||||
throw new Error(`invalid bindings ${path}: ${result.error.message}`);
|
||||
// Zod's own `.message` is the entire issue array as JSON — which renders the
|
||||
// refusals above as one long line of `\n` escapes, i.e. throws away the part
|
||||
// of them that was worth writing. Render the issues instead.
|
||||
const detail = result.error.issues
|
||||
.map((issue) => {
|
||||
// A multi-line message is one WE wrote: it already names the path, the
|
||||
// cause, and the YAML to write. A one-line message is zod's ("Required"),
|
||||
// and is useless without the path it happened at.
|
||||
if (issue.message.includes("\n")) return issue.message;
|
||||
const where = issue.path.map(String).join(".");
|
||||
return where ? `${where}: ${issue.message}` : issue.message;
|
||||
})
|
||||
.join("\n\n");
|
||||
throw new Error(`invalid bindings ${path}:\n\n${detail}`);
|
||||
}
|
||||
return result.data;
|
||||
}
|
||||
|
|
|
|||
191
src/cli.ts
191
src/cli.ts
|
|
@ -32,6 +32,17 @@ import {
|
|||
computeDiff,
|
||||
renderDiff,
|
||||
} from "./diff.js";
|
||||
import {
|
||||
type DraftProject,
|
||||
assertEmptyTarget,
|
||||
draftResourcesFrom,
|
||||
emitDraft,
|
||||
planDraft,
|
||||
renderAmbiguousEnvironments,
|
||||
renderDraftPlan,
|
||||
renderNoRecipient,
|
||||
renderRepoWithDraft,
|
||||
} from "./draft.js";
|
||||
import { assertEnvVarPolicy } from "./envtemplate.js";
|
||||
import {
|
||||
type LiveResource,
|
||||
|
|
@ -62,6 +73,7 @@ const USAGE = `usage: cast apply <org>/<repo> --env <env> [--path <dir>] [--
|
|||
cast capture <org>/<repo> --env <env> [--path <dir>] [--project <name>] [--environment <name>] [--generated <NAME>] [--override <NAME>] [--force]
|
||||
cast inventory <org>/<repo> --env <env> [--path <dir>] [--project <name>] [--environment <name>] [--resource <m>=<l>]
|
||||
cast inventory --env <env> [--instance <name>] # no repo: SWEEP the whole instance
|
||||
cast inventory --env <env> --emit-draft <dir> [--recipient age1…] [--no-secrets]
|
||||
cast server add <name> --ip <ip> --key <file> --env <env> [--user root] [--port 22]
|
||||
cast smoke [<org>/<repo>] --env <env>
|
||||
cast team [--env <env>]
|
||||
|
|
@ -107,7 +119,27 @@ capture (adopt a hand-built instance into the age secret store):
|
|||
--override <NAME> supply NAME yourself instead of copying the source's value.
|
||||
The VALUE is read from \$CAST_CAPTURE_<NAME>, never from the
|
||||
command line — argv is visible in \`ps\`. Repeatable.
|
||||
--force overwrite an existing store (refused by default).`;
|
||||
--force overwrite an existing store (refused by default).
|
||||
|
||||
inventory --emit-draft (write down what a box has, as a PROPOSAL):
|
||||
--emit-draft <dir> emit what the sweep saw as a draft of cast's own inputs — a
|
||||
manifest per project, env templates, an environments.yaml
|
||||
carrying the \`projects:\` registry, an age store per project,
|
||||
and UNCAPTURED.md. Into a NEW directory, always: a draft is a
|
||||
proposal, reviewed by a human and landed as a PR, and \`apply\`
|
||||
never reads one. SWEEP MODE ONLY — with a repo there is already
|
||||
a manifest, and a manifest regenerated from a live box would
|
||||
overwrite a reviewed spec with that box's accumulated cruft.
|
||||
--recipient age1… the age recipient the draft's stores are encrypted to. Defaults
|
||||
to the environment's \`age_recipient\` binding.
|
||||
--no-secrets emit no stores. Required when no recipient is available: cast
|
||||
will not silently drop the values it read off the box.
|
||||
--environment <name> a TIEBREAK, not a filter: which environment to draft for a
|
||||
project that has resources in more than one (cast refuses to
|
||||
pick). A project with only one populated environment is drafted
|
||||
from it either way — filtering the instance by an environment
|
||||
name would drop whole projects out of a blueprint that claims to
|
||||
describe the box.`;
|
||||
|
||||
// cast is stateless: every instance-scoped input is read from the state
|
||||
// directory it is pointed at, never from a location the tool itself knows.
|
||||
|
|
@ -479,7 +511,7 @@ export function aliasLive<T extends { name: string }>(
|
|||
// capture.
|
||||
async function fetchEnv(
|
||||
client: CoolifyClient,
|
||||
l: Live,
|
||||
l: Pick<Live, "kind" | "uuid">,
|
||||
): Promise<Record<string, string>> {
|
||||
const base = l.kind === "database" ? "databases" : `${l.kind}s`;
|
||||
const envs = (await client.get(`/${base}/${l.uuid}/envs`).catch((err) => {
|
||||
|
|
@ -887,6 +919,9 @@ async function main(): Promise<number> {
|
|||
environment: { type: "string" },
|
||||
resource: { type: "string", multiple: true },
|
||||
instance: { type: "string" },
|
||||
"emit-draft": { type: "string" },
|
||||
recipient: { type: "string" },
|
||||
"no-secrets": { type: "boolean", default: false },
|
||||
},
|
||||
});
|
||||
const orgRepo = positionals[0];
|
||||
|
|
@ -895,6 +930,17 @@ async function main(): Promise<number> {
|
|||
console.error(USAGE);
|
||||
return 2;
|
||||
}
|
||||
const draftDir = values["emit-draft"];
|
||||
// A draft is emitted from the SWEEP, and only from the sweep. With a repo,
|
||||
// inventory is reconciling against a manifest that already exists — which is
|
||||
// exactly the case where a draft must not be written: for a declared project
|
||||
// the manifest IS the truth, and one regenerated from a live box would let
|
||||
// that box's accumulated cruft overwrite the reviewed spec. Adoption is
|
||||
// one-way, so the two flags cannot be combined at all.
|
||||
if (draftDir && orgRepo) {
|
||||
console.error(renderRepoWithDraft(orgRepo, draftDir));
|
||||
return 2;
|
||||
}
|
||||
const stateDir = stateDirFrom(values.state);
|
||||
const sweepBindings = loadBindings(join(stateDir, "environments.yaml"));
|
||||
const sweepBinding = sweepBindings.environments[envName];
|
||||
|
|
@ -908,6 +954,28 @@ async function main(): Promise<number> {
|
|||
// made inventory a discovery verb that needed you to have already
|
||||
// discovered.
|
||||
if (!orgRepo) {
|
||||
// Both refusals BEFORE the first live call. A draft that is going to be
|
||||
// refused should be refused before an operator watches a whole instance be
|
||||
// swept for it — and, more to the point, before cast reads every env var on
|
||||
// a box it is then not going to write down.
|
||||
const recipient = values.recipient ?? sweepBinding.age_recipient;
|
||||
if (draftDir) {
|
||||
try {
|
||||
assertEmptyTarget(draftDir);
|
||||
} catch (err) {
|
||||
console.error(err instanceof Error ? err.message : String(err));
|
||||
return 2;
|
||||
}
|
||||
// No recipient, no store — and cast will not make that decision quietly.
|
||||
// Silently skipping the secrets would emit a draft that LOOKS complete: a
|
||||
// manifest, templates full of ${REF}s, and nothing anywhere holding a
|
||||
// single value. You would find out when `apply` refused, having already
|
||||
// deleted the box the values were on.
|
||||
if (!recipient && !values["no-secrets"]) {
|
||||
console.error(renderNoRecipient(envName));
|
||||
return 2;
|
||||
}
|
||||
}
|
||||
const { instance, client } = openCoolify(
|
||||
stateDir,
|
||||
values.instance,
|
||||
|
|
@ -918,8 +986,9 @@ async function main(): Promise<number> {
|
|||
// truthfully reports that it is empty.
|
||||
const team = await assertTeam(client, sweepBinding.team, envName);
|
||||
console.log(`team ${formatTeam(team)} ✓`);
|
||||
const live = await client.projects();
|
||||
const projects: SweepProject[] = [];
|
||||
for (const p of await client.projects()) {
|
||||
for (const p of live) {
|
||||
const environments: SweepEnvironment[] = [];
|
||||
for (const name of await client.environments(p.uuid)) {
|
||||
const found = await fetchLive(client, p.name, name);
|
||||
|
|
@ -938,6 +1007,122 @@ async function main(): Promise<number> {
|
|||
baseUrl: instance.baseUrl,
|
||||
}),
|
||||
);
|
||||
if (!draftDir) return 0;
|
||||
|
||||
// --- The draft (#27) ---
|
||||
//
|
||||
// The sweep above is a DOCUMENT. This is the same reading, written into the
|
||||
// shape of cast's own inputs — and it is still a proposal, not desired
|
||||
// state. See draft.ts for the boundary that lets this verb exist at all.
|
||||
//
|
||||
// A project with resources in TWO environments cannot be drafted without
|
||||
// picking one, and cast does not pick: a blueprint of half a box, silently
|
||||
// chosen, is the failure mode this whole issue is about. --environment says
|
||||
// which.
|
||||
//
|
||||
// --environment is a TIEBREAK here, not a filter. A project with resources
|
||||
// in exactly one environment has no tie to break, and is drafted from it
|
||||
// whatever the flag says — filtering the instance by an environment NAME
|
||||
// would drop the projects that most need drafting (the third-party sites,
|
||||
// each sitting in its own Coolify-default `production`) out of a blueprint
|
||||
// that claims to describe the box.
|
||||
const populatedIn = (p: SweepProject) =>
|
||||
p.environments.filter((e) => e.resources.length > 0);
|
||||
const pick = (p: SweepProject) => {
|
||||
const populated = populatedIn(p);
|
||||
return populated.length === 1
|
||||
? populated[0]
|
||||
: populated.find((e) => e.name === values.environment);
|
||||
};
|
||||
const ambiguous = projects.filter(
|
||||
(p) => populatedIn(p).length > 1 && !pick(p),
|
||||
);
|
||||
if (ambiguous.length > 0) {
|
||||
console.error(
|
||||
renderAmbiguousEnvironments(
|
||||
ambiguous.map((p) => ({
|
||||
name: p.name,
|
||||
environments: populatedIn(p).map(
|
||||
(e) => `${e.name} (${e.resources.length})`,
|
||||
),
|
||||
})),
|
||||
envName,
|
||||
),
|
||||
);
|
||||
return 2;
|
||||
}
|
||||
const draftProjects: DraftProject[] = [];
|
||||
for (const p of live) {
|
||||
const swept = projects.find((s) => s.name === p.name);
|
||||
const populated = swept ? populatedIn(swept) : [];
|
||||
const chosen = swept ? pick(swept) : undefined;
|
||||
const others = populated
|
||||
.filter((e) => e.name !== chosen?.name)
|
||||
.map((e) => ({ name: e.name, resources: e.resources.length }));
|
||||
if (!chosen) {
|
||||
// Not drafted — and SAID, in UNCAPTURED.md, rather than left out of a
|
||||
// blueprint that a reader would take for the whole box.
|
||||
draftProjects.push({
|
||||
name: p.name,
|
||||
coolifyEnv: "(none)",
|
||||
resources: [],
|
||||
unreadable: [],
|
||||
otherEnvironments: others,
|
||||
skipReason: "every environment on it is empty",
|
||||
});
|
||||
continue;
|
||||
}
|
||||
// The RAW environment document, not fetchLive's projection: the uncaptured
|
||||
// pass's whole job is to notice fields cast has no home for, and it cannot
|
||||
// notice what a projection has already thrown away.
|
||||
const raw = (await client.get(
|
||||
`/projects/${p.uuid}/${chosen.name}`,
|
||||
)) as Record<string, unknown> | null;
|
||||
const { resources, unreadable } = draftResourcesFrom(raw ?? {});
|
||||
for (const r of resources) {
|
||||
// Databases hold no manifest-templated env of their own — their URL is
|
||||
// what the APPS reference, and that name is generated, not captured.
|
||||
if (r.kind === "database") continue;
|
||||
r.env = await fetchEnv(client, { kind: r.kind, uuid: r.uuid });
|
||||
}
|
||||
draftProjects.push({
|
||||
name: p.name,
|
||||
coolifyEnv: chosen.name,
|
||||
resources,
|
||||
unreadable,
|
||||
otherEnvironments: others,
|
||||
});
|
||||
}
|
||||
// Which GitHub App clones a repo is NOT a property of any resource — no
|
||||
// field Coolify returns about an application says so. What the instance can
|
||||
// answer is which Apps exist; with exactly one, there is no other it could
|
||||
// be. Best-effort: an instance that will not list them still gets a draft,
|
||||
// with a REVIEW marker where the binding goes.
|
||||
const githubApps = (await client.get("/github-apps").catch(() => [])) as
|
||||
| Array<{ name?: unknown }>
|
||||
| undefined;
|
||||
const draftCtx = {
|
||||
env: envName,
|
||||
instance: instance.name,
|
||||
baseUrl: instance.baseUrl,
|
||||
team,
|
||||
server: sweepBinding.server,
|
||||
githubApps: (Array.isArray(githubApps) ? githubApps : [])
|
||||
.map((a) => a?.name)
|
||||
.filter((n): n is string => typeof n === "string"),
|
||||
recipient,
|
||||
generatedAt: new Date().toISOString(),
|
||||
};
|
||||
const storeRecipient = values["no-secrets"] ? undefined : recipient;
|
||||
const plan = planDraft(draftProjects, draftCtx);
|
||||
const written = emitDraft(draftDir, plan, { recipient: storeRecipient });
|
||||
console.log(
|
||||
renderDraftPlan(plan, draftCtx, {
|
||||
dir: draftDir,
|
||||
recipient: storeRecipient,
|
||||
written,
|
||||
}),
|
||||
);
|
||||
return 0;
|
||||
}
|
||||
const repoShort = orgRepo.split("/")[1];
|
||||
|
|
|
|||
1321
src/draft.ts
Normal file
1321
src/draft.ts
Normal file
File diff suppressed because it is too large
Load diff
|
|
@ -5,6 +5,7 @@ import {
|
|||
githubAppNameFor,
|
||||
loadBindings,
|
||||
projectBindingFor,
|
||||
projectsIn,
|
||||
smokeTargetFor,
|
||||
} from "../src/bindings.js";
|
||||
|
||||
|
|
@ -223,3 +224,233 @@ github_apps: {}
|
|||
).toThrow(/invalid bindings/);
|
||||
});
|
||||
});
|
||||
|
||||
// The registry: the list of which projects exist at all. Everything it is FOR
|
||||
// (fleet iteration, rebuild-from-state) depends on it being true, and the way it
|
||||
// stops being true is silent — see the refusals below.
|
||||
describe("the project registry", () => {
|
||||
const twoEnvs = `
|
||||
environments:
|
||||
prod:
|
||||
server: shared-box
|
||||
team: { id: 0, name: Root Team }
|
||||
staging:
|
||||
server: staging-box
|
||||
team: { id: 0, name: Root Team }
|
||||
`;
|
||||
|
||||
it("registers projects per environment, keyed by the full slug", () => {
|
||||
const b = loadBindings("environments.yaml", {
|
||||
overrideText: `${twoEnvs}
|
||||
projects:
|
||||
heavy-duty/incubator:
|
||||
environments: [prod, staging]
|
||||
acme/client-site:
|
||||
environments: [prod]
|
||||
github_apps: {}
|
||||
`,
|
||||
});
|
||||
expect(b.projects).toEqual({
|
||||
"heavy-duty/incubator": { environments: ["prod", "staging"] },
|
||||
"acme/client-site": { environments: ["prod"] },
|
||||
});
|
||||
});
|
||||
|
||||
describe("projectsIn", () => {
|
||||
const b = loadBindings("environments.yaml", {
|
||||
overrideText: `${twoEnvs}
|
||||
projects:
|
||||
heavy-duty/incubator:
|
||||
environments: [prod, staging]
|
||||
acme/client-site:
|
||||
environments: [prod]
|
||||
github_apps: {}
|
||||
`,
|
||||
});
|
||||
|
||||
// Sorted, not file-order: a fleet run's output is read by a human and diffed
|
||||
// by CI, and must not reshuffle because someone appended a project.
|
||||
it("gives an environment's projects, sorted", () => {
|
||||
expect(projectsIn(b, "prod")).toEqual([
|
||||
"acme/client-site",
|
||||
"heavy-duty/incubator",
|
||||
]);
|
||||
});
|
||||
|
||||
it("gives only the projects registered for that environment", () => {
|
||||
expect(projectsIn(b, "staging")).toEqual(["heavy-duty/incubator"]);
|
||||
});
|
||||
|
||||
it("is empty for an environment no project is registered in", () => {
|
||||
expect(projectsIn(b, "nowhere")).toEqual([]);
|
||||
});
|
||||
});
|
||||
|
||||
// The refusal the issue is actually about. A typo'd environment name makes the
|
||||
// project real and its environment imaginary: `cast diff --all` visits nothing
|
||||
// for it, reports nothing, and exits clean — and a silently skipped project
|
||||
// reads exactly like a clean one.
|
||||
it("refuses an environment that does not exist, naming the ones that do", () => {
|
||||
const err = () =>
|
||||
loadBindings("environments.yaml", {
|
||||
overrideText: `${twoEnvs}
|
||||
projects:
|
||||
heavy-duty/incubator:
|
||||
environments: [prod, stagng]
|
||||
github_apps: {}
|
||||
`,
|
||||
});
|
||||
expect(err).toThrow(/environment "stagng", which does not exist/);
|
||||
expect(err).toThrow(/known envs:\s+prod, staging/);
|
||||
});
|
||||
|
||||
// The other direction, and the one that rots quietly: per-environment state
|
||||
// (#21) sitting in an environment the registry does not register the project
|
||||
// for. The two blocks then describe two different fleets.
|
||||
it("refuses a binding in an environment the registry does not register", () => {
|
||||
const err = () =>
|
||||
loadBindings("environments.yaml", {
|
||||
overrideText: `
|
||||
environments:
|
||||
prod:
|
||||
server: shared-box
|
||||
team: { id: 0, name: Root Team }
|
||||
staging:
|
||||
server: staging-box
|
||||
team: { id: 0, name: Root Team }
|
||||
projects:
|
||||
heavy-duty/incubator:
|
||||
destination_uuid: dest-abc
|
||||
projects:
|
||||
heavy-duty/incubator:
|
||||
environments: [prod]
|
||||
github_apps: {}
|
||||
`,
|
||||
});
|
||||
expect(err).toThrow(
|
||||
/environments\.staging\.projects\["heavy-duty\/incubator"\]/,
|
||||
);
|
||||
expect(err).toThrow(/registered for:\s+prod/);
|
||||
});
|
||||
|
||||
it("refuses a binding for a project the registry does not carry at all", () => {
|
||||
const err = () =>
|
||||
loadBindings("environments.yaml", {
|
||||
overrideText: `
|
||||
environments:
|
||||
prod:
|
||||
server: shared-box
|
||||
team: { id: 0, name: Root Team }
|
||||
projects:
|
||||
acme/client-site:
|
||||
destination_uuid: dest-client
|
||||
projects:
|
||||
heavy-duty/incubator:
|
||||
environments: [prod]
|
||||
github_apps: {}
|
||||
`,
|
||||
});
|
||||
expect(err).toThrow(/environments\.prod\.projects\["acme\/client-site"\]/);
|
||||
expect(err).toThrow(/registry has:\s+heavy-duty\/incubator/);
|
||||
});
|
||||
|
||||
// A legacy bare-<repo> binding key (projectBindingFor still resolves one) under
|
||||
// a slug-keyed registry is drift with an obvious fix — say which fix.
|
||||
it("tells a legacy bare-<repo> binding key which slug to rename to", () => {
|
||||
const err = () =>
|
||||
loadBindings("environments.yaml", {
|
||||
overrideText: `
|
||||
environments:
|
||||
prod:
|
||||
server: shared-box
|
||||
team: { id: 0, name: Root Team }
|
||||
projects:
|
||||
incubator:
|
||||
smoke_target: core
|
||||
projects:
|
||||
heavy-duty/incubator:
|
||||
environments: [prod]
|
||||
github_apps: {}
|
||||
`,
|
||||
});
|
||||
expect(err).toThrow(/registry has:\s+projects\["heavy-duty\/incubator"\]/);
|
||||
expect(err).toThrow(/legacy bare-<repo> key/);
|
||||
});
|
||||
|
||||
// No fallback here, unlike github_apps: this block is new, so it has no state
|
||||
// files in the wild to keep working, and a bare <repo> is unique only within an
|
||||
// org — which is exactly why it is not a key.
|
||||
it("refuses a bare <repo> registry key — the org is not optional", () => {
|
||||
const err = () =>
|
||||
loadBindings("environments.yaml", {
|
||||
overrideText: `${twoEnvs}
|
||||
projects:
|
||||
incubator:
|
||||
environments: [prod]
|
||||
github_apps: {}
|
||||
`,
|
||||
});
|
||||
expect(err).toThrow(/projects\["incubator"\] is not a repo/);
|
||||
expect(err).toThrow(/full <org>\/<repo> slug/);
|
||||
});
|
||||
|
||||
// A project registered into nothing is a line of YAML that reads like a
|
||||
// registration and is skipped by every fleet run.
|
||||
it("refuses a project registered into no environment", () => {
|
||||
expect(() =>
|
||||
loadBindings("environments.yaml", {
|
||||
overrideText: `${twoEnvs}
|
||||
projects:
|
||||
heavy-duty/incubator:
|
||||
environments: []
|
||||
github_apps: {}
|
||||
`,
|
||||
}),
|
||||
).toThrow(/invalid bindings/);
|
||||
});
|
||||
|
||||
it("rejects an unknown key inside a registry entry", () => {
|
||||
expect(() =>
|
||||
loadBindings("environments.yaml", {
|
||||
overrideText: `${twoEnvs}
|
||||
projects:
|
||||
heavy-duty/incubator:
|
||||
environments: [prod]
|
||||
repo: heavy-duty/incubator
|
||||
github_apps: {}
|
||||
`,
|
||||
}),
|
||||
).toThrow(/invalid bindings/);
|
||||
});
|
||||
|
||||
// Back-compat: the registry is optional, and every state file written before it
|
||||
// existed has no `projects:` block. Such a file loads unchanged — including its
|
||||
// per-environment bindings, which are NOT checked against a registry that is
|
||||
// not there.
|
||||
describe("with no registry at all", () => {
|
||||
const b = loadBindings("environments.yaml", {
|
||||
overrideText: `
|
||||
environments:
|
||||
prod:
|
||||
server: shared-box
|
||||
team: { id: 0, name: Root Team }
|
||||
projects:
|
||||
heavy-duty/incubator:
|
||||
destination_uuid: dest-abc
|
||||
smoke_target: core
|
||||
github_apps: {}
|
||||
`,
|
||||
});
|
||||
|
||||
it("loads, and keeps its per-environment bindings working", () => {
|
||||
expect(b.projects).toBe(undefined);
|
||||
expect(
|
||||
projectBindingFor(b, "prod", "heavy-duty/incubator")?.destination_uuid,
|
||||
).toBe("dest-abc");
|
||||
});
|
||||
|
||||
it("has no projects registered in any environment", () => {
|
||||
expect(projectsIn(b, "prod")).toEqual([]);
|
||||
});
|
||||
});
|
||||
});
|
||||
|
|
|
|||
574
test/draft-cli.test.ts
Normal file
574
test/draft-cli.test.ts
Normal file
|
|
@ -0,0 +1,574 @@
|
|||
import { execFileSync, spawn } from "node:child_process";
|
||||
import {
|
||||
existsSync,
|
||||
mkdirSync,
|
||||
mkdtempSync,
|
||||
readFileSync,
|
||||
readdirSync,
|
||||
writeFileSync,
|
||||
} from "node:fs";
|
||||
import { createServer } from "node:http";
|
||||
import type { AddressInfo } from "node:net";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { afterEach, beforeAll, describe, expect, it } from "vitest";
|
||||
import { loadBindings } from "../src/bindings.js";
|
||||
import { GENERATED_PLACEHOLDER } from "../src/capture.js";
|
||||
import { decryptSecrets } from "../src/secrets.js";
|
||||
|
||||
// `cast inventory --emit-draft` against a stub shaped like the box that made it
|
||||
// necessary: a Coolify nobody declared, holding our stack under names someone
|
||||
// typed, two third-party client sites, and a database cast cannot even model.
|
||||
//
|
||||
// The single most important assertion in this file is that POISON — the source
|
||||
// box's real DATABASE_URL — appears in NO emitted artifact. A draft that carried
|
||||
// it would rebuild a box that comes up WORKING, reading and writing the old
|
||||
// box's database, and nobody would find out until the old box was deleted.
|
||||
|
||||
// The live values. If any of these reaches an emitted file, the test fails.
|
||||
const POISON =
|
||||
"postgres://postgres:s3cr3t@incubator-database-v2.box-b.internal:5432/app";
|
||||
const POISON_REDIS = "redis://:r3d1s@incubator-redis.box-b.internal:6379/0";
|
||||
const POISON_SERVICE_PW = "umami-generated-9f3a1c";
|
||||
// …and one that MUST survive: an ordinary secret is captured, not placeheld.
|
||||
const MAILGUN = "key-1a2b3c-real-mailgun";
|
||||
|
||||
type Stub = { url: string; close: () => Promise<void> };
|
||||
const stubs: Stub[] = [];
|
||||
|
||||
const ENVS: Record<string, Array<{ key: string; value: string }>> = {
|
||||
a1: [
|
||||
{ key: "DATABASE_URL", value: POISON },
|
||||
{ key: "REDIS_URL", value: POISON_REDIS },
|
||||
{ key: "MAILGUN_KEY", value: MAILGUN },
|
||||
{ key: "NODE_ENV", value: "production" },
|
||||
// Not a name a cast env template can hold. Reported, never dropped in silence.
|
||||
{ key: "legacy.flag", value: "on" },
|
||||
],
|
||||
s1: [
|
||||
{ key: "SERVICE_PASSWORD_UMAMI", value: POISON_SERVICE_PW },
|
||||
{ key: "SERVICE_FQDN_UMAMI", value: "https://umami.box-b.example.com" },
|
||||
{ key: "UMAMI_APP_SECRET", value: "umami-app-secret-xyz" },
|
||||
],
|
||||
a9: [{ key: "WP_HOME", value: "https://lafamilia.example.com" }],
|
||||
};
|
||||
|
||||
// One project with resources in TWO environments — cast must refuse to pick.
|
||||
async function stubCoolify(opts: { ambiguous?: boolean } = {}): Promise<Stub> {
|
||||
const server = createServer((req, res) => {
|
||||
const path = (req.url ?? "").replace("/api/v1", "");
|
||||
const json = (body: unknown) => {
|
||||
res.writeHead(200, { "content-type": "application/json" });
|
||||
res.end(JSON.stringify(body));
|
||||
};
|
||||
if (path === "/teams/current") return json({ id: 0, name: "Root Team" });
|
||||
// Exactly one App: there is no other one an application could have been
|
||||
// cloned by, so cast binds every repo to it rather than leave a marker.
|
||||
if (path === "/github-apps")
|
||||
return json([{ uuid: "g1", name: "hdb-coolify" }]);
|
||||
if (path === "/projects")
|
||||
return json([
|
||||
{ uuid: "p1", name: "Incubator" },
|
||||
{ uuid: "p2", name: "La Familia Site" },
|
||||
{ uuid: "p3", name: "Martin Reyes Barber Shop" },
|
||||
]);
|
||||
if (path === "/projects/p1/environments")
|
||||
return json([{ name: "production" }, { name: "staging" }]);
|
||||
if (path === "/projects/p2/environments")
|
||||
return json([{ name: "production" }]);
|
||||
if (path === "/projects/p3/environments")
|
||||
return json([{ name: "production" }]);
|
||||
|
||||
// Coolify auto-creates `production`. It is empty (unless the ambiguous
|
||||
// variant puts a resource in it too). Everything real lives in `staging`.
|
||||
if (path === "/projects/p1/production")
|
||||
return json(
|
||||
opts.ambiguous
|
||||
? { applications: [{ name: "old-core", uuid: "a7", fqdn: "" }] }
|
||||
: {},
|
||||
);
|
||||
if (path === "/projects/p1/staging")
|
||||
return json({
|
||||
applications: [
|
||||
{
|
||||
name: "Incubator Stack v2",
|
||||
uuid: "a1",
|
||||
git_repository: "heavy-duty/incubator",
|
||||
git_branch: "main",
|
||||
build_pack: "dockercompose",
|
||||
base_directory: "/",
|
||||
docker_compose_location: "/docker-compose.yaml",
|
||||
docker_compose_domains: JSON.stringify([
|
||||
{ name: "core", domain: "https://app.example.com" },
|
||||
]),
|
||||
destination_id: 3,
|
||||
// Basic Auth lives here, and cast's manifest has no field for it.
|
||||
custom_labels:
|
||||
"traefik.http.middlewares.auth.basicauth.users=admin:$2y$05$x",
|
||||
},
|
||||
],
|
||||
postgresqls: [
|
||||
{
|
||||
name: "Incubator Database v2",
|
||||
uuid: "d1",
|
||||
database_type: "standalone-postgresql",
|
||||
image: "postgres:16-alpine",
|
||||
destination_id: 3,
|
||||
},
|
||||
],
|
||||
services: [
|
||||
{
|
||||
name: "Incubator Umami",
|
||||
uuid: "s1",
|
||||
service_type: "umami",
|
||||
destination_id: 3,
|
||||
},
|
||||
],
|
||||
// A database cast's manifest cannot express at all.
|
||||
mysqls: [{ name: "legacy-analytics", uuid: "m1" }],
|
||||
});
|
||||
if (path === "/projects/p2/production")
|
||||
return json({
|
||||
applications: [
|
||||
{
|
||||
name: "lafamilia-web",
|
||||
uuid: "a9",
|
||||
git_repository: "https://github.com/third-party/la-familia.git",
|
||||
git_branch: "main",
|
||||
build_pack: "static",
|
||||
base_directory: "/",
|
||||
publish_directory: "/dist",
|
||||
fqdn: "https://lafamilia.example.com",
|
||||
destination_id: 4,
|
||||
},
|
||||
],
|
||||
});
|
||||
// A project with no application at all: nothing on the box knows its repo.
|
||||
if (path === "/projects/p3/production")
|
||||
return json({
|
||||
services: [
|
||||
{ name: "barber-site", uuid: "s9", service_type: "wordpress" },
|
||||
],
|
||||
});
|
||||
|
||||
const envs = path.match(/^\/[a-z]+\/([a-z0-9]+)\/envs$/);
|
||||
if (envs) return json(ENVS[envs[1]] ?? []);
|
||||
res.writeHead(404);
|
||||
res.end("{}");
|
||||
});
|
||||
await new Promise<void>((r) => {
|
||||
server.listen(0, "127.0.0.1", r);
|
||||
});
|
||||
const stub: Stub = {
|
||||
url: `http://127.0.0.1:${(server.address() as AddressInfo).port}`,
|
||||
close: () =>
|
||||
new Promise<void>((r) => {
|
||||
server.close(() => r());
|
||||
}),
|
||||
};
|
||||
stubs.push(stub);
|
||||
return stub;
|
||||
}
|
||||
|
||||
afterEach(async () => {
|
||||
await Promise.all(stubs.splice(0).map((s) => s.close()));
|
||||
});
|
||||
|
||||
let KEY_FILE = "";
|
||||
let RECIPIENT = "";
|
||||
|
||||
beforeAll(() => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "cast-age-"));
|
||||
KEY_FILE = join(dir, "key.txt");
|
||||
execFileSync("age-keygen", ["-o", KEY_FILE], { stdio: "ignore" });
|
||||
RECIPIENT = execFileSync("age-keygen", ["-y", KEY_FILE], {
|
||||
encoding: "utf8",
|
||||
}).trim();
|
||||
});
|
||||
|
||||
function fixture(url: string, opts: { recipient?: string } = {}) {
|
||||
const state = mkdtempSync(join(tmpdir(), "cast-state-"));
|
||||
writeFileSync(
|
||||
join(state, ".coolify.env"),
|
||||
`COOLIFY_BASE_URL="${url}"\nCOOLIFY_ACCESS_TOKEN="t"\n`,
|
||||
);
|
||||
writeFileSync(
|
||||
join(state, "environments.yaml"),
|
||||
[
|
||||
"environments:",
|
||||
" prod:",
|
||||
" server: box-b",
|
||||
" team: { id: 0, name: Root Team }",
|
||||
...(opts.recipient ? [` age_recipient: ${opts.recipient}`] : []),
|
||||
"github_apps:",
|
||||
" heavy-duty/incubator: hdb-coolify",
|
||||
"",
|
||||
].join("\n"),
|
||||
);
|
||||
const out = join(mkdtempSync(join(tmpdir(), "cast-out-")), "draft");
|
||||
return { state, out };
|
||||
}
|
||||
|
||||
function run(args: string[]): Promise<{ code: number; output: string }> {
|
||||
return new Promise((resolve) => {
|
||||
const child = spawn("node", ["dist/cli.js", "inventory", ...args], {
|
||||
stdio: ["pipe", "pipe", "pipe"],
|
||||
});
|
||||
let output = "";
|
||||
child.stdout.on("data", (d) => {
|
||||
output += String(d);
|
||||
});
|
||||
child.stderr.on("data", (d) => {
|
||||
output += String(d);
|
||||
});
|
||||
child.stdin.end();
|
||||
child.on("close", (code) => resolve({ code: code ?? 0, output }));
|
||||
});
|
||||
}
|
||||
|
||||
// Every file in the emitted tree, path -> bytes as text.
|
||||
function tree(dir: string, prefix = ""): Record<string, string> {
|
||||
const out: Record<string, string> = {};
|
||||
for (const entry of readdirSync(dir, { withFileTypes: true })) {
|
||||
const path = join(dir, entry.name);
|
||||
const rel = prefix ? `${prefix}/${entry.name}` : entry.name;
|
||||
if (entry.isDirectory()) Object.assign(out, tree(path, rel));
|
||||
else out[rel] = readFileSync(path, "utf8");
|
||||
}
|
||||
return out;
|
||||
}
|
||||
|
||||
describe("cast inventory --emit-draft (#27)", () => {
|
||||
it("emits the tree: bindings + registry, a manifest per project, templates, stores, UNCAPTURED", async () => {
|
||||
const f = fixture((await stubCoolify()).url);
|
||||
const r = await run([
|
||||
"--env",
|
||||
"prod",
|
||||
"--state",
|
||||
f.state,
|
||||
"--emit-draft",
|
||||
f.out,
|
||||
"--recipient",
|
||||
RECIPIENT,
|
||||
]);
|
||||
expect(r.code).toBe(0);
|
||||
expect(Object.keys(tree(f.out)).sort()).toEqual([
|
||||
"UNCAPTURED.md",
|
||||
"environments.yaml",
|
||||
// One per project — INCLUDING the two third-party client sites nobody ever
|
||||
// declared. They each sit alone in a Coolify-default `production`, and a
|
||||
// draft that filtered the instance by one environment name would drop
|
||||
// exactly them: the projects the verb exists to bootstrap.
|
||||
"incubator/.infra/env/incubator-stack-v2.prod.env.template",
|
||||
"incubator/.infra/env/incubator-umami.prod.env.template",
|
||||
"incubator/.infra/manifest.yaml",
|
||||
"la-familia/.infra/env/lafamilia-web.prod.env.template",
|
||||
"la-familia/.infra/manifest.yaml",
|
||||
"martin-reyes-barber-shop/.infra/manifest.yaml",
|
||||
"secrets/incubator.prod.env.age",
|
||||
"secrets/la-familia.prod.env.age",
|
||||
]);
|
||||
// Every emitted file says what it is, in its own body.
|
||||
for (const [path, body] of Object.entries(tree(f.out))) {
|
||||
if (path.endsWith(".age")) continue;
|
||||
expect(body, path).toContain("PROPOSAL");
|
||||
expect(body, path).toContain("apply` does not read");
|
||||
}
|
||||
});
|
||||
|
||||
it("NEVER copies a provider-generated value — not into any artifact", async () => {
|
||||
const f = fixture((await stubCoolify()).url);
|
||||
const r = await run([
|
||||
"--env",
|
||||
"prod",
|
||||
"--state",
|
||||
f.state,
|
||||
"--emit-draft",
|
||||
f.out,
|
||||
"--recipient",
|
||||
RECIPIENT,
|
||||
]);
|
||||
expect(r.code).toBe(0);
|
||||
|
||||
// 1. Not in any emitted FILE.
|
||||
for (const [path, body] of Object.entries(tree(f.out))) {
|
||||
expect(body, path).not.toContain(POISON);
|
||||
expect(body, path).not.toContain(POISON_REDIS);
|
||||
expect(body, path).not.toContain(POISON_SERVICE_PW);
|
||||
}
|
||||
// 2. Not in the age store either, once decrypted — the store is the one place
|
||||
// a copied value would actually be, and the one place you cannot grep.
|
||||
const store = decryptSecrets(
|
||||
join(f.out, "secrets", "incubator.prod.env.age"),
|
||||
KEY_FILE,
|
||||
);
|
||||
expect(store.DATABASE_URL).toBe(GENERATED_PLACEHOLDER);
|
||||
expect(store.REDIS_URL).toBe(GENERATED_PLACEHOLDER);
|
||||
expect(store.SERVICE_PASSWORD_UMAMI).toBe(GENERATED_PLACEHOLDER);
|
||||
expect(store.SERVICE_FQDN_UMAMI).toBe(GENERATED_PLACEHOLDER);
|
||||
expect(Object.values(store)).not.toContain(POISON);
|
||||
// 3. Not on stdout.
|
||||
expect(r.output).not.toContain(POISON);
|
||||
|
||||
// …and an ORDINARY secret IS carried. The discipline is placeholding the
|
||||
// provider's names, not refusing to capture anything.
|
||||
expect(store.MAILGUN_KEY).toBe(MAILGUN);
|
||||
expect(store.UMAMI_APP_SECRET).toBe("umami-app-secret-xyz");
|
||||
|
||||
// The manifest declares the placeheld names, so a later `capture` placeholds
|
||||
// them again with no flag to remember.
|
||||
const manifest = readFileSync(
|
||||
join(f.out, "incubator", ".infra", "manifest.yaml"),
|
||||
"utf8",
|
||||
);
|
||||
expect(manifest).toContain("generated_secrets:");
|
||||
expect(manifest).toContain("- DATABASE_URL");
|
||||
expect(manifest).toContain("- REDIS_URL");
|
||||
// The plan says so out loud, in names and provenance — never values.
|
||||
expect(r.output).toContain("generated");
|
||||
expect(r.output).toContain(GENERATED_PLACEHOLDER);
|
||||
expect(r.output).toContain("captured");
|
||||
});
|
||||
|
||||
it("UNCAPTURED.md lists every known-inexpressible setting it SAW", async () => {
|
||||
const f = fixture((await stubCoolify()).url);
|
||||
const r = await run([
|
||||
"--env",
|
||||
"prod",
|
||||
"--state",
|
||||
f.state,
|
||||
"--emit-draft",
|
||||
f.out,
|
||||
"--recipient",
|
||||
RECIPIENT,
|
||||
]);
|
||||
expect(r.code).toBe(0);
|
||||
const md = readFileSync(join(f.out, "UNCAPTURED.md"), "utf8");
|
||||
|
||||
// Seen on the box, and inexpressible:
|
||||
expect(md).toContain("Incubator Umami"); // service hostnames (#21 / 4.1.2)
|
||||
expect(md).toContain("domains (hostnames)");
|
||||
expect(md).toContain("destination"); // which Docker network (#21)
|
||||
expect(md).toContain("destination_id 3");
|
||||
expect(md).toContain("legacy-analytics"); // a MySQL cast cannot model
|
||||
expect(md).toContain("custom_labels"); // Basic Auth / Traefik labels
|
||||
expect(md).toContain("backup"); // not exposed by the API
|
||||
expect(md).toContain("legacy.flag"); // not a name a template can hold
|
||||
|
||||
// And the standing sections, emitted on every run whatever was found:
|
||||
expect(md).toContain("no API coverage in Coolify 4.1.2");
|
||||
expect(md).toContain("Include Source Commit in Build");
|
||||
expect(md).toContain("What a rebuild from this draft still cannot restore");
|
||||
expect(md).toContain("the GitHub App private key");
|
||||
expect(md).toContain("S3 access keys");
|
||||
|
||||
// The run points at it rather than leaving it to be found.
|
||||
expect(r.output).toContain("UNCAPTURED.md");
|
||||
});
|
||||
|
||||
it("writes the projects: registry — the list of what exists", async () => {
|
||||
const f = fixture((await stubCoolify()).url);
|
||||
await run([
|
||||
"--env",
|
||||
"prod",
|
||||
"--state",
|
||||
f.state,
|
||||
"--emit-draft",
|
||||
f.out,
|
||||
"--recipient",
|
||||
RECIPIENT,
|
||||
]);
|
||||
const yaml = readFileSync(join(f.out, "environments.yaml"), "utf8");
|
||||
expect(yaml).toContain("projects:");
|
||||
// Read off the application's git remote — the only place a box knows its repo.
|
||||
expect(yaml).toContain("third-party/la-familia:");
|
||||
expect(yaml).toContain("environments:\n - prod");
|
||||
// The bindings the sweep could actually read.
|
||||
expect(yaml).toContain("server: box-b");
|
||||
expect(yaml).toContain("name: Root Team");
|
||||
// Which GitHub App clones a repo is not on any resource — but this instance
|
||||
// has exactly one, and there is no other it could be.
|
||||
expect(yaml).toContain("github_apps:");
|
||||
expect(yaml).toContain("third-party/la-familia: hdb-coolify");
|
||||
|
||||
// The barber shop has no application, so the box knows no repo for it. cast
|
||||
// writes the bare project name — and the registry's own parse-time refusal
|
||||
// (#25: "a registry key has no meaning without its org") then stops the file
|
||||
// being used until a human supplies the org. That refusal IS the design: the
|
||||
// alternatives are inventing an org, or dropping a project from the list —
|
||||
// and a project missing from the registry is one every fleet run skips in
|
||||
// silence.
|
||||
expect(yaml).toContain("martin-reyes-barber-shop:");
|
||||
const path = join(f.out, "environments.yaml");
|
||||
expect(() => loadBindings(path)).toThrow(/has no meaning without its org/);
|
||||
});
|
||||
|
||||
it("refuses --emit-draft together with a repo — that is the reconcile path", async () => {
|
||||
const f = fixture((await stubCoolify()).url);
|
||||
const r = await run([
|
||||
"heavy-duty/incubator",
|
||||
"--env",
|
||||
"prod",
|
||||
"--state",
|
||||
f.state,
|
||||
"--emit-draft",
|
||||
f.out,
|
||||
]);
|
||||
expect(r.code).toBe(2);
|
||||
expect(r.output).toContain("SWEEP-mode flag");
|
||||
expect(r.output).toContain("Adoption is one-way");
|
||||
expect(existsSync(f.out)).toBe(false);
|
||||
});
|
||||
|
||||
it("refuses a target directory that already holds a repo", async () => {
|
||||
const f = fixture((await stubCoolify()).url);
|
||||
mkdirSync(join(f.out, ".infra"), { recursive: true });
|
||||
writeFileSync(
|
||||
join(f.out, ".infra", "manifest.yaml"),
|
||||
"project: incubator\nenvironments: {}\n",
|
||||
);
|
||||
const r = await run([
|
||||
"--env",
|
||||
"prod",
|
||||
"--state",
|
||||
f.state,
|
||||
"--emit-draft",
|
||||
f.out,
|
||||
"--recipient",
|
||||
RECIPIENT,
|
||||
]);
|
||||
expect(r.code).toBe(2);
|
||||
expect(r.output).toContain("is not empty");
|
||||
// For a declared project the manifest is the truth — a draft must not be able
|
||||
// to overwrite a reviewed spec with a live box's accumulated cruft.
|
||||
expect(r.output).toContain("Adoption is one-way");
|
||||
expect(readFileSync(join(f.out, ".infra", "manifest.yaml"), "utf8")).toBe(
|
||||
"project: incubator\nenvironments: {}\n",
|
||||
);
|
||||
});
|
||||
|
||||
it("refuses to emit secrets nobody can decrypt — and takes an explicit opt-out", async () => {
|
||||
const f = fixture((await stubCoolify()).url);
|
||||
const refused = await run([
|
||||
"--env",
|
||||
"prod",
|
||||
"--state",
|
||||
f.state,
|
||||
"--emit-draft",
|
||||
f.out,
|
||||
]);
|
||||
// No --recipient, no age_recipient binding. Silently skipping the store would
|
||||
// emit a draft that LOOKS complete and holds not one value.
|
||||
expect(refused.code).toBe(2);
|
||||
expect(refused.output).toContain("no age recipient");
|
||||
expect(refused.output).toContain("--no-secrets");
|
||||
expect(existsSync(f.out)).toBe(false);
|
||||
|
||||
const optOut = await run([
|
||||
"--env",
|
||||
"prod",
|
||||
"--state",
|
||||
f.state,
|
||||
"--emit-draft",
|
||||
f.out,
|
||||
"--no-secrets",
|
||||
]);
|
||||
expect(optOut.code).toBe(0);
|
||||
expect(existsSync(join(f.out, "secrets"))).toBe(false);
|
||||
expect(
|
||||
existsSync(join(f.out, "incubator", ".infra", "manifest.yaml")),
|
||||
).toBe(true);
|
||||
expect(optOut.output).toContain("NOT WRITTEN");
|
||||
});
|
||||
|
||||
it("takes the recipient from the environment's binding when no flag is given", async () => {
|
||||
const f = fixture((await stubCoolify()).url, { recipient: RECIPIENT });
|
||||
const r = await run([
|
||||
"--env",
|
||||
"prod",
|
||||
"--state",
|
||||
f.state,
|
||||
"--emit-draft",
|
||||
f.out,
|
||||
]);
|
||||
expect(r.code).toBe(0);
|
||||
const store = decryptSecrets(
|
||||
join(f.out, "secrets", "incubator.prod.env.age"),
|
||||
KEY_FILE,
|
||||
);
|
||||
expect(store.MAILGUN_KEY).toBe(MAILGUN);
|
||||
});
|
||||
|
||||
it("refuses to pick between two environments that both have resources", async () => {
|
||||
const f = fixture((await stubCoolify({ ambiguous: true })).url);
|
||||
const r = await run([
|
||||
"--env",
|
||||
"prod",
|
||||
"--state",
|
||||
f.state,
|
||||
"--emit-draft",
|
||||
f.out,
|
||||
"--recipient",
|
||||
RECIPIENT,
|
||||
]);
|
||||
// Picking would emit a blueprint of half a box that says nothing about the
|
||||
// other half — and this box is where that already happened once (#22).
|
||||
expect(r.code).toBe(2);
|
||||
expect(r.output).toContain("MORE THAN ONE environment");
|
||||
expect(r.output).toContain("Incubator");
|
||||
expect(r.output).toContain("--environment");
|
||||
expect(existsSync(f.out)).toBe(false);
|
||||
|
||||
// …and --environment breaks the tie. For the projects that HAVE no tie (the
|
||||
// client sites, each alone in its own `production`), it changes nothing: they
|
||||
// are drafted either way.
|
||||
const tied = await run([
|
||||
"--env",
|
||||
"prod",
|
||||
"--state",
|
||||
f.state,
|
||||
"--emit-draft",
|
||||
f.out,
|
||||
"--recipient",
|
||||
RECIPIENT,
|
||||
"--environment",
|
||||
"staging",
|
||||
]);
|
||||
expect(tied.code).toBe(0);
|
||||
const files = Object.keys(tree(f.out));
|
||||
expect(files).toContain("incubator/.infra/manifest.yaml");
|
||||
expect(files).toContain("la-familia/.infra/manifest.yaml");
|
||||
// The environment it did NOT draft is named, not dropped in silence.
|
||||
expect(readFileSync(join(f.out, "UNCAPTURED.md"), "utf8")).toContain(
|
||||
"other environments",
|
||||
);
|
||||
});
|
||||
|
||||
it("still sweeps, and still asserts the team, before it writes anything", async () => {
|
||||
const f = fixture((await stubCoolify()).url);
|
||||
writeFileSync(
|
||||
join(f.state, "environments.yaml"),
|
||||
[
|
||||
"environments:",
|
||||
" prod:",
|
||||
" server: box-b",
|
||||
" team: { id: 9, name: Some Other Team }",
|
||||
"",
|
||||
].join("\n"),
|
||||
);
|
||||
const r = await run([
|
||||
"--env",
|
||||
"prod",
|
||||
"--state",
|
||||
f.state,
|
||||
"--emit-draft",
|
||||
f.out,
|
||||
"--recipient",
|
||||
RECIPIENT,
|
||||
]);
|
||||
// A wrong-team token reads back an empty instance — and would draft a
|
||||
// blueprint of nothing at all, confidently.
|
||||
expect(r.code).not.toBe(0);
|
||||
expect(existsSync(f.out)).toBe(false);
|
||||
});
|
||||
});
|
||||
317
test/draft.test.ts
Normal file
317
test/draft.test.ts
Normal file
|
|
@ -0,0 +1,317 @@
|
|||
import { mkdirSync, mkdtempSync, writeFileSync } from "node:fs";
|
||||
import { tmpdir } from "node:os";
|
||||
import { join } from "node:path";
|
||||
import { describe, expect, it } from "vitest";
|
||||
import { GENERATED_PLACEHOLDER } from "../src/capture.js";
|
||||
import {
|
||||
type DraftProject,
|
||||
assertEmptyTarget,
|
||||
assertNoExistingManifest,
|
||||
draftResourcesFrom,
|
||||
isProviderGenerated,
|
||||
planDraft,
|
||||
repoFromGitUrl,
|
||||
} from "../src/draft.js";
|
||||
import { templateKeys, templateRefs } from "../src/envtemplate.js";
|
||||
import { loadManifest } from "../src/manifest.js";
|
||||
|
||||
const ctx = {
|
||||
env: "prod",
|
||||
instance: "box-b",
|
||||
baseUrl: "https://coolify.example.com",
|
||||
team: { id: 0, name: "Root Team" },
|
||||
server: "box-b",
|
||||
recipient: "age1example",
|
||||
generatedAt: "2026-07-13T00:00:00.000Z",
|
||||
};
|
||||
|
||||
// The value that must never leave the box it was read from.
|
||||
const POISON = "postgres://postgres:pw@incubator-db-v2.box-b.internal:5432/app";
|
||||
|
||||
const project = (over: Partial<DraftProject> = {}): DraftProject => ({
|
||||
name: "Incubator",
|
||||
coolifyEnv: "staging",
|
||||
resources: [
|
||||
{
|
||||
kind: "application",
|
||||
name: "Incubator Stack v2",
|
||||
uuid: "a1",
|
||||
raw: {
|
||||
git_repository: "https://github.com/heavy-duty/incubator",
|
||||
git_branch: "main",
|
||||
build_pack: "nixpacks",
|
||||
base_directory: "/",
|
||||
ports_exposes: "3000",
|
||||
fqdn: "https://app.example.com",
|
||||
destination_id: 3,
|
||||
},
|
||||
env: {
|
||||
DATABASE_URL: POISON,
|
||||
MAILGUN_KEY: "key-abc123",
|
||||
NODE_ENV: "production",
|
||||
},
|
||||
},
|
||||
],
|
||||
unreadable: [],
|
||||
otherEnvironments: [],
|
||||
...over,
|
||||
});
|
||||
|
||||
describe("isProviderGenerated — the one judgment that must not be wrong", () => {
|
||||
it("recognizes the datastore families whose value points at the SOURCE box", () => {
|
||||
for (const key of [
|
||||
"DATABASE_URL",
|
||||
"DATABASE_URL_PROD",
|
||||
"UMAMI_DATABASE_URL",
|
||||
"REDIS_URL",
|
||||
"POSTGRES_PASSWORD",
|
||||
"DB_HOST",
|
||||
"MONGO_URI",
|
||||
]) {
|
||||
expect(isProviderGenerated(key), key).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it("recognizes Coolify's own per-instance magic vars", () => {
|
||||
for (const key of [
|
||||
"SERVICE_FQDN_UMAMI",
|
||||
"SERVICE_URL_UMAMI",
|
||||
"SERVICE_PASSWORD_POSTGRES",
|
||||
"SERVICE_USER_UMAMI",
|
||||
"SERVICE_BASE64_KEY",
|
||||
]) {
|
||||
expect(isProviderGenerated(key), key).toBe(true);
|
||||
}
|
||||
});
|
||||
|
||||
it("leaves an ordinary secret alone — it is captured, not placeheld", () => {
|
||||
for (const key of [
|
||||
"MAILGUN_KEY",
|
||||
"OPENROUTER_KEY",
|
||||
"ADMIN_EMAIL",
|
||||
"NODE_ENV",
|
||||
"SERVICE_NAME",
|
||||
"PORT",
|
||||
]) {
|
||||
expect(isProviderGenerated(key), key).toBe(false);
|
||||
}
|
||||
});
|
||||
});
|
||||
|
||||
describe("repoFromGitUrl — the only place a box knows which repo it is", () => {
|
||||
it("reads a slug out of every remote shape Coolify stores", () => {
|
||||
expect(repoFromGitUrl("https://github.com/heavy-duty/incubator")).toBe(
|
||||
"heavy-duty/incubator",
|
||||
);
|
||||
expect(repoFromGitUrl("https://github.com/heavy-duty/incubator.git")).toBe(
|
||||
"heavy-duty/incubator",
|
||||
);
|
||||
expect(repoFromGitUrl("git@github.com:heavy-duty/incubator.git")).toBe(
|
||||
"heavy-duty/incubator",
|
||||
);
|
||||
expect(repoFromGitUrl("heavy-duty/incubator")).toBe("heavy-duty/incubator");
|
||||
});
|
||||
|
||||
it("answers undefined rather than guessing", () => {
|
||||
expect(repoFromGitUrl("")).toBeUndefined();
|
||||
expect(repoFromGitUrl(undefined)).toBeUndefined();
|
||||
expect(repoFromGitUrl("not-a-remote")).toBeUndefined();
|
||||
});
|
||||
});
|
||||
|
||||
describe("draftResourcesFrom — including what cast cannot model", () => {
|
||||
it("names a MySQL rather than silently omitting it", () => {
|
||||
const { resources, unreadable } = draftResourcesFrom({
|
||||
applications: [{ name: "web", uuid: "a1" }],
|
||||
postgresqls: [{ name: "db", uuid: "d1" }],
|
||||
redis: [{ name: "cache", uuid: "d2" }],
|
||||
services: [{ name: "umami", uuid: "s1" }],
|
||||
mysqls: [{ name: "legacy-mysql", uuid: "m1" }],
|
||||
mongodbs: [{ name: "old-mongo", uuid: "m2" }],
|
||||
});
|
||||
expect(resources.map((r) => [r.kind, r.name])).toEqual([
|
||||
["application", "web"],
|
||||
["database", "db"],
|
||||
["database", "cache"],
|
||||
["service", "umami"],
|
||||
]);
|
||||
expect(unreadable).toEqual([
|
||||
{ kind: "mysql", name: "legacy-mysql" },
|
||||
{ kind: "mongodb", name: "old-mongo" },
|
||||
]);
|
||||
});
|
||||
});
|
||||
|
||||
describe("planDraft — the emitted shape", () => {
|
||||
it("writes a manifest cast itself can load, keyed by --env", () => {
|
||||
const plan = planDraft([project()], ctx);
|
||||
const manifest = plan.files.find((f) => f.path.endsWith("manifest.yaml"));
|
||||
expect(manifest?.path).toBe("incubator/.infra/manifest.yaml");
|
||||
// A file that says what it is. It leaves this process and is read by someone
|
||||
// deciding whether to trust it.
|
||||
expect(manifest?.content).toContain("PROPOSAL");
|
||||
expect(manifest?.content).toContain("`apply` does not read this file");
|
||||
expect(manifest?.content).toContain("box-b");
|
||||
|
||||
const dir = mkdtempSync(join(tmpdir(), "cast-draft-"));
|
||||
const path = join(dir, "manifest.yaml");
|
||||
writeFileSync(path, manifest?.content ?? "");
|
||||
const loaded = loadManifest(path);
|
||||
expect(loaded.project).toBe("Incubator");
|
||||
const env = loaded.environments.prod;
|
||||
// The resource keeps the BOX's name — renaming it here would make the file
|
||||
// unusable against the UI it was read from.
|
||||
expect(Object.keys(env.applications)).toEqual(["Incubator Stack v2"]);
|
||||
expect(env.applications["Incubator Stack v2"].source).toEqual({
|
||||
repo: "heavy-duty/incubator",
|
||||
branch: "main",
|
||||
});
|
||||
// And the manifest DECLARES the placeheld name, so a later `capture` does the
|
||||
// same placeholding with no flag to remember.
|
||||
expect(env.generated_secrets).toEqual(["DATABASE_URL"]);
|
||||
});
|
||||
|
||||
it("emits an env template every cast reader can parse", () => {
|
||||
const plan = planDraft([project()], ctx);
|
||||
const tpl = plan.files.find((f) => f.path.endsWith(".env.template"));
|
||||
expect(tpl?.path).toBe(
|
||||
"incubator/.infra/env/incubator-stack-v2.prod.env.template",
|
||||
);
|
||||
const body = tpl?.content ?? "";
|
||||
expect(templateKeys(body).sort()).toEqual([
|
||||
"DATABASE_URL",
|
||||
"MAILGUN_KEY",
|
||||
"NODE_ENV",
|
||||
]);
|
||||
// EVERY var is a ${REF}: values live in the store, never as a literal in a
|
||||
// file that is about to be committed to a product repo.
|
||||
expect(
|
||||
templateRefs(body)
|
||||
.map((r) => r.ref)
|
||||
.sort(),
|
||||
).toEqual(["DATABASE_URL", "MAILGUN_KEY", "NODE_ENV"]);
|
||||
expect(body).not.toContain(POISON);
|
||||
expect(body).not.toContain("key-abc123");
|
||||
});
|
||||
|
||||
it("PLACEHOLDS a provider-generated value and captures an ordinary one", () => {
|
||||
const plan = planDraft([project()], ctx);
|
||||
const byRef = Object.fromEntries(plan.dispositions.map((d) => [d.ref, d]));
|
||||
expect(byRef.DATABASE_URL.provenance).toBe("generated");
|
||||
expect(byRef.DATABASE_URL.value).toBe(GENERATED_PLACEHOLDER);
|
||||
expect(byRef.MAILGUN_KEY.provenance).toBe("captured");
|
||||
expect(byRef.MAILGUN_KEY.value).toBe("key-abc123");
|
||||
|
||||
// The store carries the placeholder, NOT the source box's Postgres.
|
||||
const store = plan.stores[0];
|
||||
expect(store.path).toBe("secrets/incubator.prod.env.age");
|
||||
expect(store.vars.DATABASE_URL).toBe(GENERATED_PLACEHOLDER);
|
||||
expect(JSON.stringify(plan.files)).not.toContain(POISON);
|
||||
});
|
||||
|
||||
it("splits one name carrying two values rather than picking", () => {
|
||||
const p = project();
|
||||
p.resources.push({
|
||||
kind: "application",
|
||||
name: "Landing",
|
||||
uuid: "a2",
|
||||
raw: {
|
||||
git_repository: "https://github.com/heavy-duty/incubator",
|
||||
git_branch: "main",
|
||||
build_pack: "static",
|
||||
base_directory: "/",
|
||||
fqdn: "https://www.example.com",
|
||||
},
|
||||
env: { MAILGUN_KEY: "key-DIFFERENT" },
|
||||
});
|
||||
const plan = planDraft([p], ctx);
|
||||
const refs = plan.dispositions.map((d) => d.ref);
|
||||
// One store holds one value per name (capture refuses a CONFLICT for exactly
|
||||
// this reason). cast will not pick, so both survive under distinct names.
|
||||
expect(refs).toContain("INCUBATOR_STACK_V2_MAILGUN_KEY");
|
||||
expect(refs).toContain("LANDING_MAILGUN_KEY");
|
||||
expect(refs).not.toContain("MAILGUN_KEY");
|
||||
expect(
|
||||
plan.uncaptured.some((u) => u.detail.includes("DIFFERENT values")),
|
||||
).toBe(true);
|
||||
});
|
||||
|
||||
it("always emits UNCAPTURED.md — even with little to say", () => {
|
||||
const bare = project({
|
||||
resources: [
|
||||
{
|
||||
kind: "application",
|
||||
name: "web",
|
||||
uuid: "a1",
|
||||
raw: {
|
||||
git_repository: "git@github.com:acme/web.git",
|
||||
git_branch: "main",
|
||||
build_pack: "nixpacks",
|
||||
base_directory: "/",
|
||||
fqdn: "https://web.example.com",
|
||||
},
|
||||
env: {},
|
||||
},
|
||||
],
|
||||
});
|
||||
const md = planDraft([bare], ctx).files.find(
|
||||
(f) => f.path === "UNCAPTURED.md",
|
||||
);
|
||||
expect(md).toBeDefined();
|
||||
// The standing sections are unconditional: what cast CANNOT SEE does not
|
||||
// depend on what it happened to find.
|
||||
expect(md?.content).toContain("no API coverage in Coolify 4.1.2");
|
||||
expect(md?.content).toContain("Include Source Commit in Build");
|
||||
expect(md?.content).toContain("the GitHub App private key");
|
||||
expect(md?.content).toContain("S3 access keys");
|
||||
});
|
||||
|
||||
it("registers what it drafted, and only what it drafted", () => {
|
||||
const empty = project({
|
||||
name: "Empty Project",
|
||||
resources: [],
|
||||
skipReason: "every environment on it is empty",
|
||||
});
|
||||
const bindings = planDraft([project(), empty], ctx).files.find(
|
||||
(f) => f.path === "environments.yaml",
|
||||
);
|
||||
expect(bindings?.content).toContain("heavy-duty/incubator");
|
||||
expect(bindings?.content).toContain("environments:\n - prod");
|
||||
// Not registered — a registry entry for a project with no manifest sends
|
||||
// every future fleet run at nothing.
|
||||
expect(bindings?.content).not.toContain("Empty Project");
|
||||
// …but it is not lost either.
|
||||
const md = planDraft([project(), empty], ctx).files.find(
|
||||
(f) => f.path === "UNCAPTURED.md",
|
||||
);
|
||||
expect(md?.content).toContain("Empty Project");
|
||||
});
|
||||
});
|
||||
|
||||
describe("the emit refusals — adoption is one-way", () => {
|
||||
it("refuses a target directory that is not empty", () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "cast-draft-"));
|
||||
writeFileSync(join(dir, "README.md"), "a repo lives here\n");
|
||||
expect(() => assertEmptyTarget(dir)).toThrow(/is not empty/);
|
||||
expect(() => assertEmptyTarget(dir)).toThrow(/Adoption is one-way/);
|
||||
});
|
||||
|
||||
it("allows a directory that does not exist yet, and an empty one", () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "cast-draft-"));
|
||||
expect(() => assertEmptyTarget(dir)).not.toThrow();
|
||||
expect(() => assertEmptyTarget(join(dir, "new"))).not.toThrow();
|
||||
});
|
||||
|
||||
it("refuses to write a manifest over one that already exists", () => {
|
||||
const dir = mkdtempSync(join(tmpdir(), "cast-draft-"));
|
||||
mkdirSync(join(dir, ".infra"), { recursive: true });
|
||||
const path = join(dir, ".infra", "manifest.yaml");
|
||||
writeFileSync(path, "project: incubator\n");
|
||||
// For a declared project the manifest IS the truth: regenerating it from a
|
||||
// live box would let that box's cruft overwrite a reviewed spec.
|
||||
expect(() => assertNoExistingManifest(path)).toThrow(/already exists/);
|
||||
expect(() => assertNoExistingManifest(path)).toThrow(
|
||||
/the manifest IS the truth/,
|
||||
);
|
||||
});
|
||||
});
|
||||
Loading…
Reference in a new issue