heavy-duty/cast

Fork 0

inventory --emit-draft: a reviewable blueprint of a live instance (and an honest list of what it could not capture) #27

New issue

Closed

opened 2026-07-13 19:23:54 +00:00 by dan-claude-bot · 1 comment

dan-claude-bot commented

2026-07-13 19:23:54 +00:00

(Migrated from github.com)

The idea

cast inventory can already see a whole instance (#22). Let it emit what it
sees as a draft of cast's own inputs:

cast inventory --env prod --instance box-b --emit-draft ./draft

draft/
  environments.yaml                    # bindings, as far as they can be read
  incubator/.infra/manifest.yaml       # one per project
  incubator/.infra/env/*.env.template
  la-familia/.infra/manifest.yaml
  secrets/<project>.<env>.env.age      # encrypted to a recipient you name
  UNCAPTURED.md                        # ← the important file. See below.

Two uses, and they are different:

Bootstrap a project that has no manifest. Two third-party client sites on
the box being drained have never been declared and never will be unless
something writes the first draft. Hand-transcribing them from a UI is exactly
the work cast exists to eliminate.
A point-in-time blueprint — run it any time, get a snapshot you could
rebuild an instance from. From the draft you raise adjustment PRs against the
respective repos, and the repos stay the source of truth.

The boundary (non-negotiable)

A draft is a PROPOSAL. It is never desired state, and apply never reads it.
It is emitted, reviewed by a human, and lands in a repo as a PR. Same shape
as terraform import → HCL. The path is:

sweep → emit draft → human reviews → manifest PR → capture → apply

And it must never emit into a repo that already has a manifest. For a
declared project the manifest is the truth; regenerating it from a live box
would let that box's accumulated cruft overwrite the reviewed spec. Adoption is
one-way.

Two things that would make the draft actively dangerous

1. Copied provider-generated values. A DATABASE_URL read off the source
points at the source box's Postgres. Emit that into a draft, rebuild elsewhere,
and the new box comes up working — reading and writing the old box's
database. You find out the day the old box is deleted. Same for REDIS_URL, and
for Coolify's own magic vars (SERVICE_FQDN_*, SERVICE_URL_*,
SERVICE_PASSWORD_*), which Coolify generates per-instance and which mean
nothing anywhere else.

So the draft applies capture's discipline: provider-generated names are
placeheld and flagged for disposition, never copied. A draft that is
confidently wrong in four entries out of seventeen is worse than one that is
obviously incomplete.

2. Silent losses. cast cannot currently express everything a Coolify holds:

destinations (#21) — which network a resource sits on
service hostnames (Coolify 4.1.2 has no flat domains on services)
Basic Auth / custom Traefik labels
the "Include Source Commit in Build" toggle (no API coverage in 4.1.2)
anything else configured in the UI that has no manifest field

A blueprint that omits these without saying so is worse than no blueprint,
because in a disaster you would trust it and rebuild a different box. Hence
UNCAPTURED.md: a first-class output listing, per resource, every live setting
cast saw and could not express. Emitting it is not optional.

What a blueprint still cannot restore

Worth writing into the docs at the same time, because "rebuild from the repo" is
routinely over-claimed:


control plane	`rig coolify install` ✅
structure	draft/manifests → `apply` ✅
secret values	the age store + your key ✅
data	Coolify's DB backups → S3 ✅ (separate path)
the GitHub App private key	❌ re-create by hand
S3 access keys	❌ re-mint by hand

The last two are not in the repo (correctly — it holds no live credentials) and
cannot be regenerated from it. A DR runbook has to say so.

Depends on

The projects: registry (#25) — a whole-instance draft needs somewhere to write
the project list it discovered.

## The idea `cast inventory` can already see a whole instance (#22). Let it **emit what it sees** as a draft of cast's own inputs: ```sh cast inventory --env prod --instance box-b --emit-draft ./draft ``` ``` draft/ environments.yaml # bindings, as far as they can be read incubator/.infra/manifest.yaml # one per project incubator/.infra/env/*.env.template la-familia/.infra/manifest.yaml secrets/<project>.<env>.env.age # encrypted to a recipient you name UNCAPTURED.md # ← the important file. See below. ``` Two uses, and they are different: 1. **Bootstrap a project that has no manifest.** Two third-party client sites on the box being drained have never been declared and never will be unless something writes the first draft. Hand-transcribing them from a UI is exactly the work cast exists to eliminate. 2. **A point-in-time blueprint** — run it any time, get a snapshot you could rebuild an instance from. From the draft you raise adjustment PRs against the respective repos, and the repos stay the source of truth. ## The boundary (non-negotiable) **A draft is a PROPOSAL. It is never desired state, and `apply` never reads it.** It is emitted, reviewed by a human, and lands in a repo as a **PR**. Same shape as `terraform import` → HCL. The path is: **sweep → emit draft → human reviews → manifest PR → `capture` → `apply`** And it must **never emit into a repo that already has a manifest**. For a declared project the manifest *is* the truth; regenerating it from a live box would let that box's accumulated cruft overwrite the reviewed spec. Adoption is one-way. ## Two things that would make the draft actively dangerous **1. Copied provider-generated values.** A `DATABASE_URL` read off the source points at the *source box's* Postgres. Emit that into a draft, rebuild elsewhere, and the new box comes up **working** — reading and writing the old box's database. You find out the day the old box is deleted. Same for `REDIS_URL`, and for Coolify's own magic vars (`SERVICE_FQDN_*`, `SERVICE_URL_*`, `SERVICE_PASSWORD_*`), which Coolify generates per-instance and which mean nothing anywhere else. So the draft applies `capture`'s discipline: provider-generated names are **placeheld and flagged for disposition**, never copied. A draft that is confidently wrong in four entries out of seventeen is worse than one that is obviously incomplete. **2. Silent losses.** cast cannot currently express everything a Coolify holds: - **destinations** (#21) — which network a resource sits on - **service hostnames** (Coolify 4.1.2 has no flat `domains` on services) - Basic Auth / custom Traefik labels - the **"Include Source Commit in Build"** toggle (no API coverage in 4.1.2) - anything else configured in the UI that has no manifest field A blueprint that omits these **without saying so** is worse than no blueprint, because in a disaster you would trust it and rebuild a *different box*. Hence `UNCAPTURED.md`: a first-class output listing, per resource, every live setting cast saw and could not express. Emitting it is not optional. ## What a blueprint still cannot restore Worth writing into the docs at the same time, because "rebuild from the repo" is routinely over-claimed: | | | | --- | --- | | control plane | `rig coolify install` ✅ | | structure | draft/manifests → `apply` ✅ | | secret **values** | the age store + your key ✅ | | **data** | Coolify's DB backups → S3 ✅ (separate path) | | **the GitHub App private key** | ❌ re-create by hand | | **S3 access keys** | ❌ re-mint by hand | The last two are not in the repo (correctly — it holds no live credentials) and cannot be regenerated from it. A DR runbook has to say so. ## Depends on The `projects:` registry (#25) — a whole-instance draft needs somewhere to write the project list it discovered.

dan-claude-bot commented

2026-07-13 19:47:53 +00:00

(Migrated from github.com)

#28 adds a hard limit to what a draft can honestly emit — and it lands squarely in UNCAPTURED.md.

A destination cannot be round-tripped. Coolify 4.1.2 takes destination_uuid on write and returns destination_id (an integer PK) on read, with no endpoint mapping one to the other, and no destinations API at all.

So a draft emitted from a live instance cannot produce a working destination_uuid. It can see that core sits on destination 5; it cannot learn the UUID that 5 corresponds to, because nothing exposes it. That is not a rough edge in the emitter — it is a hole in the blueprint, and exactly the kind this issue says must never be silent:

a blueprint that omits these without saying so is worse than no blueprint, because in a disaster you would trust it and rebuild a different box.

Concretely, the draft must:

emit projects.<slug>.destination_uuid: as a TODO with the observed destination_id beside it — never a guess, never omitted;
record in UNCAPTURED.md that placement was observed but not resolvable, and that the UUIDs must be read out of the source instance's UI before the draft can be applied anywhere;
carry over smoke_target (now project-scoped, per #28).

And the consequence for the DR story, which is the reason this issue exists: a rebuild from a blueprint alone will place every resource on the target server's default network unless a human supplies the UUIDs by hand. On a single-destination server that is silently fine. On a multi-destination server it is a hard 400 — Coolify refuses a create that omits destination_uuid there. Either way the blueprint did not rebuild the box it described, and only UNCAPTURED.md would have said so.

Worth adding to the table in this issue's body: destinations join the GitHub App private key and the S3 access keys as things a blueprint cannot restore. All three are re-supplied by hand.

**#28 adds a hard limit to what a draft can honestly emit — and it lands squarely in `UNCAPTURED.md`.** A destination **cannot be round-tripped.** Coolify 4.1.2 takes `destination_uuid` on write and returns `destination_id` (an integer PK) on read, with **no endpoint mapping one to the other**, and no destinations API at all. So a draft emitted from a live instance **cannot produce a working `destination_uuid`**. It can see that `core` sits on destination `5`; it cannot learn the UUID that `5` corresponds to, because nothing exposes it. That is not a rough edge in the emitter — it is a **hole in the blueprint**, and exactly the kind this issue says must never be silent: > a blueprint that omits these *without saying so* is worse than no blueprint, because in a disaster you would trust it and rebuild a *different box*. Concretely, the draft must: - emit `projects.<slug>.destination_uuid:` as a **`TODO` with the observed `destination_id` beside it** — never a guess, never omitted; - record in `UNCAPTURED.md` that placement was observed but not resolvable, and that the UUIDs must be read out of the source instance's UI **before the draft can be applied anywhere**; - carry over `smoke_target` (now project-scoped, per #28). And the consequence for the DR story, which is the reason this issue exists: **a rebuild from a blueprint alone will place every resource on the target server's default network** unless a human supplies the UUIDs by hand. On a single-destination server that is silently fine. On a multi-destination server it is a **hard 400** — Coolify refuses a create that omits `destination_uuid` there. Either way the blueprint did not rebuild the box it described, and only `UNCAPTURED.md` would have said so. Worth adding to the table in this issue's body: **destinations join the GitHub App private key and the S3 access keys as things a blueprint cannot restore.** All three are re-supplied by hand.

No milestone

No project

No assignees

1 participant

Notifications

Due date

The due date is invalid or out of range. Please use the format "yyyy-mm-dd".

No due date set.

Dependencies

No dependencies set.

Reference: heavy-duty/cast#27

No description provided.