Merge pull request #106 from dan-claude-bot/fix/greenfield-empty-store

fix: a manifest with no ${…} refs applies without a store (#104)

CHANGELOG.md

@@ -22,6 +22,23 @@ actually cutting it, and this file starts there.
   contains an application; a manifest that does declare one still refuses on
   a missing binding exactly as before, clean plan or not, because that
   binding is state the next create will need.
+- **A manifest with no `${…}` refs applies without a store** (#104) — the
+  greenfield manifest-first bootstrap was a chicken-and-egg with no exit,
+  found by the 2026-07-19 release drill against two fresh Coolify 4.1.2
+  instances: a registered project whose manifest declared databases only
+  (zero `${…}` refs) could not take its first `apply` — apply refused with
+  `no secret store for <org>/<repo> in <env>`, and `capture`, the documented
+  way to get a store, rightly refuses a project that is absent on the box,
+  because apply is the verb that would create it. The drill unblocked with a
+  hand-rolled empty store (`printf '' | age -r … -o secrets/….env.age`),
+  documented nowhere. Now `diff`/`apply` gate that refusal on the manifest
+  actually *referencing* a secret, asked via the same parser resolution
+  uses: when the templates resolve zero `${…}` refs, an absent store is
+  treated as empty and the run proceeds, printing a loud one-line note
+  naming the path the store would live at — and since there is nothing to
+  decrypt, the age key is not demanded either. The moment any template
+  gains a `${…}` ref, the refusal returns byte-identical to before.
+  `capture` and `destroy` are untouched.
 
 ### Added
 

README.md

@@ -431,6 +431,13 @@ apply-from-nothing, so every generated secret in every store is a placeholder
 again. This used to be a hand `age` re-encrypt against production, with the prod
 key in a process substitution.
 
+**Greenfield needs zero passes** (#104): a manifest whose templates hold no
+`${…}` refs at all — databases only, or apps whose env is pure literals —
+applies from nothing. No store, no age key: `diff` and `apply` say out loud that
+the store is absent and was not needed, and proceed. The store appears the first
+time `capture` writes it, or the first time a template gains a placeholder —
+from then on, an absent store refuses exactly as before.
+
 `--generated-only` **inverts** capture's rule and changes nothing else: it fills
 the `generated_secrets` names and leaves every other name in the store **exactly as
 it is, byte for byte** — never re-read from the box, so a secret you rotated by

src/cli.ts

@@ -1158,7 +1158,24 @@ async function runProject(
   // store was never written is a project a fleet run cannot read — and under
   // --all the headline of this message is what the summary carries, so it has to
   // say which project and which file rather than "Command failed: age -d".
-  if (!existsSync(store)) {
+  //
+  // Unless nothing would ever be read from it (#104). A manifest whose templates
+  // hold no ${…} refs resolves against an absent store exactly as it would
+  // against an empty one — and the greenfield first apply (fresh box, databases
+  // only, no secret authored yet) is precisely that manifest. Refusing there is
+  // a chicken-and-egg with no exit: apply demands a store, and capture — the
+  // documented way to get one — rightly refuses a project that does not exist on
+  // the box yet, because apply is the verb that would create it. So the refusal
+  // is gated on the manifest actually REFERENCING a secret, asked via
+  // requiredSecrets — the same parser resolution itself uses, so the two cannot
+  // disagree about what "no refs" means — and the zero-refs case proceeds on {}
+  // with a loud note. No store also means no decrypt, so the age key is not
+  // demanded either: nothing to open, nothing to protect yet. The store appears
+  // the first time capture writes it, or a template gains a placeholder.
+  let secrets: Record<string, string>;
+  if (existsSync(store)) {
+    secrets = decryptSecrets(store, keyFileFor(ctx.envName));
+  } else if (requiredSecrets(checkout, ctx.envName).required.length > 0) {
     throw new Error(
       [
         `no secret store for ${orgRepo} in ${ctx.envName}`,
@@ -1169,8 +1186,12 @@ async function runProject(
         "diff or apply without it. `cast capture` writes one from a live box.",
       ].join("\n"),
     );
+  } else {
+    console.log(
+      `NOTE: no secret store for ${orgRepo} in ${ctx.envName} (looked for ${store}) — and none is needed: this manifest's templates hold no \${…} refs, so cast proceeds without a store or an age key. The store appears the first time capture writes it, or a template gains a placeholder.`,
+    );
+    secrets = {};
   }
-  const secrets = decryptSecrets(store, keyFileFor(ctx.envName));
   let { desired, resolvedEnvs } = desiredFromManifest(
     checkout,
     ctx.envName,

test/fleet-cli.test.ts

@@ -114,7 +114,7 @@ afterEach(async () => {
   await Promise.all(stubs.splice(0).map((s) => s.close()));
 });
 
-const manifest = (repo: string) => `project: ${repo}
+const manifest = (repo: string, withRef = false) => `project: ${repo}
 environments:
   staging:
     applications:
@@ -122,20 +122,33 @@ environments:
         source: { repo: heavy-duty/${repo}, branch: main }
         build: { pack: nixpacks, base_directory: / }
         domains: ["http://${repo}.example.com"]
-`;
+${withRef ? "        env_template: core.env\n" : ""}`;
 
 // A state dir + three clonable product repos. `registry` is the knob: which
 // slugs the `projects:` block registers for staging — undefined writes no
 // `projects:` block at all (a state file from before the registry existed).
+// `refIn` gives ONE repo a template with a ${…} ref: since #104 gated the
+// missing-store refusal on the manifest actually referencing a secret, a
+// fixture that wants to exercise that refusal has to reference one.
 function fixture(
   url: string,
-  opts: { registry?: string[]; registryEnv?: string } = {},
+  opts: { registry?: string[]; registryEnv?: string; refIn?: string } = {},
 ) {
   const root = mkdtempSync(join(tmpdir(), "cast-fleet-"));
   for (const repo of REPOS) {
     const dir = join(root, "repos", "heavy-duty", `${repo}.git`);
     mkdirSync(join(dir, ".infra"), { recursive: true });
-    writeFileSync(join(dir, ".infra", "manifest.yaml"), manifest(repo));
+    writeFileSync(
+      join(dir, ".infra", "manifest.yaml"),
+      manifest(repo, repo === opts.refIn),
+    );
+    if (repo === opts.refIn) {
+      mkdirSync(join(dir, ".infra", "env"));
+      writeFileSync(
+        join(dir, ".infra", "env", "core.env"),
+        "API_KEY=${API_KEY}\n",
+      );
+    }
     const git = (...args: string[]) =>
       execFileSync("git", args, { cwd: dir, stdio: "pipe" });
     git("init", "-q");
@@ -157,9 +170,10 @@ function fixture(
     `COOLIFY_BASE_URL="${url}"\nCOOLIFY_ACCESS_TOKEN="t"\n`,
   );
   for (const repo of REPOS) {
-    // No template refs a secret, but the store still has to exist and open —
+    // No template refs a secret, so since #104 none of these stores is strictly
-    // a project whose store is missing is a project cast cannot read, which is
+    // required — but the pre-greenfield shape (store on disk, key injected) is
-    // a fleet ERROR, not a fleet skip (asserted below).
+    // the shape most fleets are in, and it has to keep working unchanged. The
+    // missing-store-with-a-ref refusal is exercised below by deleting one.
     execFileSync("age", ["-r", recipient, "-o", `${repo}.staging.env.age`], {
       input: "\n",
       cwd: join(state, "secrets"),
@@ -292,8 +306,11 @@ describe("cast diff --all (#26)", () => {
     expect(r.output).toContain("500");
   });
 
-  it("treats a missing secret store as unreachable, naming the file", async () => {
+  // `refIn` matters: since #104 the refusal is gated on the manifest actually
-    const f = fixture((await stubCoolify()).url);
+  // referencing a secret, so the missing store is only an error because beta's
+  // template holds a ${…} ref. The gate's other half is the test after this one.
+  it("treats a missing secret store as unreachable when a template refs a secret, naming the file", async () => {
+    const f = fixture((await stubCoolify()).url, { refIn: "beta" });
     execFileSync("rm", [join(f.state, "secrets", "beta.staging.env.age")]);
     const r = await run("diff", fleet(f), f);
     expect(r.code).toBe(2);
@@ -302,6 +319,22 @@ describe("cast diff --all (#26)", () => {
     );
     expect(r.output).toContain("UNREACHABLE:  1  heavy-duty/beta");
   });
+
+  // The greenfield gate (#104) under --all: a zero-refs project whose store was
+  // never written is READ, not failed — the note prints, the fleet stays whole.
+  it("reads a zero-refs project with no store, noting the absence instead of failing it", async () => {
+    const f = fixture((await stubCoolify()).url);
+    execFileSync("rm", [join(f.state, "secrets", "beta.staging.env.age")]);
+    const r = await run("diff", fleet(f), f);
+    expect(r.code).toBe(0);
+    expect(r.output).toContain(
+      "NOTE: no secret store for heavy-duty/beta in staging",
+    );
+    expect(r.output).toContain("read:         3 of 3");
+    expect(r.output).toContain(
+      "all 3 registered project(s) were read, and every one is clean.",
+    );
+  });
 });
 
 describe("cast diff --all — an empty fleet is not a clean fleet", () => {

test/greenfield-cli.test.ts

@@ -0,0 +1,237 @@
+import { execFileSync, spawn } from "node:child_process";
+import { mkdirSync, mkdtempSync, writeFileSync } from "node:fs";
+import { createServer } from "node:http";
+import type { AddressInfo } from "node:net";
+import { tmpdir } from "node:os";
+import { join } from "node:path";
+import { afterEach, beforeAll, describe, expect, it } from "vitest";
+
+// The greenfield manifest-first bootstrap (#104), end to end: fresh box,
+// registered project, a manifest that declares databases only and refs no
+// secret — the exact shape the 2026-07-19 release drill hit. There used to be
+// no path to the first apply: `apply` refused on the absent store, `capture`
+// rightly refused the absent project, and the drill unblocked with a hand-made
+// empty store nothing documented.
+//
+// The rule under test: the "no secret store" refusal is gated on the manifest
+// actually REFERENCING a secret. Zero ${…} refs → an absent store is treated
+// as empty, a loud note names the path, and no age key is demanded (nothing to
+// decrypt, nothing to protect yet). One ${…} ref → the refusal, byte-identical
+// to what it always said. A present store keeps decrypting exactly as before.
+
+let recipient: string;
+let keyFile: string;
+
+beforeAll(() => {
+  const dir = mkdtempSync(join(tmpdir(), "cast-age-"));
+  keyFile = join(dir, "age.key");
+  execFileSync("age-keygen", ["-o", keyFile], { stdio: "pipe" });
+  recipient = execFileSync("age-keygen", ["-y", keyFile], {
+    encoding: "utf8",
+  }).trim();
+});
+
+type Stub = { url: string; close: () => Promise<void> };
+const stubs: Stub[] = [];
+
+// A fresh box: the project is registered on Coolify but its environment holds
+// nothing — the state one API call after `cast project create`, and the state
+// the drill's first apply ran against.
+async function stubCoolify(): Promise<Stub> {
+  const server = createServer((req, res) => {
+    const path = (req.url ?? "").replace("/api/v1", "");
+    const json = (body: unknown) => {
+      res.writeHead(200, { "content-type": "application/json" });
+      res.end(JSON.stringify(body));
+    };
+    if (path === "/teams/current") return json({ id: 0, name: "Root Team" });
+    if (path === "/projects") return json([{ uuid: "p1", name: "fresh" }]);
+    if (path === "/projects/p1/staging") return json({ applications: [] });
+    res.writeHead(404);
+    res.end("{}");
+  });
+  await new Promise<void>((r) => {
+    server.listen(0, "127.0.0.1", r);
+  });
+  const stub: Stub = {
+    url: `http://127.0.0.1:${(server.address() as AddressInfo).port}`,
+    close: () =>
+      new Promise<void>((r) => {
+        server.close(() => r());
+      }),
+  };
+  stubs.push(stub);
+  return stub;
+}
+
+afterEach(async () => {
+  await Promise.all(stubs.splice(0).map((s) => s.close()));
+});
+
+// The drill's manifest, minimized: databases only, zero ${…} refs anywhere.
+const ZERO_REFS_MANIFEST = `project: fresh
+environments:
+  staging:
+    applications: {}
+    databases:
+      fresh-db:
+        type: postgresql
+`;
+
+// The same environment the moment one template gains a placeholder — the
+// boundary at which the old refusal must return, word for word.
+const ONE_REF_MANIFEST = `project: fresh
+environments:
+  staging:
+    applications:
+      core:
+        source: { repo: heavy-duty/fresh, branch: main }
+        build: { pack: nixpacks, base_directory: / }
+        domains: ["http://core.example.com"]
+        env_template: core.env
+    databases:
+      fresh-db:
+        type: postgresql
+`;
+
+function fixture(
+  url: string,
+  opts: { manifest: string; store?: boolean } = {
+    manifest: ZERO_REFS_MANIFEST,
+  },
+) {
+  const checkout = mkdtempSync(join(tmpdir(), "cast-co-"));
+  mkdirSync(join(checkout, ".infra", "env"), { recursive: true });
+  writeFileSync(join(checkout, ".infra", "manifest.yaml"), opts.manifest);
+  writeFileSync(
+    join(checkout, ".infra", "env", "core.env"),
+    "API_KEY=${API_KEY}\n",
+  );
+
+  const state = mkdtempSync(join(tmpdir(), "cast-state-"));
+  mkdirSync(join(state, "secrets"));
+  writeFileSync(
+    join(state, ".coolify.env"),
+    `COOLIFY_BASE_URL="${url}"\nCOOLIFY_ACCESS_TOKEN="t"\n`,
+  );
+  if (opts.store) {
+    execFileSync("age", ["-r", recipient, "-o", "fresh.staging.env.age"], {
+      input: "\n",
+      cwd: join(state, "secrets"),
+      stdio: ["pipe", "pipe", "pipe"],
+    });
+  }
+  writeFileSync(
+    join(state, "environments.yaml"),
+    [
+      "environments:",
+      "  staging:",
+      "    server: fresh-box",
+      "    team: { id: 0, name: Root Team }",
+      "github_apps:",
+      "  fresh: hdb-coolify",
+      "",
+    ].join("\n"),
+  );
+  return { checkout, state };
+}
+
+// `withKey: false` is the greenfield claim itself: the run is spawned with no
+// CAST_AGE_KEY_FILE_STAGING and a HOME that holds no standing key, so if cast
+// so much as ASKS for the age key, the run dies "no age key for staging" and
+// the assertion on the output catches it.
+function run(
+  args: string[],
+  opts: { withKey: boolean },
+): Promise<{ code: number; output: string }> {
+  return new Promise((resolve) => {
+    const { CAST_AGE_KEY_FILE_STAGING: _dropped, ...inherited } = process.env;
+    const env = opts.withKey
+      ? { ...inherited, CAST_AGE_KEY_FILE_STAGING: keyFile }
+      : { ...inherited, HOME: mkdtempSync(join(tmpdir(), "cast-home-")) };
+    const child = spawn("node", ["dist/cli.js", "diff", ...args], {
+      stdio: ["pipe", "pipe", "pipe"],
+      env,
+    });
+    let output = "";
+    child.stdout.on("data", (d) => {
+      output += String(d);
+    });
+    child.stderr.on("data", (d) => {
+      output += String(d);
+    });
+    child.on("close", (code) => resolve({ code: code ?? 0, output }));
+  });
+}
+
+const base = (f: { checkout: string; state: string }) => [
+  "heavy-duty/fresh",
+  "--env",
+  "staging",
+  "--path",
+  f.checkout,
+  "--state",
+  f.state,
+];
+
+describe("greenfield: zero ${…} refs and no store (#104)", () => {
+  it("proceeds to a full plan, says the store was absent and unneeded, and never asks for an age key", async () => {
+    const f = fixture((await stubCoolify()).url, {
+      manifest: ZERO_REFS_MANIFEST,
+    });
+    const r = await run(base(f), { withKey: false });
+    // The plan, not a refusal: the database is there to create, so this is an
+    // ordinary drift exit — which is the whole point, the first apply's plan.
+    expect(r.code).toBe(1);
+    expect(r.output).toContain("fresh-db");
+    expect(r.output).toContain(
+      "NOTE: no secret store for heavy-duty/fresh in staging",
+    );
+    expect(r.output).toContain(
+      join(f.state, "secrets", "fresh.staging.env.age"),
+    );
+    expect(r.output).toContain("proceeds without a store or an age key");
+    // The two ways the old behavior would have surfaced, both absent: the
+    // refusal (whose body, unlike the note, tells you what the store is FOR),
+    // and — with no key in the spawn env at all — the key demand.
+    expect(r.output).not.toContain("resolved from that store");
+    expect(r.output).not.toContain("no age key for staging");
+  });
+
+  it("keeps the original refusal, word for word, the moment a template holds a ${…} ref", async () => {
+    const f = fixture((await stubCoolify()).url, {
+      manifest: ONE_REF_MANIFEST,
+    });
+    const r = await run(base(f), { withKey: false });
+    // Exit 1 is what a single-project refusal has always exited with: the
+    // throw lands in main()'s rejection handler, same as before #104. The
+    // fleet flavor of this refusal (exit 2, UNREACHABLE) is fleet-cli.test.ts.
+    expect(r.code).toBe(1);
+    expect(r.output).toContain(
+      "no secret store for heavy-duty/fresh in staging",
+    );
+    expect(r.output).toContain(
+      `looked for:  ${join(f.state, "secrets", "fresh.staging.env.age")}`,
+    );
+    expect(r.output).toContain(
+      "The manifest's ${…} refs are resolved from that store",
+    );
+    expect(r.output).toContain("`cast capture` writes one from a live box.");
+    // The refusal, not the plan and not the note.
+    expect(r.output).not.toContain("NOTE:");
+    expect(r.output).not.toContain("fresh-db");
+  });
+
+  it("still opens a store that DOES exist — zero refs or not, a written store is decrypted as before", async () => {
+    const f = fixture((await stubCoolify()).url, {
+      manifest: ZERO_REFS_MANIFEST,
+      store: true,
+    });
+    // The pre-#104 shape: store on disk, key injected. Same plan as the
+    // greenfield run, and no note — the store was there, so nothing to say.
+    const r = await run(base(f), { withKey: true });
+    expect(r.code).toBe(1);
+    expect(r.output).toContain("fresh-db");
+    expect(r.output).not.toContain("NOTE: no secret store");
+  });
+});