README.md

@@ -51,6 +51,8 @@ environments.yaml               # bindings: the team each env's token must belon
                                 #   which server it deploys onto, the S3 destination,
                                 #   GitHub App name, guards — and, per project,
                                 #   the destination it deploys onto + its smoke target
+                                #   …plus `projects:`, the registry: which projects
+                                #   exist, and in which environments
 secrets/<repo>.<env>.env.age    # age-encrypted values for the ${…} placeholders
 .coolify.env                    # COOLIFY_BASE_URL + COOLIFY_ACCESS_TOKEN (never commit)
 .coolify/<name>.env             # …the same, for a NAMED instance (see below)
@@ -357,6 +359,54 @@ own (it hangs off a project) and no API path scopes by one: **Coolify
 environments are an organizational construct, not an auth boundary.** The team
 is the only boundary there is, so it is the one cast asserts.
 
+## The registry: which projects exist
+
+`environments.yaml` says where things deploy *to*, and how a project you have
+already named is placed once it is there. Until the `projects:` block, nothing in
+it said **which projects exist at all** — "every project" was a thing the
+operator remembered:
+
+```yaml
+projects:
+  heavy-duty/incubator:
+    environments: [prod, staging]
+  acme/client-site:
+    environments: [prod]
+```
+
+Keyed by the **full `<org>/<repo>` slug**, and the key *is* the repo — there is
+no `repo:` field inside, because a second place to write the same string is a
+second place for it to be wrong. Unlike `github_apps`, a bare `<repo>` key is
+**refused** rather than resolved: this block is new, so it has no state files in
+the wild to keep working, and a bare `<repo>` is unique only *within* an org —
+which is exactly why it is not a key. `environments:` lists **our** environment
+names (the values `--env` takes), never Coolify's.
+
+The block is optional; a state file written before it loads unchanged.
+
+**It has to be true, so cast checks that it is — at parse time, for every verb.**
+Two ways it could quietly stop being true, both refused:
+
+- an environment name that no `environments:` block defines (a typo). The project
+  is real and its environment imaginary, so a fleet run visits nothing for it,
+  reports nothing, and exits clean.
+- an `environments.<env>.projects.<repo>` binding — a destination, a smoke target
+  — in an environment the registry does not register that project for. The two
+  blocks then describe two different fleets: state real enough for a direct
+  `cast apply` to use, invisible to every fleet run. (Checked only when
+  `projects:` is present.)
+
+Both refusals defend one failure: **a silently skipped project reads exactly like
+a clean one.** Silence is the one report that must never be ambiguous.
+
+What it unlocks, neither of which was possible without a list to iterate:
+
+- **fleet operations** — `cast diff --all` / `apply --all` over every project in
+  an environment.
+- **rebuild-from-state** — "restore this Coolify from the state repo" cannot even
+  be *attempted* without knowing what was on it. The registry is the difference
+  between a documented recovery and an archaeology exercise.
+
 ## Two projects, one box: destinations
 
 A **destination** is the Docker network a resource is created on. A server has a

docs/semantics.md

@@ -158,6 +158,51 @@ softened by an implementation detail):
   refuses `--path` combined with `--env prod`: prod always reads the default
   branch, so a feature-branch checkout can never reach it.
 
+## The registry (`projects:`)
+
+The top-level `projects:` block is the list of **which projects exist**, and in
+which environments. It is the only place that says so: `environments:` says where
+things deploy to, `environments.<env>.projects.<repo>` says how an already-named
+project is placed, `github_apps` says how to clone one you have already named.
+
+```yaml
+projects:
+  heavy-duty/incubator:
+    environments: [prod, staging]
+```
+
+- **Keyed by the full `<org>/<repo>` slug, with no bare-`<repo>` fallback.** The
+  key is the repo; there is no `repo:` field. `github_apps` and
+  `environments.<env>.projects` accept a bare key because state files in the wild
+  are written that way; this block is new and has none, so it requires the slug —
+  a bare `<repo>` is unique only *within* an org.
+- **`environments:` names OUR environments** — the keys of the `environments:`
+  block, the values `--env` takes — never Coolify's. Non-empty.
+- **Optional.** A state file with no `projects:` block loads unchanged, and
+  `projectsIn` reports `[]` for every environment.
+
+**Validated at parse time, so every verb refuses a registry that lies.** Two
+refusals, both defending the same failure — *a silently skipped project reads
+exactly like a clean one*, which makes silence, the most common report there is,
+ambiguous:
+
+1. **An environment that does not exist** (a typo in `projects.<slug>.environments`)
+   is an error naming the unknown environment and listing the known ones. Left
+   alone, the project would be registered into an environment no command can
+   visit: a fleet run skips it, reports nothing, exits clean.
+2. **A binding the registry does not register.** Every
+   `environments.<env>.projects.<slug>` key must be a project the registry
+   registers *for that environment*. Otherwise the two blocks describe two
+   different fleets: a `destination_uuid` or `smoke_target` real enough for a
+   direct `cast apply <repo> --env <env>` to act on, and invisible to every
+   fleet run over that environment. Enforced **only when `projects:` is
+   present**, so pre-registry state files keep loading.
+
+The registry is what makes two things possible, neither of which can be attempted
+without a list to iterate: **fleet operations** (`--all`), and
+**rebuild-from-state** — restoring a Coolify from the state repo, which is
+otherwise an assumption, since you cannot restore what you cannot enumerate.
+
 ## Placement (destinations)
 
 A **destination** is the Docker network a resource is created on. It is declared

src/bindings.ts

@@ -58,6 +58,32 @@ const ProjectBindingSchema = z
 
 export type ProjectBinding = z.infer<typeof ProjectBindingSchema>;
 
+// One project in the registry — the top-level `projects:` block, which is the
+// list of what EXISTS. Nothing else in this file says that: `environments:`
+// says where things are deployed to, `environments.<env>.projects.<repo>` says
+// how one project is placed once you already know it is there, and
+// `github_apps` says how to clone one you have already named. "Every project"
+// was, until this block, a thing the operator remembered.
+//
+// Two things need the list, and neither can be built without it: fleet
+// operations (`cast diff --all`, #26 — iterating "every project in this
+// environment") and rebuild-from-state (#27 — a Coolify restored from the state
+// repo, which cannot even be attempted without knowing what was on it).
+const RegisteredProjectSchema = z
+  .object({
+    // OUR environment names — the values `--env` takes, the keys of the
+    // `environments:` block above — never Coolify's. The distinction is the same
+    // one `--env` vs `--environment` draws everywhere else in cast.
+    //
+    // Non-empty: a project registered into no environment is not a registration,
+    // it is a line of YAML that reads like one. It would be skipped by every
+    // fleet run silently.
+    environments: z.array(z.string().min(1)).nonempty(),
+  })
+  .strict();
+
+export type RegisteredProject = z.infer<typeof RegisteredProjectSchema>;
+
 const BindingsSchema = z
   .object({
     environments: z.record(
@@ -100,6 +126,22 @@ const BindingsSchema = z
         })
         .strict(),
     ),
+    // THE REGISTRY: which projects exist at all, keyed by the full `<org>/<repo>`
+    // slug. The key IS the repo — there is no `repo:` field inside, because a
+    // second place to write the same string is a second place for it to be
+    // wrong.
+    //
+    // Full slug REQUIRED, with no bare-`<repo>` fallback — the one place in this
+    // file where that fallback does not exist. `github_apps` and
+    // `environments.<env>.projects` carry one because they predate the lesson
+    // (#12) and there are state files in the wild keyed the old way; this block
+    // is new, has no such files, and so gets to be right from the start. A bare
+    // `<repo>` is unique only *within* an org, which is precisely why it is not
+    // a key.
+    //
+    // Optional: a state file written before the registry existed keeps loading
+    // untouched, and `projectsIn` answers `[]` for it.
+    projects: z.record(RegisteredProjectSchema).optional(),
     // Keyed by the repo the App clones for. Prefer the FULL `<org>/<repo>`
     // slug; a bare `<repo>` key still resolves (see githubAppNameFor) so
     // existing state files keep working.
@@ -112,7 +154,136 @@ const BindingsSchema = z
     // cannot distinguish prod's app from staging's either.
     smoke_target: z.string().optional(),
   })
-  .strict();
+  .strict()
+  // The registry only earns its keep if it is TRUE. Every check here defends the
+  // same failure: a project that a fleet run never visits, because a fleet run
+  // that skips a project prints exactly what a fleet run over a clean project
+  // prints — nothing. Silence is the one report that must never be ambiguous, so
+  // these are parse-time errors (every verb loads bindings, so every verb refuses
+  // a registry that lies) rather than warnings some command might print.
+  .superRefine((bindings, ctx) => {
+    const registry = bindings.projects;
+    if (!registry) return;
+
+    const knownEnvs = Object.keys(bindings.environments);
+    const knownEnvList = knownEnvs.join(", ") || "(none)";
+
+    for (const [slug, project] of Object.entries(registry)) {
+      // A key with no `/` is not a repo. See the schema note above: no fallback.
+      if (!slug.includes("/")) {
+        ctx.addIssue({
+          code: z.ZodIssueCode.custom,
+          path: ["projects", slug],
+          message: [
+            `projects["${slug}"] is not a repo — a registry key has no meaning without its org`,
+            "",
+            `  found:     projects["${slug}"]`,
+            "  wanted:    a full <org>/<repo> slug",
+            "",
+            "A bare <repo> is unique only *within* an org: heavy-duty/incubator and",
+            "acme/incubator collapse onto one entry, and the registry then claims one",
+            "project where there are two. Unlike github_apps, this block is new and has",
+            "no legacy state files to support, so there is no bare-<repo> fallback.",
+            "",
+            "  projects:",
+            `    <org>/${slug}:`,
+            `      environments: [${project.environments.join(", ")}]`,
+          ].join("\n"),
+        });
+      }
+
+      // Every environment named here must be one that actually exists. A typo
+      // makes the project real but its environment imaginary — so `--all` visits
+      // nothing for it, reports nothing about it, and exits clean.
+      for (const envName of project.environments) {
+        if (!(envName in bindings.environments)) {
+          ctx.addIssue({
+            code: z.ZodIssueCode.custom,
+            path: ["projects", slug, "environments"],
+            message: [
+              `projects["${slug}"] is registered in environment "${envName}", which does not exist`,
+              "",
+              `  registered:  projects["${slug}"].environments  →  ${project.environments.join(", ")}`,
+              `  known envs:  ${knownEnvList}`,
+              "",
+              "An environment nothing defines is one that no command can visit: a fleet",
+              "run would skip this project, and a silently skipped project reads exactly",
+              "like a clean one. Fix the name, or declare the environment:",
+              "",
+              "  environments:",
+              `    ${envName}:`,
+              "      server: <server>",
+              "      team: { id: <id>, name: <name> }",
+            ].join("\n"),
+          });
+        }
+      }
+    }
+
+    // The other direction, and the one that rots quietly: a project carrying
+    // per-environment state (destination_uuid, smoke_target — #21) in an
+    // environment the registry does not register it for. That state is then real
+    // enough to be used by a direct `cast apply <repo> --env <env>` and invisible
+    // to every fleet run — the two blocks describing two different fleets.
+    //
+    // Only checked when `projects:` is present, so state files written before the
+    // registry keep loading exactly as they did.
+    for (const [envName, env] of Object.entries(bindings.environments)) {
+      for (const key of Object.keys(env.projects ?? {})) {
+        const entry = registry[key];
+        if (entry?.environments.includes(envName)) continue;
+
+        // The likeliest cause, worth saying out loud: a legacy bare-<repo> key
+        // (which projectBindingFor still resolves) under a registry that is
+        // correctly keyed by slug. The fix is a rename, not a registration.
+        const slugFor = key.includes("/")
+          ? undefined
+          : Object.keys(registry).find((s) => s.endsWith(`/${key}`));
+
+        const cause = slugFor
+          ? [
+              `  registry has:  projects["${slugFor}"]`,
+              "",
+              "The binding uses the legacy bare-<repo> key. The registry is keyed by the",
+              `full slug, so rename it to match — environments.${envName}.projects["${slugFor}"].`,
+            ]
+          : entry
+            ? [
+                `  registered for:  ${entry.environments.join(", ")}`,
+                "",
+                "Register the project in this environment, or drop the binding — state that",
+                "no fleet run will ever visit is state that stops being true without anyone",
+                "finding out:",
+                "",
+                "  projects:",
+                `    ${key}:`,
+                `      environments: [${[...entry.environments, envName].join(", ")}]`,
+              ]
+            : [
+                `  registry has:  ${Object.keys(registry).join(", ") || "(nothing)"}`,
+                "",
+                "The project is not in the registry at all, so no fleet run will ever visit",
+                "it — while this binding says it is deployed here. Register it, or drop the",
+                "binding:",
+                "",
+                "  projects:",
+                `    ${key}:`,
+                `      environments: [${envName}]`,
+              ];
+
+        ctx.addIssue({
+          code: z.ZodIssueCode.custom,
+          path: ["environments", envName, "projects", key],
+          message: [
+            `environments.${envName}.projects["${key}"] is bound in an environment the registry does not register it for`,
+            "",
+            `  bound at:      environments.${envName}.projects["${key}"]`,
+            ...cause,
+          ].join("\n"),
+        });
+      }
+    }
+  });
 
 export type Bindings = z.infer<typeof BindingsSchema>;
 
@@ -188,6 +359,27 @@ export function smokeTargetFor(
   return undefined;
 }
 
+// The `<org>/<repo>` slugs registered for one environment — the list a fleet
+// operation iterates (`cast diff --all`, #26) and a rebuild reads (#27).
+//
+// SORTED, deliberately: the order of keys in a YAML file is an accident of who
+// typed what when, and a fleet run's output — which a human reads top to bottom,
+// and CI diffs — must not reshuffle because someone appended a project. The
+// registry is a set; this returns it as one.
+//
+// `[]` when there is no registry, which is every state file written before this
+// block existed. That is the honest answer for "which projects are registered
+// here" when nothing is registered anywhere — and it makes a fleet verb over an
+// unmigrated state file a clean no-op rather than a crash.
+export function projectsIn(bindings: Bindings, envName: string): string[] {
+  const registry = bindings.projects;
+  if (!registry) return [];
+  return Object.entries(registry)
+    .filter(([, project]) => project.environments.includes(envName))
+    .map(([slug]) => slug)
+    .sort();
+}
+
 export function loadBindings(
   path: string,
   opts: { overrideText?: string } = {},
@@ -195,7 +387,20 @@ export function loadBindings(
   const text = opts.overrideText ?? readFileSync(path, "utf8");
   const result = BindingsSchema.safeParse(parse(text));
   if (!result.success) {
-    throw new Error(`invalid bindings ${path}: ${result.error.message}`);
+    // Zod's own `.message` is the entire issue array as JSON — which renders the
+    // refusals above as one long line of `\n` escapes, i.e. throws away the part
+    // of them that was worth writing. Render the issues instead.
+    const detail = result.error.issues
+      .map((issue) => {
+        // A multi-line message is one WE wrote: it already names the path, the
+        // cause, and the YAML to write. A one-line message is zod's ("Required"),
+        // and is useless without the path it happened at.
+        if (issue.message.includes("\n")) return issue.message;
+        const where = issue.path.map(String).join(".");
+        return where ? `${where}: ${issue.message}` : issue.message;
+      })
+      .join("\n\n");
+    throw new Error(`invalid bindings ${path}:\n\n${detail}`);
   }
   return result.data;
 }

test/bindings.test.ts

@@ -5,6 +5,7 @@ import {
   githubAppNameFor,
   loadBindings,
   projectBindingFor,
+  projectsIn,
   smokeTargetFor,
 } from "../src/bindings.js";
 
@@ -223,3 +224,233 @@ github_apps: {}
     ).toThrow(/invalid bindings/);
   });
 });
+
+// The registry: the list of which projects exist at all. Everything it is FOR
+// (fleet iteration, rebuild-from-state) depends on it being true, and the way it
+// stops being true is silent — see the refusals below.
+describe("the project registry", () => {
+  const twoEnvs = `
+environments:
+  prod:
+    server: shared-box
+    team: { id: 0, name: Root Team }
+  staging:
+    server: staging-box
+    team: { id: 0, name: Root Team }
+`;
+
+  it("registers projects per environment, keyed by the full slug", () => {
+    const b = loadBindings("environments.yaml", {
+      overrideText: `${twoEnvs}
+projects:
+  heavy-duty/incubator:
+    environments: [prod, staging]
+  acme/client-site:
+    environments: [prod]
+github_apps: {}
+`,
+    });
+    expect(b.projects).toEqual({
+      "heavy-duty/incubator": { environments: ["prod", "staging"] },
+      "acme/client-site": { environments: ["prod"] },
+    });
+  });
+
+  describe("projectsIn", () => {
+    const b = loadBindings("environments.yaml", {
+      overrideText: `${twoEnvs}
+projects:
+  heavy-duty/incubator:
+    environments: [prod, staging]
+  acme/client-site:
+    environments: [prod]
+github_apps: {}
+`,
+    });
+
+    // Sorted, not file-order: a fleet run's output is read by a human and diffed
+    // by CI, and must not reshuffle because someone appended a project.
+    it("gives an environment's projects, sorted", () => {
+      expect(projectsIn(b, "prod")).toEqual([
+        "acme/client-site",
+        "heavy-duty/incubator",
+      ]);
+    });
+
+    it("gives only the projects registered for that environment", () => {
+      expect(projectsIn(b, "staging")).toEqual(["heavy-duty/incubator"]);
+    });
+
+    it("is empty for an environment no project is registered in", () => {
+      expect(projectsIn(b, "nowhere")).toEqual([]);
+    });
+  });
+
+  // The refusal the issue is actually about. A typo'd environment name makes the
+  // project real and its environment imaginary: `cast diff --all` visits nothing
+  // for it, reports nothing, and exits clean — and a silently skipped project
+  // reads exactly like a clean one.
+  it("refuses an environment that does not exist, naming the ones that do", () => {
+    const err = () =>
+      loadBindings("environments.yaml", {
+        overrideText: `${twoEnvs}
+projects:
+  heavy-duty/incubator:
+    environments: [prod, stagng]
+github_apps: {}
+`,
+      });
+    expect(err).toThrow(/environment "stagng", which does not exist/);
+    expect(err).toThrow(/known envs:\s+prod, staging/);
+  });
+
+  // The other direction, and the one that rots quietly: per-environment state
+  // (#21) sitting in an environment the registry does not register the project
+  // for. The two blocks then describe two different fleets.
+  it("refuses a binding in an environment the registry does not register", () => {
+    const err = () =>
+      loadBindings("environments.yaml", {
+        overrideText: `
+environments:
+  prod:
+    server: shared-box
+    team: { id: 0, name: Root Team }
+  staging:
+    server: staging-box
+    team: { id: 0, name: Root Team }
+    projects:
+      heavy-duty/incubator:
+        destination_uuid: dest-abc
+projects:
+  heavy-duty/incubator:
+    environments: [prod]
+github_apps: {}
+`,
+      });
+    expect(err).toThrow(
+      /environments\.staging\.projects\["heavy-duty\/incubator"\]/,
+    );
+    expect(err).toThrow(/registered for:\s+prod/);
+  });
+
+  it("refuses a binding for a project the registry does not carry at all", () => {
+    const err = () =>
+      loadBindings("environments.yaml", {
+        overrideText: `
+environments:
+  prod:
+    server: shared-box
+    team: { id: 0, name: Root Team }
+    projects:
+      acme/client-site:
+        destination_uuid: dest-client
+projects:
+  heavy-duty/incubator:
+    environments: [prod]
+github_apps: {}
+`,
+      });
+    expect(err).toThrow(/environments\.prod\.projects\["acme\/client-site"\]/);
+    expect(err).toThrow(/registry has:\s+heavy-duty\/incubator/);
+  });
+
+  // A legacy bare-<repo> binding key (projectBindingFor still resolves one) under
+  // a slug-keyed registry is drift with an obvious fix — say which fix.
+  it("tells a legacy bare-<repo> binding key which slug to rename to", () => {
+    const err = () =>
+      loadBindings("environments.yaml", {
+        overrideText: `
+environments:
+  prod:
+    server: shared-box
+    team: { id: 0, name: Root Team }
+    projects:
+      incubator:
+        smoke_target: core
+projects:
+  heavy-duty/incubator:
+    environments: [prod]
+github_apps: {}
+`,
+      });
+    expect(err).toThrow(/registry has:\s+projects\["heavy-duty\/incubator"\]/);
+    expect(err).toThrow(/legacy bare-<repo> key/);
+  });
+
+  // No fallback here, unlike github_apps: this block is new, so it has no state
+  // files in the wild to keep working, and a bare <repo> is unique only within an
+  // org — which is exactly why it is not a key.
+  it("refuses a bare <repo> registry key — the org is not optional", () => {
+    const err = () =>
+      loadBindings("environments.yaml", {
+        overrideText: `${twoEnvs}
+projects:
+  incubator:
+    environments: [prod]
+github_apps: {}
+`,
+      });
+    expect(err).toThrow(/projects\["incubator"\] is not a repo/);
+    expect(err).toThrow(/full <org>\/<repo> slug/);
+  });
+
+  // A project registered into nothing is a line of YAML that reads like a
+  // registration and is skipped by every fleet run.
+  it("refuses a project registered into no environment", () => {
+    expect(() =>
+      loadBindings("environments.yaml", {
+        overrideText: `${twoEnvs}
+projects:
+  heavy-duty/incubator:
+    environments: []
+github_apps: {}
+`,
+      }),
+    ).toThrow(/invalid bindings/);
+  });
+
+  it("rejects an unknown key inside a registry entry", () => {
+    expect(() =>
+      loadBindings("environments.yaml", {
+        overrideText: `${twoEnvs}
+projects:
+  heavy-duty/incubator:
+    environments: [prod]
+    repo: heavy-duty/incubator
+github_apps: {}
+`,
+      }),
+    ).toThrow(/invalid bindings/);
+  });
+
+  // Back-compat: the registry is optional, and every state file written before it
+  // existed has no `projects:` block. Such a file loads unchanged — including its
+  // per-environment bindings, which are NOT checked against a registry that is
+  // not there.
+  describe("with no registry at all", () => {
+    const b = loadBindings("environments.yaml", {
+      overrideText: `
+environments:
+  prod:
+    server: shared-box
+    team: { id: 0, name: Root Team }
+    projects:
+      heavy-duty/incubator:
+        destination_uuid: dest-abc
+        smoke_target: core
+github_apps: {}
+`,
+    });
+
+    it("loads, and keeps its per-environment bindings working", () => {
+      expect(b.projects).toBe(undefined);
+      expect(
+        projectBindingFor(b, "prod", "heavy-duty/incubator")?.destination_uuid,
+      ).toBe("dest-abc");
+    });
+
+    it("has no projects registered in any environment", () => {
+      expect(projectsIn(b, "prod")).toEqual([]);
+    });
+  });
+});