cast/src/envtemplate.ts
claude-hdb 6f0b0f6fc1 feat(resolve): derive base-URL env vars from manifest domains via ${domain:...} (#66)
A public base URL an app reads (LANDING_BASE_URL, ADMIN_WEB_BASE_URL) is a
fact the manifest already states in `domains`/`service_domains` — the same
fields cast parses to reconcile Coolify domains. Hand-transcribing it into an
env template is a second copy that drifts (incubator's prod LANDING_BASE_URL
silently kept a pre-apex host). So a template can now say it directly:

    LANDING_BASE_URL=${domain:landing}
    ADMIN_WEB_BASE_URL=${domain:core.admin}

- ${domain:<app>}            -> applications.<app>.domains[0]
- ${domain:<app>.<service>}  -> applications.<app>.service_domains.<service>[0]

Symmetric with ${resource:...} (#60) — parse -> sentinel -> validate -> fill —
but a domain is PURE MANIFEST DATA, known at plan time, so it resolves fully in
desiredFromManifest against a map built from the manifest: no live read, no
executor deferral, no unresolved-at-write path. Domains are PUBLIC, so they
resolve to secret:false (printed in diffs) and read as plain literals
downstream (no diff.ts change). Not secrets: excluded from templateRefs, never
captured. assertDomainRefs is the single validation gate (apply/diff/capture),
refusing an undeclared app/service, a wrong-shape ref, or an empty/blank domain
list before the sentinel can escape. Applications only (Coolify 4.1.2 can't set
service domains). REPORTING_TZ-style operator literals stay literal.

Closes #66.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-15 12:02:33 +00:00

346 lines
15 KiB
TypeScript

// A ${resource:<name>.url} reference — the internal URL of a database THIS
// manifest declares, read back from the resource Coolify created. It is not a
// secret: a secret is something a human authored and the age store holds, while
// this is a FACT about a resource cast itself made, readable from the API that
// made it, at any time, for free (#60). So it never enters the store, never
// enters a terminal, and needs no age key to read back — it is resolved from the
// live box, not decrypted.
export type ResourceRef = { resource: string; attr: string };
// A ${domain:<app>[.<service>]} reference — a public domain THIS manifest
// already declares (`applications.<app>.domains[0]`, or a compose app's
// `service_domains.<service>[0]`). Symmetric with ${resource:…} but simpler in
// the one way that matters: a resource URL is a fact read back from the live box
// (#60), while a domain is PURE MANIFEST DATA, known the instant the manifest is
// parsed. So it needs no live read, no age key, no deferral — it resolves at
// plan time, from the same manifest that declared it. It is also PUBLIC, not a
// secret: it prints in a diff like any literal (unlike the secret-flagged
// resource URL).
export type DomainRef = { app: string; service?: string };
// The value a derived var carries until it is resolved against the live
// resource. NEVER a legal thing to WRITE — the executor refuses it, the same
// fail-closed shape as the generated-secret placeholder — because a database URL
// that resolved to nothing boots every consumer pointed at nothing (the exact
// trap fetchGeneratedSources guards with its `never a fill of ""` rule).
//
// Collision-safe by construction, not by exotic bytes: this string only ever
// occupies a `derived` var's value, and such a value is only ever this sentinel
// or a real `postgres://…`/`redis://…` URL that fillDerivedEnv put there — so it
// cannot be mistaken for a resolved value, and `unresolvedDerived` gates on the
// `derived` flag anyway. (A plain string, no NUL byte: a NUL makes git treat the
// whole source file as binary and its diff unreviewable.)
export const DERIVED_UNRESOLVED = "cast:unresolved-derived-resource-url";
// The value a domain var carries between resolveTemplate and fillDomainEnv.
// Unlike DERIVED_UNRESOLVED, which is a legitimate transient state a derived var
// can be diffed and applied in (the from-nothing case, resolved later by the
// executor), this sentinel must NEVER survive validation: a domain is manifest
// data, so a ref that does not resolve is a ref that names an app/service the
// manifest does not declare, and `assertDomainRefs` throws on it before the
// desired set is ever returned. It exists for one reason only — so
// resolveTemplate need not know the manifest (the domain map is built later, in
// resolve.ts) — and fillDomainEnv always replaces it in the same plan.
// (A plain string, no NUL byte: a NUL makes git treat the source file as binary
// and its diff unreviewable — same reasoning as DERIVED_UNRESOLVED.)
export const DOMAIN_UNRESOLVED = "cast:unresolved-domain-ref";
// `secret` is true for both a ${SECRET} and a resolved ${resource:…} — both are
// values that must never be printed. `derived` is set (and stays set after
// resolution) so the diff can say "derived from database X" rather than mistaking
// a routine URL change for a secret rotation, and so the executor knows which
// vars to resolve from the live resource before it writes.
export type ResolvedEnv = {
vars: Record<
string,
{
value: string;
secret: boolean;
derived?: ResourceRef;
domain?: DomainRef;
}
>;
};
// A template line, parsed but not resolved: `ref` is set when the whole RHS is a
// single ${NAME} placeholder (a store secret); `resourceRef` when it is a single
// ${resource:<name>.attr} placeholder (a derived value); `domainRef` when it is
// a single ${domain:<app>[.<service>]} placeholder (a declared public domain).
// The three are mutually exclusive — a secret name is UPPER_SNAKE, a resource
// ref starts `resource:`, a domain ref starts `domain:`.
export type TemplateVar = {
key: string;
rhs: string;
ref?: string;
resourceRef?: ResourceRef;
domainRef?: DomainRef;
};
// ONE grammar, shared by both readers of a template — resolveTemplate (which
// needs the values) and templateRefs (which needs only the names). Keeping
// them on separate parsers would let the two drift, and a drift here is not
// cosmetic: `capture` would collect a different set of names than `apply` will
// later demand, which is precisely the "a name silently missed" failure the
// capture verb exists to remove.
function parseTemplate(text: string): TemplateVar[] {
const vars: TemplateVar[] = [];
const lines = text.split("\n");
for (let i = 0; i < lines.length; i++) {
const line = lines[i].trim();
if (line === "" || line.startsWith("#")) continue;
const m = line.match(/^([A-Z][A-Z0-9_]*)=(.*)$/);
if (!m)
throw new Error(
`env template line ${i + 1}: expected KEY=value, got "${line}"`,
);
const [, key, rhs] = m;
const placeholder = rhs.match(/^\$\{([A-Z][A-Z0-9_]*)\}$/);
// A resource ref names a Coolify resource (lower-kebab/snake, as the manifest
// and Coolify spell it) and one dotted attribute. Only `.url` is derivable
// today; an unknown attr is not rejected HERE — parsing stays dumb — but at
// validation time (resolve.ts), where the message can name the declared
// resources, exactly as an undeclared-resource ref is.
const resource = rhs.match(
/^\$\{resource:([a-z0-9][a-z0-9_-]*)\.([a-z_]+)\}$/,
);
// A domain ref names a manifest application (lower-kebab/snake, as the
// manifest spells it) and, optionally, one of that app's compose services.
// Parsing stays dumb — an unknown app or service is not rejected HERE but at
// validation time (resolve.ts), where the message can name the applications
// and services the manifest actually declares, exactly as an undeclared
// resource ref is.
const domain = rhs.match(
/^\$\{domain:([a-z0-9][a-z0-9_-]*)(?:\.([a-z0-9][a-z0-9_-]*))?\}$/,
);
vars.push({
key,
rhs,
...(placeholder ? { ref: placeholder[1] } : {}),
...(resource
? { resourceRef: { resource: resource[1], attr: resource[2] } }
: {}),
...(domain
? {
domainRef: {
app: domain[1],
...(domain[2] ? { service: domain[2] } : {}),
},
}
: {}),
});
}
return vars;
}
export function resolveTemplate(
text: string,
secrets: Record<string, string>,
): ResolvedEnv {
const vars: ResolvedEnv["vars"] = {};
for (const { key, rhs, ref, resourceRef, domainRef } of parseTemplate(text)) {
// A derived value is resolved from the live resource, not the store, and not
// here — the resource may not exist yet (a from-nothing apply creates it in
// the same run). So it lands UNRESOLVED, carrying the ref; fillDerivedEnv
// fills it once a URL map is known, and the executor refuses to write one
// that never resolved. This is BEFORE the ref/store branch on purpose: a
// resource ref's RHS is not UPPER_SNAKE, so it would otherwise be mistaken
// for a literal and written through as the text "${resource:…}".
if (resourceRef) {
vars[key] = {
value: DERIVED_UNRESOLVED,
secret: true,
derived: resourceRef,
};
continue;
}
// A domain is manifest data, and resolveTemplate is deliberately
// manifest-unaware (it takes secrets, not a domains map — the map is built
// later, in resolve.ts). So the var lands carrying the transient sentinel and
// its ref; fillDomainEnv resolves it against the domain map, and
// assertDomainRefs throws before an unresolved one can escape the plan. Like
// resourceRef, this is BEFORE the ref/literal branch on purpose: a domain
// ref's RHS is not UPPER_SNAKE, so it would otherwise be written through as
// the literal text "${domain:…}". A domain is PUBLIC — secret: false.
if (domainRef) {
vars[key] = {
value: DOMAIN_UNRESOLVED,
secret: false,
domain: domainRef,
};
continue;
}
if (ref === undefined) {
vars[key] = { value: rhs, secret: false };
continue;
}
const value = secrets[ref];
if (value === undefined) {
throw new Error(`secret ${ref} (for ${key}) missing from the age store`);
}
vars[key] = { value, secret: true };
}
return { vars };
}
// Resolve every derived var whose resource now has a known URL, leaving the rest
// unresolved (a from-nothing apply has no live URL to resolve against until the
// database is created — the executor fills those, post-create). Pure, and the
// single place a derived value is turned into a real one, so the diff-time fill
// (against pre-existing live) and the apply-time fill (against a just-created
// resource) can never resolve the same ref two different ways.
//
// `urls` is keyed by the resource's MANIFEST name — the same name the ref
// carries. The caller keys it that way (see fillDesiredDerived / the executor):
// aliasing has already renamed live resources to the manifest's vocabulary by
// the time a URL map is built.
export function fillDerivedEnv(
env: ResolvedEnv,
urls: Record<string, string>,
): ResolvedEnv {
const vars: ResolvedEnv["vars"] = {};
for (const [key, v] of Object.entries(env.vars)) {
const url =
v.derived && v.derived.attr === "url"
? urls[v.derived.resource]
: undefined;
vars[key] =
url !== undefined && url !== ""
? { value: url, secret: true, derived: v.derived }
: v;
}
return { vars };
}
// Resolve every domain var against a map of the domains the manifest declares,
// keyed `<app>` or `<app>.<service>` (built in resolve.ts). Pure, and mirrors
// fillDerivedEnv — but where a derived var may legitimately stay unresolved (a
// from-nothing apply has no live database yet), a domain ALWAYS resolves here:
// the map is manifest data, and assertDomainRefs has already thrown on any ref
// that would miss it. So a hit becomes a plain `{ value, secret: false }` and
// DROPS the `domain` marker — a resolved domain is indistinguishable from a
// literal downstream, which is why the diff needs no domain-awareness at all. A
// miss (only reachable if validation was skipped) is left untouched, sentinel
// and marker intact, so nothing silently writes the sentinel as a value.
export function fillDomainEnv(
env: ResolvedEnv,
domains: Record<string, string>,
): ResolvedEnv {
const vars: ResolvedEnv["vars"] = {};
for (const [key, v] of Object.entries(env.vars)) {
const domain = v.domain
? domains[
v.domain.service
? `${v.domain.app}.${v.domain.service}`
: v.domain.app
]
: undefined;
vars[key] =
domain !== undefined && domain !== ""
? { value: domain, secret: false }
: v;
}
return { vars };
}
// The derived vars still holding the unresolved sentinel — what the executor
// must resolve from the live box before it can write this env, and what it
// refuses on if it cannot. Names the env key and the resource it derives from;
// never a value (there is none yet, and there never will be one to print).
export function unresolvedDerived(
env: ResolvedEnv,
): Array<{ key: string; resource: string }> {
return Object.entries(env.vars).flatMap(([key, v]) =>
v.derived && v.value === DERIVED_UNRESOLVED
? [{ key, resource: v.derived.resource }]
: [],
);
}
// The ${NAME} refs a template declares: the secret names the manifest requires,
// paired with the env var each one lands on. `capture` reads these to learn
// what to go and fetch — at capture time there is no store to resolve against
// yet, which is the whole point of the verb.
export function templateRefs(
text: string,
): Array<{ key: string; ref: string }> {
return parseTemplate(text).flatMap(({ key, ref }) =>
ref === undefined ? [] : [{ key, ref }],
);
}
// The ${resource:<name>.attr} refs a template declares — the derived edges. NOT
// returned by templateRefs above, and that separation is the point: a derived
// value is not a secret to be captured, so `capture` must never go looking for a
// store name called `resource:postgres.url`. resolve.ts reads these to validate
// each edge against the databases the manifest actually declares.
export function templateResourceRefs(
text: string,
): Array<{ key: string; resource: string; attr: string }> {
return parseTemplate(text).flatMap(({ key, resourceRef }) =>
resourceRef === undefined
? []
: [{ key, resource: resourceRef.resource, attr: resourceRef.attr }],
);
}
// The ${domain:<app>[.<service>]} refs a template declares — the declared-domain
// edges. NOT returned by templateRefs (a domain is not a secret to capture) and
// NOT part of the required-secret set, exactly like templateResourceRefs.
// resolve.ts reads these to validate each ref against the applications and
// service_domains the manifest actually declares.
export function templateDomainRefs(
text: string,
): Array<{ key: string; app: string; service?: string }> {
return parseTemplate(text).flatMap(({ key, domainRef }) =>
domainRef === undefined
? []
: [
{
key,
app: domainRef.app,
...(domainRef.service ? { service: domainRef.service } : {}),
},
],
);
}
// Every env var key a template declares — refs and literals alike. `capture`
// only cares about the ${...} refs (the secrets); `inventory` needs all of them,
// because the question it answers is "what does the manifest put on this
// resource, and what does the box actually have?", and a literal the manifest
// sets (a feature flag, NODE_ENV) is just as much a difference as a secret.
//
// Same parser as everything else in this file — see parseTemplate.
export function templateKeys(text: string): string[] {
return parseTemplate(text).map(({ key }) => key);
}
// An environment may forbid variables by name pattern, declared as
// `environments.<env>.forbidden_var_patterns` in the state repo. The rule is
// PRESENCE, not value: a forbidden var set to "false" still refuses the apply,
// because a var that exists can be flipped on later in the Coolify UI without
// touching a manifest — "off" has to mean absent.
//
// The policy lives in the operator's private state, never in a product's
// manifest: a product-side change must not be able to lower its own guard.
export function assertEnvVarPolicy(
envName: string,
resolved: Record<string, ResolvedEnv>,
forbiddenPatterns: string[] | undefined,
): void {
if (!forbiddenPatterns?.length) return;
const patterns = forbiddenPatterns.map((p) => ({
src: p,
re: new RegExp(p),
}));
for (const [resource, env] of Object.entries(resolved)) {
for (const key of Object.keys(env.vars)) {
const hit = patterns.find((p) => p.re.test(key));
if (hit) {
throw new Error(
`refusing ${envName} apply: ${key} is present on ${resource} regardless of value — forbidden by forbidden_var_patterns /${hit.src}/ ("off" means absent, not false)`,
);
}
}
}
}