A public base URL an app reads (LANDING_BASE_URL, ADMIN_WEB_BASE_URL) is a
fact the manifest already states in `domains`/`service_domains` — the same
fields cast parses to reconcile Coolify domains. Hand-transcribing it into an
env template is a second copy that drifts (incubator's prod LANDING_BASE_URL
silently kept a pre-apex host). So a template can now say it directly:
LANDING_BASE_URL=${domain:landing}
ADMIN_WEB_BASE_URL=${domain:core.admin}
- ${domain:<app>} -> applications.<app>.domains[0]
- ${domain:<app>.<service>} -> applications.<app>.service_domains.<service>[0]
Symmetric with ${resource:...} (#60) — parse -> sentinel -> validate -> fill —
but a domain is PURE MANIFEST DATA, known at plan time, so it resolves fully in
desiredFromManifest against a map built from the manifest: no live read, no
executor deferral, no unresolved-at-write path. Domains are PUBLIC, so they
resolve to secret:false (printed in diffs) and read as plain literals
downstream (no diff.ts change). Not secrets: excluded from templateRefs, never
captured. assertDomainRefs is the single validation gate (apply/diff/capture),
refusing an undeclared app/service, a wrong-shape ref, or an empty/blank domain
list before the sentinel can escape. Applications only (Coolify 4.1.2 can't set
service domains). REPORTING_TZ-style operator literals stay literal.
Closes #66.
Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
346 lines
15 KiB
TypeScript
346 lines
15 KiB
TypeScript
// A ${resource:<name>.url} reference — the internal URL of a database THIS
|
|
// manifest declares, read back from the resource Coolify created. It is not a
|
|
// secret: a secret is something a human authored and the age store holds, while
|
|
// this is a FACT about a resource cast itself made, readable from the API that
|
|
// made it, at any time, for free (#60). So it never enters the store, never
|
|
// enters a terminal, and needs no age key to read back — it is resolved from the
|
|
// live box, not decrypted.
|
|
export type ResourceRef = { resource: string; attr: string };
|
|
|
|
// A ${domain:<app>[.<service>]} reference — a public domain THIS manifest
|
|
// already declares (`applications.<app>.domains[0]`, or a compose app's
|
|
// `service_domains.<service>[0]`). Symmetric with ${resource:…} but simpler in
|
|
// the one way that matters: a resource URL is a fact read back from the live box
|
|
// (#60), while a domain is PURE MANIFEST DATA, known the instant the manifest is
|
|
// parsed. So it needs no live read, no age key, no deferral — it resolves at
|
|
// plan time, from the same manifest that declared it. It is also PUBLIC, not a
|
|
// secret: it prints in a diff like any literal (unlike the secret-flagged
|
|
// resource URL).
|
|
export type DomainRef = { app: string; service?: string };
|
|
|
|
// The value a derived var carries until it is resolved against the live
|
|
// resource. NEVER a legal thing to WRITE — the executor refuses it, the same
|
|
// fail-closed shape as the generated-secret placeholder — because a database URL
|
|
// that resolved to nothing boots every consumer pointed at nothing (the exact
|
|
// trap fetchGeneratedSources guards with its `never a fill of ""` rule).
|
|
//
|
|
// Collision-safe by construction, not by exotic bytes: this string only ever
|
|
// occupies a `derived` var's value, and such a value is only ever this sentinel
|
|
// or a real `postgres://…`/`redis://…` URL that fillDerivedEnv put there — so it
|
|
// cannot be mistaken for a resolved value, and `unresolvedDerived` gates on the
|
|
// `derived` flag anyway. (A plain string, no NUL byte: a NUL makes git treat the
|
|
// whole source file as binary and its diff unreviewable.)
|
|
export const DERIVED_UNRESOLVED = "cast:unresolved-derived-resource-url";
|
|
|
|
// The value a domain var carries between resolveTemplate and fillDomainEnv.
|
|
// Unlike DERIVED_UNRESOLVED, which is a legitimate transient state a derived var
|
|
// can be diffed and applied in (the from-nothing case, resolved later by the
|
|
// executor), this sentinel must NEVER survive validation: a domain is manifest
|
|
// data, so a ref that does not resolve is a ref that names an app/service the
|
|
// manifest does not declare, and `assertDomainRefs` throws on it before the
|
|
// desired set is ever returned. It exists for one reason only — so
|
|
// resolveTemplate need not know the manifest (the domain map is built later, in
|
|
// resolve.ts) — and fillDomainEnv always replaces it in the same plan.
|
|
// (A plain string, no NUL byte: a NUL makes git treat the source file as binary
|
|
// and its diff unreviewable — same reasoning as DERIVED_UNRESOLVED.)
|
|
export const DOMAIN_UNRESOLVED = "cast:unresolved-domain-ref";
|
|
|
|
// `secret` is true for both a ${SECRET} and a resolved ${resource:…} — both are
|
|
// values that must never be printed. `derived` is set (and stays set after
|
|
// resolution) so the diff can say "derived from database X" rather than mistaking
|
|
// a routine URL change for a secret rotation, and so the executor knows which
|
|
// vars to resolve from the live resource before it writes.
|
|
export type ResolvedEnv = {
|
|
vars: Record<
|
|
string,
|
|
{
|
|
value: string;
|
|
secret: boolean;
|
|
derived?: ResourceRef;
|
|
domain?: DomainRef;
|
|
}
|
|
>;
|
|
};
|
|
|
|
// A template line, parsed but not resolved: `ref` is set when the whole RHS is a
|
|
// single ${NAME} placeholder (a store secret); `resourceRef` when it is a single
|
|
// ${resource:<name>.attr} placeholder (a derived value); `domainRef` when it is
|
|
// a single ${domain:<app>[.<service>]} placeholder (a declared public domain).
|
|
// The three are mutually exclusive — a secret name is UPPER_SNAKE, a resource
|
|
// ref starts `resource:`, a domain ref starts `domain:`.
|
|
export type TemplateVar = {
|
|
key: string;
|
|
rhs: string;
|
|
ref?: string;
|
|
resourceRef?: ResourceRef;
|
|
domainRef?: DomainRef;
|
|
};
|
|
|
|
// ONE grammar, shared by both readers of a template — resolveTemplate (which
|
|
// needs the values) and templateRefs (which needs only the names). Keeping
|
|
// them on separate parsers would let the two drift, and a drift here is not
|
|
// cosmetic: `capture` would collect a different set of names than `apply` will
|
|
// later demand, which is precisely the "a name silently missed" failure the
|
|
// capture verb exists to remove.
|
|
function parseTemplate(text: string): TemplateVar[] {
|
|
const vars: TemplateVar[] = [];
|
|
const lines = text.split("\n");
|
|
for (let i = 0; i < lines.length; i++) {
|
|
const line = lines[i].trim();
|
|
if (line === "" || line.startsWith("#")) continue;
|
|
const m = line.match(/^([A-Z][A-Z0-9_]*)=(.*)$/);
|
|
if (!m)
|
|
throw new Error(
|
|
`env template line ${i + 1}: expected KEY=value, got "${line}"`,
|
|
);
|
|
const [, key, rhs] = m;
|
|
const placeholder = rhs.match(/^\$\{([A-Z][A-Z0-9_]*)\}$/);
|
|
// A resource ref names a Coolify resource (lower-kebab/snake, as the manifest
|
|
// and Coolify spell it) and one dotted attribute. Only `.url` is derivable
|
|
// today; an unknown attr is not rejected HERE — parsing stays dumb — but at
|
|
// validation time (resolve.ts), where the message can name the declared
|
|
// resources, exactly as an undeclared-resource ref is.
|
|
const resource = rhs.match(
|
|
/^\$\{resource:([a-z0-9][a-z0-9_-]*)\.([a-z_]+)\}$/,
|
|
);
|
|
// A domain ref names a manifest application (lower-kebab/snake, as the
|
|
// manifest spells it) and, optionally, one of that app's compose services.
|
|
// Parsing stays dumb — an unknown app or service is not rejected HERE but at
|
|
// validation time (resolve.ts), where the message can name the applications
|
|
// and services the manifest actually declares, exactly as an undeclared
|
|
// resource ref is.
|
|
const domain = rhs.match(
|
|
/^\$\{domain:([a-z0-9][a-z0-9_-]*)(?:\.([a-z0-9][a-z0-9_-]*))?\}$/,
|
|
);
|
|
vars.push({
|
|
key,
|
|
rhs,
|
|
...(placeholder ? { ref: placeholder[1] } : {}),
|
|
...(resource
|
|
? { resourceRef: { resource: resource[1], attr: resource[2] } }
|
|
: {}),
|
|
...(domain
|
|
? {
|
|
domainRef: {
|
|
app: domain[1],
|
|
...(domain[2] ? { service: domain[2] } : {}),
|
|
},
|
|
}
|
|
: {}),
|
|
});
|
|
}
|
|
return vars;
|
|
}
|
|
|
|
export function resolveTemplate(
|
|
text: string,
|
|
secrets: Record<string, string>,
|
|
): ResolvedEnv {
|
|
const vars: ResolvedEnv["vars"] = {};
|
|
for (const { key, rhs, ref, resourceRef, domainRef } of parseTemplate(text)) {
|
|
// A derived value is resolved from the live resource, not the store, and not
|
|
// here — the resource may not exist yet (a from-nothing apply creates it in
|
|
// the same run). So it lands UNRESOLVED, carrying the ref; fillDerivedEnv
|
|
// fills it once a URL map is known, and the executor refuses to write one
|
|
// that never resolved. This is BEFORE the ref/store branch on purpose: a
|
|
// resource ref's RHS is not UPPER_SNAKE, so it would otherwise be mistaken
|
|
// for a literal and written through as the text "${resource:…}".
|
|
if (resourceRef) {
|
|
vars[key] = {
|
|
value: DERIVED_UNRESOLVED,
|
|
secret: true,
|
|
derived: resourceRef,
|
|
};
|
|
continue;
|
|
}
|
|
// A domain is manifest data, and resolveTemplate is deliberately
|
|
// manifest-unaware (it takes secrets, not a domains map — the map is built
|
|
// later, in resolve.ts). So the var lands carrying the transient sentinel and
|
|
// its ref; fillDomainEnv resolves it against the domain map, and
|
|
// assertDomainRefs throws before an unresolved one can escape the plan. Like
|
|
// resourceRef, this is BEFORE the ref/literal branch on purpose: a domain
|
|
// ref's RHS is not UPPER_SNAKE, so it would otherwise be written through as
|
|
// the literal text "${domain:…}". A domain is PUBLIC — secret: false.
|
|
if (domainRef) {
|
|
vars[key] = {
|
|
value: DOMAIN_UNRESOLVED,
|
|
secret: false,
|
|
domain: domainRef,
|
|
};
|
|
continue;
|
|
}
|
|
if (ref === undefined) {
|
|
vars[key] = { value: rhs, secret: false };
|
|
continue;
|
|
}
|
|
const value = secrets[ref];
|
|
if (value === undefined) {
|
|
throw new Error(`secret ${ref} (for ${key}) missing from the age store`);
|
|
}
|
|
vars[key] = { value, secret: true };
|
|
}
|
|
return { vars };
|
|
}
|
|
|
|
// Resolve every derived var whose resource now has a known URL, leaving the rest
|
|
// unresolved (a from-nothing apply has no live URL to resolve against until the
|
|
// database is created — the executor fills those, post-create). Pure, and the
|
|
// single place a derived value is turned into a real one, so the diff-time fill
|
|
// (against pre-existing live) and the apply-time fill (against a just-created
|
|
// resource) can never resolve the same ref two different ways.
|
|
//
|
|
// `urls` is keyed by the resource's MANIFEST name — the same name the ref
|
|
// carries. The caller keys it that way (see fillDesiredDerived / the executor):
|
|
// aliasing has already renamed live resources to the manifest's vocabulary by
|
|
// the time a URL map is built.
|
|
export function fillDerivedEnv(
|
|
env: ResolvedEnv,
|
|
urls: Record<string, string>,
|
|
): ResolvedEnv {
|
|
const vars: ResolvedEnv["vars"] = {};
|
|
for (const [key, v] of Object.entries(env.vars)) {
|
|
const url =
|
|
v.derived && v.derived.attr === "url"
|
|
? urls[v.derived.resource]
|
|
: undefined;
|
|
vars[key] =
|
|
url !== undefined && url !== ""
|
|
? { value: url, secret: true, derived: v.derived }
|
|
: v;
|
|
}
|
|
return { vars };
|
|
}
|
|
|
|
// Resolve every domain var against a map of the domains the manifest declares,
|
|
// keyed `<app>` or `<app>.<service>` (built in resolve.ts). Pure, and mirrors
|
|
// fillDerivedEnv — but where a derived var may legitimately stay unresolved (a
|
|
// from-nothing apply has no live database yet), a domain ALWAYS resolves here:
|
|
// the map is manifest data, and assertDomainRefs has already thrown on any ref
|
|
// that would miss it. So a hit becomes a plain `{ value, secret: false }` and
|
|
// DROPS the `domain` marker — a resolved domain is indistinguishable from a
|
|
// literal downstream, which is why the diff needs no domain-awareness at all. A
|
|
// miss (only reachable if validation was skipped) is left untouched, sentinel
|
|
// and marker intact, so nothing silently writes the sentinel as a value.
|
|
export function fillDomainEnv(
|
|
env: ResolvedEnv,
|
|
domains: Record<string, string>,
|
|
): ResolvedEnv {
|
|
const vars: ResolvedEnv["vars"] = {};
|
|
for (const [key, v] of Object.entries(env.vars)) {
|
|
const domain = v.domain
|
|
? domains[
|
|
v.domain.service
|
|
? `${v.domain.app}.${v.domain.service}`
|
|
: v.domain.app
|
|
]
|
|
: undefined;
|
|
vars[key] =
|
|
domain !== undefined && domain !== ""
|
|
? { value: domain, secret: false }
|
|
: v;
|
|
}
|
|
return { vars };
|
|
}
|
|
|
|
// The derived vars still holding the unresolved sentinel — what the executor
|
|
// must resolve from the live box before it can write this env, and what it
|
|
// refuses on if it cannot. Names the env key and the resource it derives from;
|
|
// never a value (there is none yet, and there never will be one to print).
|
|
export function unresolvedDerived(
|
|
env: ResolvedEnv,
|
|
): Array<{ key: string; resource: string }> {
|
|
return Object.entries(env.vars).flatMap(([key, v]) =>
|
|
v.derived && v.value === DERIVED_UNRESOLVED
|
|
? [{ key, resource: v.derived.resource }]
|
|
: [],
|
|
);
|
|
}
|
|
|
|
// The ${NAME} refs a template declares: the secret names the manifest requires,
|
|
// paired with the env var each one lands on. `capture` reads these to learn
|
|
// what to go and fetch — at capture time there is no store to resolve against
|
|
// yet, which is the whole point of the verb.
|
|
export function templateRefs(
|
|
text: string,
|
|
): Array<{ key: string; ref: string }> {
|
|
return parseTemplate(text).flatMap(({ key, ref }) =>
|
|
ref === undefined ? [] : [{ key, ref }],
|
|
);
|
|
}
|
|
|
|
// The ${resource:<name>.attr} refs a template declares — the derived edges. NOT
|
|
// returned by templateRefs above, and that separation is the point: a derived
|
|
// value is not a secret to be captured, so `capture` must never go looking for a
|
|
// store name called `resource:postgres.url`. resolve.ts reads these to validate
|
|
// each edge against the databases the manifest actually declares.
|
|
export function templateResourceRefs(
|
|
text: string,
|
|
): Array<{ key: string; resource: string; attr: string }> {
|
|
return parseTemplate(text).flatMap(({ key, resourceRef }) =>
|
|
resourceRef === undefined
|
|
? []
|
|
: [{ key, resource: resourceRef.resource, attr: resourceRef.attr }],
|
|
);
|
|
}
|
|
|
|
// The ${domain:<app>[.<service>]} refs a template declares — the declared-domain
|
|
// edges. NOT returned by templateRefs (a domain is not a secret to capture) and
|
|
// NOT part of the required-secret set, exactly like templateResourceRefs.
|
|
// resolve.ts reads these to validate each ref against the applications and
|
|
// service_domains the manifest actually declares.
|
|
export function templateDomainRefs(
|
|
text: string,
|
|
): Array<{ key: string; app: string; service?: string }> {
|
|
return parseTemplate(text).flatMap(({ key, domainRef }) =>
|
|
domainRef === undefined
|
|
? []
|
|
: [
|
|
{
|
|
key,
|
|
app: domainRef.app,
|
|
...(domainRef.service ? { service: domainRef.service } : {}),
|
|
},
|
|
],
|
|
);
|
|
}
|
|
|
|
// Every env var key a template declares — refs and literals alike. `capture`
|
|
// only cares about the ${...} refs (the secrets); `inventory` needs all of them,
|
|
// because the question it answers is "what does the manifest put on this
|
|
// resource, and what does the box actually have?", and a literal the manifest
|
|
// sets (a feature flag, NODE_ENV) is just as much a difference as a secret.
|
|
//
|
|
// Same parser as everything else in this file — see parseTemplate.
|
|
export function templateKeys(text: string): string[] {
|
|
return parseTemplate(text).map(({ key }) => key);
|
|
}
|
|
|
|
// An environment may forbid variables by name pattern, declared as
|
|
// `environments.<env>.forbidden_var_patterns` in the state repo. The rule is
|
|
// PRESENCE, not value: a forbidden var set to "false" still refuses the apply,
|
|
// because a var that exists can be flipped on later in the Coolify UI without
|
|
// touching a manifest — "off" has to mean absent.
|
|
//
|
|
// The policy lives in the operator's private state, never in a product's
|
|
// manifest: a product-side change must not be able to lower its own guard.
|
|
export function assertEnvVarPolicy(
|
|
envName: string,
|
|
resolved: Record<string, ResolvedEnv>,
|
|
forbiddenPatterns: string[] | undefined,
|
|
): void {
|
|
if (!forbiddenPatterns?.length) return;
|
|
const patterns = forbiddenPatterns.map((p) => ({
|
|
src: p,
|
|
re: new RegExp(p),
|
|
}));
|
|
for (const [resource, env] of Object.entries(resolved)) {
|
|
for (const key of Object.keys(env.vars)) {
|
|
const hit = patterns.find((p) => p.re.test(key));
|
|
if (hit) {
|
|
throw new Error(
|
|
`refusing ${envName} apply: ${key} is present on ${resource} regardless of value — forbidden by forbidden_var_patterns /${hit.src}/ ("off" means absent, not false)`,
|
|
);
|
|
}
|
|
}
|
|
}
|
|
}
|