cast/src/resolve.ts

272 lines
9.7 KiB
TypeScript
Raw Normal View History

feat: cast — the Coolify executor, extracted from the infra state repo Public tool, private state. cast holds no hostnames, no bindings, no secrets: it joins a product repo's .infra/ manifest with a state directory you point it at, and makes Coolify match. Extracted from heavy-duty/infra, which was half tool and half state — the inconsistency that made it impossible to say whether "infra" named a CLI or a runbook. rig builds the boxes; cast fills them; infra is what they are filled with. Two changes were required to make it genuinely stateless and publishable: - The implicit cwd contract (environments.yaml / secrets/ / .coolify.env resolved against the working directory, silently reading the wrong file from the wrong place) is now an explicit --state <dir> / $CAST_STATE. - BANNED_IN_PROD — a hardcoded list of one product's ALLOW_* flags, the only product knowledge in the executor — becomes the generic, operator- owned environments.<env>.forbidden_var_patterns. The guard now lives in private state, so a product-side change cannot lower its own guard, and it is a pattern rather than a list, so it catches unforeseen siblings. Age identities resolve as $CAST_AGE_KEY_FILE_<ENV> then ~/.config/cast/age-<env>.key — which is the entire attended-vs-unattended apply mechanism, with no environment names known to the tool. Instance identity (org names, the GitHub App name, founder domains) is out of the fixtures and out of register-github-app.sh, which took APP_NAME and ORG as arguments rather than baking them in. 69 tests green; bin/cast + curl installer mirror rig's shape. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 12:25:44 +00:00
import { execFileSync } from "node:child_process";
import { existsSync, mkdtempSync, readFileSync } from "node:fs";
import { tmpdir } from "node:os";
import { join } from "node:path";
import type { Desired } from "./diff.js";
import { type ResolvedEnv, resolveTemplate } from "./envtemplate.js";
import { loadManifest } from "./manifest.js";
fix: authenticate clones via gh / token, never fall into git's prompt (#13) resolveCheckout shelled out to a bare `git clone` and relied entirely on the ambient credential helper. On a workstation with none configured, git falls through to its interactive username/password prompt — which GitHub no longer accepts — and the resulting error talks about *the repository* rather than about cast's missing credentials. Being logged into `gh` does not help: `gh auth login` alone does not wire git's helper (that is `gh auth setup-git`, a separate act most people never run). Not routable around for prod: resolveCheckout refuses --path with --env prod, so the clone is the only path and its auth is mandatory. cast now resolves credentials itself, in order: `gh` borrowed as a per-invocation credential helper (no mutation of the user's global git config), then GITHUB_TOKEN / GH_TOKEN, then the ambient helper. The token is never embedded in the clone URL or in http.extraheader — both leak it into `ps`, and the latter persists it into the clone's git config. The helper reads it from the environment at run time, so what lands in argv is the literal text `$CAST_GIT_TOKEN`, never its value. GIT_TERMINAL_PROMPT=0 on every path: whichever credential was used, git may never fall through to a prompt it cannot satisfy — it can only hang, or hide the real fault. When there were no credentials at all, cast now says so, and names the fix. Note that the empty `credential.helper=` reset clears URL-scoped helpers (`credential.https://github.com.helper`, what `gh auth setup-git` writes) as well as generic ones — verified against a live private clone, along with all three acceptance criteria. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 16:30:04 +00:00
// How cast authenticated (or failed to authenticate) a clone.
//
// gh — `gh` is installed and holds a token; borrowed as a credential
// helper for this invocation only
// token — GITHUB_TOKEN / GH_TOKEN in the environment (the CI path)
// ambient — neither; whatever git's own credential helper does, if anything
export type GitAuth = {
source: "gh" | "token" | "ambient";
configArgs: string[];
env: Record<string, string>;
};
// A credential helper reads the token from the ENVIRONMENT at run time. The
// alternatives both leak it: a token in the clone URL shows up in `ps` and in
// git's own error messages, and `http.extraheader` additionally persists into
// the clone's .git/config. What lands in argv here is the literal text
// `$CAST_GIT_TOKEN`, never its value.
const TOKEN_HELPER =
'!f() { test "$1" = get || exit 0; echo username=x-access-token; echo "password=$CAST_GIT_TOKEN"; }; f';
// `gh auth login` alone does NOT wire git's credential helper — that is
// `gh auth setup-git`, a separate act most people never run. So being logged
// into `gh` does not make `git clone` work, which is exactly the trap #13
// fell into. Borrowing gh as a helper for this one invocation closes that gap
// without mutating the operator's global git config.
const GH_HELPER = "!gh auth git-credential";
function ghHasToken(): boolean {
try {
// A local keyring/config read, not a network call. We never keep the
// value — the helper re-reads it inside git.
execFileSync("gh", ["auth", "token"], { stdio: "pipe" });
return true;
} catch {
return false;
}
}
// Resolve clone credentials INSIDE cast, in a fixed order, rather than leaving
// it to whatever the ambient git config happens to do. `credential.helper=`
// (empty) first RESETS the inherited helper list — otherwise a helper
// configured globally is consulted before ours and silently decides the
// outcome, which is the same "the connection target is implicit in a file's
// contents" problem #14 is about.
export function resolveGitAuth(
env: NodeJS.ProcessEnv = process.env,
hasGh: () => boolean = ghHasToken,
): GitAuth {
if (hasGh()) {
return {
source: "gh",
configArgs: [
"-c",
"credential.helper=",
"-c",
`credential.helper=${GH_HELPER}`,
],
env: {},
};
}
const token = env.GITHUB_TOKEN || env.GH_TOKEN;
if (token) {
return {
source: "token",
configArgs: [
"-c",
"credential.helper=",
"-c",
`credential.helper=${TOKEN_HELPER}`,
],
env: { CAST_GIT_TOKEN: token },
};
}
return { source: "ambient", configArgs: [], env: {} };
}
// GitHub answers "you cannot see this" with a 404, not a 403 — so a private
// repo you lack access to and a repo that does not exist are the same message
// on the wire. The failure text must not pick one; it has to name both, and
// name the credential cast actually used, or the operator debugs the wrong
// half. (The original bug reported *the repository* when the real fault was
// cast's missing credentials.)
export function cloneFailureMessage(
orgRepo: string,
auth: GitAuth,
stderr: string,
): string {
const detail = stderr.trim();
const tail = detail
? ["", "git said:", ...detail.split("\n").map((l) => ` ${l}`)]
: [];
if (auth.source === "ambient") {
return [
`cannot clone ${orgRepo}: no GitHub credentials.`,
"",
"cast looked for, in order:",
" 1. `gh` — not installed, or not logged in (`gh auth token` failed)",
" 2. GITHUB_TOKEN / GH_TOKEN — not set in the environment",
" 3. git's own credential helper — did not supply credentials either",
"",
"Run `gh auth login`, or set GITHUB_TOKEN. (`gh auth setup-git` also works,",
"but cast borrows `gh` as a credential helper on its own, so logging in is",
"enough — you do not need to change your global git config.)",
...tail,
].join("\n");
}
const used =
auth.source === "gh"
? "`gh` (borrowed as a credential helper for this clone)"
: "GITHUB_TOKEN / GH_TOKEN from the environment";
return [
`cannot clone ${orgRepo}: authenticated with ${used}, and GitHub still refused.`,
"",
"GitHub answers 'you cannot see this' with a 404, so this is one of:",
` - ${orgRepo} does not exist (check the slug)`,
" - it is private and this credential has no access to it",
" - the credential is expired, or lacks the `repo` scope",
...tail,
].join("\n");
}
feat: cast — the Coolify executor, extracted from the infra state repo Public tool, private state. cast holds no hostnames, no bindings, no secrets: it joins a product repo's .infra/ manifest with a state directory you point it at, and makes Coolify match. Extracted from heavy-duty/infra, which was half tool and half state — the inconsistency that made it impossible to say whether "infra" named a CLI or a runbook. rig builds the boxes; cast fills them; infra is what they are filled with. Two changes were required to make it genuinely stateless and publishable: - The implicit cwd contract (environments.yaml / secrets/ / .coolify.env resolved against the working directory, silently reading the wrong file from the wrong place) is now an explicit --state <dir> / $CAST_STATE. - BANNED_IN_PROD — a hardcoded list of one product's ALLOW_* flags, the only product knowledge in the executor — becomes the generic, operator- owned environments.<env>.forbidden_var_patterns. The guard now lives in private state, so a product-side change cannot lower its own guard, and it is a pattern rather than a list, so it catches unforeseen siblings. Age identities resolve as $CAST_AGE_KEY_FILE_<ENV> then ~/.config/cast/age-<env>.key — which is the entire attended-vs-unattended apply mechanism, with no environment names known to the tool. Instance identity (org names, the GitHub App name, founder domains) is out of the fixtures and out of register-github-app.sh, which took APP_NAME and ORG as arguments rather than baking them in. 69 tests green; bin/cast + curl installer mirror rig's shape. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 12:25:44 +00:00
export function resolveCheckout(
orgRepo: string,
opts: { env: string; path?: string },
): string {
if (opts.path && opts.env === "prod") {
throw new Error(
"apply refuses --path with --env prod: prod always reads the default branch",
);
}
if (opts.path) return opts.path;
const dir = mkdtempSync(join(tmpdir(), "infra-checkout-"));
fix: authenticate clones via gh / token, never fall into git's prompt (#13) resolveCheckout shelled out to a bare `git clone` and relied entirely on the ambient credential helper. On a workstation with none configured, git falls through to its interactive username/password prompt — which GitHub no longer accepts — and the resulting error talks about *the repository* rather than about cast's missing credentials. Being logged into `gh` does not help: `gh auth login` alone does not wire git's helper (that is `gh auth setup-git`, a separate act most people never run). Not routable around for prod: resolveCheckout refuses --path with --env prod, so the clone is the only path and its auth is mandatory. cast now resolves credentials itself, in order: `gh` borrowed as a per-invocation credential helper (no mutation of the user's global git config), then GITHUB_TOKEN / GH_TOKEN, then the ambient helper. The token is never embedded in the clone URL or in http.extraheader — both leak it into `ps`, and the latter persists it into the clone's git config. The helper reads it from the environment at run time, so what lands in argv is the literal text `$CAST_GIT_TOKEN`, never its value. GIT_TERMINAL_PROMPT=0 on every path: whichever credential was used, git may never fall through to a prompt it cannot satisfy — it can only hang, or hide the real fault. When there were no credentials at all, cast now says so, and names the fix. Note that the empty `credential.helper=` reset clears URL-scoped helpers (`credential.https://github.com.helper`, what `gh auth setup-git` writes) as well as generic ones — verified against a live private clone, along with all three acceptance criteria. Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 16:30:04 +00:00
const auth = resolveGitAuth();
try {
execFileSync(
"git",
[
...auth.configArgs,
"clone",
"--depth",
"1",
`https://github.com/${orgRepo}.git`,
dir,
],
{
stdio: "pipe",
env: {
...process.env,
...auth.env,
// Belt and braces: whatever credential path we took, git may NEVER
// fall through to its interactive username/password prompt. GitHub
// stopped accepting passwords there years ago, so it cannot succeed
// — it can only hang cast, or (in the original report) hand back an
// error about the repository that hides the real fault.
GIT_TERMINAL_PROMPT: "0",
},
},
);
} catch (err) {
const stderr = String((err as { stderr?: Buffer | string })?.stderr ?? "");
throw new Error(cloneFailureMessage(orgRepo, auth, stderr));
}
feat: cast — the Coolify executor, extracted from the infra state repo Public tool, private state. cast holds no hostnames, no bindings, no secrets: it joins a product repo's .infra/ manifest with a state directory you point it at, and makes Coolify match. Extracted from heavy-duty/infra, which was half tool and half state — the inconsistency that made it impossible to say whether "infra" named a CLI or a runbook. rig builds the boxes; cast fills them; infra is what they are filled with. Two changes were required to make it genuinely stateless and publishable: - The implicit cwd contract (environments.yaml / secrets/ / .coolify.env resolved against the working directory, silently reading the wrong file from the wrong place) is now an explicit --state <dir> / $CAST_STATE. - BANNED_IN_PROD — a hardcoded list of one product's ALLOW_* flags, the only product knowledge in the executor — becomes the generic, operator- owned environments.<env>.forbidden_var_patterns. The guard now lives in private state, so a product-side change cannot lower its own guard, and it is a pattern rather than a list, so it catches unforeseen siblings. Age identities resolve as $CAST_AGE_KEY_FILE_<ENV> then ~/.config/cast/age-<env>.key — which is the entire attended-vs-unattended apply mechanism, with no environment names known to the tool. Instance identity (org names, the GitHub App name, founder domains) is out of the fixtures and out of register-github-app.sh, which took APP_NAME and ORG as arguments rather than baking them in. 69 tests green; bin/cast + curl installer mirror rig's shape. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-11 12:25:44 +00:00
return dir;
}
export function desiredFromManifest(
checkoutDir: string,
envName: string,
secrets: Record<string, string>,
): {
desired: Desired[];
resolvedEnvs: Record<string, ResolvedEnv>;
backupSchedules: Record<string, { frequency: string; retention: number }>;
} {
const manifest = loadManifest(join(checkoutDir, ".infra", "manifest.yaml"));
const envSpec = manifest.environments[envName];
if (!envSpec) {
throw new Error(
`environment ${envName} not in manifest (has: ${Object.keys(manifest.environments).join(", ") || "none"})`,
);
}
const desired: Desired[] = [];
const resolvedEnvs: Record<string, ResolvedEnv> = {};
const backupSchedules: Record<
string,
{ frequency: string; retention: number }
> = {};
const resolveEnvFile = (
name: string,
template?: string,
): ResolvedEnv | undefined => {
if (!template) return undefined;
const file = join(checkoutDir, ".infra", "env", template);
if (!existsSync(file))
throw new Error(`env template missing: ${file} (referenced by ${name})`);
const env = resolveTemplate(readFileSync(file, "utf8"), secrets);
resolvedEnvs[name] = env;
return env;
};
for (const [name, app] of Object.entries(envSpec.applications)) {
desired.push({
kind: "application",
name,
fields: {
git_repository: app.source.repo,
git_branch: app.source.branch,
build_pack: app.build.pack,
base_directory: app.build.base_directory,
...(app.build.publish_directory
? { publish_directory: app.build.publish_directory }
: {}),
...(app.build.pack === "dockercompose"
? {
docker_compose_location: app.build.compose_file,
docker_compose_domains: app.service_domains,
}
: {
...(app.port !== undefined ? { port: app.port } : {}),
...(app.healthcheck ? { healthcheck: app.healthcheck } : {}),
domains: app.domains,
}),
},
env: resolveEnvFile(name, app.env_template),
});
}
for (const [name, db] of Object.entries(envSpec.databases ?? {})) {
desired.push({
kind: "database",
name,
fields: { type: db.type, ...(db.version ? { version: db.version } : {}) },
});
if (db.backup)
backupSchedules[name] = {
frequency: db.backup.frequency,
retention: db.backup.retention,
};
}
for (const [name, svc] of Object.entries(envSpec.services ?? {})) {
if (svc.domains && svc.domains.length > 0) {
// Coolify 4.1.2's service executor has no flat `domains` concept —
// hostnames live per-container on `urls` (see serviceApiFields in
// cli.ts) — so a manifest-declared service `domains` list is silently
// unhonorable by apply. Warn at build time, once per run, while the
// service name is still in scope.
console.warn(
`service ${name} declares domains (${svc.domains.join(", ")}), but apply cannot set them on Coolify 4.1.2 services — configure hostnames manually in the Coolify UI`,
);
}
desired.push({
kind: "service",
name,
// domains dropped from fields, same as database `backup` above: the
// live side (projectLiveFields in cli.ts) can't read service domains
// and the write side (serviceApiFields) drops them, so keeping
// domains in fields makes every domain-bearing service diff as a
// perpetual update. Hostnames stay a manual Coolify UI act (warned
// above).
fields: { type: svc.type },
env: resolveEnvFile(name, svc.env_template),
});
}
return { desired, resolvedEnvs, backupSchedules };
}