feat(db): bring ad-hoc dump/restore on-box as rig db (Closes #15)
#18
4 changed files with 307 additions and 0 deletions
63
README.md
63
README.md
|
|
@ -124,6 +124,69 @@ journalctl -u coolify-dump.service -n 20 --no-pager
|
|||
|
||||
A backup you have never read back is not yet a backup.
|
||||
|
||||
### `rig db <dump|restore>`
|
||||
|
||||
Ad-hoc PostgreSQL dump/restore for a container running on this box. Run as
|
||||
root.
|
||||
|
||||
```sh
|
||||
rig db dump coolify-db # -> coolify-db-20260717T041500Z.sql.gz
|
||||
rig db dump my-app-db /srv/snapshots/pre-migrate.sql.gz
|
||||
rig db restore pre-migrate.sql.gz my-app-db # prompts before overwriting
|
||||
rig db restore umami.sql.gz shared-pg umami --yes
|
||||
```
|
||||
|
||||
This is **imperative on-box tooling** — the "give me a copy of that database
|
||||
right now", "put this artifact back" verbs you reach for by hand. It is the
|
||||
counterpart to [`rig coolify backup install`](#rig-coolify-backup-install),
|
||||
which is the *scheduled, declarative, forensics-only* path; `db` is
|
||||
interactive, targets any container, and (on restore) overwrites live data
|
||||
behind a confirm gate. Declarative convergence lives elsewhere by design — this
|
||||
verb exists precisely for the moments that are not convergent.
|
||||
|
||||
**`dump`** pipes `pg_dump` straight into `gzip`:
|
||||
|
||||
```sh
|
||||
docker exec <container> sh -c \
|
||||
'pg_dump -U "$POSTGRES_USER" --clean --if-exists --no-owner --no-acl "$POSTGRES_DB"' | gzip
|
||||
```
|
||||
|
||||
- **`--no-owner --no-acl` is mandatory, not cosmetic.** A cross-instance restore
|
||||
runs as the *target's* superuser, and Coolify randomizes that role per
|
||||
database — so the source's `ALTER OWNER`/`GRANT` statements name a role that
|
||||
does not exist on the target and, under `ON_ERROR_STOP=1`, abort the whole
|
||||
restore on the first one. Stripping ownership and ACLs makes the dump describe
|
||||
*data and schema*, portable onto any instance.
|
||||
- **`$POSTGRES_USER` / `$POSTGRES_DB` are read inside the container** — that is
|
||||
why the command is a *single-quoted* `sh -c`: it evaluates the container's own
|
||||
environment, never the host's. The host never hardcodes `postgres`; on the
|
||||
next container that role name is simply wrong.
|
||||
- With no `[outfile]`, it writes `<container>-<UTC-timestamp>.sql.gz` in the
|
||||
current directory (same timestamp shape as the nightly dump). `pipefail` is
|
||||
load-bearing: without it a failing `pg_dump` still exits 0 through the pipe and
|
||||
`gzip` compresses the truncated output into a valid `.gz` that looks exactly
|
||||
like a good backup. rig dumps to a sibling temp, promotes it only on success,
|
||||
and refuses to keep an empty artifact.
|
||||
|
||||
**`restore <artifact> <container> [db]`** streams the artifact back in:
|
||||
|
||||
```sh
|
||||
gunzip -c <artifact> | docker exec -i <container> sh -c \
|
||||
'psql -U "$POSTGRES_USER" -d "${db:-$POSTGRES_DB}" -v ON_ERROR_STOP=1'
|
||||
```
|
||||
|
||||
- It connects as the container's **own superuser** (`$POSTGRES_USER`), again
|
||||
never a hardcoded role, and runs with `ON_ERROR_STOP=1` so a bad restore fails
|
||||
loudly instead of limping to a half-applied state and reporting success.
|
||||
- **`[db]` targets a named database in a shared container** — e.g. a `umami`
|
||||
database living in a Postgres that also hosts other apps. Omit it to restore
|
||||
into the container's default `$POSTGRES_DB`. rig passes the name *into* the
|
||||
container as an env var rather than splicing it into the command string.
|
||||
- **Restore overwrites the target**, so it prompts `y/N` first. `--yes` (or
|
||||
`--force`) is the automation bypass. The artifact is checked for existence and
|
||||
non-emptiness *before* the prompt and before anything touches the database, so
|
||||
a fat-fingered path fails cheaply.
|
||||
|
||||
### `rig runner install --repo <owner/repo>`
|
||||
|
||||
Runner box only, run after `rig bootstrap runner` (the same two-step rhythm
|
||||
|
|
|
|||
17
bin/rig
17
bin/rig
|
|
@ -20,6 +20,11 @@ commands:
|
|||
systemd timer. rig installs the machinery and templates an empty
|
||||
0600 bindings file; you fill in the age recipient and S3 details.
|
||||
Control-plane box only. Run as root.
|
||||
db <dump|restore> ...
|
||||
Ad-hoc PostgreSQL dump/restore for a container on this box. `dump`
|
||||
writes a gzipped SQL artifact (--no-owner --no-acl, so it restores
|
||||
onto a different instance); `restore` loads one back, connecting as
|
||||
the container's own superuser, behind a confirm gate. Run as root.
|
||||
runner install --repo <owner/repo> [options]
|
||||
GitHub Actions runner as a systemd service under an unprivileged
|
||||
user — outbound-only, no Docker. Prompts for the short-lived
|
||||
|
|
@ -71,6 +76,18 @@ case "$cmd" in
|
|||
;;
|
||||
esac
|
||||
;;
|
||||
db)
|
||||
shift
|
||||
case "${1:-}" in
|
||||
dump|restore|-h|--help)
|
||||
exec "$ROOT/commands/db.sh" "$@"
|
||||
;;
|
||||
*)
|
||||
usage >&2
|
||||
exit 2
|
||||
;;
|
||||
esac
|
||||
;;
|
||||
runner)
|
||||
shift
|
||||
sub="${1:-}"
|
||||
|
|
|
|||
181
commands/db.sh
Executable file
181
commands/db.sh
Executable file
|
|
@ -0,0 +1,181 @@
|
|||
#!/usr/bin/env bash
|
||||
# rig db — ad-hoc PostgreSQL dump/restore for the containers running on THIS box.
|
||||
#
|
||||
# Imperative on-box tooling, deliberately: this is the "I need a copy of that
|
||||
# database right now" / "put this artifact back" verb an operator reaches for by
|
||||
# hand. It is the counterpart to `coolify backup install`, which is the
|
||||
# scheduled, declarative, forensics-only path — this one is interactive, targets
|
||||
# any container, and (for restore) overwrites live data behind a confirm gate.
|
||||
#
|
||||
# Two rules run through everything below and are non-negotiable:
|
||||
# * $POSTGRES_USER / $POSTGRES_DB are read INSIDE the container (that is why
|
||||
# every pg_dump/psql lives in a SINGLE-quoted `sh -c '...'` — the container's
|
||||
# own environment, not the host's). Coolify randomizes the superuser per
|
||||
# database, so the host must never hardcode `postgres`; a hardcoded role is
|
||||
# simply wrong on the next container.
|
||||
# * dumps carry `--no-owner --no-acl`. Without them a cross-instance restore
|
||||
# aborts under ON_ERROR_STOP=1 the moment psql hits a GRANT/ALTER OWNER for
|
||||
# a role that does not exist on the target (source and target superusers
|
||||
# differ by construction). The dump must describe *data and schema*, not the
|
||||
# source box's role graph.
|
||||
set -euo pipefail
|
||||
|
||||
log() { printf 'rig-db: %s\n' "$*"; }
|
||||
warn() { printf 'rig-db: WARNING: %s\n' "$*" >&2; }
|
||||
die() { printf 'rig-db: ERROR: %s\n' "$1" >&2; exit "${2:-1}"; }
|
||||
|
||||
usage() {
|
||||
cat <<'EOF'
|
||||
usage: rig db <dump|restore> ...
|
||||
|
||||
dump <container> [outfile]
|
||||
Dump the PostgreSQL database inside <container> to a gzipped SQL file.
|
||||
When [outfile] is omitted, writes <container>-<UTC-timestamp>.sql.gz in
|
||||
the current directory. The dump is taken with --clean --if-exists
|
||||
--no-owner --no-acl, so it restores cleanly onto a different instance
|
||||
whose superuser differs from this one's.
|
||||
|
||||
restore <artifact> <container> [db] [--yes]
|
||||
Restore a gzipped SQL artifact into <container>, connecting as the
|
||||
container's OWN superuser. [db] targets a NAMED database in a shared
|
||||
container (e.g. a `umami` database in a shared Postgres); omit it to
|
||||
restore into the container's default database. Restore OVERWRITES the
|
||||
target and prompts before doing so — pass --yes (or --force) to run
|
||||
non-interactively.
|
||||
EOF
|
||||
}
|
||||
|
||||
# --- common guards ----------------------------------------------------------
|
||||
# Called AFTER arg validation (arg errors must stay testable without root).
|
||||
require_common_guards() {
|
||||
[ "$(id -u)" -eq 0 ] || die "must run as root"
|
||||
if [ -r /etc/os-release ]; then
|
||||
# Sourced in a subshell — /etc/os-release defines VERSION and would clobber
|
||||
# a caller's variables (see test/cli.sh's regression check).
|
||||
local OS_FAMILY
|
||||
# shellcheck source=/dev/null
|
||||
OS_FAMILY="$(. /etc/os-release && printf '%s %s' "${ID:-}" "${ID_LIKE:-}")"
|
||||
case "$OS_FAMILY" in
|
||||
*debian*) ;;
|
||||
*) warn "not a Debian-family system (${OS_FAMILY:-unknown}); proceeding anyway" ;;
|
||||
esac
|
||||
else
|
||||
warn "cannot read /etc/os-release; proceeding anyway"
|
||||
fi
|
||||
# dump/restore only make sense on a box actually running the containers.
|
||||
command -v docker >/dev/null \
|
||||
|| die "docker not found — rig db operates on containers running on this box"
|
||||
}
|
||||
|
||||
# --- db dump ----------------------------------------------------------------
|
||||
cmd_dump() {
|
||||
local container="" outfile="" tmp
|
||||
local -a pos=()
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
-h|--help) usage; exit 0 ;;
|
||||
--*|-*) die "unknown flag: $1" 2 ;;
|
||||
*) pos+=("$1"); shift ;;
|
||||
esac
|
||||
done
|
||||
|
||||
# Args first, so these errors are reachable without docker or root.
|
||||
[ "${#pos[@]}" -le 2 ] || die "dump takes at most <container> [outfile]" 2
|
||||
container="${pos[0]:-}"
|
||||
outfile="${pos[1]:-}"
|
||||
[ -n "$container" ] || die "dump needs a container name" 2
|
||||
# Mirror coolify-dump.sh's timestamp style for the default name.
|
||||
[ -n "$outfile" ] || outfile="${container}-$(date -u +%Y%m%dT%H%M%SZ).sql.gz"
|
||||
|
||||
require_common_guards
|
||||
command -v gzip >/dev/null || die "gzip not found (part of coreutils on Debian)"
|
||||
|
||||
log "dumping database in container ${container} -> ${outfile}"
|
||||
# Write to a sibling temp and promote on success, so a failed dump never
|
||||
# leaves a plausible-looking artifact in place of a real one. Same directory
|
||||
# as the target keeps the final mv atomic.
|
||||
tmp="$(mktemp "${outfile}.XXXXXX")" || die "could not create a temp file next to ${outfile}"
|
||||
trap 'rm -f "$tmp"' EXIT
|
||||
|
||||
# pipefail (set above) is load-bearing: without it a failing pg_dump still
|
||||
# exits 0 through the pipe and gzip faithfully compresses the truncated output
|
||||
# into a valid .gz that looks exactly like a good backup. The `sh -c` is
|
||||
# single-quoted on purpose — $POSTGRES_USER/$POSTGRES_DB are the CONTAINER's.
|
||||
# shellcheck disable=SC2016
|
||||
if ! docker exec "$container" \
|
||||
sh -c 'pg_dump -U "$POSTGRES_USER" --clean --if-exists --no-owner --no-acl "$POSTGRES_DB"' \
|
||||
| gzip > "$tmp"; then
|
||||
die "pg_dump failed — no artifact written"
|
||||
fi
|
||||
# Even an empty dump gzips to a ~20-byte VALID file; refuse to keep one.
|
||||
[ -s "$tmp" ] || die "refusing to keep an empty dump artifact"
|
||||
mv "$tmp" "$outfile"
|
||||
trap - EXIT
|
||||
log "wrote ${outfile} ($(stat -c %s "$outfile") bytes)"
|
||||
}
|
||||
|
||||
# --- db restore -------------------------------------------------------------
|
||||
cmd_restore() {
|
||||
local artifact="" container="" target_db="" assume_yes=0 reply=""
|
||||
local -a pos=()
|
||||
while [ $# -gt 0 ]; do
|
||||
case "$1" in
|
||||
--yes|--force) assume_yes=1; shift ;;
|
||||
-h|--help) usage; exit 0 ;;
|
||||
--*|-*) die "unknown flag: $1" 2 ;;
|
||||
*) pos+=("$1"); shift ;;
|
||||
esac
|
||||
done
|
||||
|
||||
# Args first: required-arg errors are exit 2 and reachable without root.
|
||||
[ "${#pos[@]}" -le 3 ] || die "restore takes at most <artifact> <container> [db]" 2
|
||||
artifact="${pos[0]:-}"
|
||||
container="${pos[1]:-}"
|
||||
target_db="${pos[2]:-}"
|
||||
[ -n "$artifact" ] || die "restore needs an artifact file" 2
|
||||
[ -n "$container" ] || die "restore needs a target container" 2
|
||||
# Artifact existence/non-emptiness is checkable without docker or root, so we
|
||||
# check it HERE — a fat-fingered path fails clearly and cheaply, before the
|
||||
# confirm gate and before anything touches the database.
|
||||
[ -e "$artifact" ] || die "artifact not found: ${artifact}"
|
||||
[ -f "$artifact" ] || die "artifact is not a regular file: ${artifact}"
|
||||
[ -s "$artifact" ] || die "artifact is empty: ${artifact}"
|
||||
|
||||
require_common_guards
|
||||
command -v gunzip >/dev/null || die "gunzip not found (part of coreutils on Debian)"
|
||||
|
||||
# Destructive: restore overwrites the target database in place. Default to
|
||||
# prompting; --yes/--force is the automation bypass.
|
||||
if [ "$assume_yes" -eq 0 ]; then
|
||||
printf 'rig-db: restore OVERWRITES the database in container %s%s. Continue? [y/N] ' \
|
||||
"$container" "${target_db:+ (database ${target_db})}" >&2
|
||||
read -r reply || reply=""
|
||||
case "$reply" in
|
||||
y|Y|yes|YES|Yes) ;;
|
||||
*) die "aborted — no changes made" ;;
|
||||
esac
|
||||
fi
|
||||
|
||||
log "restoring ${artifact} into container ${container}${target_db:+ (database ${target_db})}"
|
||||
# Connect as the container's OWN superuser ($POSTGRES_USER); never hardcode
|
||||
# postgres. RIG_TARGET_DB carries the optional [db] arg INTO the container so
|
||||
# a shared Postgres can be restored into a NAMED database; empty falls back to
|
||||
# the container's own $POSTGRES_DB. ON_ERROR_STOP=1 aborts on the first error
|
||||
# rather than limping to a half-applied restore and reporting success.
|
||||
# shellcheck disable=SC2016
|
||||
if ! gunzip -c "$artifact" \
|
||||
| docker exec -i -e RIG_TARGET_DB="$target_db" "$container" \
|
||||
sh -c 'psql -U "$POSTGRES_USER" -d "${RIG_TARGET_DB:-$POSTGRES_DB}" -v ON_ERROR_STOP=1'; then
|
||||
die "restore failed — the database may be partially applied; inspect container ${container}"
|
||||
fi
|
||||
log "restore complete"
|
||||
}
|
||||
|
||||
# --- dispatch ---------------------------------------------------------------
|
||||
sub="${1:-}"
|
||||
case "$sub" in
|
||||
dump) shift; cmd_dump "$@" ;;
|
||||
restore) shift; cmd_restore "$@" ;;
|
||||
-h|--help) usage; exit 0 ;;
|
||||
*) usage >&2; exit 2 ;;
|
||||
esac
|
||||
46
test/cli.sh
46
test/cli.sh
|
|
@ -66,6 +66,52 @@ else
|
|||
echo "skip: coolify backup non-root refusal (running as root)"
|
||||
fi
|
||||
|
||||
# --- rig db (ad-hoc dump/restore) -------------------------------------------
|
||||
check "bare db shows usage, exit 2" 2 "usage:" "$ROOT/bin/rig" db
|
||||
check "db --help exits 0" 0 "usage:" "$ROOT/bin/rig" db --help
|
||||
check "db bad subcommand exits 2" 2 "usage:" "$ROOT/bin/rig" db frobnicate
|
||||
check "db dump: --help exits 0" 0 "usage:" "$ROOT/commands/db.sh" dump --help
|
||||
check "db dump: container required, exit 2" 2 "needs a container" "$ROOT/commands/db.sh" dump
|
||||
check "db dump: unknown flag exits 2" 2 "unknown flag" "$ROOT/commands/db.sh" dump --nope
|
||||
check "db restore: artifact required, exit 2" 2 "needs an artifact" "$ROOT/commands/db.sh" restore
|
||||
check "db restore: container required, exit 2" 2 "needs a target container" \
|
||||
"$ROOT/commands/db.sh" restore /tmp/whatever.sql.gz
|
||||
check "db restore: unknown flag exits 2" 2 "unknown flag" "$ROOT/commands/db.sh" restore --nope
|
||||
# Artifact existence is checked BEFORE docker/root, so a fat-fingered path fails
|
||||
# clearly and cheaply — and is testable here without root or a live container.
|
||||
check "db restore: missing artifact fails before the docker/root path" \
|
||||
1 "artifact not found" "$ROOT/commands/db.sh" restore /no/such/artifact.sql.gz somecontainer --yes
|
||||
|
||||
# The two DB invariants live as embedded command strings (single-quoted sh -c),
|
||||
# not an extractable heredoc, so guard them directly: dropping --no-owner/--no-acl
|
||||
# breaks every cross-instance restore, and hardcoding a role instead of the
|
||||
# container's own $POSTGRES_USER/$POSTGRES_DB is wrong on Coolify's randomized
|
||||
# superuser. ON_ERROR_STOP=1 is what makes a bad restore fail instead of limp.
|
||||
check "db dump embeds --no-owner --no-acl" 0 "" \
|
||||
grep -qF -- "--no-owner --no-acl" "$ROOT/commands/db.sh"
|
||||
# The $POSTGRES_USER below is a LITERAL we grep for in db.sh (it must read the
|
||||
# container's env, not the host's) — single quotes are the point here.
|
||||
# shellcheck disable=SC2016
|
||||
check "db dump reads the container's own \$POSTGRES_USER/\$POSTGRES_DB" 0 "" \
|
||||
grep -qF 'pg_dump -U "$POSTGRES_USER"' "$ROOT/commands/db.sh"
|
||||
# shellcheck disable=SC2016
|
||||
check "db restore connects as the container's own \$POSTGRES_USER" 0 "" \
|
||||
grep -qF 'psql -U "$POSTGRES_USER"' "$ROOT/commands/db.sh"
|
||||
check "db restore uses ON_ERROR_STOP=1" 0 "" \
|
||||
grep -qF "ON_ERROR_STOP=1" "$ROOT/commands/db.sh"
|
||||
if [ "$(id -u)" -ne 0 ]; then
|
||||
# Valid args, so validation passes and we reach the root guard.
|
||||
check "db dump: refuses non-root" 1 "must run as root" "$ROOT/commands/db.sh" dump somecontainer
|
||||
# Restore needs a real, non-empty artifact to get PAST the artifact check and
|
||||
# reach the root guard; --yes skips the confirm prompt so the check is exit-clean.
|
||||
DB_ART="$(mktemp)"; printf 'SELECT 1;\n' > "$DB_ART"
|
||||
check "db restore: refuses non-root" 1 "must run as root" \
|
||||
"$ROOT/commands/db.sh" restore "$DB_ART" somecontainer --yes
|
||||
rm -f "$DB_ART"
|
||||
else
|
||||
echo "skip: db non-root refusals (running as root)"
|
||||
fi
|
||||
|
||||
check "bare runner shows usage, exit 2" 2 "usage:" "$ROOT/bin/rig" runner
|
||||
check "runner: --help exits 0" 0 "usage:" "$ROOT/commands/runner-install.sh" --help
|
||||
check "runner: repo required, exit 2" 2 "--repo" "$ROOT/commands/runner-install.sh" --version 2.335.1
|
||||
|
|
|
|||
Loading…
Reference in a new issue