feat(db): bring ad-hoc dump/restore on-box as rig db (Closes #15)
#18
6 changed files with 536 additions and 0 deletions
13
.github/workflows/ci.yml
vendored
13
.github/workflows/ci.yml
vendored
|
|
@ -20,3 +20,16 @@ jobs:
|
||||||
shellcheck -x "${files[@]}"
|
shellcheck -x "${files[@]}"
|
||||||
- name: cli tests
|
- name: cli tests
|
||||||
run: bash test/cli.sh
|
run: bash test/cli.sh
|
||||||
|
|
||||||
|
# Kept SEPARATE from `check` on purpose: this job pulls a Postgres image and
|
||||||
|
# stands up throwaway containers, and a slow image pull must never delay the
|
||||||
|
# fast shellcheck + cli.sh feedback above. ubuntu-latest ships Docker running
|
||||||
|
# and passwordless sudo, so test/db-integration.sh EXECUTES here (it only
|
||||||
|
# skips where Docker is absent). It is the automated proof that dump/restore
|
||||||
|
# actually round-trips, not just that the args parse.
|
||||||
|
db-integration:
|
||||||
|
runs-on: ubuntu-latest
|
||||||
|
steps:
|
||||||
|
- uses: actions/checkout@v4
|
||||||
|
- name: db dump/restore round-trip
|
||||||
|
run: bash test/db-integration.sh
|
||||||
|
|
|
||||||
101
README.md
101
README.md
|
|
@ -124,6 +124,107 @@ journalctl -u coolify-dump.service -n 20 --no-pager
|
||||||
|
|
||||||
A backup you have never read back is not yet a backup.
|
A backup you have never read back is not yet a backup.
|
||||||
|
|
||||||
|
### `rig db <dump|restore>`
|
||||||
|
|
||||||
|
Ad-hoc PostgreSQL dump/restore for a container running on this box. Run as
|
||||||
|
root.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
rig db dump coolify-db # -> coolify-db-20260717T041500Z.sql.gz
|
||||||
|
rig db dump my-app-db /srv/snapshots/pre-migrate.sql.gz
|
||||||
|
rig db restore pre-migrate.sql.gz my-app-db # prompts before overwriting
|
||||||
|
rig db restore umami.sql.gz shared-pg umami --yes
|
||||||
|
```
|
||||||
|
|
||||||
|
This is **imperative on-box tooling** — the "give me a copy of that database
|
||||||
|
right now", "put this artifact back" verbs you reach for by hand. It is the
|
||||||
|
counterpart to [`rig coolify backup install`](#rig-coolify-backup-install),
|
||||||
|
which is the *scheduled, declarative, forensics-only* path; `db` is
|
||||||
|
interactive, targets any container, and (on restore) overwrites live data
|
||||||
|
behind a confirm gate. Declarative convergence lives elsewhere by design — this
|
||||||
|
verb exists precisely for the moments that are not convergent.
|
||||||
|
|
||||||
|
**`dump`** pipes `pg_dump` straight into `gzip`:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
docker exec <container> sh -c \
|
||||||
|
'pg_dump -U "$POSTGRES_USER" --clean --if-exists --no-owner --no-acl "$POSTGRES_DB"' | gzip
|
||||||
|
```
|
||||||
|
|
||||||
|
- **`--no-owner --no-acl` is mandatory, not cosmetic.** A cross-instance restore
|
||||||
|
runs as the *target's* superuser, and Coolify randomizes that role per
|
||||||
|
database — so the source's `ALTER OWNER`/`GRANT` statements name a role that
|
||||||
|
does not exist on the target and, under `ON_ERROR_STOP=1`, abort the whole
|
||||||
|
restore on the first one. Stripping ownership and ACLs makes the dump describe
|
||||||
|
*data and schema*, portable onto any instance.
|
||||||
|
- **`$POSTGRES_USER` / `$POSTGRES_DB` are read inside the container** — that is
|
||||||
|
why the command is a *single-quoted* `sh -c`: it evaluates the container's own
|
||||||
|
environment, never the host's. The host never hardcodes `postgres`; on the
|
||||||
|
next container that role name is simply wrong.
|
||||||
|
- With no `[outfile]`, it writes `<container>-<UTC-timestamp>.sql.gz` in the
|
||||||
|
current directory (same timestamp shape as the nightly dump). `pipefail` is
|
||||||
|
load-bearing: without it a failing `pg_dump` still exits 0 through the pipe and
|
||||||
|
`gzip` compresses the truncated output into a valid `.gz` that looks exactly
|
||||||
|
like a good backup. rig dumps to a sibling temp, promotes it only on success,
|
||||||
|
and refuses to keep an empty artifact.
|
||||||
|
|
||||||
|
**`restore <artifact> <container> [db]`** streams the artifact back in:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
gunzip -c <artifact> | docker exec -i <container> sh -c \
|
||||||
|
'psql -U "$POSTGRES_USER" -d "${db:-$POSTGRES_DB}" -v ON_ERROR_STOP=1'
|
||||||
|
```
|
||||||
|
|
||||||
|
- It connects as the container's **own superuser** (`$POSTGRES_USER`), again
|
||||||
|
never a hardcoded role, and runs with `ON_ERROR_STOP=1` so a bad restore fails
|
||||||
|
loudly instead of limping to a half-applied state and reporting success.
|
||||||
|
- **`[db]` targets a named database in a shared container** — e.g. a `umami`
|
||||||
|
database living in a Postgres that also hosts other apps. Omit it to restore
|
||||||
|
into the container's default `$POSTGRES_DB`. rig passes the name *into* the
|
||||||
|
container as an env var rather than splicing it into the command string.
|
||||||
|
- **Restore overwrites the target**, so it prompts `y/N` first. `--yes` (or
|
||||||
|
`--force`) is the automation bypass. The artifact is checked for existence and
|
||||||
|
non-emptiness *before* the prompt and before anything touches the database, so
|
||||||
|
a fat-fingered path fails cheaply.
|
||||||
|
|
||||||
|
#### Verifying a dump/restore actually works
|
||||||
|
|
||||||
|
A `.gz` that opens without error is not proof of a good backup: a `pg_dump`
|
||||||
|
truncated mid-stream still compresses into a perfectly valid gzip file that
|
||||||
|
*looks* exactly like a complete one. The same ethos as the nightly dump applies
|
||||||
|
here — **a backup you have never read back is not yet a backup.** The only
|
||||||
|
fully-trustworthy proof is to restore the artifact and read the rows back out.
|
||||||
|
|
||||||
|
On a real Coolify box you can do that **without touching prod data** by
|
||||||
|
restoring into a fresh *scratch* database rather than over the live one:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
# 1. dump the live database (read-only; harmless)
|
||||||
|
rig db dump coolify-db /srv/snapshots/verify.sql.gz
|
||||||
|
|
||||||
|
# 2. create a throwaway database as the container's OWN superuser
|
||||||
|
docker exec coolify-db sh -c 'createdb -U "$POSTGRES_USER" rig_verify'
|
||||||
|
|
||||||
|
# 3. restore the artifact INTO the scratch db (not the live one)
|
||||||
|
rig db restore /srv/snapshots/verify.sql.gz coolify-db rig_verify --yes
|
||||||
|
|
||||||
|
# 4. spot-check a table you expect to see
|
||||||
|
docker exec coolify-db sh -c \
|
||||||
|
'psql -U "$POSTGRES_USER" -d rig_verify -c "\dt"'
|
||||||
|
docker exec coolify-db sh -c \
|
||||||
|
'psql -U "$POSTGRES_USER" -d rig_verify -c "SELECT count(*) FROM <some-table>"'
|
||||||
|
|
||||||
|
# 5. drop the scratch db — live data was never touched
|
||||||
|
docker exec coolify-db sh -c 'dropdb -U "$POSTGRES_USER" rig_verify'
|
||||||
|
```
|
||||||
|
|
||||||
|
If the counts and tables are there, the artifact is real. This is exactly the
|
||||||
|
round-trip `test/db-integration.sh` automates in CI (the `db-integration` job):
|
||||||
|
it seeds a known table in a source container whose superuser is *not* the
|
||||||
|
default, dumps it, restores into a second container whose superuser differs, and
|
||||||
|
asserts the rows and an ordered checksum survived — the same proof, done against
|
||||||
|
throwaway containers on every push.
|
||||||
|
|
||||||
### `rig runner install --repo <owner/repo>`
|
### `rig runner install --repo <owner/repo>`
|
||||||
|
|
||||||
Runner box only, run after `rig bootstrap runner` (the same two-step rhythm
|
Runner box only, run after `rig bootstrap runner` (the same two-step rhythm
|
||||||
|
|
|
||||||
17
bin/rig
17
bin/rig
|
|
@ -20,6 +20,11 @@ commands:
|
||||||
systemd timer. rig installs the machinery and templates an empty
|
systemd timer. rig installs the machinery and templates an empty
|
||||||
0600 bindings file; you fill in the age recipient and S3 details.
|
0600 bindings file; you fill in the age recipient and S3 details.
|
||||||
Control-plane box only. Run as root.
|
Control-plane box only. Run as root.
|
||||||
|
db <dump|restore> ...
|
||||||
|
Ad-hoc PostgreSQL dump/restore for a container on this box. `dump`
|
||||||
|
writes a gzipped SQL artifact (--no-owner --no-acl, so it restores
|
||||||
|
onto a different instance); `restore` loads one back, connecting as
|
||||||
|
the container's own superuser, behind a confirm gate. Run as root.
|
||||||
runner install --repo <owner/repo> [options]
|
runner install --repo <owner/repo> [options]
|
||||||
GitHub Actions runner as a systemd service under an unprivileged
|
GitHub Actions runner as a systemd service under an unprivileged
|
||||||
user — outbound-only, no Docker. Prompts for the short-lived
|
user — outbound-only, no Docker. Prompts for the short-lived
|
||||||
|
|
@ -71,6 +76,18 @@ case "$cmd" in
|
||||||
;;
|
;;
|
||||||
esac
|
esac
|
||||||
;;
|
;;
|
||||||
|
db)
|
||||||
|
shift
|
||||||
|
case "${1:-}" in
|
||||||
|
dump|restore|-h|--help)
|
||||||
|
exec "$ROOT/commands/db.sh" "$@"
|
||||||
|
;;
|
||||||
|
*)
|
||||||
|
usage >&2
|
||||||
|
exit 2
|
||||||
|
;;
|
||||||
|
esac
|
||||||
|
;;
|
||||||
runner)
|
runner)
|
||||||
shift
|
shift
|
||||||
sub="${1:-}"
|
sub="${1:-}"
|
||||||
|
|
|
||||||
181
commands/db.sh
Executable file
181
commands/db.sh
Executable file
|
|
@ -0,0 +1,181 @@
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
# rig db — ad-hoc PostgreSQL dump/restore for the containers running on THIS box.
|
||||||
|
#
|
||||||
|
# Imperative on-box tooling, deliberately: this is the "I need a copy of that
|
||||||
|
# database right now" / "put this artifact back" verb an operator reaches for by
|
||||||
|
# hand. It is the counterpart to `coolify backup install`, which is the
|
||||||
|
# scheduled, declarative, forensics-only path — this one is interactive, targets
|
||||||
|
# any container, and (for restore) overwrites live data behind a confirm gate.
|
||||||
|
#
|
||||||
|
# Two rules run through everything below and are non-negotiable:
|
||||||
|
# * $POSTGRES_USER / $POSTGRES_DB are read INSIDE the container (that is why
|
||||||
|
# every pg_dump/psql lives in a SINGLE-quoted `sh -c '...'` — the container's
|
||||||
|
# own environment, not the host's). Coolify randomizes the superuser per
|
||||||
|
# database, so the host must never hardcode `postgres`; a hardcoded role is
|
||||||
|
# simply wrong on the next container.
|
||||||
|
# * dumps carry `--no-owner --no-acl`. Without them a cross-instance restore
|
||||||
|
# aborts under ON_ERROR_STOP=1 the moment psql hits a GRANT/ALTER OWNER for
|
||||||
|
# a role that does not exist on the target (source and target superusers
|
||||||
|
# differ by construction). The dump must describe *data and schema*, not the
|
||||||
|
# source box's role graph.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
log() { printf 'rig-db: %s\n' "$*"; }
|
||||||
|
warn() { printf 'rig-db: WARNING: %s\n' "$*" >&2; }
|
||||||
|
die() { printf 'rig-db: ERROR: %s\n' "$1" >&2; exit "${2:-1}"; }
|
||||||
|
|
||||||
|
usage() {
|
||||||
|
cat <<'EOF'
|
||||||
|
usage: rig db <dump|restore> ...
|
||||||
|
|
||||||
|
dump <container> [outfile]
|
||||||
|
Dump the PostgreSQL database inside <container> to a gzipped SQL file.
|
||||||
|
When [outfile] is omitted, writes <container>-<UTC-timestamp>.sql.gz in
|
||||||
|
the current directory. The dump is taken with --clean --if-exists
|
||||||
|
--no-owner --no-acl, so it restores cleanly onto a different instance
|
||||||
|
whose superuser differs from this one's.
|
||||||
|
|
||||||
|
restore <artifact> <container> [db] [--yes]
|
||||||
|
Restore a gzipped SQL artifact into <container>, connecting as the
|
||||||
|
container's OWN superuser. [db] targets a NAMED database in a shared
|
||||||
|
container (e.g. a `umami` database in a shared Postgres); omit it to
|
||||||
|
restore into the container's default database. Restore OVERWRITES the
|
||||||
|
target and prompts before doing so — pass --yes (or --force) to run
|
||||||
|
non-interactively.
|
||||||
|
EOF
|
||||||
|
}
|
||||||
|
|
||||||
|
# --- common guards ----------------------------------------------------------
|
||||||
|
# Called AFTER arg validation (arg errors must stay testable without root).
|
||||||
|
require_common_guards() {
|
||||||
|
[ "$(id -u)" -eq 0 ] || die "must run as root"
|
||||||
|
if [ -r /etc/os-release ]; then
|
||||||
|
# Sourced in a subshell — /etc/os-release defines VERSION and would clobber
|
||||||
|
# a caller's variables (see test/cli.sh's regression check).
|
||||||
|
local OS_FAMILY
|
||||||
|
# shellcheck source=/dev/null
|
||||||
|
OS_FAMILY="$(. /etc/os-release && printf '%s %s' "${ID:-}" "${ID_LIKE:-}")"
|
||||||
|
case "$OS_FAMILY" in
|
||||||
|
*debian*) ;;
|
||||||
|
*) warn "not a Debian-family system (${OS_FAMILY:-unknown}); proceeding anyway" ;;
|
||||||
|
esac
|
||||||
|
else
|
||||||
|
warn "cannot read /etc/os-release; proceeding anyway"
|
||||||
|
fi
|
||||||
|
# dump/restore only make sense on a box actually running the containers.
|
||||||
|
command -v docker >/dev/null \
|
||||||
|
|| die "docker not found — rig db operates on containers running on this box"
|
||||||
|
}
|
||||||
|
|
||||||
|
# --- db dump ----------------------------------------------------------------
|
||||||
|
cmd_dump() {
|
||||||
|
local container="" outfile="" tmp
|
||||||
|
local -a pos=()
|
||||||
|
while [ $# -gt 0 ]; do
|
||||||
|
case "$1" in
|
||||||
|
-h|--help) usage; exit 0 ;;
|
||||||
|
--*|-*) die "unknown flag: $1" 2 ;;
|
||||||
|
*) pos+=("$1"); shift ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
# Args first, so these errors are reachable without docker or root.
|
||||||
|
[ "${#pos[@]}" -le 2 ] || die "dump takes at most <container> [outfile]" 2
|
||||||
|
container="${pos[0]:-}"
|
||||||
|
outfile="${pos[1]:-}"
|
||||||
|
[ -n "$container" ] || die "dump needs a container name" 2
|
||||||
|
# Mirror coolify-dump.sh's timestamp style for the default name.
|
||||||
|
[ -n "$outfile" ] || outfile="${container}-$(date -u +%Y%m%dT%H%M%SZ).sql.gz"
|
||||||
|
|
||||||
|
require_common_guards
|
||||||
|
command -v gzip >/dev/null || die "gzip not found (part of coreutils on Debian)"
|
||||||
|
|
||||||
|
log "dumping database in container ${container} -> ${outfile}"
|
||||||
|
# Write to a sibling temp and promote on success, so a failed dump never
|
||||||
|
# leaves a plausible-looking artifact in place of a real one. Same directory
|
||||||
|
# as the target keeps the final mv atomic.
|
||||||
|
tmp="$(mktemp "${outfile}.XXXXXX")" || die "could not create a temp file next to ${outfile}"
|
||||||
|
trap 'rm -f "$tmp"' EXIT
|
||||||
|
|
||||||
|
# pipefail (set above) is load-bearing: without it a failing pg_dump still
|
||||||
|
# exits 0 through the pipe and gzip faithfully compresses the truncated output
|
||||||
|
# into a valid .gz that looks exactly like a good backup. The `sh -c` is
|
||||||
|
# single-quoted on purpose — $POSTGRES_USER/$POSTGRES_DB are the CONTAINER's.
|
||||||
|
# shellcheck disable=SC2016
|
||||||
|
if ! docker exec "$container" \
|
||||||
|
sh -c 'pg_dump -U "$POSTGRES_USER" --clean --if-exists --no-owner --no-acl "$POSTGRES_DB"' \
|
||||||
|
| gzip > "$tmp"; then
|
||||||
|
die "pg_dump failed — no artifact written"
|
||||||
|
fi
|
||||||
|
# Even an empty dump gzips to a ~20-byte VALID file; refuse to keep one.
|
||||||
|
[ -s "$tmp" ] || die "refusing to keep an empty dump artifact"
|
||||||
|
mv "$tmp" "$outfile"
|
||||||
|
trap - EXIT
|
||||||
|
log "wrote ${outfile} ($(stat -c %s "$outfile") bytes)"
|
||||||
|
}
|
||||||
|
|
||||||
|
# --- db restore -------------------------------------------------------------
|
||||||
|
cmd_restore() {
|
||||||
|
local artifact="" container="" target_db="" assume_yes=0 reply=""
|
||||||
|
local -a pos=()
|
||||||
|
while [ $# -gt 0 ]; do
|
||||||
|
case "$1" in
|
||||||
|
--yes|--force) assume_yes=1; shift ;;
|
||||||
|
-h|--help) usage; exit 0 ;;
|
||||||
|
--*|-*) die "unknown flag: $1" 2 ;;
|
||||||
|
*) pos+=("$1"); shift ;;
|
||||||
|
esac
|
||||||
|
done
|
||||||
|
|
||||||
|
# Args first: required-arg errors are exit 2 and reachable without root.
|
||||||
|
[ "${#pos[@]}" -le 3 ] || die "restore takes at most <artifact> <container> [db]" 2
|
||||||
|
artifact="${pos[0]:-}"
|
||||||
|
container="${pos[1]:-}"
|
||||||
|
target_db="${pos[2]:-}"
|
||||||
|
[ -n "$artifact" ] || die "restore needs an artifact file" 2
|
||||||
|
[ -n "$container" ] || die "restore needs a target container" 2
|
||||||
|
# Artifact existence/non-emptiness is checkable without docker or root, so we
|
||||||
|
# check it HERE — a fat-fingered path fails clearly and cheaply, before the
|
||||||
|
# confirm gate and before anything touches the database.
|
||||||
|
[ -e "$artifact" ] || die "artifact not found: ${artifact}"
|
||||||
|
[ -f "$artifact" ] || die "artifact is not a regular file: ${artifact}"
|
||||||
|
[ -s "$artifact" ] || die "artifact is empty: ${artifact}"
|
||||||
|
|
||||||
|
require_common_guards
|
||||||
|
command -v gunzip >/dev/null || die "gunzip not found (part of coreutils on Debian)"
|
||||||
|
|
||||||
|
# Destructive: restore overwrites the target database in place. Default to
|
||||||
|
# prompting; --yes/--force is the automation bypass.
|
||||||
|
if [ "$assume_yes" -eq 0 ]; then
|
||||||
|
printf 'rig-db: restore OVERWRITES the database in container %s%s. Continue? [y/N] ' \
|
||||||
|
"$container" "${target_db:+ (database ${target_db})}" >&2
|
||||||
|
read -r reply || reply=""
|
||||||
|
case "$reply" in
|
||||||
|
y|Y|yes|YES|Yes) ;;
|
||||||
|
*) die "aborted — no changes made" ;;
|
||||||
|
esac
|
||||||
|
fi
|
||||||
|
|
||||||
|
log "restoring ${artifact} into container ${container}${target_db:+ (database ${target_db})}"
|
||||||
|
# Connect as the container's OWN superuser ($POSTGRES_USER); never hardcode
|
||||||
|
# postgres. RIG_TARGET_DB carries the optional [db] arg INTO the container so
|
||||||
|
# a shared Postgres can be restored into a NAMED database; empty falls back to
|
||||||
|
# the container's own $POSTGRES_DB. ON_ERROR_STOP=1 aborts on the first error
|
||||||
|
# rather than limping to a half-applied restore and reporting success.
|
||||||
|
# shellcheck disable=SC2016
|
||||||
|
if ! gunzip -c "$artifact" \
|
||||||
|
| docker exec -i -e RIG_TARGET_DB="$target_db" "$container" \
|
||||||
|
sh -c 'psql -U "$POSTGRES_USER" -d "${RIG_TARGET_DB:-$POSTGRES_DB}" -v ON_ERROR_STOP=1'; then
|
||||||
|
die "restore failed — the database may be partially applied; inspect container ${container}"
|
||||||
|
fi
|
||||||
|
log "restore complete"
|
||||||
|
}
|
||||||
|
|
||||||
|
# --- dispatch ---------------------------------------------------------------
|
||||||
|
sub="${1:-}"
|
||||||
|
case "$sub" in
|
||||||
|
dump) shift; cmd_dump "$@" ;;
|
||||||
|
restore) shift; cmd_restore "$@" ;;
|
||||||
|
-h|--help) usage; exit 0 ;;
|
||||||
|
*) usage >&2; exit 2 ;;
|
||||||
|
esac
|
||||||
46
test/cli.sh
46
test/cli.sh
|
|
@ -66,6 +66,52 @@ else
|
||||||
echo "skip: coolify backup non-root refusal (running as root)"
|
echo "skip: coolify backup non-root refusal (running as root)"
|
||||||
fi
|
fi
|
||||||
|
|
||||||
|
# --- rig db (ad-hoc dump/restore) -------------------------------------------
|
||||||
|
check "bare db shows usage, exit 2" 2 "usage:" "$ROOT/bin/rig" db
|
||||||
|
check "db --help exits 0" 0 "usage:" "$ROOT/bin/rig" db --help
|
||||||
|
check "db bad subcommand exits 2" 2 "usage:" "$ROOT/bin/rig" db frobnicate
|
||||||
|
check "db dump: --help exits 0" 0 "usage:" "$ROOT/commands/db.sh" dump --help
|
||||||
|
check "db dump: container required, exit 2" 2 "needs a container" "$ROOT/commands/db.sh" dump
|
||||||
|
check "db dump: unknown flag exits 2" 2 "unknown flag" "$ROOT/commands/db.sh" dump --nope
|
||||||
|
check "db restore: artifact required, exit 2" 2 "needs an artifact" "$ROOT/commands/db.sh" restore
|
||||||
|
check "db restore: container required, exit 2" 2 "needs a target container" \
|
||||||
|
"$ROOT/commands/db.sh" restore /tmp/whatever.sql.gz
|
||||||
|
check "db restore: unknown flag exits 2" 2 "unknown flag" "$ROOT/commands/db.sh" restore --nope
|
||||||
|
# Artifact existence is checked BEFORE docker/root, so a fat-fingered path fails
|
||||||
|
# clearly and cheaply — and is testable here without root or a live container.
|
||||||
|
check "db restore: missing artifact fails before the docker/root path" \
|
||||||
|
1 "artifact not found" "$ROOT/commands/db.sh" restore /no/such/artifact.sql.gz somecontainer --yes
|
||||||
|
|
||||||
|
# The two DB invariants live as embedded command strings (single-quoted sh -c),
|
||||||
|
# not an extractable heredoc, so guard them directly: dropping --no-owner/--no-acl
|
||||||
|
# breaks every cross-instance restore, and hardcoding a role instead of the
|
||||||
|
# container's own $POSTGRES_USER/$POSTGRES_DB is wrong on Coolify's randomized
|
||||||
|
# superuser. ON_ERROR_STOP=1 is what makes a bad restore fail instead of limp.
|
||||||
|
check "db dump embeds --no-owner --no-acl" 0 "" \
|
||||||
|
grep -qF -- "--no-owner --no-acl" "$ROOT/commands/db.sh"
|
||||||
|
# The $POSTGRES_USER below is a LITERAL we grep for in db.sh (it must read the
|
||||||
|
# container's env, not the host's) — single quotes are the point here.
|
||||||
|
# shellcheck disable=SC2016
|
||||||
|
check "db dump reads the container's own \$POSTGRES_USER/\$POSTGRES_DB" 0 "" \
|
||||||
|
grep -qF 'pg_dump -U "$POSTGRES_USER"' "$ROOT/commands/db.sh"
|
||||||
|
# shellcheck disable=SC2016
|
||||||
|
check "db restore connects as the container's own \$POSTGRES_USER" 0 "" \
|
||||||
|
grep -qF 'psql -U "$POSTGRES_USER"' "$ROOT/commands/db.sh"
|
||||||
|
check "db restore uses ON_ERROR_STOP=1" 0 "" \
|
||||||
|
grep -qF "ON_ERROR_STOP=1" "$ROOT/commands/db.sh"
|
||||||
|
if [ "$(id -u)" -ne 0 ]; then
|
||||||
|
# Valid args, so validation passes and we reach the root guard.
|
||||||
|
check "db dump: refuses non-root" 1 "must run as root" "$ROOT/commands/db.sh" dump somecontainer
|
||||||
|
# Restore needs a real, non-empty artifact to get PAST the artifact check and
|
||||||
|
# reach the root guard; --yes skips the confirm prompt so the check is exit-clean.
|
||||||
|
DB_ART="$(mktemp)"; printf 'SELECT 1;\n' > "$DB_ART"
|
||||||
|
check "db restore: refuses non-root" 1 "must run as root" \
|
||||||
|
"$ROOT/commands/db.sh" restore "$DB_ART" somecontainer --yes
|
||||||
|
rm -f "$DB_ART"
|
||||||
|
else
|
||||||
|
echo "skip: db non-root refusals (running as root)"
|
||||||
|
fi
|
||||||
|
|
||||||
check "bare runner shows usage, exit 2" 2 "usage:" "$ROOT/bin/rig" runner
|
check "bare runner shows usage, exit 2" 2 "usage:" "$ROOT/bin/rig" runner
|
||||||
check "runner: --help exits 0" 0 "usage:" "$ROOT/commands/runner-install.sh" --help
|
check "runner: --help exits 0" 0 "usage:" "$ROOT/commands/runner-install.sh" --help
|
||||||
check "runner: repo required, exit 2" 2 "--repo" "$ROOT/commands/runner-install.sh" --version 2.335.1
|
check "runner: repo required, exit 2" 2 "--repo" "$ROOT/commands/runner-install.sh" --version 2.335.1
|
||||||
|
|
|
||||||
178
test/db-integration.sh
Executable file
178
test/db-integration.sh
Executable file
|
|
@ -0,0 +1,178 @@
|
||||||
|
#!/usr/bin/env bash
|
||||||
|
# test/db-integration.sh — a REAL dump/restore round-trip for `rig db`.
|
||||||
|
#
|
||||||
|
# test/cli.sh proves the arg parsing; this proves the actual thing works. It
|
||||||
|
# stands up throwaway PostgreSQL containers, seeds a known table, runs the real
|
||||||
|
# `rig db dump` / `rig db restore`, and reads the rows back out the far side —
|
||||||
|
# because a dump piped through gzip can look perfectly valid while being
|
||||||
|
# truncated, and only restoring it and reading it back proves otherwise.
|
||||||
|
#
|
||||||
|
# It exercises the two invariants db.sh is built around:
|
||||||
|
# * the source superuser is a NON-default name (src_super), so a green run
|
||||||
|
# proves the code reads the CONTAINER's own $POSTGRES_USER/$POSTGRES_DB and
|
||||||
|
# never a hardcoded `postgres`;
|
||||||
|
# * the destination superuser is a DIFFERENT name (dst_super), so the restore
|
||||||
|
# only succeeds because --no-owner --no-acl stripped the source role graph —
|
||||||
|
# a plain dump would abort under ON_ERROR_STOP=1 on the first missing role.
|
||||||
|
#
|
||||||
|
# Skips cleanly (exit 0) when it cannot run — no Docker, no reachable daemon, or
|
||||||
|
# no way to become root (rig db requires root) — so it never reddens a dev box
|
||||||
|
# that simply has no Docker. On ubuntu-latest CI, Docker is preinstalled and
|
||||||
|
# running and passwordless sudo works, so it EXECUTES for real there.
|
||||||
|
set -euo pipefail
|
||||||
|
|
||||||
|
ROOT="$(cd "$(dirname "${BASH_SOURCE[0]}")/.." && pwd)"
|
||||||
|
RIG="$ROOT/bin/rig"
|
||||||
|
PG_IMAGE="${RIG_DBIT_PG_IMAGE:-postgres:16-alpine}"
|
||||||
|
|
||||||
|
PASS=0 FAIL=0
|
||||||
|
ok() { echo "ok: $*"; PASS=$((PASS + 1)); }
|
||||||
|
bad() { echo "FAIL: $*"; FAIL=$((FAIL + 1)); }
|
||||||
|
skip() { echo "skip: $*"; exit 0; }
|
||||||
|
die() { echo "FAIL: $*" >&2; exit 1; } # trap still runs cleanup
|
||||||
|
|
||||||
|
# --- privilege / docker preflight -------------------------------------------
|
||||||
|
# rig db requires root (require_common_guards). CI's runner user is not root but
|
||||||
|
# has passwordless sudo and docker-group access; mirror that exactly. EVERYTHING
|
||||||
|
# that touches Docker or rig goes through as_root so containers and the artifact
|
||||||
|
# share one owner and cleanup is uniform (root can always reach the socket).
|
||||||
|
if [ "$(id -u)" -eq 0 ]; then
|
||||||
|
as_root() { "$@"; }
|
||||||
|
elif sudo -n true >/dev/null 2>&1; then
|
||||||
|
as_root() { sudo "$@"; }
|
||||||
|
else
|
||||||
|
skip "not root and no passwordless sudo — rig db requires root; cannot run the round-trip here"
|
||||||
|
fi
|
||||||
|
|
||||||
|
command -v docker >/dev/null 2>&1 || skip "docker not installed — nothing to exercise"
|
||||||
|
as_root docker info >/dev/null 2>&1 || skip "docker daemon not reachable — skipping the live round-trip"
|
||||||
|
# Pull up front so an offline box SKIPS (not FAILS): a missing network is not a
|
||||||
|
# regression in rig. On CI the image is fetched here and the run below is fast.
|
||||||
|
as_root docker pull "$PG_IMAGE" >/dev/null 2>&1 || skip "could not pull ${PG_IMAGE} (offline?) — skipping"
|
||||||
|
|
||||||
|
# --- unique names + guaranteed cleanup --------------------------------------
|
||||||
|
PREFIX="rig_dbit_$$"
|
||||||
|
SRC="${PREFIX}_src"
|
||||||
|
DST="${PREFIX}_dst"
|
||||||
|
WORKDIR="$(mktemp -d)"
|
||||||
|
ARTIFACT="$WORKDIR/roundtrip.sql.gz"
|
||||||
|
|
||||||
|
cleanup() {
|
||||||
|
as_root docker rm -f "$SRC" "$DST" >/dev/null 2>&1 || true
|
||||||
|
as_root rm -rf "$WORKDIR" >/dev/null 2>&1 || true
|
||||||
|
}
|
||||||
|
trap cleanup EXIT
|
||||||
|
|
||||||
|
# A stale run must never collide with this one.
|
||||||
|
as_root docker rm -f "$SRC" "$DST" >/dev/null 2>&1 || true
|
||||||
|
|
||||||
|
# --- helpers ----------------------------------------------------------------
|
||||||
|
# Poll until <container> answers a real query as <user>/<db>. pg_isready alone
|
||||||
|
# reports "ready" during the entrypoint's temp-server phase, so gate on an
|
||||||
|
# actual SELECT succeeding instead. Bounded (~60s) so a wedged box fails, loudly.
|
||||||
|
wait_ready() { # <container> <user> <db>
|
||||||
|
local c="$1" u="$2" d="$3" i=0
|
||||||
|
while [ "$i" -lt 60 ]; do
|
||||||
|
if as_root docker exec "$c" psql -U "$u" -d "$d" -tAc 'SELECT 1' >/dev/null 2>&1; then
|
||||||
|
return 0
|
||||||
|
fi
|
||||||
|
i=$((i + 1))
|
||||||
|
sleep 1
|
||||||
|
done
|
||||||
|
die "container ${c} never started accepting connections as ${u}/${d}"
|
||||||
|
}
|
||||||
|
|
||||||
|
# An ordered, checksummable fingerprint of the fixture table — the thing that
|
||||||
|
# must survive the round-trip byte for byte.
|
||||||
|
fingerprint() { # <container> <user> <db>
|
||||||
|
as_root docker exec "$1" psql -U "$2" -d "$3" -tAc \
|
||||||
|
"SELECT id||'|'||name||'|'||qty FROM widgets ORDER BY id" | md5sum | cut -d' ' -f1
|
||||||
|
}
|
||||||
|
rowcount() { # <container> <user> <db>
|
||||||
|
as_root docker exec "$1" psql -U "$2" -d "$3" -tAc 'SELECT count(*) FROM widgets' | tr -d '[:space:]'
|
||||||
|
}
|
||||||
|
|
||||||
|
# --- source: NON-default superuser, seeded fixture --------------------------
|
||||||
|
as_root docker run -d --name "$SRC" \
|
||||||
|
-e POSTGRES_USER=src_super -e POSTGRES_DB=src_appdb -e POSTGRES_PASSWORD=srcpw \
|
||||||
|
"$PG_IMAGE" >/dev/null
|
||||||
|
wait_ready "$SRC" src_super src_appdb
|
||||||
|
|
||||||
|
as_root docker exec "$SRC" psql -U src_super -d src_appdb -v ON_ERROR_STOP=1 -c "
|
||||||
|
CREATE TABLE widgets (id int PRIMARY KEY, name text NOT NULL, qty int NOT NULL);
|
||||||
|
INSERT INTO widgets VALUES (1,'alpha',10),(2,'beta',20),(3,'gamma',30);
|
||||||
|
" >/dev/null
|
||||||
|
|
||||||
|
# assert_eq <desc> <want> <got>
|
||||||
|
assert_eq() {
|
||||||
|
if [ "$2" = "$3" ]; then ok "$1"; else bad "$1 — wanted [$2] got [$3]"; fi
|
||||||
|
}
|
||||||
|
|
||||||
|
SRC_FP="$(fingerprint "$SRC" src_super src_appdb)"
|
||||||
|
assert_eq "source seeded with 3 known rows" 3 "$(rowcount "$SRC" src_super src_appdb)"
|
||||||
|
|
||||||
|
# --- dump (explicit outfile) ------------------------------------------------
|
||||||
|
if as_root "$RIG" db dump "$SRC" "$ARTIFACT" >/dev/null 2>&1; then
|
||||||
|
ok "rig db dump exited 0"
|
||||||
|
else
|
||||||
|
bad "rig db dump failed"
|
||||||
|
fi
|
||||||
|
if as_root test -s "$ARTIFACT"; then
|
||||||
|
ok "dump wrote a non-empty artifact"
|
||||||
|
else
|
||||||
|
die "dump produced no usable artifact — nothing to restore"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# --- dump (default outfile naming: <container>-<UTC-timestamp>.sql.gz) -------
|
||||||
|
( cd "$WORKDIR" && as_root "$RIG" db dump "$SRC" >/dev/null 2>&1 )
|
||||||
|
if compgen -G "$WORKDIR/${SRC}-*.sql.gz" >/dev/null; then
|
||||||
|
ok "dump with no outfile wrote <container>-<timestamp>.sql.gz in cwd"
|
||||||
|
else
|
||||||
|
bad "default dump name not produced"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# --- destination: DIFFERENT superuser (the portability proof) ---------------
|
||||||
|
as_root docker run -d --name "$DST" \
|
||||||
|
-e POSTGRES_USER=dst_super -e POSTGRES_DB=dst_appdb -e POSTGRES_PASSWORD=dstpw \
|
||||||
|
"$PG_IMAGE" >/dev/null
|
||||||
|
wait_ready "$DST" dst_super dst_appdb
|
||||||
|
|
||||||
|
# --- restore into the default database --------------------------------------
|
||||||
|
if as_root "$RIG" db restore "$ARTIFACT" "$DST" --yes >/dev/null 2>&1; then
|
||||||
|
ok "rig db restore exited 0 across a differing superuser"
|
||||||
|
else
|
||||||
|
bad "rig db restore failed (a role-graph leak would abort here)"
|
||||||
|
fi
|
||||||
|
|
||||||
|
DST_FP="$(fingerprint "$DST" dst_super dst_appdb)"
|
||||||
|
assert_eq "destination has exactly 3 rows after restore" 3 "$(rowcount "$DST" dst_super dst_appdb)"
|
||||||
|
if [ "$DST_FP" = "$SRC_FP" ]; then
|
||||||
|
ok "restored fingerprint matches source ($SRC_FP)"
|
||||||
|
else
|
||||||
|
bad "restored data differs — src=$SRC_FP dst=$DST_FP"
|
||||||
|
fi
|
||||||
|
|
||||||
|
# --- idempotency: --clean --if-exists means a second restore is a no-op-ish --
|
||||||
|
if as_root "$RIG" db restore "$ARTIFACT" "$DST" --yes >/dev/null 2>&1; then
|
||||||
|
ok "second restore exited 0 (dump is --clean --if-exists)"
|
||||||
|
else
|
||||||
|
bad "second restore failed"
|
||||||
|
fi
|
||||||
|
assert_eq "still exactly 3 rows after re-restore (no duplication)" 3 "$(rowcount "$DST" dst_super dst_appdb)"
|
||||||
|
assert_eq "fingerprint stable after re-restore" "$SRC_FP" "$(fingerprint "$DST" dst_super dst_appdb)"
|
||||||
|
|
||||||
|
# --- restore into a NAMED scratch db (the [db] arg / shared-Postgres path) ---
|
||||||
|
# This is exactly the safe manual proof the README documents: restore into a
|
||||||
|
# fresh scratch database rather than over live data.
|
||||||
|
as_root docker exec "$DST" createdb -U dst_super rig_verify >/dev/null
|
||||||
|
if as_root "$RIG" db restore "$ARTIFACT" "$DST" rig_verify --yes >/dev/null 2>&1; then
|
||||||
|
ok "rig db restore into a named [db] exited 0"
|
||||||
|
else
|
||||||
|
bad "restore into named database failed"
|
||||||
|
fi
|
||||||
|
assert_eq "named-database restore reproduced the source fingerprint" \
|
||||||
|
"$SRC_FP" "$(fingerprint "$DST" dst_super rig_verify)"
|
||||||
|
|
||||||
|
echo "---"
|
||||||
|
echo "$PASS passed, $FAIL failed"
|
||||||
|
[ "$FAIL" -eq 0 ]
|
||||||
Loading…
Reference in a new issue