KoutenDB v0.14 Self-Hosted Operations

v0.14 reduces the work required to operate KoutenDB without turning the core database into a cloud-provisioning platform. KoutenDB owns database safety, recovery artifacts, topology validation, and typed execution plans. Cloud, virtualization, or bare-metal systems remain responsible for creating actual CPU, memory, disks, and hosts.

Scope

  1. official versioned linux/amd64 and linux/arm64 images on GHCR;
  2. a single-node self-host bootstrap with persistent storage, TLS/auth, health, and strong-durability defaults;
  3. failure detection and supervised restart integration;
  4. checkpoint creation, strict verification, independent restore verification, retention, and provider-neutral transfer;
  5. staged certificate rotation with preflight and rollback;
  6. safe upgrade through drain, checkpoint, replacement, health verification, resume, and rollback;
  7. capacity history and forecast from bounded operational metrics;
  8. typed scale-plan generation;
  9. immutable plan approval followed by operator-controlled execution.

Instant horizontal autoscaling is not in scope. v0.14 can identify required capacity and safely consume a prepared node, but it does not silently create an EC2 instance, resize a physical server, or execute an arbitrary provisioning shell command.

Safety Invariants

  • topology or binary replacement requires a verified recovery generation;
  • generated plans contain no password, token, secret key, or private key;
  • a plan cannot execute without an exact plan ID and explicit approval;
  • approval does not bypass current drain, epoch, quorum, or fencing checks;
  • failed operations retain a known-good binary/configuration and recovery generation;
  • transferred backups are verified at the destination before success is recorded;
  • health failure and infrastructure shortage are reported separately;
  • process supervision may restart a failed process, but cannot invent physical capacity.

Delivery Stages

Stage 1: Reproducible Image

The repository root Dockerfile builds TLS-enabled koutend, kouten, and koutencli binaries into a non-root runtime image. GitHub Releases publish the same version tag to ghcr.io/puffball1567/koutendb with build provenance.

Stage 2: Single-Node Operations

The self-host bundle generates a pinned deployment, secrets outside the configuration file, a persistent data volume, health checks, monitoring output, and recovery storage. Source compilation is not part of the normal path.

The implementation lives in deploy/self-hosted. It provides TLS/authenticated strong-durability defaults, a non-root read-only container, crash restart, and a bounded systemd health watchdog. The watchdog requires consecutive failures and stops after a configured restart limit rather than hiding a persistent fault in an infinite restart loop.

Stage 3: Recovery And Lifecycle

The self-host operator now automates a drain/snapshot checkpoint quiet window, immutable checkpoint verification, staged provider-neutral transfer, and an independent restore drill without changing the active database. Failed checkpoint creation restores service availability before returning an error, and an exported generation is not published until the restored copy passes an offline segment-aware verification. Each operation emits a bounded JSON record without paths or credentials.

The same operator now supports replacement with an explicitly versioned image or digest. It preflights the target, verifies the recovery checkpoint with both the active and target images, changes the Compose image source of truth, and rolls back to the previous image when post-replacement health fails. Certificate rotation validates the CA, chain, hostname, validity window, private key, and certificate/key match before staging the new files. Failed TLS health checks restore the previous certificate set. Managed PKI, client trust distribution, and fleet-wide rotation remain external operations concerns.

Stage 4: Capacity And Scale Plans

Capacity samples produce a provider-neutral plan containing observed growth, forecast horizon, required bytes/CPU/memory, topology intent, prerequisites, and KoutenDB actions. Terraform, Kubernetes, AWS, GCP, Proxmox, or a human may prepare the resources. KoutenDB applies only its typed portion after approval.

The single-node bundle implements the first typed action, verify-prepared-capacity. It keeps a bounded numeric history, calculates a least-squares disk-growth forecast, adds explicit disk/memory/CPU headroom, and stores a versioned plan whose ID is the SHA-256 digest of its complete content. Approval binds to that exact ID. Execution rechecks identity, expiry, sample freshness, live growth, health, and currently available resources, then records one successful execution. It never calls a provider API or arbitrary operator hook.

The same bundle also provides a daily systemd backup path. One operator lock covers the checkpoint, staged export, independent restore, restored-data verification, publication, and retention sequence. Scheduled checkpoints use a separate internal root so its one-verified-generation retention target cannot delete manual or pre-upgrade checkpoints. Export retention removes only older verified scheduled generations and preserves corrupt evidence for diagnosis.

Release Gate

v0.14 is ready when local and container matrices cover:

  • image startup as a non-root user on both declared architectures;
  • unhealthy process detection and bounded restart behavior;
  • interrupted checkpoint, transfer, restore, upgrade, and certificate rotation;
  • exact plan-ID approval, stale-plan refusal, and repeat execution;
  • rollback after post-replacement health failure;
  • capacity boundary and forecast calculations;
  • redaction of every credential-bearing input;
  • clean removal of test containers, volumes, and images.

The confidentiality audit remains a separate implementation branch and must be reviewed and integrated independently before the v0.14 release branch is cut.

Product Boundary

The official image, basic health/restart integration, local checkpoints, restore verification, typed plans, and explicit approval gates belong to the Apache-2.0 core. A future commercial operations layer may add multi-account cloud provisioning, fleet-wide policy, managed PKI/KMS, cross-region transfer, cost-aware recommendations, multi-operator approval, and a hosted control plane. Core operational safety is not paywalled.