# ColdCTL > Local archival for PostgreSQL, MySQL and MongoDB, with customer-controlled Parquet storage and explicit metadata synchronization to Cloud. Reference for the current CLI 0.1.1 development workspace. Commands and options are checked against the repository CLI; published release packages may lag this reference. Implemented: independently installed signed database connectors, discovery/analysis, retention policies, local/S3/Azure/GCS Parquet export, automatically refreshed AWS role and Azure VM identity credentials, durable checkpoints, verification, recovery/import, same-engine separate-target restore, a local archive scheduler, recurring PostgreSQL/MySQL/MongoDB-to-S3/Azure/GCS archive/delete cycles, explicit verified source deletion, Prometheus metrics, durable failure history, optional webhook alerts and opt-in background one-way Cloud metadata sync. Source-scoped production protection consumes fresh external monitoring samples for lag/load thresholds, UTC windows and adaptive delays. Read-only shadow validation compares saved source ranges and optional gateway ID lookups with durable local evidence. Opt-in per-policy automatic shadow validation refreshes the gateway catalog before scheduled deletion; job/manifest/configuration-bound receipts permit safe continuation of the same deletion after restart. Native metric polling and live application query mirroring are not implemented. Archive-only is the default. The separate customer-hosted gateway retrieves local/S3/Azure/GCS archives with browser filters, record views, bounded CSV/JSONL/Parquet downloads, tenant-scoped access, local audit events and full-object verification. Query processing runs on customer infrastructure; authorized downloads go to users. Built-in PII masking, legal holds and arbitrary nested MongoDB field search are not provided. Not implemented in the Agent workflow: CDC/incremental watermarks, mutable-source snapshots, distributed scheduling or remote Cloud execution. Paged Cloud reports include scheduler state, deletion outcomes, connector inventory and active monitor incidents, without raw errors or customer records. Never put passwords in --url. Use credential environment references. Source-stability flags are operator assertions. Verification does not by itself authorize deletion or prove historical authenticity. All links are relative to the website origin. Examples contain placeholders such as JOB_ID and console.example.com; replace these with your own values. ## Your first archive Website: /docs Configure a source, preview a policy, run a bounded export and verify it. ### Before you start This reference covers the current 0.1.1 development workspace. Download a verified Windows x64 or Linux x64 release, then install an approved connector for your database. The base CLI does not include database drivers. Match the connector version to available signed release metadata; local development may be ahead of published assets. The walkthrough uses PostgreSQL public.orders with a single integer primary key and created_at timestamp. MySQL and MongoDB have the limits described in Supported databases. ```text coldctl --version coldctl --help ``` ### 1. Initialize local state init creates or upgrades local SQLite state. Use the same --data-dir for every command if you override the OS default. Stop existing workers before migrations. ```text coldctl init coldctl status ``` ### Install the database connector Obtain root.json and its independently verified SHA-256 fingerprint from your release trust process. Replace VERIFIED_FINGERPRINT and RELEASE_VERSION. Download verification and offline installation are covered in Connector installation. ```text coldctl connector registry --url https://github.com/coldctl/coldctl/releases/latest/download/ --root root.json --root-sha256 VERIFIED_FINGERPRINT coldctl connector install postgres --version RELEASE_VERSION coldctl connector doctor ``` ### 2. Register and inspect a source Set ARCHIVE_DB_PASSWORD through your environment or secret manager. Do not put a password in --url. The PostgreSQL role needs only the source privileges described in your deployment setup; source discovery/export are read-only. ```text coldctl source add postgres --name production --url "postgresql://archive_reader@db.example.com:5432/app" --password-env ARCHIVE_DB_PASSWORD coldctl source test production coldctl source discover production coldctl analyze production ``` ### 3. Choose local storage The directory is customer-controlled. destination test creates it if missing and performs a temporary write/read/delete probe. S3, Azure Blob and Google Cloud Storage are also supported; see Customer-owned archive destinations and Cloud storage authentication. Use the chosen destination name in your policy. ```text coldctl destination add local --name local-archive --path ./archives coldctl destination test local-archive ``` ### 4. Define and preview the policy This example selects records older than 365 days whose status is completed. Saving a policy does not validate the table. Planning reads metadata and estimates; it does not run exact counts or create archive files. ```text coldctl policy create --name orders-archive --source production --destination local-archive --schema public --table orders --time-column created_at --older-than-days 365 --equals-column status --equals-value completed --batch-size 500 coldctl policy validate orders-archive coldctl archive plan orders-archive ``` ### 5. Export a bounded trial Only assert immutable-rows when both row values and eligibility are stable, including no backdated inserts. The 1000-row invocation budget may leave a paused job; inspect its returned status. The source records are retained. ```text coldctl archive run orders-archive --source-stability immutable-rows --max-rows 1000 coldctl jobs list ``` ### 6. Resume and verify Replace JOB_ID with the job UUID from jobs list. For a paused/incomplete job, first confirm the original stable source, then resume. --max-rows 0 removes the saved row budget. Verify requires a completed job. ```text coldctl jobs show JOB_ID coldctl archive resume JOB_ID --confirm-original-source --max-rows 0 coldctl archive verify JOB_ID ``` ### 7. Observe in Cloud Create a lifecycle token in Cloud Settings / Agent tokens and supply it as COLDCTL_CLOUD_TOKEN. Inspect the dry run before upload. Use your actual HTTPS Cloud origin; refresh the console after sync. No database rows are uploaded. ```text coldctl agent sync --dry-run coldctl agent sync --cloud-url https://console.example.com ``` ### Continue with long-running work Use the durable local scheduler for an archive that needs multiple invocations. Add --delete-after-archive only when you explicitly authorize verified source deletion and have granted its required permissions. Archive-only is the default. ```text coldctl agent scheduler enqueue orders-archive --source-stability immutable-rows --confirm-original-source coldctl agent scheduler run ``` ## Installation & local state Website: /docs/concepts/installation-state The installation identity, SQLite journal and state directory form the basis of recovery. ### One installation, one state directory init creates an installation UUID and migration-managed SQLite state. The default is %LOCALAPPDATA%\coldctl\data on Windows, ~/Library/Application Support/com.coldctl.coldctl on macOS, or $XDG_DATA_HOME/coldctl (otherwise ~/.local/share/coldctl) on Linux. Use --data-dir PATH globally for isolation. status is read-only. ```text coldctl --data-dir ./test-state init coldctl --data-dir ./test-state status ``` ### Preserve the recovery set Keep the state directory, archive files and referenced staging files together. Per-job and per-policy OS locks coordinate workers sharing this state. Do not delete lock files while processes run, use copied state concurrently against the same archive, or mix old and new binaries. ### Back up safely state backup uses the SQLite backup API and includes committed WAL changes. It excludes archive files and environment credentials. For a consistent state-and-files set, stop all writers, create the backup and copy destination files while they remain stopped. Restore a trusted backup into a fresh empty state directory, not over a live database. ```text coldctl state backup --to ./backups/state.db --output json ``` ### Upgrades and service accounts The current local schema is version 17, including remote destinations and recurring schedules. Stop workers and run coldctl init before using new commands. Preserve state backups and remote archive objects. A service must use the same --data-dir, connector installation, identity or environment credentials, and storage permissions as the operator. Scheduling coordinates one machine, not a distributed fleet. ## Sources & credentials Website: /docs/concepts/sources-credentials Configure PostgreSQL, MySQL and MongoDB with local secret references and verified TLS. ### Connection choices Install the matching postgres, mysql or mongodb connector before connecting. Use a password-free --url plus --password-env, or --url-env referencing a complete environment URL. Credentials resolve only at connection time. Local add/list/show/remove operations do not connect. Protect configuration metadata in local state. ```text coldctl source add mysql --name mysql-source --url mysql://archive_reader@db.example.com:3306/app --password-env MYSQL_PASSWORD --tls-ca-env DATABASE_CA_PEM coldctl source add mongodb --name mongo-source --url mongodb://archive_reader@db.example.com:27017/app --password-env MONGO_PASSWORD --tls-ca-env DATABASE_CA_PEM ``` ### TLS and secrets Verified TLS is the default. --tls-ca-env references PEM CA text for MySQL or MongoDB, not a filename. PostgreSQL uses the platform trust store. URLs support sslmode=require|disable; use disable only for intentional local testing. MongoDB connects directly to one replica-set primary and authenticates against the configured database. Multi-host URLs, SRV URLs and automatic TLS downgrade are unsupported. ### Discovery versus analysis SQL discovery and analysis use catalogs, existing statistics and metadata estimates. MongoDB discovery may sample documents to identify fields and eligible shapes; it is not proof that every document is supported. Export validates each record. Estimates can be stale or unknown and are not exact eligible counts. None of these operations establishes source stability. ### Database credentials and cloud storage identities Database connections continue to use verified TLS and password environment references. AWS role authentication and Azure managed identity apply to storage in the agent and gateway, not to PostgreSQL/MySQL/MongoDB authentication. Cloud portal lifecycle tokens and gateway tenant bearer tokens are separate credentials. ## Datasets & retention policies Website: /docs/concepts/policies-datasets An explicit time cutoff and optional equality filter define what is eligible. ### Selection rule A policy references a saved source, destination, schema/database, table/collection, time field, retention age and batch size. Selection is strict time < cutoff. SQL connectors also accept one paired text equality predicate; MongoDB does not support equality filters. NULL or missing retention values are not eligible. Arbitrary SQL predicates are not accepted. ### Fixed bounds Each archive fixes a UTC cutoff and upper primary-key bound, then advances by key instead of OFFSET. A later run gets its own bounds and can overlap earlier exports; no incremental watermark or cross-policy deduplication is implemented. Cloud Datasets derives from saved policies rather than a complete database catalog. ### Supported export shape PostgreSQL supports ordinary non-inherited tables with a single smallint/integer/bigint key and up to 1,000 rows per batch. MySQL 8.4 InnoDB supports a single integer key, including unsigned 64-bit keys, and up to 400 rows. MongoDB 8.0 supports ordinary replica-set collections with ObjectId keys, a top-level BSON Date retention field and up to 100 documents. Engine-specific type, topology and byte limits apply; unsupported or oversized records fail without truncation. ## Parquet & the manifest Website: /docs/concepts/archive-format Each job produces bounded Parquet objects and a manifest with integrity metadata. ### Archive layout Each job publishes Parquet batches and a manifest under its job UUID. Local files are staged, synced and published without overwriting existing targets. Remote destinations use durable local pending staging, create-only uploads and full SHA-256 readback; the completed manifest is published last. It records the policy, cutoff, schema, key bounds, counts, object sizes and hashes. Job byte counters exclude manifest bytes. ### Type representation PostgreSQL format 2 uses native Parquet integer, boolean and float columns, with exact numeric/date/UUID/JSON representations stored as UTF-8 plus type metadata. MySQL format 3 preserves native scalar representations as text, including unsigned integers, exact decimals and hex binary. MongoDB format 4 stores an ObjectId envelope and hex-encoded raw BSON, preserving types, field order and binary data rather than flattening documents. These archives are not interchangeable cross-engine tables. ### Integrity and trust verify checks the completed archive against its local journal. verify-directory checks a standalone manifest and its files. A consistent checksum proves consistency with those metadata, not historical source authenticity. Importing untrusted files cannot turn them into authenticated source evidence. ### Export and deletion are separate records The manifest is immutable and describes export. Its source_deleted: false field remains unchanged after source deletion. Use archive deletion-status for the local deletion checkpoint. Keep state backups as well as archive files; imported archives cannot authorize deletion. ## Jobs, checkpoints & recovery Website: /docs/concepts/jobs-recovery Durable checkpoints allow explicit recovery without silently repeating committed batches. ### State and progress Jobs may be running, paused, completed, failed or cancelled. Counters include committed Parquet rows/bytes only. A running record can outlive a crashed process. Progress timestamps and historical rates do not prove liveness, and planner estimates are not percentages or ETAs. ### Resume versus a new run archive resume and jobs retry keep the same job, fixed bounds, schema and object names. They verify committed objects and reconcile prepared batches before continuing. New runs after prior work require --allow-repeat and can duplicate records. No cross-policy deduplication is promised. ### Safe stopping Row/time budgets and process interruption request a pause at a safe boundary. jobs cancel requests cancellation after the current batch. Files are preserved. There is no jobs pause command. To continue an incomplete job, validate the original source and pass --confirm-original-source. ### Failure investigation Use jobs show to inspect safe failure category, retryability and next_step. Fix permissions, storage, schema or integrity problems before retrying. A smaller batch does not fix an oversized row. Never discard a checkpoint to skip data. Preserve corrupt files for investigation. ```text coldctl jobs show JOB_ID --output json coldctl jobs retry JOB_ID --confirm-original-source ``` ## Execution controls Website: /docs/concepts/execution-controls Bound source load and invocation length without confusing a budget with completion. ### Invocation budgets --max-rows and --max-duration-seconds apply per invocation. Time is checked at safe boundaries, so an in-flight batch may finish after the requested duration. Zero clears a saved budget on resume/retry; unspecified controls preserve the saved value. Duration is limited to one year. ### Load and capacity --batch-delay-ms is 0–3,600,000 (default 0). --query-timeout-ms defaults to 10,000; --lock-timeout-ms defaults to 2,000. Both accept 1–300,000 ms and lock timeout must not exceed query timeout. --min-free-bytes defaults to 1 GiB and must be positive; disk checks are point-in-time, not space reservations. ### Bounded example The command below limits each invocation and adds a delay between batches. Inspect returned status and durable counters before deciding whether to resume. ```text coldctl archive run orders-archive --source-stability quiescent-copy --max-rows 10000 --max-duration-seconds 300 --batch-delay-ms 100 --query-timeout-ms 10000 --lock-timeout-ms 2000 ``` ## Restore & value validation Website: /docs/concepts/restore Restore verified archives into a separate target using the matching database connector. ### Same engine, separate target Register a target with source add postgres, mysql or mongodb matching the archive engine, and confirm separation explicitly. The target database must exist; SQL target schemas must also exist. The command verifies the archive before writing, then creates an isolated target and recovery journal. It refuses unrelated existing targets. ```text coldctl archive restore JOB_ID --target restore-target --schema restore_database --table restored_orders --confirm-separate-target ``` ### Transactional recovery Restore batches and their recovery checkpoints commit together using native engine transactions. Rerunning the identical command validates existing committed values and resumes. Preserve the companion SQL journal or MongoDB restore ledger. A --max-batches pause is successful but does not mean restore is complete. ### What is not restored Restore preserves supported archived values and key representations; it is not a complete application DDL backup. Defaults, sequences, secondary indexes, triggers, grants and application migrations need separate handling. MongoDB retains raw BSON but does not reconstruct collection-level application settings and indexes. Cross-engine restoration is unsupported. ### Validate values validate-restore compares committed values against the archive and can inspect partial restores. It does not establish equality with a source changed since export. Use the matching engine connector and its target privileges. ```text coldctl archive validate-restore JOB_ID --target restore-target --schema restore_database --table restored_orders --confirm-separate-target ``` ## Cloud synchronization Website: /docs/concepts/cloud-sync One-way metadata reports connect your local lifecycle to the Cloud console. ### Authentication Create a lifecycle token in the intended workspace, then supply its secret through COLDCTL_CLOUD_TOKEN. Cloud stores a hash and validates token expiry, revocation and creator membership. Legacy scan tokens and persisted tokens without a scope cannot sync lifecycle metadata. Legacy scan endpoints have been removed. Previously received reports remain after revocation. ### Upload boundary The allowlist includes installation ID/version, source/destination names/types, policy retention metadata and job counters/stages/verification times. It excludes endpoints, passwords, credential environment references, filter values, filesystem paths, row keys, row contents and raw errors. Metadata names can still be sensitive; inspect --dry-run before selecting a destination. ### Report semantics Sync atomically replaces one installation report within the token-derived workspace. Collection reads local tables sequentially, not in one SQLite snapshot. Cloud is an observer, not a worker controller. Refresh after sync. Completed exports can overlap because of repeats/imports; totals are not unique data or reclaimed capacity. ### Limits and retries Schema-v2 supports up to 1,024 immutable metadata pages / 102,400 records, with 100 records, 40 KiB and 24,000 UTF-16 units per page. A commit atomically publishes the complete report; interrupted uploads leave the previous report visible. Oversized reports fail without truncation. Page and commit retries are idempotent; newer reports win. Uploads expire after one hour, and clocks must be accurate. Deploy Cloud v2 before updating agents. Background retries collect fresh pages. ### Transport HTTPS with certificate/hostname verification is required. Redirects are rejected. Connection timeout is 10 seconds; the request timeout is 30 seconds. Only --allow-local-http allows numeric loopback HTTP for local test harnesses. ```text coldctl agent sync --dry-run coldctl agent sync --cloud-url http://127.0.0.1:5180 --allow-local-http ``` ### Scheduler and deletion visibility Schema-v2 reports queue retries, recurring schedules and their local deletion authorization, deletion completion, confirmed deleted rows separately from absent or uncertain outcomes, connector versions and active monitor incidents. Failed source jobs remain visible through job status. Cloud observes these states; it does not authorize deletion or execute agent commands. Counts do not establish reclaimed disk space. Raw errors and filter values stay local. ### Run background synchronization Run sync --watch as a separate supervised process using the scheduler data directory. Successful uploads repeat every 300 seconds by default (30–86400 configurable). Failures back off from 30 seconds to one hour; next attempt, last success and sanitized failures persist in cloud-sync.sqlite. Each retry collects fresh metadata; intermediate states may be coalesced. One-shot and background uploads share a per-state lock. Ctrl+C stops the loop; restart after changing token environment variables. The agent monitor exposes sync failure count and last-success time and can send cloud_sync alerts. External monitoring is needed for a stopped process. Existing payload limits still apply, with no silent truncation. ```text coldctl agent sync --cloud-url https://console.example.com --watch --interval-seconds 300 coldctl agent sync-status ``` ### Paged dashboard and cleanup Load next page retrieves bounded report fragments. Totals describe only loaded metadata until all pages are read. Refresh if a report changes during traversal. Reports older than 15 minutes are marked stale; this is not a worker liveness check. Best-effort cleanup during sync removes abandoned or superseded pages older than 24 hours while retaining active reports. Retired installations require operator cleanup. Dry-run emits an array of schema-v2 pages. ```text coldctl agent sync --dry-run coldctl agent sync --cloud-url https://console.example.com --watch ``` ## CLI conventions Website: /docs/concepts/cli-conventions Global flags, output streams, identifiers and placeholder syntax. ### Global options --data-dir PATH selects local state for any command. --help prints usage for a command. --version prints the CLI version at the root. coldctl help COMMAND also displays help. init/status have no --output option. agent sync --dry-run emits JSON; its successful upload emits a human-readable message. ```text coldctl --help coldctl help archive run coldctl --version ``` ### Structured output Commands that expose --output accept table|json. Scheduler commands and connector inventory operations emit their documented structured results without a universal --output flag. Progress and diagnostics go to stderr. Errors exit nonzero; a bounded pause can exit successfully, so inspect the returned status. Exact flags for each command appear in the command reference. ```text coldctl jobs show JOB_ID --output json ``` ### Example notation Replace JOB_ID with an actual job UUID and names such as production or orders-archive with local configuration names. Paths are examples; create required parent directories. Commands are shown one per line without shell-specific continuation characters. Supply secrets via the environment or a secret manager, not command-line URLs. ### Availability Implemented: optional PostgreSQL/MySQL/MongoDB connectors, local/S3/Azure/GCS archival, checkpoint recovery, same-engine separate-target restore, verified deletion, a durable scheduler and explicit recurring PostgreSQL/MySQL/MongoDB-to-S3/Azure/GCS archive/delete cycles. The separate customer-hosted gateway retrieves supported archives. CDC, mutable-source snapshots, distributed scheduling remain outside this workflow; optional agent sync --watch provides background metadata uploads. Use agent scheduler run for the local worker. ## Cloud HTTP API Website: /docs/concepts/cloud-api Lifecycle upload and console endpoints, authentication and response codes. ### Base URL and authentication Use your Cloud website origin followed by /api. Browser endpoints use the signed-in Static Web Apps session and verify workspace membership. Upload uses Authorization: Bearer with a lifecycle token. Do not send database credentials or forge browser identity headers. Direct Function-host access is not the supported production route. ### POST /api/agents/sync Lifecycle bearer token required. Workspace is derived from the token, never from the payload. Accepts exactly the schema-v1 allowlisted fields shown by coldctl agent sync --dry-run. The example is a report from an empty installation; replace its UUID and capturedAtUnixMs with actual values. For populated reports, use the agent serializer rather than constructing job records manually. ```json { "schemaVersion": 1, "agentId": "11111111-1111-4111-8111-111111111111", "agentVersion": "0.1.0", "capturedAtUnixMs": 1791200000000, "sources": [], "destinations": [], "policies": [], "jobs": [] } ``` ### Upload responses 200: {"accepted":true}. 400: invalid schema, fields, types, counts, size or future clock. 401: invalid token, wrong scope, expiry, revocation or lost creator membership. 409: older/duplicate report or concurrent update; collect again. A gateway may also reject an oversized HTTP body with 413. Network/5xx failures do not roll back or modify local archives. ### GET /api/agents?workspaceId=WORKSPACE_ID Signed-in browser member required. Returns an array of {receivedAt, report}; report uses the upload schema. 401 means no session; 403 means no workspace membership; 400 means invalid/missing workspace ID; 413 means the listing exceeds 100 installations. Response is not cached. ### POST /api/tokens Signed-in workspace member required. Creates a token; plaintext is returned only once. Only lifecycle scope is supported; omitting scope defaults to lifecycle. Existing legacy tokens are not upgraded. Optional expiresAt must be in the future. Store returned secrets securely; do not include them in bug reports. ```json { "workspaceId": "WORKSPACE_ID", "name": "test-installation", "scope": "lifecycle" } ``` ### GET /api/tokens?workspaceId=WORKSPACE_ID Signed-in workspace member required. Returns id, name, prefix, scope, createdAt, lastUsedAt, expiresAt and revoked. Plaintext tokens are never returned by this list endpoint. ### DELETE /api/tokens/TOKEN_ID?workspaceId=WORKSPACE_ID Signed-in workspace member required. Revokes an existing credential. Success returns {"message":"Token revoked."}. Existing reports remain available; future authentication with the revoked token fails. ### Legacy routes Legacy source, scan, finding, access and overview APIs have been removed. Database metadata comes from agent lifecycle reports. This reference documents the lifecycle integration. No HTTP endpoints start, resume, cancel, restore or delete database data. ## Long-running archive scheduler Website: /docs/concepts/scheduler Queue bounded archival work, resume checkpoints after restart and inspect persistent retry state. ### Enqueue and run Initialize state, configure the source/destination/policy, then enqueue it. Enqueue returns a queue ID; a job ID is assigned after preflight. The original source and eligible rows must stay stable through all slices, downtime and retries. MongoDB requires quiescent-copy. ```text coldctl init coldctl agent scheduler enqueue orders-archive --source-stability immutable-rows --confirm-original-source --max-duration-seconds 300 --max-rows 10000 coldctl agent scheduler run ``` ### Local execution model One foreground worker per state directory claims durable SQLite queue entries and processes one job at a time. The default slice is 60 seconds; a positive time slice is required. Budgets are checked between batches after preflight/verification, not hard wall-clock deadlines. Paused slices requeue after one second. --delay-seconds delays the first execution, while --once executes at most one due slice and exits. ### Adopt interrupted work Use --job to adopt an incomplete original job for the same policy. This preserves its cutoff, upper key, schema, connector pin and files. Saved execution controls remain except the default time slice becomes 60 seconds; explicit flags override them. The supplied stability flag cannot replace the job safety contract. Completed, imported and cancelled jobs cannot be adopted. Only one active queue entry per policy is allowed. ```text coldctl agent scheduler enqueue orders-archive --job JOB_ID --source-stability immutable-rows --confirm-original-source ``` ### Inspect, cancel and retry Queue IDs and job IDs are different. Queue output is JSON and due_at is a UTC Unix timestamp. Cancellation stops future batches; the in-flight batch may commit. Cancelled entries are terminal. Failed entries can be retried after fixing their cause. Transient failures back off from 30 seconds, doubling to at most one hour; the default is five consecutive retries, maximum 100. A successful slice resets that count. Permanent failures require intervention. ```text coldctl agent scheduler list coldctl agent scheduler show QUEUE_ID coldctl jobs show JOB_ID coldctl agent scheduler cancel QUEUE_ID coldctl agent scheduler retry QUEUE_ID --confirm-original-source ``` ### Optional deletion --delete-after-archive explicitly authorizes deletion of verified unchanged records. It is off by default. The worker verifies the completed export and deletes one original archive batch per turn, saving separate deletion progress. The queue completes only when both phases complete. Previously committed deletions remain if cancelled or a later batch fails. ```text coldctl agent scheduler enqueue orders-archive --source-stability immutable-rows --confirm-original-source --delete-after-archive ``` ### Operate as a service Use an OS service manager to keep agent scheduler run alive, supplying the same state path, environment secrets and filesystem permissions. Ctrl+C, Unix SIGTERM or Windows Ctrl+Break requests a safe stop. Restart recovers claimed entries and resumes checkpoints. Preserve lock files. The scheduler does not install a service or sync Cloud automatically. Explicit recurring schedules create fresh PostgreSQL/MySQL/MongoDB cycles to S3, Azure Blob or GCS only after completed verified deletion; they rescan eligible survivors to include late arrivals between cycles. MongoDB must remain quiescent during each cycle. Ordinary repeated exports may duplicate data. ## Verified source deletion Website: /docs/concepts/verified-deletion Explicitly remove unchanged archived records with atomic batches and durable local deletion progress. ### Authorize a completed original archive Upgrade state and install an approved connector advertising verified-delete-v1. The archive must be completed, locally created and retain its original safety contract. Imported archives cannot authorize deletion. Both confirmations are required; verification alone never enables deletion. ```text coldctl archive delete JOB_ID --confirm-delete --confirm-original-source --max-batches 1 coldctl archive deletion-status JOB_ID ``` ### Verification and comparison The host verifies the complete archive before connecting, then rechecks each batch and decodes the exact bytes whose digest was verified. The connector validates original source and dataset identity and compares each existing record to the archive inside a transaction. Any mismatch rolls back that batch, including earlier deletes in it. Already absent keys are counted. Earlier committed batches remain deleted after a later failure. ### PostgreSQL permissions and locks Use the original source role with schema USAGE and table SELECT/DELETE. Each batch, up to 1,000 rows, holds SHARE ROW EXCLUSIVE, blocking writers while allowing ordinary reads. RLS, inheritance, partitions, user triggers, rules and incoming foreign keys are refused. The original endpoint/user, database and table identity checks still apply. ### MySQL permissions and locks MySQL 8.4 InnoDB supports up to 400 rows per batch and preserves full unsigned integer keys. The role needs SELECT, DELETE, LOCK TABLES, direct TRIGGER privilege and global PROCESS for identity and foreign-key inspection. TRIGGER must be granted directly at table, database or global scope; role-only grants and partial revokes are refused. Tables with triggers or incoming foreign keys, including hidden cross-database child tables, are refused. A WRITE table lock blocks readers and writers for the batch; errors roll back before unlocking. ### MongoDB transactions MongoDB 8.0 requires a replica-set primary and the quiescent-copy contract. Add remove permission on the source collection to the connector discovery/read permissions. Each batch, up to 100 documents, compares exact raw BSON in a snapshot transaction with majority commit. ObjectId, deployment identity, collection UUID and options are checked. Changed or oversized documents fail the batch; concurrent conflicting writes abort the transaction. Standalone servers, routers and unsupported collection shapes are refused. ### Restart and uncertain commits The source commits before the local SQLite checkpoint. If acknowledgement or the process is lost, retry the same job: missing keys count as absent and remaining records must still match. rows_processed is reconciled archive rows; rows_deleted is acknowledged deletions; rows_absent includes missing rows and unrecorded source commits. Do not reuse or reinsert archived keys during deletion/retries; identical reinserts cannot be distinguished from originals. Repeat with --max-batches to bound work, or omit it to finish. Ctrl+C stops between batches. ### Completion and Cloud A completed deletion is terminal and does not delete later inserted records. Archive files and the export manifest stay unchanged; source_deleted: false describes export, not subsequent deletion. deletion-status returns null before a checkpoint exists. Keep local state with your archive backups. Deletion progress is not currently uploaded to Cloud; inspect it locally. ## Connector architecture Website: /docs/concepts/connector-architecture The local host owns recovery and storage; supervised native connectors own database-specific operations. ### Host and connector responsibilities The base CLI and core own policies, SQLite jobs and queues, execution budgets, checkpoints, Parquet encoding, publication and verification. Local, S3, Azure Blob and GCS destinations are built-in adapters behind a shared sink interface. Separately installed postgres, mysql and mongodb executables own database drivers, discovery, planning, source identity, value encodings, restore transactions and verified deletion. Sink adapters are not separately downloadable connector packages. ### Data and control flow Database records travel over supervised local process pipes to the host and into customer-controlled Parquet storage. Restore and explicitly authorized deletion use the same connector boundary. The package registry distributes code and trust metadata, not customer rows. Cloud receives only the allowlisted metadata sent by agent sync. ```text Customer database <-> native connector <-> local host | +-> SQLite jobs / queue / checkpoints +-> Parquet files / immutable manifest +-> explicit metadata sync -> Cloud ``` ### Versioned protocol and failure handling The runtime negotiates protocol version, required features and source/restore capabilities before configuration. Framed requests carry session/request identity and bounded deadlines; payload sizes and integrity are checked. stdout is protocol traffic and stderr is handled separately. Crashes, mismatched responses and timeouts end the session; recovery uses durable checkpoints rather than guessing whether a write committed. verified-delete-v1 is an optional negotiated feature required by deletion. ### Pinning and independent upgrades An archive records the connector identity, version and binary digest so resume can use its exact approved package. Changing the active connector affects new sessions, not pinned incomplete jobs. Package leases and resumable-job references prevent unsafe removal. Verified deletion may use a newer approved connector with the deletion feature while rechecking the original source and dataset identity. ### Trust boundary Connectors execute native code as the agent account. Separate processes isolate failures but are not a security sandbox. The installer checks signed TUF metadata, pinned root trust, package hashes, compatibility and revocation state before credentials are released. Use only trusted packages and restrict the service account. Normal installation does not require development bypass environment variables. ## Install and update connectors Website: /docs/concepts/connector-installation Bootstrap independent trust, install exact signed versions and retain packages needed by unfinished jobs. ### Verify the base release Windows x64 and Linux x64 GNU packages are distributed through GitHub Releases. Linux builds target Ubuntu 24.04; older glibc distributions are not qualified. Confirm release provenance and checksums before running binaries. Match release notes to your selected version; this documentation can describe changes ahead of published assets. ```text gh attestation verify SHA256SUMS --repo coldctl/coldctl ``` ### Pin trust and install Obtain the public root.json and independently verify its fingerprint. Do not trust a root just because it was inside a downloaded bundle. Replace VERIFIED_FINGERPRINT and RELEASE_VERSION using verified release information. Select postgres, mysql or mongodb. ```text coldctl init coldctl connector registry --url https://github.com/coldctl/coldctl/releases/latest/download/ --root root.json --root-sha256 VERIFIED_FINGERPRINT coldctl connector list --available coldctl connector install mysql --version RELEASE_VERSION coldctl connector doctor ``` ### Inspect and activate Install selects the package unless --no-activate is supplied. use changes the version for new sessions; a downgrade requires --allow-downgrade. Incomplete jobs retain their connector pins. doctor checks local integrity and known revocations without executing code or downloading. refresh verifies new signed metadata and retains revocation information. ```text coldctl connector inspect mysql coldctl connector refresh coldctl connector use mysql --version RELEASE_VERSION ``` ### Offline repositories Extract connector-registry.zip from the verified release and pass the directory to --file. Trust bootstrap is still mandatory. A bundle is a repository directory, not a native connector executable. Approved installed packages run offline; signed metadata checks still govern installation and refresh. ```text coldctl connector install mysql --version RELEASE_VERSION --file ./connector-registry coldctl connector bundle mysql --version RELEASE_VERSION --out ./offline-bundle ``` ### Retention and release operation Do not remove packages referenced by running sessions or resumable jobs. Published package identities must never be replaced. Version-tagged release automation builds and qualifies packages, checksums and signed registry assets; ordinary commits do not publish releases. Expired metadata or root trust requires a reviewed release/rotation process, never bypassing verification. Use preview bundles in separate local state to avoid changing production rollback state. ## Supported databases and limits Website: /docs/concepts/supported-databases Current PostgreSQL, MySQL 8.4 and MongoDB 8.0 connector scope, with local, S3, Azure Blob and GCS destinations. ### PostgreSQL Supports ordinary non-inherited tables with a single smallint, integer or bigint primary key. Supported scalar values include text, exact numeric, booleans, floats, date/time, UUID and JSON representations. RLS, unsupported types and table shapes fail explicitly. Batch limit is 1,000 rows; transport rows are bounded at 64 KiB. Source SELECT and schema USAGE cover export; deletion adds DELETE and stricter table checks. ### MySQL 8.4 Supports InnoDB with a single supported integer primary key, including unsigned BIGINT through 18446744073709551615. Bounded scalar values include exact decimal, text, binary and temporal types; unsupported schema/types fail. Batch limit is 400 rows and native row payload is capped at 30,000 bytes. SELECT and PROCESS are needed for source reads/identity inspection. Use the database name as --schema. MySQL-compatible variants such as MariaDB are not qualified. ```text coldctl policy create --name mysql-orders --source mysql-source --destination local-archive --schema app --table orders --time-column created_at --older-than-days 365 --batch-size 200 ``` ### MongoDB 8.0 Requires a replica-set primary, ordinary non-capped collections, ObjectId keys, a top-level BSON Date retention field and quiescent-copy throughout archival and resume. Standalone/secondary/router, time-series, capped, clustered and encrypted collections are unsupported. Documents are capped at 30 KiB and batches at 100. Raw BSON is preserved; equality filters and cross-engine restoration are unavailable. The connector needs read/discovery and topology-inspection permissions; read on the database plus clusterMonitor on admin is the fixture role pattern. ```text coldctl policy create --name mongo-events --source mongo-source --destination local-archive --schema app --table events --time-column created_at --older-than-days 365 --batch-size 100 coldctl archive run mongo-events --source-stability quiescent-copy ``` ### Implemented and pending All three connectors support archival, same-engine separate-target restore and explicit verified deletion within their supported shapes. The local scheduler resumes queued work and supports explicit recurring PostgreSQL/MySQL/MongoDB-to-S3/Azure/GCS archive/delete cycles. S3, Azure and GCS sinks and the separate archive gateway are implemented. Mutable-source snapshots, CDC and incremental watermarks remain outside the workflow. Development test coverage does not imply production-scale qualification. ## Customer-owned archive destinations Website: /docs/concepts/archive-destinations Choose local files, S3, Azure Blob or Google Cloud Storage in your infrastructure. ### Configure and test storage Provision the bucket or container separately. Adding a destination saves configuration without connecting. destination test writes, verifies and removes a unique probe. Supported source databases can use one sink, with separate policies and job namespaces. Set the referenced credentials through your environment or secret manager; values are not stored in configuration. ```text coldctl destination add s3 --name aws-archive --bucket YOUR_BUCKET --region ap-south-1 --prefix coldctl coldctl destination test aws-archive coldctl destination add azure --name azure-archive --account YOUR_ACCOUNT --container archives --prefix coldctl coldctl destination test azure-archive ``` ### Credential choices S3 defaults to AWS_ACCESS_KEY_ID and AWS_SECRET_ACCESS_KEY. Temporary static credentials also require --session-token-env AWS_SESSION_TOKEN; this mode does not renew them automatically. Azure defaults to AZURE_STORAGE_ACCOUNT_KEY. Use --role-auth or --managed-identity for automatic cloud credential refresh on supported deployments, as described in Cloud storage authentication. ### Durability and local disk The provider-independent sink interface uses create-only publication and full-object checksum verification. Keep the local state and staging directory on durable storage; pending batches are needed after interruption. --staging-path selects a local directory. Publication is bounded to 128 MiB per object, with batch-level retry rather than multipart resume. Allow adequate disk for pending staging and verified reads. ### Emulators and endpoints Custom HTTPS S3-compatible and Azure endpoints use environment credentials. HTTP requires --allow-http and an explicit loopback endpoint. Cloud identity modes reject all custom endpoints and HTTP. Azurite uses an account-path endpoint; provision fixture storage first. ```text coldctl destination add s3 --name moto --bucket coldctl-test --region us-east-1 --endpoint http://127.0.0.1:19000 --allow-http coldctl destination add azure --name azurite --account coldctltest --container coldctl-test --endpoint http://127.0.0.1:11000/coldctltest --allow-http ``` ### Verification, restore and removal Jobs retain their selected destination configuration for verification, restore and verified source deletion. Downloaded standalone archives are treated as local files; embedded destination settings do not authorize remote access. Removing a destination removes configuration, not archive objects. Keep backups of local state alongside protected remote objects. ### Google Cloud Storage GCS supports archive publication, verification, restore, Gateway retrieval and recurring deletion with automatic shadow gates. Use Google credentials through local environment references. ## Google Cloud Storage archives Website: /docs/concepts/google-cloud-storage Archive to a customer-owned GCS bucket, retrieve through the Gateway, and validate before scheduled deletion. ### Supported workflows Google Cloud Storage (GCS) is an archive destination, not a database connector. Supported PostgreSQL, MySQL and MongoDB sources can use it under their existing source-stability contracts. Archive, resume, verification, restore, Gateway retrieval and recurring archive/delete workflows use the same integrity checks as other sinks. Cloud receives destination metadata, not bucket credentials or archive contents. ### Prepare a private bucket and identity Create the bucket separately. Give the agent object create/get access and delete permission for its unique .coldctl-probe-* access-test objects under the selected prefix. A bucket-scoped Storage Object User role includes these permissions; a custom role can narrow them. Use a separate Gateway identity with object get/list access, such as Storage Object Viewer. Retention locks can prevent probe cleanup and cause destination test to fail. Validate IAM, retention and customer-managed encryption for your deployment. ### Configure the agent Use matching agent and connector builds that include GCS support. Stop workers and run init to upgrade local state to the current schema. Set GOOGLE_APPLICATION_CREDENTIALS in the process environment to an absolute JSON credential-file path, not JSON contents. Configuration stores only references; keep credentials out of source control. This example is PowerShell. ```text $env:GOOGLE_APPLICATION_CREDENTIALS = 'C:\credentials\coldctl-gcs.json' coldctl init coldctl destination add gcs --name gcs-archive --bucket YOUR_BUCKET --prefix coldctl coldctl destination test gcs-archive ``` ### Authentication and durable staging Use --credentials-env COLDCTL_GCS_CREDENTIALS for another environment variable containing an absolute credential-file path. Otherwise the adapter checks GOOGLE_APPLICATION_CREDENTIALS, local gcloud application-default credentials, then attached GCP instance credentials. Service-account and authorized-user JSON files are supported; external_account workload identity federation files are not supported by the pinned SDK. Explicit missing credentials fail without fallback. Keep local state and staging durable across restarts; --staging-path changes the local staging directory. Custom GCS endpoints are limited to anonymous HTTP loopback emulators. ### Configure Gateway retrieval Use the same bucket and prefix as the agent. Save this as coldctl.gcs.toml, provide a private local gateway-layout.json assigning the archives to the customer tenant, and set COLDCTL_GATEWAY_TOKEN in the Gateway environment. Its Google credentials can use a separate read-only identity. Each source dataset must have its own trusted mapping; do not combine incompatible schemas. Restart after changing credential files. ```text state_dir = "gateway-gcs-state" layout_file = "gateway-layout.json" [remote] prefix = "coldctl" [remote.provider] kind = "gcs" bucket = "YOUR_BUCKET" [server] listen = "127.0.0.1:8090" [[users]] name = "operator" token_env = "COLDCTL_GATEWAY_TOKEN" tenants = ["customer"] admin = true ``` ### Verify before enabling deletion Start with archive-only operation on a separate test collection or table. Gateway retrieval supports search, record views, pagination and CSV/JSONL/Parquet exports. Recurring deletion requires explicit authorization and stable eligible source data. Configure per-policy automatic shadow validation before enrollment: it refreshes the Gateway catalog and compares source, archive and Gateway values. Mismatches or Gateway failures block deletion. Use separate refresh and reader credentials. Follow the linked guides for dataset mappings and protection configuration. ### Real-service validation and remaining limits On October 21, 2026, tests against a real GCS bucket and separate Aiven MySQL tables passed archival, verification, Gateway retrieval/export and full shadow comparison. Deliberate source/Gateway mismatches and injected Gateway refresh/lookup 503 and lookup 401 failures blocked deletion with no receipt or deletion journal. After repair and explicit retry, the same jobs validated before deletion. Three continuous scheduled cycles deleted 2, 1 and 0 eligible rows, preserved recent rows, and stayed idle after stopping past the next due time. These were small functional fixtures, not throughput or year-long reliability qualification. Prolonged GCS outage/backoff, workload identity federation, and deployment-specific KMS/retention behavior remain unqualified. Real tests used Google credentials; emulator tests alone cannot qualify GCP safety. ## Cloud storage authentication Website: /docs/concepts/cloud-storage-auth Use refreshed AWS role credentials or Azure VM managed identity in the agent and archive gateway. ### AWS EC2 role Attach an IAM instance profile to the customer VM and allow IMDSv2 access. Configure an existing bucket and region. No static AWS key variables are needed in role mode. When both AWS_WEB_IDENTITY_TOKEN_FILE and AWS_ROLE_ARN are set, web identity takes precedence; the platform must maintain the token file. Incomplete web identity configuration fails closed. IMDSv1 fallback is disabled. ```text coldctl destination add s3 --name aws-archive --bucket YOUR_BUCKET --region ap-south-1 --prefix coldctl --role-auth coldctl destination test aws-archive ``` ### AWS permissions Scope the agent to archive-prefix GetObject and PutObject permissions. destination test additionally needs DeleteObject for unique .coldctl-probe-* keys under that prefix. The gateway needs ListBucket restricted to the prefix and GetObject for archive objects; prefer a separate read-only role. Customer-managed KMS encryption requires corresponding key permissions and key policy. Bucket creation and IAM provisioning are outside the CLI. ### Azure VM identity Enable a system-assigned identity on the VM, or attach a user-assigned identity. Assign Storage Blob Data Contributor to the agent at container scope and Storage Blob Data Reader to the gateway, preferably with a separate identity. Allow role assignments to propagate. --managed-identity-client-id takes the user-assigned identity client UUID, not its object ID. ```text coldctl destination add azure --name azure-archive --account YOUR_ACCOUNT --container archives --prefix coldctl --managed-identity coldctl destination test azure-archive # For a user-assigned identity, add: # --managed-identity-client-id YOUR_CLIENT_UUID ``` ### Gateway settings In [remote.provider], use role_auth = true for S3 or managed_identity = true for Azure. Azure accepts managed_identity_client_id for a user-assigned identity. Match the agent bucket/container, region and prefix. Static environment references are ignored in cloud identity modes. Changing authentication does not change the gateway archive namespace identity. ### Supported identity environments These modes support EC2 IMDSv2/AWS web identity and Azure VM metadata identity. They do not implement shared AWS profiles, AWS SSO, ECS task credentials, arbitrary assume-role chains, Azure CLI login, App Service identity endpoints or sovereign-cloud endpoint overrides. A laptop needs environment credentials unless it is running in a supported identity environment. Custom storage endpoints are rejected in identity modes; private networking can resolve the standard service hostname. ### Remote trial checklist Use verified TLS and database passwords for PostgreSQL and MongoDB. Test each source and destination, then run a bounded export without deletion. Verify the archive, refresh the gateway catalog, retrieve records and validate restoration. Only then authorize verified deletion or recurring archival. Exercise credential expiry and restart on the actual cloud VM. Local mock refresh tests and emulator tests do not verify your IAM/RBAC or network rules. ### Google credentials GCS supports service-account and authorized-user ADC JSON files and attached GCP instance credentials. External-account federation files are unsupported by the pinned SDK. ## Recurring archive and delete Website: /docs/concepts/recurring-archival Run PostgreSQL, MySQL or MongoDB cycles to S3, Azure Blob or GCS, with verified deletion gating each new cycle. ### Enroll a policy Create and test a policy using PostgreSQL, MySQL or MongoDB with S3, Azure Blob or GCS. Recurring enrollment requires explicit source-deletion authorization and an interval from 60 seconds to one year. Use immutable-rows for PostgreSQL only when eligible values and membership stay stable through the cycle; MongoDB requires quiescent-copy. Stability must hold during downtime and retries too. ```text coldctl agent scheduler recurring create pg-policy --every-seconds 86400 --source-stability immutable-rows --confirm-original-source --delete-after-archive coldctl agent scheduler recurring create mongo-policy --every-seconds 86400 --source-stability quiescent-copy --confirm-original-source --delete-after-archive coldctl agent scheduler run ``` ### Cycle boundaries and late arrivals A cycle freezes its retention cutoff and configuration, exports, verifies and deletes verified unchanged records. A new cycle is admitted only after durable evidence of completed deletion. The interval starts after completion is observed, not on a fixed calendar schedule. Each fresh cycle rescans eligible data, including lower-key and backdated arrivals between cycles. This is not CDC or an incremental watermark. Missed intervals coalesce rather than creating a catch-up flood. ### Inspect and recover The worker runs in the foreground; use an OS service manager to keep it running. Ctrl+C stops the worker safely and restart resumes the same saved cycle. A failed cycle holds the schedule; after fixing the cause, recurring retry retries that cycle rather than exporting a new one. MySQL sources are not supported for recurring enrollment. ```text coldctl agent scheduler recurring list coldctl agent scheduler recurring show RECURRING_ID coldctl agent scheduler recurring retry RECURRING_ID --confirm-original-source ``` ### Stop and policy safety recurring stop is terminal, cancels pending work and requests cancellation of an incomplete job. An in-flight batch may commit; already committed deletions remain. A stopped schedule reserves its policy history, so re-enrollment needs a separately reviewed policy. Manual exports for a reserved policy and overlapping policies are not a supported bypass. Use one worker per local state directory; distributed scheduling is not implemented. Run agent sync --watch separately for background Cloud metadata synchronization. ```text coldctl agent scheduler recurring stop RECURRING_ID ``` ## Retrieve archives with the gateway Website: /docs/concepts/archive-gateway Serve verified local, S3, Azure Blob and GCS archives from a customer-hosted Gateway. ### Separate customer-hosted service Run coldctl-archive on your VM with read/list access to the archive namespace and local catalog/staging disk. This service is separate from the agent and Cloud metadata console. It reads completed manifests, verifies full object size/SHA-256 and Parquet metadata, then queries verified temporary files. It does not upload or delete archive objects. Use HTTPS at ingress and a dedicated read-only storage identity. ### S3 role configuration Save this as coldctl.s3.toml, replace YOUR_BUCKET and configure the named gateway token through the environment. The storage role and gateway tenant token serve different purposes. A trusted local gateway-layout.json is required. Azure uses kind = "azure", account, container and managed_identity = true instead of the S3 provider fields. ```text state_dir = "./gateway-state" layout_file = "./gateway-layout.json" [remote] prefix = "coldctl" [remote.provider] kind = "s3" bucket = "YOUR_BUCKET" region = "ap-south-1" role_auth = true [server] listen = "127.0.0.1:8080" [[users]] name = "operator" token_env = "COLDCTL_ARCHIVE_TOKEN" tenants = ["customer"] admin = true ``` ### Map each source dataset For a shared PostgreSQL/MongoDB sink prefix, map job UUIDs explicitly to trusted tenant and dataset names in gateway-layout.json. Replace the placeholders with completed job IDs and extend the layout for new recurring jobs. Do not map a mixed-source prefix to one dataset. Missing mappings fail catalog refresh; archive data cannot assign its own tenant permissions. ```text [ {"prefix":"POSTGRES_JOB_UUID","tenant":"customer","name":"orders","source":"PostgreSQL / public.orders"}, {"prefix":"MONGO_JOB_UUID","tenant":"customer","name":"events","source":"MongoDB / app.events"} ] ``` ### Check, refresh and serve Supply COLDCTL_ARCHIVE_TOKEN as a strong token of at least 32 bytes, check configuration, refresh the catalog and start the service. Use the configured token in the UI or API. Refresh again after newly completed archives and layout changes; the cloud console does not perform these steps. ```text coldctl-archive --config coldctl.s3.toml config check coldctl-archive --config coldctl.s3.toml catalog refresh coldctl-archive --config coldctl.s3.toml serve ``` ### Current boundaries Default limits include 128 MiB per object and a shared 2 GiB temporary staging budget. Whole-object verification precedes queries and downloads; no persistent cache or ranged reads are provided. Export status and completed files persist until their configured expiry. Previews return a bounded first page; deep pagination is not implemented. Checksums establish consistency, not signed historical provenance. Test with representative archive sizes before production use. ### Durable export recovery Export status and completed files persist in exports.sqlite and the exports directory. Startup validates checksums; interrupted work becomes INTERRUPTED for explicit resubmission, and missing/corrupt completed files become FAILED. Current owner and tenant grants still control downloads. Temporary download tickets expire on restart; request a new ticket for a retained READY job. Stop the gateway before backing up its catalog, export database and export files together. ### Browser access and audit boundaries The browser offers dataset selection, typed filters, column selection, record inspection and bounded CSV/JSONL/Parquet exports. Credentials identify named users with exact tenant grants; download authorization checks current grants and export ownership. Audit events record user, tenant, dataset, operation, fingerprint and counts in the local catalog. These events are not a tamper-proof compliance ledger. Column selection is not field-level authorization. Built-in PII masking, legal holds and dataset-level grant overrides are not implemented. MongoDB results expose the raw-BSON envelope; arbitrary nested document-field search is not supported. ### Which lookups are audited? Successful searches and record views write SEARCH and RECORD_VIEW events to the local catalog.sqlite before the API returns their results. If that audit write fails, the endpoint returns an error. Events contain a UTC timestamp, configured user name, tenant, dataset, operation, fingerprint and row count. Export creation and download access are also logged; a download event does not prove the client received every byte. Failed or denied lookups are not comprehensively recorded in this audit table. It is not a record-by-record access ledger, and it contains no full returned documents. Protect and back up the catalog and arrange separate retention and central collection if required. Give each person a separate configured credential: shared tokens cannot establish which human made a request. Tenant grants cover datasets mapped to that tenant, not individual fields. ### Google Cloud Storage provider Use kind = "gcs" with the existing bucket and matching remote prefix. Provide read/list Google credentials to the Gateway. ## Operational monitoring and alert hooks Website: /docs/concepts/operational-monitoring Scrape metrics, inspect durable failures and deliver optional webhook alerts from the agent and gateway. ### Initialize and run Stop workers and run coldctl init for schema 17. The agent records failure/recovery transitions transactionally with job, queue and recurring-schedule changes, even when the monitor is stopped. Run the monitor as a separate service alongside the scheduler with the same --data-dir. Set the referenced secrets through your service manager. Omit webhook flags to disable delivery. ```text coldctl init coldctl agent monitor serve --listen 127.0.0.1:9464 --token-env COLDCTL_METRICS_TOKEN --webhook-url-env COLDCTL_ALERT_URL --webhook-token-env COLDCTL_ALERT_TOKEN ``` ### Agent endpoints GET /metrics exposes Prometheus text: job/queue/schedule counts, archive/deletion aggregates, queue age, sample freshness, active incidents and alert backlog. GET /failures returns the latest 100 transitions. GET /ready returns 503 for failed or stale samples; it reports monitor readiness, not worker liveness or archive correctness. The metrics token needs 32–4096 bytes. All endpoints require Authorization: Bearer TOKEN when configured; non-loopback binding requires a token. Use TLS at ingress. --poll-seconds defaults to 10, with a range of 1–300. ```text coldctl agent monitor status ``` ### Gateway endpoints The existing administrator bearer credential protects GET /metrics and GET /api/admin/monitoring/failures. Tenant users are denied. Metrics include durable HTTP outcome counters and available query workers. Incidents cover server errors, query/record/download execution rejections (including invalid filters and resource limits), failed catalog refreshes and background export failures. Operations resolve on their next success; this does not imply every prior request is recoverable. Streaming errors after headers are outside response outcome counters. ```text [alerts] webhook_url_env = "COLDCTL_ALERT_URL" webhook_token_env = "COLDCTL_ALERT_TOKEN" ``` ### Webhook contract and privacy The generic JSON event has id, service_id, component, kind, entity_id, state (firing/resolved) and occurred_at (Unix seconds). service_id is stable in the monitoring sidecar; map it to your deployment in the receiver. Agent entity IDs are internal UUIDs; gateway incidents aggregate per operation. No names, query values, row data, secret URLs or raw error text are sent. Metric labels are bounded and exclude resource IDs. Provider-specific Slack/PagerDuty formats require an adapter; hooks do not execute shell commands. ### Durable delivery Only 2xx acknowledges delivery. HTTPS is required, redirects are disabled, and requests time out after 10 seconds. --allow-loopback-http (gateway allow_loopback_http) permits only numeric loopback HTTP for tests. Retry starts at 30 seconds and doubles to one hour. At-least-once delivery uses Idempotency-Key equal to the stable event UUID; the receiver must deduplicate. Recovery cannot overtake undelivered firing for the same incident. Events observed with hooks disabled are not replayed when hooks are enabled; previously pending events resume. Changing the receiver redirects pending deliveries to its new URL. ### Retention and backups The agent source journal retains 10,000 transitions. The separate monitoring.sqlite sidecar retains delivered/disabled history and resolved incidents for 30 days; active incidents and pending alerts are retained. Stop its owning process before copying the sidecar. coldctl state backup includes the source journal, not monitoring delivery acknowledgements. Restore matching backups; a regressed source journal makes monitoring unready rather than silently skipping new events. Review pending alerts before replacing mismatched sidecar state. ### Outage detection and limits coldctl_agent_journal_gap and the durable coldctl_monitor_journal_gaps_total expose journal overflow after extended monitor downtime. Current failure states are reconciled after catch-up; missing history cannot be reconstructed. Use external scrape/process alerts for stopped workers/monitors, stale samples, increasing oldest-due queue age, disk pressure and pending webhook age. A dead process cannot notify its own outage. Monitoring does not restart workers, resume archives or sync Cloud metadata. Gateway delivery runs during serve; export jobs remain in memory. ## Protect production during archival Website: /docs/concepts/production-protection Source-scoped UTC windows, external lag/load telemetry, adaptive delays and optional gateway validation gates. ### Configure once per source Protection policies apply to manual export/resume, scheduled work, deletion and shadow reads. Limits are checked before connection where possible and at safe batch boundaries. In-flight work can finish. Required missing/stale/future telemetry fails closed. Scheduled protection holds retry without consuming the normal failure budget. Manual paused archives require resume. ```text coldctl protection set SOURCE --config protection.json coldctl protection sample SOURCE --file health.json coldctl protection status SOURCE ``` ### Example policy UTC minutes 1320–360 mean 22:00–06:00 daily, start inclusive/end exclusive. Reaching a threshold blocks admission. Delays increase with the highest metric-to-limit ratio. Samples are supplied by your trusted monitoring collector; native replica discovery and database polling are not implemented. Use maximum lag across every replica requiring protection and do not refresh timestamps on stale measurements. ```json { "window_start_utc_minute": 1320, "window_end_utc_minute": 360, "max_replica_lag_ms": 2000, "max_cpu_percent": 80, "sample_max_age_seconds": 30, "min_batch_delay_ms": 100, "max_batch_delay_ms": 2000 } ``` ### Publish real measurements Use the UUID shown by protection status and actual measurement time in Unix milliseconds. Missing metrics are null, not zero. The timestamp below is illustrative and must be replaced. Samples must be newer than the last sample. Operator-supplied metrics cannot prove replica coverage. ```json { "source_id": "SOURCE_UUID", "observed_at_unix_ms": 1792000000000, "replica_lag_ms": 250, "cpu_percent": 30, "lock_waiters": 0 } ``` ### Require retrieval evidence before deletion Set require_gateway_shadow=true and shadow_max_age_seconds (1–86400) to require a recent full gateway pass bound to the manifest. Source-only, partial, failed, empty or expired reports do not qualify. Explicit deletion authorization still applies. Configure automatic_shadow by archive policy name to let the scheduler refresh the gateway and run full validation before deletion; otherwise the manual gate remains. Back up protection.sqlite and shadow.sqlite with state.db while workers are stopped; protect filesystem ACLs. ## Validate archived retrieval before deletion Website: /docs/concepts/shadow-validation Compare retained source rows with verified archives and optional gateway ID lookups; persist redacted evidence. ### Archive without deleting first Create a completed archive and keep its eligible source records stable and present. Refresh and map it in the gateway catalog. Shadow validation never deletes or restores source data and rejects jobs with deletion history. It holds job/policy locks and uses the original connector identity. ```text coldctl archive shadow JOB_ID --confirm-original-source --gateway-url https://archive.example.com --dataset-id DATASET_ID --token-env COLDCTL_GATEWAY_TOKEN coldctl archive shadow-reports JOB_ID ``` ### Bounded, explicit coverage Omit all gateway flags for source/archive comparison only. --max-batches and --max-duration-seconds bound a trial; unfinished runs return incomplete and nonzero. The duration is cooperative: blocking archive I/O may finish first. Gateway mode batches typed ID membership searches (64 keys by default; --gateway-batch-size accepts 1-100), matches results by key and checks all supported columns, missing/duplicate/extra records and truncation. Reduce batch size for wide records or small Gateway result limits. Durable reports distinguish keys checked from HTTP requests. It does not mirror live application traffic or validate arbitrary predicates, joins, collation and aggregates. MongoDB uses the raw-BSON envelope. Comparisons retain timestamp/decimal precision; non-finite floats are unsupported. ### Evidence and safety Reports store manifest hash, timestamps, rows/batches, gateway lookups, source-mismatch batches and gateway-mismatch rows without record values, keys, raw errors or credentials. Crashes leave running/unfinalized evidence, never a pass. New runs restart comparison from the beginning. Responses are capped at 1 MiB and gateway calls at 30 seconds. Production protection applies. The latest 100 reports are listed; older reports remain in shadow.sqlite. Real-cloud/100-GB qualification remains pending. ## Automate validation before recurring deletion Website: /docs/concepts/automatic-shadow Refresh the gateway, compare retained records, and authorize each archive job automatically before verified deletion. ### Configure once for each archive policy Merge automatic_shadow into the source protection policy, preserving existing health/window limits. The map key is the exact archive policy name. require_gateway_shadow must be true. Supply the two token values through the worker service environment: tenant-scoped lookup credentials and separate administrator credentials for catalog refresh. Replacing protection configuration clears telemetry; submit fresh measurements. ```json { "require_gateway_shadow": true, "shadow_max_age_seconds": 3600, "automatic_shadow": { "pg_orders": { "gateway_url": "https://archive.example.com", "dataset_id": "DATASET_ID", "token_env": "COLDCTL_GATEWAY_READ_TOKEN", "refresh_token_env": "COLDCTL_GATEWAY_REFRESH_TOKEN" } } } ``` ### Map future archives to a stable dataset Configure trusted gateway layout for a dedicated policy storage prefix, including all future job directories. Do not map only one job UUID or combine unrelated tables/tenants. Obtain its stable dataset ID from the gateway. For an initially empty dataset the ID is SHA-256 of the compact UTF-8 JSON array [tenant,name]. Schema drift and duplicate IDs fail closed. ```text coldctl protection set SOURCE --config protection.json coldctl agent scheduler recurring create pg_orders --every-seconds 86400 --source-stability immutable-rows --confirm-original-source --delete-after-archive coldctl agent scheduler run ``` ### Validation and recovery Each configured cycle exports, verifies, refreshes the catalog, compares source/archive/gateway values, then deletes verified unchanged records. Authentication errors and mismatches fail the same cycle; transient failures use bounded retries. Partial/interrupted reports cannot authorize deletion. Restarted validation begins again. Empty verified cycles finish without claiming retrieval coverage. Recurring support remains PostgreSQL/MySQL/MongoDB to S3, Azure Blob or GCS under their stable-source contracts. ```text coldctl archive shadow-reports JOB_ID coldctl agent scheduler show QUEUE_ID coldctl agent scheduler recurring retry SCHEDULE_ID --confirm-original-source ``` ### Durable authorization after deletion starts A passing report creates a local job/manifest/configuration-bound receipt at deletion admission. Subsequent batches and worker restarts can resume that same deletion after report expiry, since deleted source rows cannot be compared again. Archive integrity, health and source-value checks still apply. The receipt does not continuously recheck gateway availability or authorize another job. Back up shadow.sqlite with the main state database while workers are stopped. Evidence is not synchronized to Cloud. ### Operating limits Stop is checked during validation and before deletion; an in-flight operation or delete batch can finish. Full automatic validation runs outside the export time-slice budget and checks every key in batches of up to 64. Individual refresh requests have a 120-second deadline and lookup requests 30 seconds. Long validations can delay other queued jobs; size the evidence age accordingly. Native telemetry collection, live-query mirroring and real-cloud/100-GB qualification remain outside this feature. ## Parent-child archival and recurring groups Website: /docs/concepts/parent-child-archival Archive related MySQL or PostgreSQL tables and validate every member before child-first deletion. ### Selection and verification Configure 2 to 32 distinct table policies on one source. Roots use retention rules; automatic child selection includes every descendant matching archived parent foreign-key values, including newer children. Archives are verified parent-first. Every deletion slice re-verifies the complete group. Remaining children block parent deletion, including CASCADE and SET NULL. ```text coldctl archive group-run --policies orders items --select-children --confirm-quiescent-source ``` ### Recurring cycles and Gateway shadow Run init to upgrade local state, then create a group and keep the scheduler worker running. Recurring groups always select children automatically. Configure require_gateway_shadow and automatic_shadow for every member policy, using distinct Gateway datasets. Partial coverage, mismatches and Gateway failures block initial deletion of the entire group. Retry retains the same jobs. Started deletions use manifest-bound receipts; cached passing evidence can be reused. The next cycle begins after all deletions complete plus the configured delay. ```text coldctl agent scheduler group create --policies orders items --every-seconds 3600 --confirm-quiescent-source --confirm-original-source --delete-after-archive coldctl agent scheduler run ``` ### Safety and limits Keep all member tables quiescent throughout each cycle and retries. This is not a cross-table snapshot or atomic deletion: verified children may already be deleted when a parent is blocked. Single integer primary keys are required; cycles and self references are unsupported. PostgreSQL requires ordinary tables, TEMP privilege and matching built-in FK types, modifiers and collations; inheritance, partitions and RLS are unsupported. MySQL needs temporary-table and authoritative metadata privileges. Large groups need performance qualification. ## coldctl protection set Website: /docs/commands/protection-set Configure production protection. Status: Implemented. Effect: Local configuration/reporting. ### Syntax ```text coldctl protection set [OPTIONS] --config ``` ### Arguments and options - ``: Registered source name - `--config `: Path to the protection policy JSON file - `--data-dir `: Override the OS application data directory - `-h, --help`: Print help ### Examples ```text coldctl protection set SOURCE --config protection.json ``` ### Operational notes Replaces a source-scoped policy and clears its prior sample. Limits use trusted external monitoring, not automatic database polling. ### CLI help (trailing whitespace normalized) ```text Replace source-scoped policy; clears old monitoring sample. JSON file contains no secrets Usage: coldctl protection set [OPTIONS] --config Arguments: Registered source name Options: --config Path to the protection policy JSON file --data-dir Override the OS application data directory -h, --help Print help ``` ## coldctl protection sample Website: /docs/commands/protection-sample Submit source monitoring telemetry. Status: Implemented. Effect: Local configuration/reporting. ### Syntax ```text coldctl protection sample [OPTIONS] --file ``` ### Arguments and options - ``: Registered source name - `--data-dir `: Override the OS application data directory - `--file `: Path to a fresh monitoring sample JSON file - `-h, --help`: Print help ### Examples ```text coldctl protection sample SOURCE --file health.json ``` ### Operational notes Requires the source UUID and actual measurement time. Stale, missing, future and unknown required metrics block work. ### CLI help (trailing whitespace normalized) ```text Submit a fresh source-ID-bound sample from your trusted monitoring collector Usage: coldctl protection sample [OPTIONS] --file Arguments: Registered source name Options: --data-dir Override the OS application data directory --file Path to a fresh monitoring sample JSON file -h, --help Print help ``` ## coldctl protection status Website: /docs/commands/protection-status Inspect production protection. Status: Implemented. Effect: Local configuration/reporting. ### Syntax ```text coldctl protection status [OPTIONS] ``` ### Arguments and options - ``: Registered source name - `--data-dir `: Override the OS application data directory - `-h, --help`: Print help ### Examples ```text coldctl protection status SOURCE ``` ### Operational notes Evaluates the policy and shows the saved sample and admission reason. ### CLI help (trailing whitespace normalized) ```text Show policy, telemetry and last admission decision Usage: coldctl protection status [OPTIONS] Arguments: Registered source name Options: --data-dir Override the OS application data directory -h, --help Print help ``` ## coldctl protection clear Website: /docs/commands/protection-clear Disable production protection. Status: Implemented. Effect: Local configuration/reporting. ### Syntax ```text coldctl protection clear [OPTIONS] --confirm ``` ### Arguments and options - ``: Registered source name - `--confirm`: Confirm disabling all protection for this source - `--data-dir `: Override the OS application data directory - `-h, --help`: Print help ### Examples ```text coldctl protection clear SOURCE --confirm ``` ### Operational notes Explicitly removes source-scoped admission guards. Other execution and deletion checks remain. ### CLI help (trailing whitespace normalized) ```text Disable source protection explicitly Usage: coldctl protection clear [OPTIONS] --confirm Arguments: Registered source name Options: --confirm Confirm disabling all protection for this source --data-dir Override the OS application data directory -h, --help Print help ``` ## coldctl connector registry Website: /docs/commands/connector-registry Pin connector registry trust. Status: Implemented. Effect: Local trust configuration. ### Syntax ```text coldctl connector registry [OPTIONS] --url --root --root-sha256 ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `--url `: HTTPS registry base URL. Use the verified release registry URL or your trusted registry. - `--root `: Path to the independently verified public TUF root.json. - `--root-sha256 `: Independently verified SHA-256 fingerprint of the trusted root file. - `-h, --help`: Print help ### Examples ```text coldctl connector registry --url https://github.com/coldctl/coldctl/releases/latest/download/ --root root.json --root-sha256 VERIFIED_FINGERPRINT ``` ### Operational notes Use an independently verified TUF root and approved signed packages. Installation and activation are separate from source configuration; incomplete jobs retain exact connector pins. Verify the fingerprint independently. Never bootstrap trust solely from the downloaded bundle. ### CLI help (trailing whitespace normalized) ```text Pin a registry's independently verified TUF root. Never trusts keys from a downloaded bundle Usage: coldctl connector registry [OPTIONS] --url --root --root-sha256 Options: --data-dir Override the OS application data directory --url --root --root-sha256 -h, --help Print help ``` ## coldctl connector refresh Website: /docs/commands/connector-refresh Refresh signed registry metadata. Status: Implemented. Effect: Registry read / local metadata write. ### Syntax ```text coldctl connector refresh [OPTIONS] ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `--file `: Path to an extracted signed offline repository directory. - `-h, --help`: Print help ### Examples ```text coldctl connector refresh ``` ### Operational notes Use an independently verified TUF root and approved signed packages. Installation and activation are separate from source configuration; incomplete jobs retain exact connector pins. Checks signatures, expiry and rollback protection and retains revocation evidence. --file selects an offline repository directory. ### CLI help (trailing whitespace normalized) ```text Verify current signed metadata and retain revocations; no executable is launched Usage: coldctl connector refresh [OPTIONS] Options: --data-dir Override the OS application data directory --file -h, --help Print help ``` ## coldctl connector list Website: /docs/commands/connector-list List installed or available connectors. Status: Implemented. Effect: Local read / optional registry refresh. ### Syntax ```text coldctl connector list [OPTIONS] ``` ### Arguments and options - `--available`: Refresh signed metadata and list available packages instead of installed inventory. - `--data-dir `: Override the OS application data directory - `--file `: Path to an extracted signed offline repository directory. - `-h, --help`: Print help ### Examples ```text coldctl connector list --available ``` ### Operational notes Use an independently verified TUF root and approved signed packages. Installation and activation are separate from source configuration; incomplete jobs retain exact connector pins. Without --available reads installed inventory; --available refreshes the signed catalog. --file requires --available. ### CLI help (trailing whitespace normalized) ```text List installed connectors, or refresh and list available signed packages Usage: coldctl connector list [OPTIONS] Options: --available --data-dir Override the OS application data directory --file -h, --help Print help ``` ## coldctl connector install Website: /docs/commands/connector-install Install an exact connector version. Status: Implemented. Effect: Download / local executable installation. ### Syntax ```text coldctl connector install [OPTIONS] --version ``` ### Arguments and options - ``: Connector ID: postgres, mysql or mongodb. - `--data-dir `: Override the OS application data directory - `--version `: Exact connector package version. Use a version present in verified release metadata. - `--file `: Path to an extracted signed offline repository directory. - `--no-activate`: Install without changing the active connector for new sessions. - `-h, --help`: Print help ### Examples ```text coldctl connector install mysql --version RELEASE_VERSION ``` ### Operational notes Use an independently verified TUF root and approved signed packages. Installation and activation are separate from source configuration; incomplete jobs retain exact connector pins. --file selects an offline repository directory. --no-activate keeps the current active choice. Only verified packages are installed. ### CLI help (trailing whitespace normalized) ```text Install an exact signed version. --file names an offline repository directory Usage: coldctl connector install [OPTIONS] --version Arguments: Options: --data-dir Override the OS application data directory --version --file --no-activate -h, --help Print help ``` ## coldctl connector use Website: /docs/commands/connector-use Select a connector for new sessions. Status: Implemented. Effect: Local configuration. ### Syntax ```text coldctl connector use [OPTIONS] --version ``` ### Arguments and options - ``: Connector ID: postgres, mysql or mongodb. - `--data-dir `: Override the OS application data directory - `--version `: Exact connector package version. Use a version present in verified release metadata. - `--allow-downgrade`: Explicitly authorize selecting an older installed connector version. - `-h, --help`: Print help ### Examples ```text coldctl connector use mysql --version RELEASE_VERSION ``` ### Operational notes Use an independently verified TUF root and approved signed packages. Installation and activation are separate from source configuration; incomplete jobs retain exact connector pins. Requires an installed version. Explicit --allow-downgrade is required for a downgrade. ### CLI help (trailing whitespace normalized) ```text Select an installed version for new sessions. Pinned jobs keep their exact version Usage: coldctl connector use [OPTIONS] --version Arguments: Options: --data-dir Override the OS application data directory --version --allow-downgrade -h, --help Print help ``` ## coldctl connector inspect Website: /docs/commands/connector-inspect Inspect connector versions and references. Status: Implemented. Effect: Local read. ### Syntax ```text coldctl connector inspect [OPTIONS] ``` ### Arguments and options - ``: Connector ID: postgres, mysql or mongodb. - `--data-dir `: Override the OS application data directory - `--version `: Exact connector package version. Use a version present in verified release metadata. - `-h, --help`: Print help ### Examples ```text coldctl connector inspect mysql ``` ### Operational notes Use an independently verified TUF root and approved signed packages. Installation and activation are separate from source configuration; incomplete jobs retain exact connector pins. Reports integrity, selection and resumable-job references. --version narrows the inspection. ### CLI help (trailing whitespace normalized) ```text Inspect installed versions, integrity, active selection and resumable-job references Usage: coldctl connector inspect [OPTIONS] Arguments: Options: --data-dir Override the OS application data directory --version -h, --help Print help ``` ## coldctl connector doctor Website: /docs/commands/connector-doctor Check local connector integrity. Status: Implemented. Effect: Local read. ### Syntax ```text coldctl connector doctor [OPTIONS] ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `-h, --help`: Print help ### Examples ```text coldctl connector doctor ``` ### Operational notes Use an independently verified TUF root and approved signed packages. Installation and activation are separate from source configuration; incomplete jobs retain exact connector pins. No executable is launched and no metadata is downloaded. ### CLI help (trailing whitespace normalized) ```text Check local package integrity and revocation state without executing or downloading code Usage: coldctl connector doctor [OPTIONS] Options: --data-dir Override the OS application data directory -h, --help Print help ``` ## coldctl connector remove Website: /docs/commands/connector-remove Remove an unused connector version. Status: Implemented. Effect: Local package removal. ### Syntax ```text coldctl connector remove [OPTIONS] --version ``` ### Arguments and options - ``: Connector ID: postgres, mysql or mongodb. - `--data-dir `: Override the OS application data directory - `--version `: Exact connector package version. Use a version present in verified release metadata. - `-h, --help`: Print help ### Examples ```text coldctl connector remove mysql --version RELEASE_VERSION ``` ### Operational notes Use an independently verified TUF root and approved signed packages. Installation and activation are separate from source configuration; incomplete jobs retain exact connector pins. Refused while a running session or resumable job needs the version. Does not delete source data or archives. ### CLI help (trailing whitespace normalized) ```text Remove an installed version unless running sessions or resumable jobs need it Usage: coldctl connector remove [OPTIONS] --version Arguments: Options: --data-dir Override the OS application data directory --version -h, --help Print help ``` ## coldctl connector bundle Website: /docs/commands/connector-bundle Export a signed offline repository. Status: Implemented. Effect: Local files write. ### Syntax ```text coldctl connector bundle [OPTIONS] --version --out ``` ### Arguments and options - ``: Connector ID: postgres, mysql or mongodb. - `--data-dir `: Override the OS application data directory - `--version `: Exact connector package version. Use a version present in verified release metadata. - `--out `: New output directory for the signed offline repository bundle. - `-h, --help`: Print help ### Examples ```text coldctl connector bundle mysql --version RELEASE_VERSION --out ./offline-bundle ``` ### Operational notes Use an independently verified TUF root and approved signed packages. Installation and activation are separate from source configuration; incomplete jobs retain exact connector pins. The output directory must be new. Include signed metadata; recipients must bootstrap trust independently. ### CLI help (trailing whitespace normalized) ```text Export one connector and signed metadata into a new offline repository directory Usage: coldctl connector bundle [OPTIONS] --version --out Arguments: Options: --data-dir Override the OS application data directory --version --out -h, --help Print help ``` ## coldctl init Website: /docs/commands/init Initialize or upgrade local state. Status: Implemented. Effect: Local state write. ### Syntax ```text coldctl init [OPTIONS] ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `-h, --help`: Print help ### Examples ```text coldctl --data-dir ./coldctl-state init ``` ### Operational notes Creates the local SQLite database and installation ID. Repeating init preserves identity and applies supported migrations. Stop existing workers before an upgrade; older binaries reject newer schemas. ### CLI help (trailing whitespace normalized) ```text Initialize local state (safe to run repeatedly) Usage: coldctl init [OPTIONS] Options: --data-dir Override the OS application data directory -h, --help Print help ``` ## coldctl status Website: /docs/commands/status Inspect the local installation. Status: Implemented. Effect: Local read. ### Syntax ```text coldctl status [OPTIONS] ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `-h, --help`: Print help ### Examples ```text coldctl --data-dir ./coldctl-state status ``` ### Operational notes Reads local installation metadata without contacting a source or Cloud. It is not a worker-health check. ### CLI help (trailing whitespace normalized) ```text Show local installation status without network access Usage: coldctl status [OPTIONS] Options: --data-dir Override the OS application data directory -h, --help Print help ``` ## coldctl state backup Website: /docs/commands/state-backup Create a consistent SQLite backup. Status: Implemented. Effect: Local file write. ### Syntax ```text coldctl state backup [OPTIONS] --to ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `--to `: New file outside the state directory; its parent must already exist - `-h, --help`: Print help ### Examples ```text coldctl state backup --to ./backups/state-2026-10-05.db --output json ``` ### Operational notes Create the parent directory first. The output must be outside the state directory and must not exist. Uses SQLite online backup, including committed WAL data. Archives, staging files, credentials and locks are not included; stop all writers for a matching state-and-files recovery set. ### CLI help (trailing whitespace normalized) ```text Consistent SQLite snapshot, including WAL; archive files are not copied Usage: coldctl state backup [OPTIONS] --to Options: --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] --to New file outside the state directory; its parent must already exist -h, --help Print help ``` ## coldctl source add postgres Website: /docs/commands/source-add-postgres Register a PostgreSQL connection. Status: Implemented. Effect: Local state write. ### Syntax ```text coldctl source add postgres [OPTIONS] --name <--url |--url-env > ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `--name `: Unique local source name (ASCII letters, digits, hyphens, underscores) - `--output `: Output format; JSON contains no resolved credentials [default: table] [possible values: table, json] - `--tls-ca-env `: Environment variable containing a PEM CA bundle for PostgreSQL/MySQL/MongoDB certificate validation - `--url `: Password-free postgres://, mysql:// or mongodb:// URL; supports sslmode=require|disable - `--url-env `: Name of an environment variable holding the full URL; its value is never stored - `--password-env `: Name of an environment variable holding the password, resolved only when connecting - `-h, --help`: Print help ### Examples ```text coldctl source add postgres --name production --url "postgresql://archive_reader@db.example.com:5432/app" --password-env ARCHIVE_DB_PASSWORD ``` ```text coldctl source add postgres --name managed-pg --url postgres://reader@db.example.com/app --password-env PG_PASSWORD --tls-ca-env PG_CA_PEM ``` ### Operational notes Set ARCHIVE_DB_PASSWORD in your process environment or secret manager before connecting. Alternatively use --url-env ARCHIVE_DATABASE_URL; only its variable name is persisted. Exactly one of --url or --url-env is required. Password-bearing --url values are rejected. Adding a source does not connect. For a private CA, load PEM certificate contents into PG_CA_PEM and use --tls-ca-env PG_CA_PEM with an updated PostgreSQL connector. Only the reference is saved; no OS trust-store change is needed. The configured bundle supplies this source's trusted roots; certificate and hostname verification remain enabled. Custom CAs require TLS. Without a custom CA, system trust is used. ### CLI help (trailing whitespace normalized) ```text Configure one PostgreSQL TCP endpoint. TLS is required by default Usage: coldctl source add postgres [OPTIONS] --name <--url |--url-env > Options: --data-dir Override the OS application data directory --name Unique local source name (ASCII letters, digits, hyphens, underscores) --output Output format; JSON contains no resolved credentials [default: table] [possible values: table, json] --tls-ca-env Environment variable containing a PEM CA bundle for PostgreSQL/MySQL/MongoDB certificate validation --url Password-free postgres://, mysql:// or mongodb:// URL; supports sslmode=require|disable --url-env Name of an environment variable holding the full URL; its value is never stored --password-env Name of an environment variable holding the password, resolved only when connecting -h, --help Print help ``` ## coldctl source add mysql Website: /docs/commands/source-add-mysql Configure a MySQL source. Status: Implemented. Effect: Local configuration. ### Syntax ```text coldctl source add mysql [OPTIONS] --name <--url |--url-env > ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `--name `: Unique local source name (ASCII letters, digits, hyphens, underscores) - `--output `: Output format; JSON contains no resolved credentials [default: table] [possible values: table, json] - `--tls-ca-env `: Environment variable containing a PEM CA bundle for PostgreSQL/MySQL/MongoDB certificate validation - `--url `: Password-free postgres://, mysql:// or mongodb:// URL; supports sslmode=require|disable - `--url-env `: Name of an environment variable holding the full URL; its value is never stored - `--password-env `: Name of an environment variable holding the password, resolved only when connecting - `-h, --help`: Print help ### Examples ```text coldctl source add mysql --name mysql-source --url mysql://archive_reader@db.example.com/app --password-env DATABASE_PASSWORD --tls-ca-env DATABASE_CA_PEM ``` ### Operational notes Adds configuration without connecting. Use the matching approved connector. --tls-ca-env references PEM certificate text. Exactly one URL mode is required; never put a password in --url. ### CLI help (trailing whitespace normalized) ```text Configure one MySQL TCP endpoint. TLS is required by default Usage: coldctl source add mysql [OPTIONS] --name <--url |--url-env > Options: --data-dir Override the OS application data directory --name Unique local source name (ASCII letters, digits, hyphens, underscores) --output Output format; JSON contains no resolved credentials [default: table] [possible values: table, json] --tls-ca-env Environment variable containing a PEM CA bundle for PostgreSQL/MySQL/MongoDB certificate validation --url Password-free postgres://, mysql:// or mongodb:// URL; supports sslmode=require|disable --url-env Name of an environment variable holding the full URL; its value is never stored --password-env Name of an environment variable holding the password, resolved only when connecting -h, --help Print help ``` ## coldctl source add mongodb Website: /docs/commands/source-add-mongodb Configure a MongoDB source. Status: Implemented. Effect: Local configuration. ### Syntax ```text coldctl source add mongodb [OPTIONS] --name <--url |--url-env > ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `--name `: Unique local source name (ASCII letters, digits, hyphens, underscores) - `--output `: Output format; JSON contains no resolved credentials [default: table] [possible values: table, json] - `--tls-ca-env `: Environment variable containing a PEM CA bundle for PostgreSQL/MySQL/MongoDB certificate validation - `--url `: Password-free postgres://, mysql:// or mongodb:// URL; supports sslmode=require|disable - `--url-env `: Name of an environment variable holding the full URL; its value is never stored - `--password-env `: Name of an environment variable holding the password, resolved only when connecting - `-h, --help`: Print help ### Examples ```text coldctl source add mongodb --name mongodb-source --url mongodb://archive_reader@db.example.com/app --password-env DATABASE_PASSWORD --tls-ca-env DATABASE_CA_PEM ``` ### Operational notes Adds configuration without connecting. Use the matching approved connector. --tls-ca-env references PEM certificate text. Exactly one URL mode is required; never put a password in --url. ### CLI help (trailing whitespace normalized) ```text Configure a MongoDB primary endpoint. TLS is required by default Usage: coldctl source add mongodb [OPTIONS] --name <--url |--url-env > Options: --data-dir Override the OS application data directory --name Unique local source name (ASCII letters, digits, hyphens, underscores) --output Output format; JSON contains no resolved credentials [default: table] [possible values: table, json] --tls-ca-env Environment variable containing a PEM CA bundle for PostgreSQL/MySQL/MongoDB certificate validation --url Password-free postgres://, mysql:// or mongodb:// URL; supports sslmode=require|disable --url-env Name of an environment variable holding the full URL; its value is never stored --password-env Name of an environment variable holding the password, resolved only when connecting -h, --help Print help ``` ## coldctl source list Website: /docs/commands/source-list List configured database sources. Status: Implemented. Effect: Local read. ### Syntax ```text coldctl source list [OPTIONS] ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `--output `: Output format; JSON contains no resolved credentials [default: table] [possible values: table, json] - `-h, --help`: Print help ### Examples ```text coldctl source list --output json ``` ### Operational notes Reads local configuration, not live connectivity. Names are case-sensitive. JSON configuration can include environment-variable names but never resolved password values. ### CLI help (trailing whitespace normalized) ```text List configured sources without connectivity checks Usage: coldctl source list [OPTIONS] Options: --data-dir Override the OS application data directory --output Output format; JSON contains no resolved credentials [default: table] [possible values: table, json] -h, --help Print help ``` ## coldctl source show Website: /docs/commands/source-show Inspect one configured source. Status: Implemented. Effect: Local read. ### Syntax ```text coldctl source show [OPTIONS] ``` ### Arguments and options - ``: Existing local configuration name. - `--data-dir `: Override the OS application data directory - `--output `: Output format; JSON contains no resolved credentials [default: table] [possible values: table, json] - `-h, --help`: Print help ### Examples ```text coldctl source show production --output json ``` ### Operational notes Shows the saved source identified by its local name. Does not resolve credentials or contact the database. ### CLI help (trailing whitespace normalized) ```text Show metadata and credential references, never credential values Usage: coldctl source show [OPTIONS] Arguments: Options: --data-dir Override the OS application data directory --output Output format; JSON contains no resolved credentials [default: table] [possible values: table, json] -h, --help Print help ``` ## coldctl source test Website: /docs/commands/source-test Check source authentication and access. Status: Implemented. Effect: Database read. ### Syntax ```text coldctl source test [OPTIONS] ``` ### Arguments and options - ``: Existing local configuration name. - `--data-dir `: Override the OS application data directory - `--output `: Output format; JSON contains no resolved credentials [default: table] [possible values: table, json] - `-h, --help`: Print help ### Examples ```text coldctl source test production ``` ### Operational notes Resolves environment-based credentials and opens a read-only connection. TLS is required and verified by default. Use sslmode=disable only for an intentionally local non-TLS test server. ### CLI help (trailing whitespace normalized) ```text Connect and verify database authentication and database access Usage: coldctl source test [OPTIONS] Arguments: Options: --data-dir Override the OS application data directory --output Output format; JSON contains no resolved credentials [default: table] [possible values: table, json] -h, --help Print help ``` ## coldctl source discover Website: /docs/commands/source-discover Discover accessible tables or collections. Status: Implemented. Effect: Database read. ### Syntax ```text coldctl source discover [OPTIONS] ``` ### Arguments and options - ``: Existing local configuration name. - `--data-dir `: Override the OS application data directory - `--output `: Output format; JSON contains no resolved credentials [default: table] [possible values: table, json] - `-h, --help`: Print help ### Examples ```text coldctl source discover production --output json ``` ### Operational notes Uses the selected connector. SQL discovery reads catalogs; MongoDB discovery can sample documents. Results do not prove all records meet supported export shapes or are safe to archive. ### CLI help (trailing whitespace normalized) ```text Read schemas, tables, columns, keys, indexes, and row estimates Usage: coldctl source discover [OPTIONS] Arguments: Options: --data-dir Override the OS application data directory --output Output format; JSON contains no resolved credentials [default: table] [possible values: table, json] -h, --help Print help ``` ## coldctl source remove Website: /docs/commands/source-remove Remove local source configuration. Status: Implemented. Effect: Local state write. ### Syntax ```text coldctl source remove [OPTIONS] ``` ### Arguments and options - ``: Existing local configuration name. - `--data-dir `: Override the OS application data directory - `--output `: Output format; JSON contains no resolved credentials [default: table] [possible values: table, json] - `-h, --help`: Print help ### Examples ```text coldctl source remove production ``` ### Operational notes Does not delete a database or its data. Removal is refused while a policy references the source. ### CLI help (trailing whitespace normalized) ```text Remove local configuration only; leaves the database and archives untouched Usage: coldctl source remove [OPTIONS] Arguments: Options: --data-dir Override the OS application data directory --output Output format; JSON contains no resolved credentials [default: table] [possible values: table, json] -h, --help Print help ``` ## coldctl analyze Website: /docs/commands/analyze Assess archival candidates from metadata. Status: Implemented. Effect: Database read. ### Syntax ```text coldctl analyze [OPTIONS] ``` ### Arguments and options - ``: Configured source name - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `-h, --help`: Print help ### Examples ```text coldctl analyze production --output json ``` ### Operational notes Uses connector-specific metadata and estimates to assess candidates. SQL does not run ANALYZE or exact eligible counts; MongoDB may sample document shapes. This does not decide source stability. ### CLI help (trailing whitespace normalized) ```text Assess archival candidates from metadata and existing statistics, without scanning rows Usage: coldctl analyze [OPTIONS] Arguments: Configured source name Options: --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] -h, --help Print help ``` ## coldctl destination add local Website: /docs/commands/destination-add-local Register a local archive directory. Status: Implemented. Effect: Local state write. ### Syntax ```text coldctl destination add local [OPTIONS] --name --path ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `--name `: Unique local name (ASCII letters, digits, hyphens, underscores) - `--output `: [default: table] [possible values: table, json] - `--path `: Archive directory. Saved as an absolute path; created by destination test if missing - `-h, --help`: Print help ### Examples ```text coldctl destination add local --name local-archive --path ./archives ``` ### Operational notes Resolves relative paths against the current working directory and persists an absolute path. Does not create or probe the directory. Local filesystem storage is the currently supported destination. ### CLI help (trailing whitespace normalized) ```text Configure a local directory; relative paths resolve against the current directory Usage: coldctl destination add local [OPTIONS] --name --path Options: --data-dir Override the OS application data directory --name Unique local name (ASCII letters, digits, hyphens, underscores) --output [default: table] [possible values: table, json] --path Archive directory. Saved as an absolute path; created by destination test if missing -h, --help Print help ``` ## coldctl destination add s3 Website: /docs/commands/destination-add-s3 Configure an S3 archive destination. Status: Implemented. Effect: Local destination configuration. ### Syntax ```text coldctl destination add s3 [OPTIONS] --name --bucket --region ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `--name `: Destination name. - `--output `: [default: table] [possible values: table, json] - `--bucket `: Existing S3 bucket. - `--region `: AWS region. - `--prefix `: [default: coldctl] - `--endpoint `: Optional explicit S3-compatible endpoint for environment credentials. - `--staging-path `: Durable local staging directory; only pending batches remain here - `--access-key-env `: [default: AWS_ACCESS_KEY_ID] - `--secret-key-env `: [default: AWS_SECRET_ACCESS_KEY] - `--session-token-env `: Environment variable containing the optional static AWS session token. - `--allow-http`: Permit HTTP only for an explicitly configured loopback emulator - `--role-auth`: Use automatically refreshed cloud identity credentials (standard endpoints only) - `-h, --help`: Print help ### Examples ```text coldctl destination add s3 --name archives --bucket customer-archive --region us-east-1 --role-auth ``` ### Operational notes Use --role-auth on EC2 for refreshed IAM role credentials or with AWS web identity. Custom endpoints and static session-token references are rejected in role mode. Otherwise credentials use the named environment variables. Provision storage separately and run destination test before archival. ### CLI help (trailing whitespace normalized) ```text Configure customer-owned S3 or a loopback S3 emulator Usage: coldctl destination add s3 [OPTIONS] --name --bucket --region Options: --data-dir Override the OS application data directory --name --output [default: table] [possible values: table, json] --bucket --region --prefix [default: coldctl] --endpoint --staging-path Durable local staging directory; only pending batches remain here --access-key-env [default: AWS_ACCESS_KEY_ID] --secret-key-env [default: AWS_SECRET_ACCESS_KEY] --session-token-env --allow-http Permit HTTP only for an explicitly configured loopback emulator --role-auth Use automatically refreshed cloud identity credentials (standard endpoints only) -h, --help Print help ``` ## coldctl destination add azure Website: /docs/commands/destination-add-azure Configure an Azure Blob archive destination. Status: Implemented. Effect: Local destination configuration. ### Syntax ```text coldctl destination add azure [OPTIONS] --name --account --container ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `--name `: Unique destination name. - `--output `: [default: table] [possible values: table, json] - `--account `: Azure storage account name. - `--container `: Existing private Blob container. - `--prefix `: [default: coldctl] - `--endpoint `: Optional HTTPS storage endpoint, or explicit loopback emulator URL. - `--staging-path `: Durable local staging directory; only pending batches remain here - `--account-key-env `: [default: AZURE_STORAGE_ACCOUNT_KEY] - `--allow-http`: Permit HTTP only for an explicitly configured loopback emulator - `--managed-identity`: Use automatically refreshed cloud identity credentials (standard endpoints only) - `--managed-identity-client-id `: Optional user-assigned Azure identity client UUID - `-h, --help`: Print help ### Examples ```text coldctl destination add azure --name archives --account customerstorage --container archives ``` ```text coldctl destination add azure --name archives --account customerstorage --container archives --managed-identity ``` ### Operational notes Use --managed-identity on an Azure VM, optionally with --managed-identity-client-id for a user-assigned identity. Tokens refresh automatically. Otherwise credentials use the account-key environment reference. Provision storage separately and test access before archival. HTTP is limited to loopback emulators with static credentials. ### CLI help (trailing whitespace normalized) ```text Configure customer-owned Azure Blob Storage or Azurite Usage: coldctl destination add azure [OPTIONS] --name --account --container Options: --data-dir Override the OS application data directory --name --output [default: table] [possible values: table, json] --account --container --prefix [default: coldctl] --endpoint --staging-path Durable local staging directory; only pending batches remain here --account-key-env [default: AZURE_STORAGE_ACCOUNT_KEY] --allow-http Permit HTTP only for an explicitly configured loopback emulator --managed-identity Use automatically refreshed cloud identity credentials (standard endpoints only) --managed-identity-client-id Optional user-assigned Azure identity client UUID -h, --help Print help ``` ## coldctl destination add gcs Website: /docs/commands/destination-add-gcs Configure a Google Cloud Storage archive destination. Status: Implemented. Effect: Local destination configuration. ### Syntax ```text coldctl destination add gcs [OPTIONS] --name --bucket ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `--name `: Unique destination name. - `--output `: [default: table] [possible values: table, json] - `--bucket `: Existing customer-owned GCS bucket. - `--prefix `: [default: coldctl] - `--credentials-env `: Variable containing an absolute ADC JSON file path. Defaults to Google's ADC chain - `--endpoint `: Anonymous loopback GCS emulator only; never a production endpoint override - `--allow-http`: Allow anonymous HTTP only with an explicit loopback emulator endpoint. - `--staging-path `: Durable local staging directory for pending batches - `-h, --help`: Print help ### Examples ```text coldctl destination add gcs --name gcs-archive --bucket YOUR_BUCKET --prefix coldctl ``` ```text coldctl destination add gcs --name gcs-archive --bucket YOUR_BUCKET --credentials-env COLDCTL_GCS_CREDENTIALS ``` ### Operational notes Provision the bucket separately and run destination test. Credential variables contain absolute ADC JSON file paths, not JSON contents. Service-account and authorized-user files are supported; external_account federation is unsupported. Match agent and connector builds. Custom endpoints are anonymous loopback emulators only. ### CLI help (trailing whitespace normalized) ```text Configure Google Cloud Storage using ADC or an anonymous loopback emulator Usage: coldctl destination add gcs [OPTIONS] --name --bucket Options: --data-dir Override the OS application data directory --name --output [default: table] [possible values: table, json] --bucket --prefix [default: coldctl] --credentials-env Variable containing an absolute ADC JSON file path. Defaults to Google's ADC chain --endpoint Anonymous loopback GCS emulator only; never a production endpoint override --allow-http --staging-path Durable local staging directory for pending batches -h, --help Print help ``` ## coldctl destination list Website: /docs/commands/destination-list List local archive destinations. Status: Implemented. Effect: Local read. ### Syntax ```text coldctl destination list [OPTIONS] ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `-h, --help`: Print help ### Examples ```text coldctl destination list --output json ``` ### Operational notes Reads saved configuration without checking disk availability or write permissions. ### CLI help (trailing whitespace normalized) ```text List saved destinations without checking storage access Usage: coldctl destination list [OPTIONS] Options: --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] -h, --help Print help ``` ## coldctl destination show Website: /docs/commands/destination-show Inspect an archive destination. Status: Implemented. Effect: Local read. ### Syntax ```text coldctl destination show [OPTIONS] ``` ### Arguments and options - ``: Existing local configuration name. - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `-h, --help`: Print help ### Examples ```text coldctl destination show local-archive --output json ``` ### Operational notes Shows the saved absolute directory path. No files are read or written in the destination. ### CLI help (trailing whitespace normalized) ```text Show saved storage configuration and local staging path Usage: coldctl destination show [OPTIONS] Arguments: Options: --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] -h, --help Print help ``` ## coldctl destination test Website: /docs/commands/destination-test Probe destination write and read access. Status: Implemented. Effect: Local file write. ### Syntax ```text coldctl destination test [OPTIONS] ``` ### Arguments and options - ``: Existing local configuration name. - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `-h, --help`: Print help ### Examples ```text coldctl destination test local-archive ``` ### Operational notes Creates the directory if missing, writes a unique temporary probe, syncs and reads it back, then removes the probe. Existing files are not overwritten. A created directory remains. Success is a point-in-time access check, not a capacity reservation. ### CLI help (trailing whitespace normalized) ```text Check storage access with a unique write, verified read and probe cleanup Usage: coldctl destination test [OPTIONS] Arguments: Options: --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] -h, --help Print help ``` ## coldctl destination remove Website: /docs/commands/destination-remove Remove destination configuration. Status: Implemented. Effect: Local state write. ### Syntax ```text coldctl destination remove [OPTIONS] ``` ### Arguments and options - ``: Existing local configuration name. - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `-h, --help`: Print help ### Examples ```text coldctl destination remove local-archive ``` ### Operational notes Preserves the destination directory and all archive files. Refused while a policy references the destination. ### CLI help (trailing whitespace normalized) ```text Remove configuration only; archive files and directories are preserved Usage: coldctl destination remove [OPTIONS] Arguments: Options: --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] -h, --help Print help ``` ## coldctl policy create Website: /docs/commands/policy-create Save an explicit retention policy. Status: Implemented. Effect: Local state write. ### Syntax ```text coldctl policy create [OPTIONS] --name --source --destination --table --time-column --older-than-days ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `--name `: Unique local configuration name: 1–64 ASCII letters, digits, hyphens or underscores. - `--output `: [default: table] [possible values: table, json] - `--source `: Name of a registered source. - `--destination `: Name of a registered local destination. - `--schema `: [default: public] - `--table
`: SQL table or MongoDB collection name. - `--time-column `: Date/timestamp column for the strict retention cutoff. - `--older-than-days `: Retention age from 1 to 365000 days. - `--batch-size `: [default: 500] - `--equals-column `: Optional text equality-filter column. Requires --equals-value. - `--equals-value `: Literal filter value. Requires --equals-column. Stored locally, excluded from Cloud reports. - `-h, --help`: Print help ### Examples ```text coldctl policy create --name orders-archive --source production --destination local-archive --schema public --table orders --time-column created_at --older-than-days 365 --equals-column status --equals-value completed --batch-size 500 ``` ### Operational notes Source and destination must exist. Retention is 1–365000 days. The parser accepts batch sizes 1–1000, but MySQL caps batches at 400 and MongoDB at 100. Equality column/value must be paired; MongoDB does not support equality filters. Create a new policy to change rules. ### CLI help (trailing whitespace normalized) ```text Save a retention policy. Validation against PostgreSQL is a separate read-only command Usage: coldctl policy create [OPTIONS] --name --source --destination --table
--time-column --older-than-days Options: --data-dir Override the OS application data directory --name --output [default: table] [possible values: table, json] --source --destination --schema [default: public] --table
--time-column --older-than-days --batch-size [default: 500] --equals-column --equals-value -h, --help Print help ``` ## coldctl policy list Website: /docs/commands/policy-list List saved retention policies. Status: Implemented. Effect: Local read. ### Syntax ```text coldctl policy list [OPTIONS] ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `-h, --help`: Print help ### Examples ```text coldctl policy list --output json ``` ### Operational notes Reads local policy definitions. A saved policy has not necessarily been validated against its database. ### CLI help (trailing whitespace normalized) ```text Usage: coldctl policy list [OPTIONS] Options: --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] -h, --help Print help ``` ## coldctl policy show Website: /docs/commands/policy-show Inspect a policy and its rule. Status: Implemented. Effect: Local read. ### Syntax ```text coldctl policy show [OPTIONS] ``` ### Arguments and options - ``: Existing local configuration name. - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `-h, --help`: Print help ### Examples ```text coldctl policy show orders-archive --output json ``` ### Operational notes Shows the saved source, destination, dataset, time field, retention and batch size. Deletion is authorized separately through archive delete or scheduler --delete-after-archive. ### CLI help (trailing whitespace normalized) ```text Usage: coldctl policy show [OPTIONS] Arguments: Options: --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] -h, --help Print help ``` ## coldctl policy validate Website: /docs/commands/policy-validate Validate a policy against its database. Status: Implemented. Effect: Database read. ### Syntax ```text coldctl policy validate [OPTIONS] ``` ### Arguments and options - ``: Existing local configuration name. - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `-h, --help`: Print help ### Examples ```text coldctl policy validate orders-archive ``` ### Operational notes Validates table access and supported schema/types and returns an estimated plan. It does not create archive files or probe destination write access. ### CLI help (trailing whitespace normalized) ```text Validate source/table/types and show an estimated archive plan without writing files Usage: coldctl policy validate [OPTIONS] Arguments: Options: --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] -h, --help Print help ``` ## coldctl policy remove Website: /docs/commands/policy-remove Remove a saved policy. Status: Implemented. Effect: Local state write. ### Syntax ```text coldctl policy remove [OPTIONS] ``` ### Arguments and options - ``: Existing local configuration name. - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `-h, --help`: Print help ### Examples ```text coldctl policy remove orders-archive ``` ### Operational notes Existing jobs and archive files are preserved. Removing and recreating policies does not provide deduplication across exports. ### CLI help (trailing whitespace normalized) ```text Remove policy configuration; existing job history and archive files are preserved Usage: coldctl policy remove [OPTIONS] Arguments: Options: --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] -h, --help Print help ``` ## coldctl archive outcomes Website: /docs/commands/archive-outcomes Inspect measured archive outcomes. Status: Implemented. Effect: Read-only local evidence. ### Syntax ```text coldctl archive outcomes [OPTIONS] ``` ### Arguments and options - ``: Original archive job ID. - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `-h, --help`: Print help ### Examples ```text coldctl archive outcomes JOB ``` ### Operational notes Shows deletion attribution, archive bytes, table/index observations, reusable heap space, physical allocation and optional observed billing. Unknown values are never reported as zero. ### CLI help (trailing whitespace normalized) ```text Show measured sizes, deletion attribution and optional billing observations Usage: coldctl archive outcomes [OPTIONS] Arguments: Options: --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] -h, --help Print help ``` ## coldctl archive reclaim Website: /docs/commands/archive-reclaim Reclaim PostgreSQL space in a bounded window. Status: Implemented. Effect: Source maintenance. ### Syntax ```text coldctl archive reclaim [OPTIONS] --confirm-original-source --window-start-unix-ms --window-end-unix-ms ``` ### Arguments and options - ``: Original archive job ID. - `--confirm-original-source`: Confirm maintenance targets the original source database. - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `--window-start-unix-ms `: Customer window start as UTC Unix milliseconds. - `--window-end-unix-ms `: Customer window end as UTC Unix milliseconds, at most 24 hours after start. - `--wait-timeout-seconds `: [default: 5] - `--max-duration-seconds `: Bound this attempt (1-290 seconds), also limited by the window end [default: 240] - `-h, --help`: Print help ### Examples ```text coldctl archive reclaim JOB --confirm-original-source --window-start-unix-ms START --window-end-unix-ms END ``` ### Operational notes Requires completed verified deletion. START/END are UTC Unix milliseconds. Default attempt limit 240 seconds; maximum 290. Verified server filesystem access and 2x free space are required for pg_repack. Always uses wait-timeout and no-kill-backend. Missing pg_repack falls back to VACUUM without truncation. ### CLI help (trailing whitespace normalized) ```text Reclaim within an active customer window (UTC Unix milliseconds, at most 24h) Usage: coldctl archive reclaim [OPTIONS] --confirm-original-source --window-start-unix-ms --window-end-unix-ms Arguments: Options: --confirm-original-source --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] --window-start-unix-ms --window-end-unix-ms --wait-timeout-seconds [default: 5] --max-duration-seconds Bound this attempt (1-290 seconds), also limited by the window end [default: 240] -h, --help Print help ``` ## coldctl archive reclaim-acknowledge Website: /docs/commands/archive-reclaim-acknowledge Acknowledge interrupted reclamation inspection. Status: Implemented. Effect: Local evidence update. ### Syntax ```text coldctl archive reclaim-acknowledge [OPTIONS] --confirm-inspected-source ``` ### Arguments and options - ``: Original archive job ID. - `--confirm-inspected-source`: Confirm source sessions and any pg_repack artifacts have been inspected and cleaned up. - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `-h, --help`: Print help ### Examples ```text coldctl archive reclaim-acknowledge JOB --confirm-inspected-source ``` ### Operational notes Inspect PostgreSQL sessions and pg_repack artifacts first. Clears the retry block without replaying maintenance or claiming success. ### CLI help (trailing whitespace normalized) ```text Acknowledge manual inspection/cleanup after an interrupted reclaim; never replays it Usage: coldctl archive reclaim-acknowledge [OPTIONS] --confirm-inspected-source Arguments: Options: --confirm-inspected-source --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] -h, --help Print help ``` ## coldctl archive record-billing Website: /docs/commands/archive-record-billing Record observed invoice totals. Status: Implemented. Effect: Local evidence update. ### Syntax ```text coldctl archive record-billing [OPTIONS] --currency --before-minor-units --after-minor-units --before-start-unix-ms --before-end-unix-ms --after-start-unix-ms --after-end-unix-ms ``` ### Arguments and options - ``: Original archive job ID. - `--currency `: Three-letter uppercase invoice currency code. - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `--before-minor-units `: Observed invoice total before archival in minor currency units. - `--after-minor-units `: Observed invoice total after archival in minor currency units. - `--before-start-unix-ms `: Start of the earlier completed invoice period, UTC Unix milliseconds. - `--before-end-unix-ms `: End of the earlier invoice period, UTC Unix milliseconds. - `--after-start-unix-ms `: Start of the later completed invoice period, UTC Unix milliseconds. - `--after-end-unix-ms `: End of the later invoice period, UTC Unix milliseconds. - `-h, --help`: Print help ### Examples ```text coldctl archive record-billing JOB --currency USD --before-minor-units 10000 --after-minor-units 9000 --before-start-unix-ms 1 --before-end-unix-ms 2 --after-start-unix-ms 2 --after-end-unix-ms 3 ``` ### Operational notes Supply actual completed invoice intervals in UTC Unix milliseconds. Totals are minor currency units. Observations are separate from uncalculated projections; unequal periods and concurrent workload changes prevent causal savings claims. ### CLI help (trailing whitespace normalized) ```text Record observed invoice totals in minor currency units for two completed periods Usage: coldctl archive record-billing [OPTIONS] --currency --before-minor-units --after-minor-units --before-start-unix-ms --before-end-unix-ms --after-start-unix-ms --after-end-unix-ms Arguments: Options: --currency --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] --before-minor-units --after-minor-units --before-start-unix-ms --before-end-unix-ms --after-start-unix-ms --after-end-unix-ms -h, --help Print help ``` ## coldctl archive group-run Website: /docs/commands/archive-group-run Archive and verify related tables. Status: Implemented. Effect: Local group lifecycle. ### Syntax ```text coldctl archive group-run [OPTIONS] --policies ... --confirm-quiescent-source ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `--policies ...`: Two to 32 explicit table policies on one source. - `--select-children`: Select descendants from archived parent FK values instead of child retention filters - `--confirm-quiescent-source`: Confirm all selected tables and their eligibility remain quiescent - `-h, --help`: Print help ### Examples ```text coldctl archive group-run --policies orders items --select-children --confirm-quiescent-source ``` ### Operational notes Requires matching CLI and connector builds. Groups require quiescent tables and are resumable, not atomic across tables. Configure automatic shadow validation for every member before scheduled deletion. ### CLI help (trailing whitespace normalized) ```text Archive and verify explicit PostgreSQL/MySQL table policies as a group. Never deletes Usage: coldctl archive group-run [OPTIONS] --policies ... --confirm-quiescent-source Options: --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] --policies ... --select-children Select descendants from archived parent FK values instead of child retention filters --confirm-quiescent-source Confirm all selected tables and their eligibility remain quiescent -h, --help Print help ``` ## coldctl archive group-delete Website: /docs/commands/archive-group-delete Delete a verified archive group. Status: Implemented. Effect: Local group lifecycle. ### Syntax ```text coldctl archive group-delete [OPTIONS] --jobs ... --confirm-delete --confirm-original-source ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `--jobs ...`: Completed archive job IDs for all group members. - `--output `: [default: table] [possible values: table, json] - `--confirm-delete`: Authorize verified source deletion. - `--confirm-original-source`: Authorize deletion against the original source. - `-h, --help`: Print help ### Examples ```text coldctl archive group-delete --jobs PARENT_JOB CHILD_JOB --confirm-delete --confirm-original-source ``` ### Operational notes Requires matching CLI and connector builds. Groups require quiescent tables and are resumable, not atomic across tables. Configure automatic shadow validation for every member before scheduled deletion. ### CLI help (trailing whitespace normalized) ```text Reverify every member, then delete children before parents; refuse remaining references Usage: coldctl archive group-delete [OPTIONS] --jobs ... --confirm-delete --confirm-original-source Options: --data-dir Override the OS application data directory --jobs ... --output [default: table] [possible values: table, json] --confirm-delete --confirm-original-source -h, --help Print help ``` ## coldctl archive shadow Website: /docs/commands/archive-shadow Compare source, archive and gateway. Status: Implemented. Effect: Read-only validation and local evidence. ### Syntax ```text coldctl archive shadow [OPTIONS] --confirm-original-source ``` ### Arguments and options - ``: Completed archive job ID - `--confirm-original-source`: Confirm the original source still retains the archived records - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `--max-batches `: Stop after this many batches; a partial report cannot pass - `--gateway-batch-size `: Keys per Gateway request (1-100); defaults to 64 - `--max-duration-seconds `: Stop after this duration; partial coverage cannot pass. Blocking storage I/O may finish first - `--gateway-url `: Gateway HTTPS origin (HTTP is allowed only on loopback) - `--dataset-id `: Gateway dataset ID containing this archive - `--token-env `: Environment variable containing the gateway bearer token - `-h, --help`: Print help ### Examples ```text coldctl archive shadow JOB_ID --confirm-original-source --gateway-url https://archive.example.com --dataset-id DATASET_ID --token-env COLDCTL_GATEWAY_TOKEN ``` ### Operational notes Read-only ID lookup comparison over the saved archive range. Requires retained stable source data and no deletion history. Partial, mismatch and error reports return nonzero; no secrets or row values are persisted. ### CLI help (trailing whitespace normalized) ```text Compare a completed archive with retained source rows and optional gateway ID lookups. Never deletes Usage: coldctl archive shadow [OPTIONS] --confirm-original-source Arguments: Completed archive job ID Options: --confirm-original-source Confirm the original source still retains the archived records --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] --max-batches Stop after this many batches; a partial report cannot pass --gateway-batch-size Keys per Gateway request (1-100); defaults to 64 --max-duration-seconds Stop after this duration; partial coverage cannot pass. Blocking storage I/O may finish first --gateway-url Gateway HTTPS origin (HTTP is allowed only on loopback) --dataset-id Gateway dataset ID containing this archive --token-env Environment variable containing the gateway bearer token -h, --help Print help ``` ## coldctl archive shadow-reports Website: /docs/commands/archive-shadow-reports Inspect durable shadow reports. Status: Implemented. Effect: Read-only validation and local evidence. ### Syntax ```text coldctl archive shadow-reports [OPTIONS] ``` ### Arguments and options - ``: Completed archive job ID - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `-h, --help`: Print help ### Examples ```text coldctl archive shadow-reports JOB_ID ``` ### Operational notes Shows the latest 100 reports. Running without a finish timestamp is unfinalized, never a pass. Reports are local and not tamper-proof. ### CLI help (trailing whitespace normalized) ```text Show up to 100 latest durable shadow reports; interrupted runs remain running/unfinalized Usage: coldctl archive shadow-reports [OPTIONS] Arguments: Completed archive job ID Options: --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] -h, --help Print help ``` ## coldctl archive delete Website: /docs/commands/archive-delete Delete unchanged records from a verified original archive. Status: Implemented. Effect: Explicit source deletion. ### Syntax ```text coldctl archive delete [OPTIONS] --confirm-delete --confirm-original-source ``` ### Arguments and options - ``: Archive job UUID from jobs list. - `--confirm-delete`: Required explicit authorization for irreversible source deletion. - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `--confirm-original-source`: Confirm the original source, stable rows, and immutable archive files - `--max-batches `: Positive number of source batches to reconcile in this invocation; omit to finish remaining work. - `-h, --help`: Print help ### Examples ```text coldctl archive delete JOB_ID --confirm-delete --confirm-original-source --max-batches 1 ``` ### Operational notes Supports PostgreSQL, MySQL and MongoDB with an approved verified-delete-v1 connector. Requires a completed locally created archive and original safety contract; imports cannot authorize deletion. A mismatch rolls back the entire source batch. Earlier commits remain. Retry the same job after interruption; do not reuse archived keys. max-batches must be positive. Keep archive files and local deletion checkpoints. The manifest stays unchanged. See Verified source deletion for engine permissions and locks. ### CLI help (trailing whitespace normalized) ```text Delete unchanged source records only after verifying their completed archive Usage: coldctl archive delete [OPTIONS] --confirm-delete --confirm-original-source Arguments: Options: --confirm-delete --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] --confirm-original-source Confirm the original source, stable rows, and immutable archive files --max-batches -h, --help Print help ``` ## coldctl archive deletion-status Website: /docs/commands/archive-deletion-status Inspect durable deletion progress. Status: Implemented. Effect: Local read. ### Syntax ```text coldctl archive deletion-status [OPTIONS] ``` ### Arguments and options - ``: Archive job UUID from jobs list. - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `-h, --help`: Print help ### Examples ```text coldctl archive deletion-status JOB_ID ``` ### Operational notes Returns null before deletion starts. rows_deleted counts acknowledged source commits; rows_absent also includes unrecorded source commits. Cloud export status does not include these counters. ### CLI help (trailing whitespace normalized) ```text Show durable source-deletion progress for an archive job Usage: coldctl archive deletion-status [OPTIONS] Arguments: Options: --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] -h, --help Print help ``` ## coldctl archive plan Website: /docs/commands/archive-plan Preview an estimated archive operation. Status: Implemented. Effect: Database read. ### Syntax ```text coldctl archive plan [OPTIONS] ``` ### Arguments and options - ``: Name of a saved policy. - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `-h, --help`: Print help ### Examples ```text coldctl archive plan orders-archive --output json ``` ### Operational notes Uses the source connector to validate supported shape and preview retention bounds. Planning creates no archive files and does not prove destination access or source stability. Estimates are not completion percentages. ### CLI help (trailing whitespace normalized) ```text Validate and estimate without reading table rows or writing archive files Usage: coldctl archive plan [OPTIONS] Arguments: Options: --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] -h, --help Print help ``` ## coldctl archive run Website: /docs/commands/archive-run Export a new job in bounded Parquet batches. Status: Implemented. Effect: Database read + local write. ### Syntax ```text coldctl archive run [OPTIONS] --source-stability ``` ### Arguments and options - ``: Name of a saved policy. - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `--source-stability `: Assert stable rows AND eligibility (including no backdated inserts) [possible values: immutable-rows, quiescent-copy, live-values] - `--allow-repeat`: Acknowledge that a new run can duplicate previous exports - `--max-rows `: Rows per invocation; 0 removes a saved limit - `--max-duration-seconds `: Seconds per invocation, checked at safe boundaries; 0 removes a saved limit - `--batch-delay-ms `: Delay between batches in ms, 0–3,600,000. Default: 0. - `--min-free-bytes `: Keep this many bytes free before each destination write - `--query-timeout-ms `: Statement timeout in ms, 1–300,000. Default: 10,000. - `--lock-timeout-ms `: Lock timeout in ms, 1–300,000 and no larger than query timeout. Default: 2,000. - `-h, --help`: Print help ### Examples ```text coldctl archive run orders-archive --source-stability immutable-rows --max-rows 10000 --batch-delay-ms 100 --output json ``` ### Operational notes Requires an explicit stable-source assertion. Use immutable-rows only if values and eligibility stay unchanged, including no backdated inserts; use quiescent-copy for a frozen copy. A replica alone is not frozen. A prior job requires --allow-repeat, acknowledging possible duplication. Does not delete source records. ### CLI help (trailing whitespace normalized) ```text Export eligible rows to a NEW local Parquet job directory. Never deletes source rows Usage: coldctl archive run [OPTIONS] --source-stability Arguments: Options: --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] --source-stability Assert stable rows AND eligibility (including no backdated inserts) [possible values: immutable-rows, quiescent-copy, live-values] --allow-repeat Acknowledge that a new run can duplicate previous exports --max-rows Rows per invocation; 0 removes a saved limit --max-duration-seconds Seconds per invocation, checked at safe boundaries; 0 removes a saved limit --batch-delay-ms --min-free-bytes Keep this many bytes free before each destination write --query-timeout-ms --lock-timeout-ms -h, --help Print help ``` ## coldctl archive resume Website: /docs/commands/archive-resume Continue the same durable archive job. Status: Implemented. Effect: Database read + local write. ### Syntax ```text coldctl archive resume [OPTIONS] ``` ### Arguments and options - ``: Job UUID from coldctl jobs list. - `--confirm-original-source`: Confirm the original database and continued stability of eligible rows - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `--max-rows `: Rows per invocation; 0 removes a saved limit - `--max-duration-seconds `: Seconds per invocation, checked at safe boundaries; 0 removes a saved limit - `--batch-delay-ms `: Delay between batches in ms, 0–3,600,000. Default: 0. - `--min-free-bytes `: Keep this many bytes free before each destination write - `--query-timeout-ms `: Statement timeout in ms, 1–300,000. Default: 10,000. - `--lock-timeout-ms `: Lock timeout in ms, 1–300,000 and no larger than query timeout. Default: 2,000. - `-h, --help`: Print help ### Examples ```text coldctl archive resume JOB_ID --confirm-original-source --max-rows 10000 --output json ``` ### Operational notes Replace JOB_ID with an ID from jobs list. Incomplete jobs require confirmation that the original source and stable eligibility still apply. Preserves cutoff, upper key, schema and object names; verifies committed objects before recovery. Unspecified controls keep saved values. A completed job is verified without exporting more rows. ### CLI help (trailing whitespace normalized) ```text Continue an interrupted, failed, or cancelled job using its saved checkpoint Usage: coldctl archive resume [OPTIONS] Arguments: Options: --confirm-original-source Confirm the original database and continued stability of eligible rows --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] --max-rows Rows per invocation; 0 removes a saved limit --max-duration-seconds Seconds per invocation, checked at safe boundaries; 0 removes a saved limit --batch-delay-ms --min-free-bytes Keep this many bytes free before each destination write --query-timeout-ms --lock-timeout-ms -h, --help Print help ``` ## coldctl archive verify Website: /docs/commands/archive-verify Verify a completed archive against its journal. Status: Implemented. Effect: Local read + state write. ### Syntax ```text coldctl archive verify [OPTIONS] ``` ### Arguments and options - ``: Job UUID from coldctl jobs list. - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `-h, --help`: Print help ### Examples ```text coldctl archive verify JOB_ID --output json ``` ### Operational notes Works offline without source credentials. Checks manifest, object size/SHA-256, Parquet schema/type metadata, decoded counts, key order, boundaries and totals. Success saves verified_at; failure clears it. Integrity does not prove equality to a changing source, historical authenticity, or authorization to delete data. ### CLI help (trailing whitespace normalized) ```text Verify a completed archive offline, without database credentials Usage: coldctl archive verify [OPTIONS] Arguments: Options: --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] -h, --help Print help ``` ## coldctl archive inspect Website: /docs/commands/archive-inspect Read a standalone archive manifest. Status: Implemented. Effect: Local read. ### Syntax ```text coldctl archive inspect [OPTIONS] ``` ### Arguments and options - ``: Completed archive directory named with its original job UUID. - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `-h, --help`: Print help ### Examples ```text coldctl archive inspect ./archives/JOB_ID --output json ``` ### Operational notes Keep the directory named with its original job UUID. Inspects supported formats 2 (PostgreSQL), 3 (MySQL) and 4 (MongoDB) without verifying every archive object. No source credentials are needed. ### CLI help (trailing whitespace normalized) ```text Inspect a format-v2 manifest without the original state or database Usage: coldctl archive inspect [OPTIONS] Arguments: Options: --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] -h, --help Print help ``` ## coldctl archive verify-directory Website: /docs/commands/archive-verify-directory Verify a completed archive without original state. Status: Implemented. Effect: Local read + temporary scratch. ### Syntax ```text coldctl archive verify-directory [OPTIONS] ``` ### Arguments and options - ``: Completed archive directory named with its original job UUID. - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `-h, --help`: Print help ### Examples ```text coldctl archive verify-directory ./archives/JOB_ID --output json ``` ### Operational notes Checks supported format 2/3/4 manifests and files using temporary scratch state. Rejects malformed metadata, traversal and internal symlinks/reparse points. Verification establishes consistency with the manifest, not authenticity or deletion authorization. ### CLI help (trailing whitespace normalized) ```text Check manifest/file consistency offline; this does not prove authenticity Usage: coldctl archive verify-directory [OPTIONS] Arguments: Options: --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] -h, --help Print help ``` ## coldctl archive import Website: /docs/commands/archive-import Recover a completed archive into local state. Status: Implemented. Effect: Local state write. ### Syntax ```text coldctl archive import [OPTIONS] --accept-untrusted-manifest ``` ### Arguments and options - ``: Completed archive directory named with its original job UUID. - `--accept-untrusted-manifest`: Acknowledge that supplied files and manifest do not prove authenticity - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `-h, --help`: Print help ### Examples ```text coldctl --data-dir ./recovered-state archive import ./archives/JOB_ID --accept-untrusted-manifest --output json ``` ### Operational notes Initialize the target state first. Explicitly acknowledges that supplied archive files do not prove authenticity. Verifies files before atomically importing completed-job metadata; duplicate IDs are refused. Does not recreate sources/policies or enable additional export rows. Partial exports still need their original journal and referenced files. ### CLI help (trailing whitespace normalized) ```text Verify and import a completed archive; never overwrites an existing job ID Usage: coldctl archive import [OPTIONS] --accept-untrusted-manifest Arguments: Options: --accept-untrusted-manifest Acknowledge that supplied files and manifest do not prove authenticity --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] -h, --help Print help ``` ## coldctl archive staging-list Website: /docs/commands/archive-staging-list List unreferenced staging candidates. Status: Implemented. Effect: Local read. ### Syntax ```text coldctl archive staging-list [OPTIONS] ``` ### Arguments and options - ``: Job UUID from coldctl jobs list. - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `-h, --help`: Print help ### Examples ```text coldctl archive staging-list JOB_ID --output json ``` ### Operational notes Inspection only: never deletes files. Do not remove checkpoint references or staging files to force recovery. Active workers must not be disturbed; preserve evidence for investigation. ### CLI help (trailing whitespace normalized) ```text List unreferenced staging candidates only; never deletes files Usage: coldctl archive staging-list [OPTIONS] Arguments: Options: --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] -h, --help Print help ``` ## coldctl archive restore Website: /docs/commands/archive-restore Restore verified data into a separate same-engine target. Status: Implemented. Effect: Target database write. ### Syntax ```text coldctl archive restore [OPTIONS] --target --schema --table
--confirm-separate-target ``` ### Arguments and options - ``: Job UUID from coldctl jobs list. - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `--target `: Name of a separately registered target using the archive's database engine. - `--schema `: Target SQL schema or MongoDB database; required for restore. - `--table
`: Target SQL table or MongoDB collection name. - `--confirm-separate-target`: Required acknowledgement that the target is separate from the operational source. - `--max-batches `: Optional committed-batch limit for this restore invocation. Stops at a safe boundary. - `-h, --help`: Print help ### Examples ```text coldctl archive restore JOB_ID --target restore-target --schema public --table restored_orders --confirm-separate-target --max-batches 2 --output json ``` ### Operational notes Register a separate target with the matching engine connector. The target database/schema must exist. Creates an isolated target and companion recovery journal, verifies before writing, and commits each batch with its checkpoint. Repeat the identical command to resume. ### CLI help (trailing whitespace normalized) ```text Restore into a new dedicated target table in the same database engine; rerun to resume Usage: coldctl archive restore [OPTIONS] --target --schema --table
--confirm-separate-target Arguments: Options: --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] --target --schema --table
--confirm-separate-target --max-batches -h, --help Print help ``` ## coldctl archive validate-restore Website: /docs/commands/archive-validate-restore Compare restored values with the archive. Status: Implemented. Effect: Target database read. ### Syntax ```text coldctl archive validate-restore [OPTIONS] --target --schema --table
``` ### Arguments and options - ``: Job UUID from coldctl jobs list. - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `--target `: Name of a separately registered target using the archive's database engine. - `--schema `: Target SQL schema or MongoDB database; required for validation. - `--table
`: Target SQL table or MongoDB collection name. - `-h, --help`: Print help ### Examples ```text coldctl archive validate-restore JOB_ID --target restore-target --schema public --table restored_orders --output json ``` ### Operational notes Uses the matching connector to compare committed restored values to the archive, including partial restores. It does not reconstruct application DDL or validate a changed source. ### CLI help (trailing whitespace normalized) ```text Compare every committed restored value with the verified archive Usage: coldctl archive validate-restore [OPTIONS] --target --schema --table
Arguments: Options: --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] --target --schema --table
-h, --help Print help ``` ## coldctl jobs list Website: /docs/commands/jobs-list List locally recorded archive jobs. Status: Implemented. Effect: Local read. ### Syntax ```text coldctl jobs list [OPTIONS] ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `-h, --help`: Print help ### Examples ```text coldctl jobs list --output json ``` ### Operational notes Reports durable state. A saved running status does not prove a worker is alive after a crash. Use IDs from this output for verification, recovery and cancellation. ### CLI help (trailing whitespace normalized) ```text Usage: coldctl jobs list [OPTIONS] Options: --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] -h, --help Print help ``` ## coldctl jobs show Website: /docs/commands/jobs-show Inspect job progress, checkpoints and failures. Status: Implemented. Effect: Local read. ### Syntax ```text coldctl jobs show [OPTIONS] ``` ### Arguments and options - ``: Job UUID from coldctl jobs list. - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `-h, --help`: Print help ### Examples ```text coldctl jobs show JOB_ID --output json ``` ### Operational notes Includes durable counters, saved execution controls, progress observation time, safe failure diagnostics, verification time and archive path. Rates are historical samples; no estimated completion percentage is implied. ### CLI help (trailing whitespace normalized) ```text Usage: coldctl jobs show [OPTIONS] Arguments: Options: --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] -h, --help Print help ``` ## coldctl jobs cancel Website: /docs/commands/jobs-cancel Request a cooperative stop. Status: Implemented. Effect: Local state write. ### Syntax ```text coldctl jobs cancel [OPTIONS] ``` ### Arguments and options - ``: Job UUID from coldctl jobs list. - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `-h, --help`: Print help ### Examples ```text coldctl jobs cancel JOB_ID ``` ### Operational notes Stops an active worker after its current batch is published and checkpointed. Files are retained. An idle interrupted or failed job is cancelled immediately; finalization can win a race. Resume or retry explicitly clears the cancellation request. ### CLI help (trailing whitespace normalized) ```text Request a safe stop after the current batch; published files are preserved Usage: coldctl jobs cancel [OPTIONS] Arguments: Options: --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] -h, --help Print help ``` ## coldctl jobs retry Website: /docs/commands/jobs-retry Retry in place after fixing the cause. Status: Implemented. Effect: Database read + local write. ### Syntax ```text coldctl jobs retry [OPTIONS] ``` ### Arguments and options - ``: Job UUID from coldctl jobs list. - `--confirm-original-source`: Confirm the original database and continued stability of eligible rows - `--data-dir `: Override the OS application data directory - `--output `: [default: table] [possible values: table, json] - `--max-rows `: Rows per invocation; 0 removes a saved limit - `--max-duration-seconds `: Seconds per invocation, checked at safe boundaries; 0 removes a saved limit - `--batch-delay-ms `: Delay between batches in ms, 0–3,600,000. Default: 0. - `--min-free-bytes `: Keep this many bytes free before each destination write - `--query-timeout-ms `: Statement timeout in ms, 1–300,000. Default: 10,000. - `--lock-timeout-ms `: Lock timeout in ms, 1–300,000 and no larger than query timeout. Default: 2,000. - `-h, --help`: Print help ### Examples ```text coldctl jobs retry JOB_ID --confirm-original-source --output json ``` ### Operational notes Explicitly resumes the same job after the cause is fixed and original source stability is confirmed. This command is one invocation. Use the local scheduler for bounded automatic retries. ### CLI help (trailing whitespace normalized) ```text Retry a failed or interrupted job in place (same behavior as archive resume) Usage: coldctl jobs retry [OPTIONS] Arguments: Options: --confirm-original-source Confirm the original database and continued stability of eligible rows --data-dir Override the OS application data directory --output [default: table] [possible values: table, json] --max-rows Rows per invocation; 0 removes a saved limit --max-duration-seconds Seconds per invocation, checked at safe boundaries; 0 removes a saved limit --batch-delay-ms --min-free-bytes Keep this many bytes free before each destination write --query-timeout-ms --lock-timeout-ms -h, --help Print help ``` ## coldctl agent sync-status Website: /docs/commands/agent-sync-status Inspect Cloud synchronization. Status: Implemented. Effect: Local status. ### Syntax ```text coldctl agent sync-status [OPTIONS] ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `-h, --help`: Print help ### Examples ```text coldctl agent sync-status ``` ### Operational notes Last success, sanitized error, consecutive failures and next scheduled attempt survive restart. A scheduled timestamp does not prove a worker is alive. ### CLI help (trailing whitespace normalized) ```text Show persisted Cloud sync attempts, failures and last successful upload Usage: coldctl agent sync-status [OPTIONS] Options: --data-dir Override the OS application data directory -h, --help Print help ``` ## coldctl agent monitor serve Website: /docs/commands/agent-monitor-serve Serve local metrics, durable failures and optional alert delivery. Status: Implemented. Effect: Local monitoring; explicit webhook delivery when configured. ### Syntax ```text coldctl agent monitor serve [OPTIONS] ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `--listen `: [default: 127.0.0.1:9464] - `--token-env `: Bearer token environment reference; required for non-loopback listeners - `--poll-seconds `: [default: 10] - `--webhook-url-env `: Environment variable containing the HTTPS alert receiver URL - `--webhook-token-env `: Environment reference for the optional alert receiver bearer token. - `--allow-loopback-http`: Allow only numeric loopback HTTP for webhook test fixtures - `-h, --help`: Print help ### Examples ```text coldctl agent monitor serve --token-env COLDCTL_METRICS_TOKEN --webhook-url-env COLDCTL_ALERT_URL --webhook-token-env COLDCTL_ALERT_TOKEN ``` ### Operational notes Stop workers and run coldctl init for schema 17. Run the monitor separately from the scheduler using the same --data-dir. Non-loopback listeners require a bearer token and TLS at ingress. HTTPS webhooks use a durable at-least-once outbox; deduplicate event IDs. ### CLI help (trailing whitespace normalized) ```text Serve Prometheus metrics and durable failure history independently of the worker Usage: coldctl agent monitor serve [OPTIONS] Options: --data-dir Override the OS application data directory --listen [default: 127.0.0.1:9464] --token-env Bearer token environment reference; required for non-loopback listeners --poll-seconds [default: 10] --webhook-url-env Environment variable containing the HTTPS alert receiver URL --webhook-token-env --allow-loopback-http Allow only numeric loopback HTTP for webhook test fixtures -h, --help Print help ``` ## coldctl agent monitor status Website: /docs/commands/agent-monitor-status Inspect persisted monitoring history and alert delivery state. Status: Implemented. Effect: Local monitoring history. ### Syntax ```text coldctl agent monitor status [OPTIONS] ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `-h, --help`: Print help ### Examples ```text coldctl agent monitor status ``` ### Operational notes Shows the latest 100 stored transitions, delivery attempts and last HTTP status. Does not send alerts. Monitoring must have been initialized by agent monitor serve. ### CLI help (trailing whitespace normalized) ```text Show up to 100 persisted incident transitions and alert delivery results as JSON Usage: coldctl agent monitor status [OPTIONS] Options: --data-dir Override the OS application data directory -h, --help Print help ``` ## coldctl agent scheduler group create Website: /docs/commands/agent-scheduler-group-create Schedule parent-child archival. Status: Implemented. Effect: Local group lifecycle. ### Syntax ```text coldctl agent scheduler group create [OPTIONS] --every-seconds --confirm-quiescent-source --confirm-original-source --delete-after-archive ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `--policies ...`: Two to 32 explicit table policies on one source. - `--every-seconds `: Delay after a completed cycle, at least 60 seconds. - `--confirm-quiescent-source`: Assert all member tables remain quiescent through the full cycle and retries. - `--confirm-original-source`: Authorize deletion against the original source. - `--delete-after-archive`: Authorize repeated verified source deletion. - `--max-retries `: [default: 5] - `--max-rows `: Rows per invocation; 0 removes a saved limit - `--max-duration-seconds `: Seconds per invocation, checked at safe boundaries; 0 removes a saved limit - `--batch-delay-ms `: Delay between batches in milliseconds. - `--min-free-bytes `: Keep this many bytes free before each destination write - `--query-timeout-ms `: Source query timeout in milliseconds. - `--lock-timeout-ms `: Source lock timeout in milliseconds. - `-h, --help`: Print help ### Examples ```text coldctl agent scheduler group create --policies orders items --every-seconds 3600 --confirm-quiescent-source --confirm-original-source --delete-after-archive ``` ### Operational notes Requires matching CLI and connector builds. Groups require quiescent tables and are resumable, not atomic across tables. Configure automatic shadow validation for every member before scheduled deletion. ### CLI help (trailing whitespace normalized) ```text Usage: coldctl agent scheduler group create [OPTIONS] --every-seconds --confirm-quiescent-source --confirm-original-source --delete-after-archive Options: --data-dir Override the OS application data directory --policies ... --every-seconds --confirm-quiescent-source --confirm-original-source --delete-after-archive --max-retries [default: 5] --max-rows Rows per invocation; 0 removes a saved limit --max-duration-seconds Seconds per invocation, checked at safe boundaries; 0 removes a saved limit --batch-delay-ms --min-free-bytes Keep this many bytes free before each destination write --query-timeout-ms --lock-timeout-ms -h, --help Print help ``` ## coldctl agent scheduler group list Website: /docs/commands/agent-scheduler-group-list List recurring groups. Status: Implemented. Effect: Local group lifecycle. ### Syntax ```text coldctl agent scheduler group list [OPTIONS] ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `-h, --help`: Print help ### Examples ```text coldctl agent scheduler group list ``` ### Operational notes Requires matching CLI and connector builds. Groups require quiescent tables and are resumable, not atomic across tables. Configure automatic shadow validation for every member before scheduled deletion. ### CLI help (trailing whitespace normalized) ```text Usage: coldctl agent scheduler group list [OPTIONS] Options: --data-dir Override the OS application data directory -h, --help Print help ``` ## coldctl agent scheduler group show Website: /docs/commands/agent-scheduler-group-show Inspect a recurring group. Status: Implemented. Effect: Local group lifecycle. ### Syntax ```text coldctl agent scheduler group show [OPTIONS] ``` ### Arguments and options - ``: Recurring group identifier. - `--data-dir `: Override the OS application data directory - `-h, --help`: Print help ### Examples ```text coldctl agent scheduler group show GROUP_ID ``` ### Operational notes Requires matching CLI and connector builds. Groups require quiescent tables and are resumable, not atomic across tables. Configure automatic shadow validation for every member before scheduled deletion. ### CLI help (trailing whitespace normalized) ```text Usage: coldctl agent scheduler group show [OPTIONS] Arguments: Options: --data-dir Override the OS application data directory -h, --help Print help ``` ## coldctl agent scheduler group stop Website: /docs/commands/agent-scheduler-group-stop Stop a recurring group. Status: Implemented. Effect: Local group lifecycle. ### Syntax ```text coldctl agent scheduler group stop [OPTIONS] ``` ### Arguments and options - ``: Recurring group identifier. - `--data-dir `: Override the OS application data directory - `-h, --help`: Print help ### Examples ```text coldctl agent scheduler group stop GROUP_ID ``` ### Operational notes Requires matching CLI and connector builds. Groups require quiescent tables and are resumable, not atomic across tables. Configure automatic shadow validation for every member before scheduled deletion. ### CLI help (trailing whitespace normalized) ```text Usage: coldctl agent scheduler group stop [OPTIONS] Arguments: Options: --data-dir Override the OS application data directory -h, --help Print help ``` ## coldctl agent scheduler group retry Website: /docs/commands/agent-scheduler-group-retry Retry a held group. Status: Implemented. Effect: Local group lifecycle. ### Syntax ```text coldctl agent scheduler group retry [OPTIONS] --confirm-original-source ``` ### Arguments and options - ``: Recurring group identifier. - `--confirm-original-source`: Authorize deletion against the original source. - `--data-dir `: Override the OS application data directory - `-h, --help`: Print help ### Examples ```text coldctl agent scheduler group retry GROUP_ID --confirm-original-source ``` ### Operational notes Requires matching CLI and connector builds. Groups require quiescent tables and are resumable, not atomic across tables. Configure automatic shadow validation for every member before scheduled deletion. ### CLI help (trailing whitespace normalized) ```text Usage: coldctl agent scheduler group retry [OPTIONS] --confirm-original-source Arguments: Options: --confirm-original-source --data-dir Override the OS application data directory -h, --help Print help ``` ## coldctl agent scheduler recurring create Website: /docs/commands/agent-scheduler-recurring-create Create a recurring archive and delete schedule. Status: Implemented. Effect: Local schedule write. ### Syntax ```text coldctl agent scheduler recurring create [OPTIONS] --every-seconds --source-stability --confirm-original-source --delete-after-archive ``` ### Arguments and options - ``: Existing retention policy using PostgreSQL/MySQL/MongoDB and an S3 or Azure Blob destination - `--data-dir `: Override the OS application data directory - `--every-seconds `: Delay after completed deletion before the next cycle (60..31536000 seconds) - `--source-stability `: Assert source stability during every cycle, including downtime and retries [possible values: immutable-rows, quiescent-copy, live-values] - `--confirm-original-source`: Authorize automatic resumes against the original source for every cycle - `--delete-after-archive`: Explicitly authorize repeated verified source deletion (irreversible) - `--max-retries `: Consecutive transient retries before holding the schedule for recovery [default: 5] - `--max-rows `: Rows per invocation; 0 removes a saved limit - `--max-duration-seconds `: Seconds per invocation, checked at safe boundaries; 0 removes a saved limit - `--batch-delay-ms `: Delay between archive batches, 0–3,600,000 milliseconds; default 0. - `--min-free-bytes `: Keep this many bytes free before each destination write - `--query-timeout-ms `: Archive query timeout, 1–300,000 milliseconds; default 10,000. - `--lock-timeout-ms `: Archive lock timeout, 1–300,000 milliseconds; default 2,000, no greater than query timeout. - `-h, --help`: Print help ### Examples ```text coldctl agent scheduler recurring create orders-archive --every-seconds 86400 --source-stability immutable-rows --confirm-original-source --delete-after-archive ``` ### Operational notes PostgreSQL and MongoDB to S3 or Azure Blob. Each cycle requires stable eligible rows; MongoDB requires a quiescent source. Fresh cycles wait for completed verified deletion. Late arrivals between cycles are found by a full eligible-range rescan. Run agent scheduler run under a service manager. Ctrl+C is restartable; recurring stop is terminal and an in-flight batch may commit. Failures retain the same job/cutoff. Policy/source/destination configuration is frozen. ### CLI help (trailing whitespace normalized) ```text Authorize repeated archive/verify/delete cycles until stopped Usage: coldctl agent scheduler recurring create [OPTIONS] --every-seconds --source-stability --confirm-original-source --delete-after-archive Arguments: Existing retention policy using PostgreSQL/MySQL/MongoDB and an S3 or Azure Blob destination Options: --data-dir Override the OS application data directory --every-seconds Delay after completed deletion before the next cycle (60..31536000 seconds) --source-stability Assert source stability during every cycle, including downtime and retries [possible values: immutable-rows, quiescent-copy, live-values] --confirm-original-source Authorize automatic resumes against the original source for every cycle --delete-after-archive Explicitly authorize repeated verified source deletion (irreversible) --max-retries Consecutive transient retries before holding the schedule for recovery [default: 5] --max-rows Rows per invocation; 0 removes a saved limit --max-duration-seconds Seconds per invocation, checked at safe boundaries; 0 removes a saved limit --batch-delay-ms --min-free-bytes Keep this many bytes free before each destination write --query-timeout-ms --lock-timeout-ms -h, --help Print help ``` ## coldctl agent scheduler recurring list Website: /docs/commands/agent-scheduler-recurring-list List recurring schedules. Status: Implemented. Effect: Read local state. ### Syntax ```text coldctl agent scheduler recurring list [OPTIONS] ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `-h, --help`: Print help ### Examples ```text coldctl agent scheduler recurring list ``` ### Operational notes PostgreSQL and MongoDB to S3 or Azure Blob. Each cycle requires stable eligible rows; MongoDB requires a quiescent source. Fresh cycles wait for completed verified deletion. Late arrivals between cycles are found by a full eligible-range rescan. Run agent scheduler run under a service manager. Ctrl+C is restartable; recurring stop is terminal and an in-flight batch may commit. Failures retain the same job/cutoff. Policy/source/destination configuration is frozen. ### CLI help (trailing whitespace normalized) ```text Usage: coldctl agent scheduler recurring list [OPTIONS] Options: --data-dir Override the OS application data directory -h, --help Print help ``` ## coldctl agent scheduler recurring show Website: /docs/commands/agent-scheduler-recurring-show Inspect a recurring schedule. Status: Implemented. Effect: Read local state. ### Syntax ```text coldctl agent scheduler recurring show [OPTIONS] ``` ### Arguments and options - ``: Recurring schedule ID returned by create or list - `--data-dir `: Override the OS application data directory - `-h, --help`: Print help ### Examples ```text coldctl agent scheduler recurring show SCHEDULE_ID ``` ### Operational notes PostgreSQL and MongoDB to S3 or Azure Blob. Each cycle requires stable eligible rows; MongoDB requires a quiescent source. Fresh cycles wait for completed verified deletion. Late arrivals between cycles are found by a full eligible-range rescan. Run agent scheduler run under a service manager. Ctrl+C is restartable; recurring stop is terminal and an in-flight batch may commit. Failures retain the same job/cutoff. Policy/source/destination configuration is frozen. ### CLI help (trailing whitespace normalized) ```text Usage: coldctl agent scheduler recurring show [OPTIONS] Arguments: Recurring schedule ID returned by create or list Options: --data-dir Override the OS application data directory -h, --help Print help ``` ## coldctl agent scheduler recurring stop Website: /docs/commands/agent-scheduler-recurring-stop Stop a recurring schedule. Status: Implemented. Effect: Cancel future cycles. ### Syntax ```text coldctl agent scheduler recurring stop [OPTIONS] ``` ### Arguments and options - ``: Recurring schedule ID returned by create or list - `--data-dir `: Override the OS application data directory - `-h, --help`: Print help ### Examples ```text coldctl agent scheduler recurring stop SCHEDULE_ID ``` ### Operational notes PostgreSQL and MongoDB to S3 or Azure Blob. Each cycle requires stable eligible rows; MongoDB requires a quiescent source. Fresh cycles wait for completed verified deletion. Late arrivals between cycles are found by a full eligible-range rescan. Run agent scheduler run under a service manager. Ctrl+C is restartable; recurring stop is terminal and an in-flight batch may commit. Failures retain the same job/cutoff. Policy/source/destination configuration is frozen. ### CLI help (trailing whitespace normalized) ```text Permanently stop new cycles and cancel the active cycle at a safe boundary Usage: coldctl agent scheduler recurring stop [OPTIONS] Arguments: Recurring schedule ID returned by create or list Options: --data-dir Override the OS application data directory -h, --help Print help ``` ## coldctl agent scheduler recurring retry Website: /docs/commands/agent-scheduler-recurring-retry Retry a failed recurring cycle. Status: Implemented. Effect: Local schedule write. ### Syntax ```text coldctl agent scheduler recurring retry [OPTIONS] --confirm-original-source ``` ### Arguments and options - ``: Failed recurring schedule ID - `--confirm-original-source`: Confirm continued source identity and stability before retrying - `--data-dir `: Override the OS application data directory - `-h, --help`: Print help ### Examples ```text coldctl agent scheduler recurring retry SCHEDULE_ID --confirm-original-source ``` ### Operational notes PostgreSQL and MongoDB to S3 or Azure Blob. Each cycle requires stable eligible rows; MongoDB requires a quiescent source. Fresh cycles wait for completed verified deletion. Late arrivals between cycles are found by a full eligible-range rescan. Run agent scheduler run under a service manager. Ctrl+C is restartable; recurring stop is terminal and an in-flight batch may commit. Failures retain the same job/cutoff. Policy/source/destination configuration is frozen. ### CLI help (trailing whitespace normalized) ```text Retry the same failed cycle; never skip an incomplete deletion Usage: coldctl agent scheduler recurring retry [OPTIONS] --confirm-original-source Arguments: Failed recurring schedule ID Options: --confirm-original-source Confirm continued source identity and stability before retrying --data-dir Override the OS application data directory -h, --help Print help ``` ## coldctl agent scheduler enqueue Website: /docs/commands/agent-scheduler-enqueue Queue a long-running archive. Status: Implemented. Effect: Local queue write. ### Syntax ```text coldctl agent scheduler enqueue [OPTIONS] --source-stability --confirm-original-source ``` ### Arguments and options - ``: Existing local retention policy name. - `--data-dir `: Override the OS application data directory - `--job `: Adopt this incomplete job instead of creating a new archive - `--source-stability `: [possible values: immutable-rows, quiescent-copy, live-values] - `--confirm-original-source`: Authorize automatic resumes against the original, continuously stable source - `--allow-repeat`: Acknowledge possible overlap with prior exports when creating a new archive job. - `--delete-after-archive`: Delete unchanged source records after archive verification (irreversible) - `--delay-seconds `: Delay initial execution by this many seconds (maximum one year) [default: 0] - `--max-retries `: Consecutive transient retries, with exponential backoff (maximum 100) [default: 5] - `--max-rows `: Rows per invocation; 0 removes a saved limit - `--max-duration-seconds `: Seconds per invocation, checked at safe boundaries; 0 removes a saved limit - `--batch-delay-ms `: Delay between archive batches, 0–3,600,000 milliseconds; default 0. - `--min-free-bytes `: Keep this many bytes free before each destination write - `--query-timeout-ms `: Archive query timeout, 1–300,000 milliseconds; default 10,000. - `--lock-timeout-ms `: Archive lock timeout, 1–300,000 milliseconds; default 2,000, no greater than query timeout. - `-h, --help`: Print help ### Examples ```text coldctl agent scheduler enqueue orders-archive --source-stability immutable-rows --confirm-original-source ``` ### Operational notes One active queue entry per policy. Default slice: 60 seconds; positive duration required. --max-rows is per slice. --job adopts an incomplete original job. --delete-after-archive explicitly authorizes verified deletion; MongoDB requires quiescent-copy. ### CLI help (trailing whitespace normalized) ```text Queue one archive; automatically resume checkpoints until it completes Usage: coldctl agent scheduler enqueue [OPTIONS] --source-stability --confirm-original-source Arguments: Options: --data-dir Override the OS application data directory --job Adopt this incomplete job instead of creating a new archive --source-stability [possible values: immutable-rows, quiescent-copy, live-values] --confirm-original-source Authorize automatic resumes against the original, continuously stable source --allow-repeat --delete-after-archive Delete unchanged source records after archive verification (irreversible) --delay-seconds Delay initial execution by this many seconds (maximum one year) [default: 0] --max-retries Consecutive transient retries, with exponential backoff (maximum 100) [default: 5] --max-rows Rows per invocation; 0 removes a saved limit --max-duration-seconds Seconds per invocation, checked at safe boundaries; 0 removes a saved limit --batch-delay-ms --min-free-bytes Keep this many bytes free before each destination write --query-timeout-ms --lock-timeout-ms -h, --help Print help ``` ## coldctl agent scheduler run Website: /docs/commands/agent-scheduler-run Run the local scheduler worker. Status: Implemented. Effect: Archive writes / opt-in deletion. ### Syntax ```text coldctl agent scheduler run [OPTIONS] ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `--once`: Execute at most one due slice, then exit - `-h, --help`: Print help ### Examples ```text coldctl agent scheduler run ``` ### Operational notes Foreground single worker per state directory; --once runs at most one due slice. Restart recovers claimed work. Provide credentials and approved connectors in the worker environment. Explicit recurring schedules admit fresh PostgreSQL/MongoDB cycles to S3 or Azure Blob after verified deletion. Cloud sync remains manual. ### CLI help (trailing whitespace normalized) ```text Run the foreground worker. Use a service manager for unattended operation Usage: coldctl agent scheduler run [OPTIONS] Options: --data-dir Override the OS application data directory --once Execute at most one due slice, then exit -h, --help Print help ``` ## coldctl agent scheduler list Website: /docs/commands/agent-scheduler-list List persistent queue entries. Status: Implemented. Effect: Local read. ### Syntax ```text coldctl agent scheduler list [OPTIONS] ``` ### Arguments and options - `--data-dir `: Override the OS application data directory - `-h, --help`: Print help ### Examples ```text coldctl agent scheduler list ``` ### Operational notes Outputs JSON. Queue IDs differ from archive job IDs. ### CLI help (trailing whitespace normalized) ```text List persistent queue entries as JSON Usage: coldctl agent scheduler list [OPTIONS] Options: --data-dir Override the OS application data directory -h, --help Print help ``` ## coldctl agent scheduler show Website: /docs/commands/agent-scheduler-show Inspect a scheduler queue entry. Status: Implemented. Effect: Local read. ### Syntax ```text coldctl agent scheduler show [OPTIONS] ``` ### Arguments and options - ``: Queue UUID returned by scheduler enqueue; not the archive job ID. - `--data-dir `: Override the OS application data directory - `-h, --help`: Print help ### Examples ```text coldctl agent scheduler show QUEUE_ID ``` ### Operational notes due_at is a UTC Unix timestamp; use the assigned job_id with jobs show. ### CLI help (trailing whitespace normalized) ```text Inspect one queue entry as JSON Usage: coldctl agent scheduler show [OPTIONS] Arguments: Options: --data-dir Override the OS application data directory -h, --help Print help ``` ## coldctl agent scheduler cancel Website: /docs/commands/agent-scheduler-cancel Cancel a queue entry. Status: Implemented. Effect: Local cancellation request. ### Syntax ```text coldctl agent scheduler cancel [OPTIONS] ``` ### Arguments and options - ``: Queue UUID returned by scheduler enqueue; not the archive job ID. - `--data-dir `: Override the OS application data directory - `-h, --help`: Print help ### Examples ```text coldctl agent scheduler cancel QUEUE_ID ``` ### Operational notes Stops future batches and requests active archival to stop at a safe boundary. An in-flight batch may commit. Committed deletions are not undone. Cancellation is terminal for the queue. ### CLI help (trailing whitespace normalized) ```text Cancel a queue entry and request its active job to stop at a batch boundary Usage: coldctl agent scheduler cancel [OPTIONS] Arguments: Options: --data-dir Override the OS application data directory -h, --help Print help ``` ## coldctl agent scheduler retry Website: /docs/commands/agent-scheduler-retry Retry a failed queue entry. Status: Implemented. Effect: Local queue write. ### Syntax ```text coldctl agent scheduler retry [OPTIONS] --confirm-original-source ``` ### Arguments and options - ``: Queue UUID returned by scheduler enqueue; not the archive job ID. - `--confirm-original-source`: Required confirmation that the original source and eligible records remain stable throughout retries. - `--data-dir `: Override the OS application data directory - `-h, --help`: Print help ### Examples ```text coldctl agent scheduler retry QUEUE_ID --confirm-original-source ``` ### Operational notes Only failed entries can be retried. Fix the cause first; the existing archive job and deletion checkpoint are retained. ### CLI help (trailing whitespace normalized) ```text Requeue a failed entry after fixing its cause, keeping the same archive job Usage: coldctl agent scheduler retry [OPTIONS] --confirm-original-source Arguments: Options: --confirm-original-source --data-dir Override the OS application data directory -h, --help Print help ``` ## coldctl agent sync Website: /docs/commands/agent-sync Send allowlisted metadata to Cloud. Status: Implemented. Effect: Cloud metadata upload. ### Syntax ```text coldctl agent sync [OPTIONS] ``` ### Arguments and options - `--cloud-url `: Cloud origin, for example https://your-console.example.com - `--data-dir `: Override the OS application data directory - `--token-env `: Environment variable containing a lifecycle-scoped Cloud token [default: COLDCTL_CLOUD_TOKEN] - `--dry-run`: Print the exact metadata payload without network access or token lookup - `--allow-local-http`: Permit HTTP to a numeric loopback address for local testing only - `--watch`: Keep synchronizing until stopped; run under your service manager - `--interval-seconds `: Seconds between successful uploads in watch mode (30..86400) [default: 300] - `-h, --help`: Print help ### Examples ```text coldctl agent sync --dry-run coldctl agent sync --cloud-url https://console.example.com ``` ```text coldctl agent sync --cloud-url https://console.example.com --watch --interval-seconds 300 ``` ### Operational notes Create a lifecycle token in Cloud Settings and supply it through COLDCTL_CLOUD_TOKEN, or choose its environment-variable name with --token-env. Dry-run prints the exact payload without token lookup or network access. HTTPS is required; explicit --allow-local-http permits numeric loopback only. Redirects are rejected. Sync is one-way; --watch enables background uploads. Watch runs until stopped; supervise it separately from archival. Retries back off from 30 seconds to one hour and persist across restarts. Each attempt collects fresh metadata; intermediate states are coalesced. Schema-v2 uses up to 1,024 pages of 100 records / 40 KiB / 24,000 UTF-16 units each; commit publishes the complete generation. Upgrade Cloud first. Dry-run emits the page array. Restart after changing token environment variables. Use agent sync-status to inspect progress. ### CLI help (trailing whitespace normalized) ```text Explicitly upload allowlisted lifecycle metadata to your Cloud workspace Usage: coldctl agent sync [OPTIONS] Options: --cloud-url Cloud origin, for example https://your-console.example.com --data-dir Override the OS application data directory --token-env Environment variable containing a lifecycle-scoped Cloud token [default: COLDCTL_CLOUD_TOKEN] --dry-run Print the exact metadata payload without network access or token lookup --allow-local-http Permit HTTP to a numeric loopback address for local testing only --watch Keep synchronizing until stopped; run under your service manager --interval-seconds Seconds between successful uploads in watch mode (30..86400) [default: 300] -h, --help Print help ```