# ColdCTL

> Local archival for PostgreSQL, MySQL and MongoDB, with customer-controlled Parquet storage and explicit metadata synchronization to Cloud.

Reference for the current CLI 0.1.1 development workspace. Commands and options are checked against the repository CLI; published release packages may lag this reference.

Implemented: independently installed signed database connectors, discovery/analysis, retention policies, local/S3/Azure/GCS Parquet export, automatically refreshed AWS role and Azure VM identity credentials, durable checkpoints, verification, recovery/import, same-engine separate-target restore, a local archive scheduler, recurring PostgreSQL/MySQL/MongoDB-to-S3/Azure/GCS archive/delete cycles, explicit verified source deletion, Prometheus metrics, durable failure history, optional webhook alerts and opt-in background one-way Cloud metadata sync. Source-scoped production protection consumes fresh external monitoring samples for lag/load thresholds, UTC windows and adaptive delays. Read-only shadow validation compares saved source ranges and optional gateway ID lookups with durable local evidence. Opt-in per-policy automatic shadow validation refreshes the gateway catalog before scheduled deletion; job/manifest/configuration-bound receipts permit safe continuation of the same deletion after restart. Native metric polling and live application query mirroring are not implemented. Archive-only is the default.

The separate customer-hosted gateway retrieves local/S3/Azure/GCS archives with browser filters, record views, bounded CSV/JSONL/Parquet downloads, tenant-scoped access, local audit events and full-object verification. Query processing runs on customer infrastructure; authorized downloads go to users. Built-in PII masking, legal holds and arbitrary nested MongoDB field search are not provided. Not implemented in the Agent workflow: CDC/incremental watermarks, mutable-source snapshots, distributed scheduling or remote Cloud execution. Paged Cloud reports include scheduler state, deletion outcomes, connector inventory and active monitor incidents, without raw errors or customer records.

Never put passwords in --url. Use credential environment references. Source-stability flags are operator assertions. Verification does not by itself authorize deletion or prove historical authenticity.

## Full documentation

- [Full command and concept reference](/llms-full.txt): complete descriptions, examples, option lists and CLI help.
- [LLM.txt](/LLM.txt): compatibility filename containing the same full reference.

## Guides and concepts

- [Your first archive](/docs): Configure a source, preview a policy, run a bounded export and verify it.
- [Installation & local state](/docs/concepts/installation-state): The installation identity, SQLite journal and state directory form the basis of recovery.
- [Sources & credentials](/docs/concepts/sources-credentials): Configure PostgreSQL, MySQL and MongoDB with local secret references and verified TLS.
- [Datasets & retention policies](/docs/concepts/policies-datasets): An explicit time cutoff and optional equality filter define what is eligible.
- [Parquet & the manifest](/docs/concepts/archive-format): Each job produces bounded Parquet objects and a manifest with integrity metadata.
- [Jobs, checkpoints & recovery](/docs/concepts/jobs-recovery): Durable checkpoints allow explicit recovery without silently repeating committed batches.
- [Execution controls](/docs/concepts/execution-controls): Bound source load and invocation length without confusing a budget with completion.
- [Restore & value validation](/docs/concepts/restore): Restore verified archives into a separate target using the matching database connector.
- [Cloud synchronization](/docs/concepts/cloud-sync): One-way metadata reports connect your local lifecycle to the Cloud console.
- [CLI conventions](/docs/concepts/cli-conventions): Global flags, output streams, identifiers and placeholder syntax.
- [Cloud HTTP API](/docs/concepts/cloud-api): Lifecycle upload and console endpoints, authentication and response codes.
- [Long-running archive scheduler](/docs/concepts/scheduler): Queue bounded archival work, resume checkpoints after restart and inspect persistent retry state.
- [Verified source deletion](/docs/concepts/verified-deletion): Explicitly remove unchanged archived records with atomic batches and durable local deletion progress.
- [Connector architecture](/docs/concepts/connector-architecture): The local host owns recovery and storage; supervised native connectors own database-specific operations.
- [Install and update connectors](/docs/concepts/connector-installation): Bootstrap independent trust, install exact signed versions and retain packages needed by unfinished jobs.
- [Supported databases and limits](/docs/concepts/supported-databases): Current PostgreSQL, MySQL 8.4 and MongoDB 8.0 connector scope, with local, S3, Azure Blob and GCS destinations.
- [Customer-owned archive destinations](/docs/concepts/archive-destinations): Choose local files, S3, Azure Blob or Google Cloud Storage in your infrastructure.
- [Google Cloud Storage archives](/docs/concepts/google-cloud-storage): Archive to a customer-owned GCS bucket, retrieve through the Gateway, and validate before scheduled deletion.
- [Cloud storage authentication](/docs/concepts/cloud-storage-auth): Use refreshed AWS role credentials or Azure VM managed identity in the agent and archive gateway.
- [Recurring archive and delete](/docs/concepts/recurring-archival): Run PostgreSQL, MySQL or MongoDB cycles to S3, Azure Blob or GCS, with verified deletion gating each new cycle.
- [Retrieve archives with the gateway](/docs/concepts/archive-gateway): Serve verified local, S3, Azure Blob and GCS archives from a customer-hosted Gateway.
- [Operational monitoring and alert hooks](/docs/concepts/operational-monitoring): Scrape metrics, inspect durable failures and deliver optional webhook alerts from the agent and gateway.
- [Protect production during archival](/docs/concepts/production-protection): Source-scoped UTC windows, external lag/load telemetry, adaptive delays and optional gateway validation gates.
- [Validate archived retrieval before deletion](/docs/concepts/shadow-validation): Compare retained source rows with verified archives and optional gateway ID lookups; persist redacted evidence.
- [Automate validation before recurring deletion](/docs/concepts/automatic-shadow): Refresh the gateway, compare retained records, and authorize each archive job automatically before verified deletion.
- [Parent-child archival and recurring groups](/docs/concepts/parent-child-archival): Archive related MySQL or PostgreSQL tables and validate every member before child-first deletion.

## CLI commands

- [coldctl protection set](/docs/commands/protection-set): Configure production protection. Local configuration/reporting.
- [coldctl protection sample](/docs/commands/protection-sample): Submit source monitoring telemetry. Local configuration/reporting.
- [coldctl protection status](/docs/commands/protection-status): Inspect production protection. Local configuration/reporting.
- [coldctl protection clear](/docs/commands/protection-clear): Disable production protection. Local configuration/reporting.
- [coldctl connector registry](/docs/commands/connector-registry): Pin connector registry trust. Local trust configuration.
- [coldctl connector refresh](/docs/commands/connector-refresh): Refresh signed registry metadata. Registry read / local metadata write.
- [coldctl connector list](/docs/commands/connector-list): List installed or available connectors. Local read / optional registry refresh.
- [coldctl connector install](/docs/commands/connector-install): Install an exact connector version. Download / local executable installation.
- [coldctl connector use](/docs/commands/connector-use): Select a connector for new sessions. Local configuration.
- [coldctl connector inspect](/docs/commands/connector-inspect): Inspect connector versions and references. Local read.
- [coldctl connector doctor](/docs/commands/connector-doctor): Check local connector integrity. Local read.
- [coldctl connector remove](/docs/commands/connector-remove): Remove an unused connector version. Local package removal.
- [coldctl connector bundle](/docs/commands/connector-bundle): Export a signed offline repository. Local files write.
- [coldctl init](/docs/commands/init): Initialize or upgrade local state. Local state write.
- [coldctl status](/docs/commands/status): Inspect the local installation. Local read.
- [coldctl state backup](/docs/commands/state-backup): Create a consistent SQLite backup. Local file write.
- [coldctl source add postgres](/docs/commands/source-add-postgres): Register a PostgreSQL connection. Local state write.
- [coldctl source add mysql](/docs/commands/source-add-mysql): Configure a MySQL source. Local configuration.
- [coldctl source add mongodb](/docs/commands/source-add-mongodb): Configure a MongoDB source. Local configuration.
- [coldctl source list](/docs/commands/source-list): List configured database sources. Local read.
- [coldctl source show](/docs/commands/source-show): Inspect one configured source. Local read.
- [coldctl source test](/docs/commands/source-test): Check source authentication and access. Database read.
- [coldctl source discover](/docs/commands/source-discover): Discover accessible tables or collections. Database read.
- [coldctl source remove](/docs/commands/source-remove): Remove local source configuration. Local state write.
- [coldctl analyze](/docs/commands/analyze): Assess archival candidates from metadata. Database read.
- [coldctl destination add local](/docs/commands/destination-add-local): Register a local archive directory. Local state write.
- [coldctl destination add s3](/docs/commands/destination-add-s3): Configure an S3 archive destination. Local destination configuration.
- [coldctl destination add azure](/docs/commands/destination-add-azure): Configure an Azure Blob archive destination. Local destination configuration.
- [coldctl destination add gcs](/docs/commands/destination-add-gcs): Configure a Google Cloud Storage archive destination. Local destination configuration.
- [coldctl destination list](/docs/commands/destination-list): List local archive destinations. Local read.
- [coldctl destination show](/docs/commands/destination-show): Inspect an archive destination. Local read.
- [coldctl destination test](/docs/commands/destination-test): Probe destination write and read access. Local file write.
- [coldctl destination remove](/docs/commands/destination-remove): Remove destination configuration. Local state write.
- [coldctl policy create](/docs/commands/policy-create): Save an explicit retention policy. Local state write.
- [coldctl policy list](/docs/commands/policy-list): List saved retention policies. Local read.
- [coldctl policy show](/docs/commands/policy-show): Inspect a policy and its rule. Local read.
- [coldctl policy validate](/docs/commands/policy-validate): Validate a policy against its database. Database read.
- [coldctl policy remove](/docs/commands/policy-remove): Remove a saved policy. Local state write.
- [coldctl archive outcomes](/docs/commands/archive-outcomes): Inspect measured archive outcomes. Read-only local evidence.
- [coldctl archive reclaim](/docs/commands/archive-reclaim): Reclaim PostgreSQL space in a bounded window. Source maintenance.
- [coldctl archive reclaim-acknowledge](/docs/commands/archive-reclaim-acknowledge): Acknowledge interrupted reclamation inspection. Local evidence update.
- [coldctl archive record-billing](/docs/commands/archive-record-billing): Record observed invoice totals. Local evidence update.
- [coldctl archive group-run](/docs/commands/archive-group-run): Archive and verify related tables. Local group lifecycle.
- [coldctl archive group-delete](/docs/commands/archive-group-delete): Delete a verified archive group. Local group lifecycle.
- [coldctl archive shadow](/docs/commands/archive-shadow): Compare source, archive and gateway. Read-only validation and local evidence.
- [coldctl archive shadow-reports](/docs/commands/archive-shadow-reports): Inspect durable shadow reports. Read-only validation and local evidence.
- [coldctl archive delete](/docs/commands/archive-delete): Delete unchanged records from a verified original archive. Explicit source deletion.
- [coldctl archive deletion-status](/docs/commands/archive-deletion-status): Inspect durable deletion progress. Local read.
- [coldctl archive plan](/docs/commands/archive-plan): Preview an estimated archive operation. Database read.
- [coldctl archive run](/docs/commands/archive-run): Export a new job in bounded Parquet batches. Database read + local write.
- [coldctl archive resume](/docs/commands/archive-resume): Continue the same durable archive job. Database read + local write.
- [coldctl archive verify](/docs/commands/archive-verify): Verify a completed archive against its journal. Local read + state write.
- [coldctl archive inspect](/docs/commands/archive-inspect): Read a standalone archive manifest. Local read.
- [coldctl archive verify-directory](/docs/commands/archive-verify-directory): Verify a completed archive without original state. Local read + temporary scratch.
- [coldctl archive import](/docs/commands/archive-import): Recover a completed archive into local state. Local state write.
- [coldctl archive staging-list](/docs/commands/archive-staging-list): List unreferenced staging candidates. Local read.
- [coldctl archive restore](/docs/commands/archive-restore): Restore verified data into a separate same-engine target. Target database write.
- [coldctl archive validate-restore](/docs/commands/archive-validate-restore): Compare restored values with the archive. Target database read.
- [coldctl jobs list](/docs/commands/jobs-list): List locally recorded archive jobs. Local read.
- [coldctl jobs show](/docs/commands/jobs-show): Inspect job progress, checkpoints and failures. Local read.
- [coldctl jobs cancel](/docs/commands/jobs-cancel): Request a cooperative stop. Local state write.
- [coldctl jobs retry](/docs/commands/jobs-retry): Retry in place after fixing the cause. Database read + local write.
- [coldctl agent sync-status](/docs/commands/agent-sync-status): Inspect Cloud synchronization. Local status.
- [coldctl agent monitor serve](/docs/commands/agent-monitor-serve): Serve local metrics, durable failures and optional alert delivery. Local monitoring; explicit webhook delivery when configured.
- [coldctl agent monitor status](/docs/commands/agent-monitor-status): Inspect persisted monitoring history and alert delivery state. Local monitoring history.
- [coldctl agent scheduler group create](/docs/commands/agent-scheduler-group-create): Schedule parent-child archival. Local group lifecycle.
- [coldctl agent scheduler group list](/docs/commands/agent-scheduler-group-list): List recurring groups. Local group lifecycle.
- [coldctl agent scheduler group show](/docs/commands/agent-scheduler-group-show): Inspect a recurring group. Local group lifecycle.
- [coldctl agent scheduler group stop](/docs/commands/agent-scheduler-group-stop): Stop a recurring group. Local group lifecycle.
- [coldctl agent scheduler group retry](/docs/commands/agent-scheduler-group-retry): Retry a held group. Local group lifecycle.
- [coldctl agent scheduler recurring create](/docs/commands/agent-scheduler-recurring-create): Create a recurring archive and delete schedule. Local schedule write.
- [coldctl agent scheduler recurring list](/docs/commands/agent-scheduler-recurring-list): List recurring schedules. Read local state.
- [coldctl agent scheduler recurring show](/docs/commands/agent-scheduler-recurring-show): Inspect a recurring schedule. Read local state.
- [coldctl agent scheduler recurring stop](/docs/commands/agent-scheduler-recurring-stop): Stop a recurring schedule. Cancel future cycles.
- [coldctl agent scheduler recurring retry](/docs/commands/agent-scheduler-recurring-retry): Retry a failed recurring cycle. Local schedule write.
- [coldctl agent scheduler enqueue](/docs/commands/agent-scheduler-enqueue): Queue a long-running archive. Local queue write.
- [coldctl agent scheduler run](/docs/commands/agent-scheduler-run): Run the local scheduler worker. Archive writes / opt-in deletion.
- [coldctl agent scheduler list](/docs/commands/agent-scheduler-list): List persistent queue entries. Local read.
- [coldctl agent scheduler show](/docs/commands/agent-scheduler-show): Inspect a scheduler queue entry. Local read.
- [coldctl agent scheduler cancel](/docs/commands/agent-scheduler-cancel): Cancel a queue entry. Local cancellation request.
- [coldctl agent scheduler retry](/docs/commands/agent-scheduler-retry): Retry a failed queue entry. Local queue write.
- [coldctl agent sync](/docs/commands/agent-sync): Send allowlisted metadata to Cloud. Cloud metadata upload.