Your databases. Your infrastructure. Your archive.

Separate the data path from Cloud visibility

The current ColdCTL agent runs database operations locally. Database credentials and database contents are not part of the Cloud synchronization payload. Operators explicitly synchronize lifecycle metadata such as configured sources, policy metadata, job counters and verification timestamps.

Cloud visibility therefore has its own boundary. Metadata can still reveal source names, database structure and operational activity. Inspect the dry-run report before synchronization and decide whether it is appropriate for your environment. Self-hosted execution does not mean that every optional control-plane component runs without an external service.

coldctl agent sync --dry-run

• Read the Cloud synchronization contract

Own the destination and its lifecycle

Choose your S3 bucket or Azure Blob container, prefix and access policy. Both providers are implemented in the agent and gateway, alongside local Parquet. MinIO and Azurite are used for local emulator tests; real-cloud identity and representative-scale qualification remain release gates.

Ownership also means operating encryption keys, access logs, retention policies, recovery copies and deletion controls. Storage lifecycle rules must not remove files that an archive catalog still considers available. Decide how archive expiration, holds and catalog changes will be coordinated.

• Design the Parquet files and manifest

Give each operation an explicit access boundary

Use source credentials scoped to the discovery and export work. Deletion requires separate explicit authorization and source permissions. Archive verification alone does not enable deletion; unchanged records are checked again in source transactions.

Separate archive writers from retrieval readers where practical. Avoid arbitrary bucket paths and unrestricted SQL in an exposed query endpoint. Authenticate callers, restrict datasets and columns, constrain resource use, and record access. Customer-controlled infrastructure still needs a defined application security model.

Plan for interrupts, upgrades and recovery

Place journal state and archive files on durable storage. Decide who monitors incomplete jobs, backs up local state and restores an installation after failure. Keep software versions and schema mapping versions with the archive evidence so upgrades do not silently change interpretation.

Test restart behavior and restore on a disposable database. Build a retrieval exercise into archive acceptance: knowing where files are stored is not the same as being able to answer a historical query. Keep recovery backups for the operational database; archival has a different purpose.

• Start with the implemented agent workflow

When this architecture fits

A customer-operated archive is a useful design when retained history must stay in approved infrastructure and infrequent queries can tolerate on-demand compute. It requires a team able to own storage, operational recovery and access controls. It does not eliminate those responsibilities or guarantee a particular cost saving.

ColdCTL’s aim is to make that lifecycle deliberate: discover, preview, export, verify and retrieve historical data without a second always-running database for every archive. Evaluate the current development scope before adopting it for production retention.

• Explore on-demand archive retrieval
• Review product availability