Documentation CLI 0.1.1 · development llms.txt

Automate validation before recurring deletion

Refresh the gateway, compare retained records, and authorize each archive job automatically before verified deletion.

Configure once for each archive policy

Merge automatic_shadow into the source protection policy, preserving existing health/window limits. The map key is the exact archive policy name. require_gateway_shadow must be true. Supply the two token values through the worker service environment: tenant-scoped lookup credentials and separate administrator credentials for catalog refresh. Replacing protection configuration clears telemetry; submit fresh measurements.

JSON

{
  "require_gateway_shadow": true,
  "shadow_max_age_seconds": 3600,
  "automatic_shadow": {
    "pg_orders": {
      "gateway_url": "https://archive.example.com",
      "dataset_id": "DATASET_ID",
      "token_env": "COLDCTL_GATEWAY_READ_TOKEN",
      "refresh_token_env": "COLDCTL_GATEWAY_REFRESH_TOKEN"
    }
  }
}

Map future archives to a stable dataset

Configure trusted gateway layout for a dedicated policy storage prefix, including all future job directories. Do not map only one job UUID or combine unrelated tables/tenants. Obtain its stable dataset ID from the gateway. For an initially empty dataset the ID is SHA-256 of the compact UTF-8 JSON array [tenant,name]. Schema drift and duplicate IDs fail closed.

CLI

coldctl protection set SOURCE --config protection.json
coldctl agent scheduler recurring create pg_orders --every-seconds 86400 --source-stability immutable-rows --confirm-original-source --delete-after-archive
coldctl agent scheduler run

Validation and recovery

Each configured cycle exports, verifies, refreshes the catalog, compares source/archive/gateway values, then deletes verified unchanged records. Authentication errors and mismatches fail the same cycle; transient failures use bounded retries. Partial/interrupted reports cannot authorize deletion. Restarted validation begins again. Empty verified cycles finish without claiming retrieval coverage. Recurring support remains PostgreSQL/MySQL/MongoDB to S3, Azure Blob or GCS under their stable-source contracts.

CLI

coldctl archive shadow-reports JOB_ID
coldctl agent scheduler show QUEUE_ID
coldctl agent scheduler recurring retry SCHEDULE_ID --confirm-original-source

Durable authorization after deletion starts

A passing report creates a local job/manifest/configuration-bound receipt at deletion admission. Subsequent batches and worker restarts can resume that same deletion after report expiry, since deleted source rows cannot be compared again. Archive integrity, health and source-value checks still apply. The receipt does not continuously recheck gateway availability or authorize another job. Back up shadow.sqlite with the main state database while workers are stopped. Evidence is not synchronized to Cloud.

Operating limits

Stop is checked during validation and before deletion; an in-flight operation or delete batch can finish. Full automatic validation runs outside the export time-slice budget and checks every key in batches of up to 64. Individual refresh requests have a 120-second deadline and lookup requests 30 seconds. Long validations can delay other queued jobs; size the evidence age accordingly. Native telemetry collection, live-query mirroring and real-cloud/100-GB qualification remain outside this feature.