Google Cloud Storage archives
Archive to a customer-owned GCS bucket, retrieve through the Gateway, and validate before scheduled deletion.
Supported workflows
Google Cloud Storage (GCS) is an archive destination, not a database connector. Supported PostgreSQL, MySQL and MongoDB sources can use it under their existing source-stability contracts. Archive, resume, verification, restore, Gateway retrieval and recurring archive/delete workflows use the same integrity checks as other sinks. Cloud receives destination metadata, not bucket credentials or archive contents.
Prepare a private bucket and identity
Create the bucket separately. Give the agent object create/get access and delete permission for its unique .coldctl-probe-* access-test objects under the selected prefix. A bucket-scoped Storage Object User role includes these permissions; a custom role can narrow them. Use a separate Gateway identity with object get/list access, such as Storage Object Viewer. Retention locks can prevent probe cleanup and cause destination test to fail. Validate IAM, retention and customer-managed encryption for your deployment.
Configure the agent
Use matching agent and connector builds that include GCS support. Stop workers and run init to upgrade local state to the current schema. Set GOOGLE_APPLICATION_CREDENTIALS in the process environment to an absolute JSON credential-file path, not JSON contents. Configuration stores only references; keep credentials out of source control. This example is PowerShell.
$env:GOOGLE_APPLICATION_CREDENTIALS = 'C:\credentials\coldctl-gcs.json'
coldctl init
coldctl destination add gcs --name gcs-archive --bucket YOUR_BUCKET --prefix coldctl
coldctl destination test gcs-archiveAuthentication and durable staging
Use --credentials-env COLDCTL_GCS_CREDENTIALS for another environment variable containing an absolute credential-file path. Otherwise the adapter checks GOOGLE_APPLICATION_CREDENTIALS, local gcloud application-default credentials, then attached GCP instance credentials. Service-account and authorized-user JSON files are supported; external_account workload identity federation files are not supported by the pinned SDK. Explicit missing credentials fail without fallback. Keep local state and staging durable across restarts; --staging-path changes the local staging directory. Custom GCS endpoints are limited to anonymous HTTP loopback emulators.
Configure Gateway retrieval
Use the same bucket and prefix as the agent. Save this as coldctl.gcs.toml, provide a private local gateway-layout.json assigning the archives to the customer tenant, and set COLDCTL_GATEWAY_TOKEN in the Gateway environment. Its Google credentials can use a separate read-only identity. Each source dataset must have its own trusted mapping; do not combine incompatible schemas. Restart after changing credential files.
state_dir = "gateway-gcs-state"
layout_file = "gateway-layout.json"
[remote]
prefix = "coldctl"
[remote.provider]
kind = "gcs"
bucket = "YOUR_BUCKET"
[server]
listen = "127.0.0.1:8090"
[[users]]
name = "operator"
token_env = "COLDCTL_GATEWAY_TOKEN"
tenants = ["customer"]
admin = trueVerify before enabling deletion
Start with archive-only operation on a separate test collection or table. Gateway retrieval supports search, record views, pagination and CSV/JSONL/Parquet exports. Recurring deletion requires explicit authorization and stable eligible source data. Configure per-policy automatic shadow validation before enrollment: it refreshes the Gateway catalog and compares source, archive and Gateway values. Mismatches or Gateway failures block deletion. Use separate refresh and reader credentials. Follow the linked guides for dataset mappings and protection configuration.
Real-service validation and remaining limits
On October 21, 2026, tests against a real GCS bucket and separate Aiven MySQL tables passed archival, verification, Gateway retrieval/export and full shadow comparison. Deliberate source/Gateway mismatches and injected Gateway refresh/lookup 503 and lookup 401 failures blocked deletion with no receipt or deletion journal. After repair and explicit retry, the same jobs validated before deletion. Three continuous scheduled cycles deleted 2, 1 and 0 eligible rows, preserved recent rows, and stayed idle after stopping past the next due time. These were small functional fixtures, not throughput or year-long reliability qualification. Prolonged GCS outage/backoff, workload identity federation, and deployment-specific KMS/retention behavior remain unqualified. Real tests used Google credentials; emulator tests alone cannot qualify GCP safety.