Long-running archive scheduler
Queue bounded archival work, resume checkpoints after restart and inspect persistent retry state.
Enqueue and run
Initialize state, configure the source/destination/policy, then enqueue it. Enqueue returns a queue ID; a job ID is assigned after preflight. The original source and eligible rows must stay stable through all slices, downtime and retries. MongoDB requires quiescent-copy.
coldctl init
coldctl agent scheduler enqueue orders-archive --source-stability immutable-rows --confirm-original-source --max-duration-seconds 300 --max-rows 10000
coldctl agent scheduler runLocal execution model
One foreground worker per state directory claims durable SQLite queue entries and processes one job at a time. The default slice is 60 seconds; a positive time slice is required. Budgets are checked between batches after preflight/verification, not hard wall-clock deadlines. Paused slices requeue after one second. --delay-seconds delays the first execution, while --once executes at most one due slice and exits.
Adopt interrupted work
Use --job to adopt an incomplete original job for the same policy. This preserves its cutoff, upper key, schema, connector pin and files. Saved execution controls remain except the default time slice becomes 60 seconds; explicit flags override them. The supplied stability flag cannot replace the job safety contract. Completed, imported and cancelled jobs cannot be adopted. Only one active queue entry per policy is allowed.
coldctl agent scheduler enqueue orders-archive --job JOB_ID --source-stability immutable-rows --confirm-original-sourceInspect, cancel and retry
Queue IDs and job IDs are different. Queue output is JSON and due_at is a UTC Unix timestamp. Cancellation stops future batches; the in-flight batch may commit. Cancelled entries are terminal. Failed entries can be retried after fixing their cause. Transient failures back off from 30 seconds, doubling to at most one hour; the default is five consecutive retries, maximum 100. A successful slice resets that count. Permanent failures require intervention.
coldctl agent scheduler list
coldctl agent scheduler show QUEUE_ID
coldctl jobs show JOB_ID
coldctl agent scheduler cancel QUEUE_ID
coldctl agent scheduler retry QUEUE_ID --confirm-original-sourceOptional deletion
--delete-after-archive explicitly authorizes deletion of verified unchanged records. It is off by default. The worker verifies the completed export and deletes one original archive batch per turn, saving separate deletion progress. The queue completes only when both phases complete. Previously committed deletions remain if cancelled or a later batch fails.
coldctl agent scheduler enqueue orders-archive --source-stability immutable-rows --confirm-original-source --delete-after-archiveOperate as a service
Use an OS service manager to keep agent scheduler run alive, supplying the same state path, environment secrets and filesystem permissions. Ctrl+C, Unix SIGTERM or Windows Ctrl+Break requests a safe stop. Restart recovers claimed entries and resumes checkpoints. Preserve lock files. The scheduler does not install a service or sync Cloud automatically. Explicit recurring schedules create fresh PostgreSQL/MySQL/MongoDB cycles to S3, Azure Blob or GCS only after completed verified deletion; they rescan eligible survivors to include late arrivals between cycles. MongoDB must remain quiescent during each cycle. Ordinary repeated exports may duplicate data.