Jobs, checkpoints & recovery
Durable checkpoints allow explicit recovery without silently repeating committed batches.
State and progress
Jobs may be running, paused, completed, failed or cancelled. Counters include committed Parquet rows/bytes only. A running record can outlive a crashed process. Progress timestamps and historical rates do not prove liveness, and planner estimates are not percentages or ETAs.
Resume versus a new run
archive resume and jobs retry keep the same job, fixed bounds, schema and object names. They verify committed objects and reconcile prepared batches before continuing. New runs after prior work require --allow-repeat and can duplicate records. No cross-policy deduplication is promised.
Safe stopping
Row/time budgets and process interruption request a pause at a safe boundary. jobs cancel requests cancellation after the current batch. Files are preserved. There is no jobs pause command. To continue an incomplete job, validate the original source and pass --confirm-original-source.
Failure investigation
Use jobs show to inspect safe failure category, retryability and next_step. Fix permissions, storage, schema or integrity problems before retrying. A smaller batch does not fix an oversized row. Never discard a checkpoint to skip data. Preserve corrupt files for investigation.
coldctl jobs show JOB_ID --output json
coldctl jobs retry JOB_ID --confirm-original-source