Patterns for Data Migration You Can Trust

The pattern

Operational data is rarely clean at the edge. Dates drift, names vary, blank fields carry hidden meanings, and a value that looks obvious in one context can be wrong in another.

Trust comes from treating messy data as normal, not exceptional. The goal is not to assume cleanliness; it is to design a process that can detect, explain, and recover from messiness.

The principle

• Make the process idempotent so a retry does not create a second version of the same truth.
• Validate before writing so bad rows are explained instead of silently absorbed.
• Keep a dry-run mode so the proposed change can be reviewed before it becomes real.
• Reconcile after each run: rows read, rows accepted, rows rejected, and why.

Why it matters

People do not trust a data process because it says it succeeded. They trust it because the process can show what changed, what did not, and what needs a human decision.