Designing a Repeatable Import That Survives Schema Changes

Editorial illustration in kirigami-cutout style depicting a repeatable import pipeline with pinned version dependencies, drift detection, and metadata capture at each stage

A one-off FHIR import runs once. A repeatable import runs weekly for years, and every schema change on either side threatens to break it silently. Designing for repeatability is the difference between a pipeline that survives IG upgrades and one that fails on the second Tuesday of every month. The site's Bulk import time estimator helps size repeat cadences. For the wider FHIR framing, related FHIR breakdowns for clinic teams has more.

What Repeatability Requires

  • The same command produces the same result
  • Schema changes on either side are detected explicitly
  • Version-pinned dependencies (profiles, value sets, code)
  • Automatic drift detection

Every one is a design choice. Skipping any of them turns repeatability into intermittent success.

Version Pin The IG

  • US Core 7.0.0, not "US Core latest"
  • Data model version pinned in the import code
  • Terminology version pinned to a specific release

Version drift is the most common source of silent breakage. Pin explicitly; upgrade deliberately.

For the staging framing, when an import needs a staging environment is the entry.

Detect Schema Changes On Ingestion

Every import should verify:

  • Expected resource types are present
  • Expected fields are populated
  • Value distributions look reasonable
  • No new fields have appeared that the import ignores

That is a schema check upfront. Failing the check surfaces changes before the import processes bad data.

Emit Deterministic Output

Given the same input, the import produces the same output. That means:

  • Deterministic id assignment (UUID v5 from stable inputs, not v4)
  • Stable ordering of operations
  • No timestamp-derived logic that varies per run

Deterministic pipelines are testable. Non-deterministic pipelines are not.

Idempotent By Design

Every write is safe to repeat. Every import can be re-run without producing duplicates. That is the base of every recovery pattern too. For the recovery side, recovering from a failed import without starting over is the entry.

Test On Every Schema Change

  • Source system upgrades — re-test the import
  • Target FHIR version upgrades — re-test the import
  • IG version upgrades — re-test the import
  • Profile changes — re-test the import

Automate the test. Manual re-testing gets skipped when the team is busy.

Alert On Distribution Drift

If the count of resources per patient changes significantly week-over-week, something is different. That might be:

  • Source system change
  • Data source change
  • Schema drift
  • Real change in patient population

Distribution alerts catch this. Volume-only alerts do not.

For the driver analysis, the four dominant drivers of import duration is the entry.

Feature Flags For Behavior Changes

When the import logic needs to change:

  • Ship the new behavior behind a feature flag
  • Turn the flag on for one environment first
  • Roll out to production after verification

That way behavior changes are reversible without a code deploy.

Version The Import Itself

Every import run should record:

  • Import code version
  • Source system version
  • Target FHIR version
  • Profile/IG versions

That version-metadata makes historical reasoning possible. "This row was imported by import v3.2 against IG 7.0.0" is a real, useful piece of provenance.

Track The Interface Contract

Between source and target, there is an implicit contract:

  • What fields the source emits
  • What fields the target expects
  • What transformations happen between

Write the contract down. Store it next to the import code. When either end changes, the contract update is the review point.

The Sunset Plan

Repeatable imports eventually get retired. Design for retirement:

  • Data-migration story from this import to the next
  • Backfill plan for records loaded under this import
  • Decommission steps for the pipeline itself

Imports that never get retired accumulate dead code and outdated dependencies. Plan the sunset.

The Short Version

Version pin dependencies. Detect schema changes on ingestion. Deterministic and idempotent by design. Test on every upgrade. Alert on distribution drift. Feature flags for behavior changes. Version the import runs. Contract with source and target. Plan retirement. That is the import that survives schema changes.

Kirigami-cutout diagram of a repeatable import pipeline with version-pin dependencies, drift detection, and version-metadata capture stages, drawn as clean paper-cutout shapes with deep-blue accents on ivory

Sources

  • HTML - HTML, HL7 FHIR - Versioning FHIR (releases, semver, backwards compat)

Emily Tran

HIM specialist from San Diego. Covers clinical document exchange, C-CDA, and the long tail of EHR migration projects.