Versioning and Migrating the DVIR Schema

A DVIR schema is never finished. A new inspection item is mandated, a defect enum gains a value, a field that was optional becomes required — and yet the records already sealed under the old schema must stay valid, retrievable, and unchanged for their full retention window. This is the tension at the center of schema evolution for a compliance pipeline: the contract must move forward while the immutable history behind it stays exactly as written. This guide specifies a versioning strategy — a semantic schema_version on every record, a clear line between additive and breaking changes, and a migration approach that reads old records forward without ever mutating the sealed original or its audit_hash. It extends Standardized DVIR JSON Schema Design within the Core DVIR Architecture & FMCSA Compliance Mapping pipeline.

The rule that governs everything here comes from the audit trail: a record sealed in WORM storage cannot be rewritten, and its audit_hash — the SHA-256 over its canonical JSON — must keep verifying for years. So migration is not a database UPDATE. It is either a read-time transformation that presents an old record under the new model, or a re-materialization that writes a new record while preserving a reference to the original and its original hash. Confusing the two is how a schema upgrade quietly corrupts an audit trail sealed under Immutable Audit Trail and WORM Retention.

• Python 3.10+ — X | Y unions and match/case for version dispatch.
• Pydantic 2.x — one model per schema version, with validators.
• hashlib and json (stdlib) — recomputing and verifying the original audit_hash.
• The canonical DVIRRecord contract and the validation gate from JSON Schema Validation for Electronic DVIRs.
• The sealed-record and audit_hash conventions from the Immutable Audit Trail and WORM Retention section.

Stamp each record with a schema_version in MAJOR.MINOR form. Increment MINOR for additive, backward-compatible changes (a new optional field, a new enum value tolerated by readers); increment MAJOR for breaking changes (a new required field, a removed or retyped field, a narrowed enum). The version is what lets a reader pick the right model years later.

from pydantic import BaseModel


class DVIRRecordV1_2(BaseModel):
    schema_version: str = "1.2"
    record_id: str
    defect_category: str
    severity_score: int          # 0-100; bands 0-34 / 35-69 / 70-100
    compliance_guardrail: str    # OOS | conditional | monitor_only


class DVIRRecordV2_0(BaseModel):
    schema_version: str = "2.0"
    record_id: str
    defect_category: str
    severity_score: int
    compliance_guardrail: str
    vmrs_sac: str                # NEW required field — a breaking (MAJOR) change

Decide additive vs breaking deliberately, because it dictates whether old readers keep working. Adding an optional field or an enum value is MINOR and needs no migration for existing readers. Adding a required field, removing a field, or narrowing an enum is MAJOR and needs a migration that supplies a defensible value for records that predate it — never a silent default that changes meaning.

Keep one migration function per version step, registered in order, each a pure function old-dict → new-dict. Chaining them walks any record from its stored version to current. Idempotence matters: re-running a migration on an already-migrated record must be a no-op, so a re-materialization job can safely retry.

from collections.abc import Callable

Migration = Callable[[dict], dict]
_REGISTRY: dict[str, tuple[str, Migration]] = {}


def register(from_v: str, to_v: str):
    def deco(fn: Migration) -> Migration:
        _REGISTRY[from_v] = (to_v, fn)
        return fn
    return deco


@register("1.2", "1.3")
def _v12_to_v13(rec: dict) -> dict:
    rec = dict(rec)
    rec.setdefault("inspection_ts", rec.get("sealed_at"))  # additive, tolerant
    rec["schema_version"] = "1.3"
    return rec


@register("1.3", "2.0")
def _v13_to_v20(rec: dict) -> dict:
    rec = dict(rec)
    # breaking: vmrs_sac now required. Old records had no code — mark for review,
    # never invent a category that changes routing.
    rec.setdefault("vmrs_sac", "PENDING-REVIEW")
    rec["schema_version"] = "2.0"
    return rec


def migrate_to_current(rec: dict, current: str = "2.0") -> dict:
    while rec["schema_version"] != current:
        step = _REGISTRY.get(rec["schema_version"])
        if step is None:
            raise ValueError(f"no migration path from {rec['schema_version']}")
        _, fn = step
        rec = fn(rec)
    return rec

Migrate-on-read is the default: read the sealed record, validate it under its own version’s model, chain migrations to the current shape in memory, and hand that to consumers. The sealed bytes and their audit_hash are untouched. Verify the original hash before migrating so you never migrate a record that has been tampered with.

import hashlib
import json


def canonical_bytes(rec: dict) -> bytes:
    return json.dumps(rec, sort_keys=True, separators=(",", ":"),
                      ensure_ascii=False).encode("utf-8")


def read_migrated(sealed: dict, stored_hash: str, current: str = "2.0") -> dict:
    # 1. prove the stored record is intact under ITS canonical form
    if hashlib.sha256(canonical_bytes(sealed)).hexdigest() != stored_hash:
        raise IntegrityError(f"{sealed['record_id']}: original audit_hash mismatch")
    # 2. transform forward in memory only
    migrated = migrate_to_current(dict(sealed), current)
    # 3. carry the original hash so provenance survives
    migrated["_origin_audit_hash"] = stored_hash
    return migrated


class IntegrityError(Exception):
    ...

If a downstream store genuinely needs the migrated shape persisted (a reporting warehouse, a re-indexed search store), write it as a new sealed record that references the original’s audit_hash and record_id. Never overwrite the WORM original. The original stays as the source of truth; the re-materialized copy is a derived artifact whose lineage points back to it.

Assert three properties: an old payload validates under its own version’s model, migration is idempotent, and the original audit_hash is preserved through a migrate-on-read.

import pytest

OLD = {"schema_version": "1.2", "record_id": "r1", "defect_category": "brakes_air_service",
       "severity_score": 72, "compliance_guardrail": "OOS"}


def test_old_payload_validates_under_old_model():
    m = DVIRRecordV1_2(**OLD)
    assert m.schema_version == "1.2"
    assert m.severity_score == 72


def test_migration_is_idempotent():
    once = migrate_to_current(dict(OLD))
    twice = migrate_to_current(dict(once))
    assert once == twice
    assert once["schema_version"] == "2.0"


def test_original_hash_is_preserved_on_read():
    stored_hash = hashlib.sha256(canonical_bytes(OLD)).hexdigest()
    migrated = read_migrated(OLD, stored_hash)
    assert migrated["_origin_audit_hash"] == stored_hash
    # the OOS floor of 70 survives the migration untouched
    assert migrated["severity_score"] == 72 and migrated["compliance_guardrail"] == "OOS"


def test_tampered_record_is_rejected_before_migration():
    stored_hash = hashlib.sha256(canonical_bytes(OLD)).hexdigest()
    tampered = OLD | {"severity_score": 10}   # someone lowered an OOS score
    with pytest.raises(IntegrityError):
        read_migrated(tampered, stored_hash)

• Mutating a WORM record in place. The cardinal sin. A migration that rewrites the stored record breaks its audit_hash, destroys the tamper-evidence property, and — under compliance-mode Object Lock — will simply fail the write. Migrate on read, or re-materialize as a new record; never UPDATE the sealed original.
• Enum additions treated as breaking. Adding a new value to a defect or guardrail enum is additive for writers but can break readers that pattern-match exhaustively and have no case for the new value. Make readers tolerant (a default monitor_only for an unknown guardrail is safe; a default that lowers an OOS score is not), and bump MINOR, not MAJOR, when the addition is genuinely backward-compatible.
• Required-field additions with silent defaults. When a new required field is added, old records have no value for it. Supplying a plausible-looking default can change how the record routes or scores — inventing a vmrs_sac that maps to a benign category could mask a real defect. Fill with an explicit PENDING-REVIEW sentinel that forces human attention instead of a value that reads as real.
• Migration order and gaps. A registry with a missing step (1.3 present, 1.4 absent) strands records at an intermediate version. Test that every stored version has an unbroken path to current, and fail loudly on a gap rather than returning a half-migrated record.

• Standardized DVIR JSON Schema Design — the canonical contract this versioning strategy evolves.
• JSON Schema Validation for Electronic DVIRs — the gate that validates each version at the boundary.
• Immutable Audit Trail and WORM Retention — why a sealed record can never be migrated in place.
• WORM Retention and SHA-256 Hashing for DVIR Records — the audit_hash a migration must preserve.
• Mapping VMRS Codes to DVIR Defect Taxonomy — where a migrated record’s PENDING-REVIEW code gets resolved.

Back to Standardized DVIR JSON Schema Design, part of Core DVIR Architecture & FMCSA Compliance Mapping.