Writing migrations

This page covers how to author a patch: the Rust API, what each verb checks, and what to do when a check fails. For the full rollout procedure (adopt, register, dry-run, apply, roll back), read Patching workflows first.

Patches are written against the hopskip-versioning crate and registered with hop migration. The command surface is pre-GA and may change; the checks underneath it are covered in Patching workflows.

What a patch contains

A patch has three inputs:

1. A list of verbs. Each verb is one change to the workflow’s event vocabulary: rename a type, convert a payload, add a type, drop a type. Each verb has a static check that runs before anything touches a fleet.
2. The functions the verbs refer to: payload conversions and backfills, registered by name.
3. The target version’s code, registered so each execution can be replayed against it before it switches.

You do not write the event graph (the workflow’s event types and the derivations between them). It is derived from the workflow’s signature: the commands it issues, the signals it listens on, its input and output.

Register types and functions

Everything a patch refers to by name lives in a registry:

use hopskip_versioning::registry::Registries;
use hopskip_versioning::value::{SemanticTypeName, Value};
use hopskip_versioning::patch::{BackfillRef, CoercionRef};

let mut regs = Registries::new();

// Semantic types come with sample values. Every totality check below runs
// against these samples, so include the values production actually
// records: the zero amount, the empty list, not just the happy case.
regs.types.register(
    SemanticTypeName::new("Money"),
    vec![Value::Int(250), Value::Int(0)],
)?;
regs.types.register(
    SemanticTypeName::new("MoneyV2"),
    vec![Value::record([("amount", Value::Int(250)), ("currency", Value::text("USD"))])],
)?;

// A payload conversion: old representation in, new representation out.
regs.register_coercion(CoercionRef::new("money-widen"), |v| match v {
    Value::Int(n) => Ok(Value::record([
        ("amount", Value::Int(*n)),
        ("currency", Value::text("USD")),
    ])),
    other => Err(format!("expected an integer amount, got {other:?}")),
});

// Its inverse, needed for exact rollback through this step.
regs.register_coercion(CoercionRef::new("money-narrow"), |v| {
    v.field("amount").cloned().ok_or_else(|| "missing amount".to_string())
});

Conversions return Result. A conversion that cannot handle a value returns an error, and the execution carrying that value is parked with the error message instead of being migrated to a wrong value.

Certify with elaborate

A patch is a Vec<CombinatorStep> passed to elaborate, which checks every verb and produces the patch manifest:

use hopskip_versioning::combinator::{CombinatorStep, elaborate};
use hopskip_versioning::patch::VersionId;

let elab = elaborate(
    "billing-4",                    // the patch id
    VersionId::new("billing", 3),   // from
    VersionId::new("billing", 4),   // to
    &from_manifest.schema,          // the old version's event graph
    &from_manifest.signature,       //   ...and its signature
    &steps,                         // the verbs, in order
    &regs,
)?;
// elab.manifest:     the certified patch, ready to register
// elab.to_signature: the new version's signature, derived
// elab.to_schema:    the new version's event graph, derived

On failure, Err names the first verb whose check failed and the fact that is missing.

Certification levels

• Certified: every verb’s check passed. Registration re-checks the manifest, and a manifest whose fields disagree with its own steps is refused.
• CorpusTested: the change does not decompose into verbs, but it was shadow-replayed against recorded histories.
• Unchecked: registered without evidence; applying it to a fleet requires an operator override.

The per-execution check at apply time runs regardless of the level. The level records how much was verified before the fleet was touched.

The verbs

Rename an event type

CombinatorStep::RenameEvent {
    old: TypeName::new("PaymentCmd"),
    new: TypeName::new("ChargeCmd"),
}

• Checked: the new name is not already used in the target version.
• Rollback: exact.

A command’s result event is a separate type. Rename both:

CombinatorStep::RenameEvent { old: TypeName::new("PaymentResult"),
                              new: TypeName::new("ChargeResult") }

If you rename only the command, the result event keeps its old name.

Convert a payload

CombinatorStep::RetypePayload {
    ty: TypeName::new("ChargeCmd"),
    new_type: SemanticTypeName::new("MoneyV2"),
    fwd: CoercionRef::new("money-widen"),
    bwd: Some(CoercionRef::new("money-narrow")),   // None = rollback not exact here
}

• Checked: money-widen succeeds on every registered sample of the old type. If bwd is given, applying bwd after fwd returns the original, checked on the same samples.
• If it fails: money-widen failed on sample Int(0): expected an integer amount.... Fix the conversion or the samples.
• Rollback: exact if and only if bwd is present and inverts.

Add an event type computed from old data

For every new event type, the dry run needs to know where old executions get their value from. A backfill computes it:

CombinatorStep::AddEventBackfilled {
    ty: TypeName::new("RiskCheck"),
    anchor: (EdgeName::new("riskOf"), TypeName::start()),  // what it derives from
    backfill: BackfillRef("risk-from-order".to_string()),
}

The function is registered against a typed view of the old history:

regs.register_backfill(
    BackfillRef("risk-from-order".to_string()),
    vec![TypeName::start()],            // the event types it reads
    |view, codecs| {
        let start = view.occurrences(&TypeName::start()).next()
            .ok_or_else(|| "no Start event".to_string())?;
        let payload = view.payloads.get(&start).ok_or("Start has no payload")?;
        let order = payload.decode(codecs).map_err(|e| e.message)?;
        Ok(Value::record([("score", order.field("amount").cloned()
            .unwrap_or(Value::Int(0)))]))
    },
);

• Checked: the declared read set only contains event types every history is guaranteed to have, and the function is total on the corpus you test against.
• Declining: a backfill may return Err("started before the risk table existed"). Those executions appear in the dry run as needing attention, with your reason, instead of being migrated with a wrong value.
• Rollback: exact.

Writing backfills is the hardest part of authoring a patch. Use the typed view for code completion, and test the candidate against a sample of recorded executions (200 by default) with a single command; failures come back in the same message format as every other check.

Add an event type with a placeholder

CombinatorStep::AddEventPlaceholder {
    ty: TypeName::new("Memo"),
    anchor: (EdgeName::new("memoOf"), TypeName::start()),
}

Old executions carry an explicit placeholder until the value is first written.

• Checked: the new code reads the type through a guarded accessor and handles “not there yet.”

Use a placeholder when no honest computation from old data exists. If old executions must not proceed without the value, add no verb at all: the dry run parks them.

Drop an event type

CombinatorStep::DropEvent { ty: TypeName::new("LegacyStamp"), archive: false } // deleted
CombinatorStep::DropEvent { ty: TypeName::new("LegacyStamp"), archive: true }  // kept, hidden

• Checked: nothing surviving still reads the type. An execution that recorded occurrences a surviving derivation reads is blocked with the DroppedButRead error, which offers archiving as the fix.
• Rollback: archive: false closes the exact-rollback window (the data is deleted); archive: true keeps it open.

Change the stored format

CombinatorStep::ChangeCodec {
    ty: TypeName::new("ChargeCmd"),
    codec: CodecRef::json(),
}

• Checked: the codec’s read-back law (what it writes, it reads back) and decode totality over recorded payloads.

A patch made only of ChangeCodec steps changes no meaning. It applies without a replay check and is fully reversible. The format/meaning split is computed from the patch, not declared, and the apply path enforces it.

Restructure commands

SplitCommand turns one command into two issued in sequence, with the second result synthesized from the recorded first. Parallelize turns two sequential commands into one concurrent batch. Sequentialize is the reverse.

• Checked: Parallelize requires that the second command’s payload does not depend on the first’s result (checked against the graph’s derivations).
• Per execution: Sequentialize migrates only executions whose recorded arrival order already fits and flags the rest.

Executions paused exactly where a restructuring verb rewrites cannot be migrated in place. The dry run flags them as must be replaced; fork them instead.

Three more verbs act on non-command event types (derived data and markers; command and result structure goes through SplitCommand): SplitEvent and MergeEvents split or merge a type through a registered conversion, and ReorderIndependent declares two types order-independent.

The signature verbs (AddInput, DropInput, ChangeInput, ChangeOutput) change the workflow’s input or output type. They are part of the deploy-order check, covered in Patching workflows.

Register and dry-run

hop migration register-version billing-v4-manifest.json
hop migration register-patch  billing-4-patch.json
hop migration dry-run billing-4

The same routes over raw HTTP ($VAPI is the versioning API base: versioning-api-dev, or a HOPSKIP_VERSIONING_API_BIND listener):

curl -X POST $VAPI/versions -d @billing-v4-manifest.json
curl -X POST $VAPI/patches  -d @billing-4-patch.json
curl -X POST $VAPI/patches/billing-4/dry-run

The dry run performs every migration step except the final switch, for every execution, and writes nothing. Each execution lands in one bucket:

Bucket	Meaning	What to do
ready	every check passed	nothing; apply covers these
needs backfill	a new type has no registered computation for this execution	register the backfill, dry-run again
new code disagrees with the record	replaying the target version did not reproduce a recorded step	see Debugging a divergence
at a boundary	paused exactly where a restructuring verb rewrites	fork it; it cannot be migrated in place
at another version	already migrated, or mid-chain	migrate the chain in order
blocked	a named check refused	follow the message

Apply

hop migration apply billing-4        # raw form: POST $VAPI/patches/billing-4/apply

Apply commits one switch per execution. Each switch is confirmed by re-reading the log before it is acknowledged. An execution that appended a step in the race window reads back refused_stale: nothing happened, and you can re-run when quiet. Executions that cannot migrate keep running on the old version.

The per-execution check replays the new version against each translated history, so the target version’s code must be registered first. Until it is, apply returns 409 with that reason.

Rollback

Every migrated execution carries a rollback status. The window is exact when the patch loses nothing (renames, reversible conversions, backfills) and stays open until the first appended event that has no place in the old version. From that point, rollback requires a registered reverse patch.

Two constraints:

• Rollback is a deploy step. If a service you talk to has already upgraded past you, rollback is refused until it rolls back first, and the refusal states the required order. Plan validation checks up-then-down sequences for this reason.
• A stale rollback request is a no-op, like a stale apply: the commit is refused by every reader, and retrying is safe.

Debugging a divergence

new code disagrees with the record at step 3 means: given the recorded events, the new version’s third reaction issued a different command set than the history records. A command was missing, extra, or carried a different payload. The usual causes, most frequent first:

1. Half-finished rename. You renamed the command but the new code still issues the old name, or the reverse. The diff names the command ids.
2. The payload really changed. The new code sends {amount, currency} where the record holds 250. If that is the point of the migration, add a RetypePayload step so the translated history matches what the new code issues.
3. The behavior changed. The new version issues a command the old one never did. That is not a migration. Let old executions finish on the old version, or replace them explicitly.

Two cases are not divergences: the execution’s live frontier (nothing is durable past the last resolved step, so the new code’s next move is taken as the frontier rather than compared against anything), and anything after an execution completed.

Deploy order across services

If another service consumes what your workflow emits, or you changed your workflow’s input or output type (a contract with its callers), validate the rollout order before deploying:

curl -X POST $VAPI/plans/validate -d @rollout.json

rollout.json carries the current versions, each service’s patch summary, the contracts at each version pair, and the steps in the order you intend, rollback steps included:

{
  "current":  { "ledger": {"workflow":"ledger","version":1},
                "payments": {"workflow":"payments","version":1} },
  "patches":  [ { "service": "ledger",
                  "from": {"workflow":"ledger","version":1},
                  "to":   {"workflow":"ledger","version":2},
                  "summary": { "renames": {"LedgerEntry": "LedgerEntry2"},
                               "reversible": {"LedgerEntry2": null} } },
                { "service": "payments",
                  "from": {"workflow":"payments","version":1},
                  "to":   {"workflow":"payments","version":2},
                  "summary": { "dropped": ["PaymentRecord"],
                               "introduced": ["PaymentRecordNew"] } } ],
  "contracts": [ /* who reads whom, at which version pairs */ ],
  "steps":    [ { "service": "payments", "to": {"workflow":"payments","version":2} },
                { "service": "ledger",   "to": {"workflow":"ledger","version":2} } ]
}

The verdict is one sentence per problem with its fix: step 0 breaks contract ledger-payments: deploy ledger before payments.... For each service pair the answer is one of: either order is safe; A first; B first; or no order is safe, in which case cut over together or add a conversion so mixed versions can talk. For input/output changes, carry conversions in both directions and either order is safe.

Existing workflows: the typing patch

Recorded histories of existing workflows are opaque bytes to every check. The first patch for each existing workflow type assigns a semantic type and codec to every payload position and changes nothing else:

let legacy = service.derive_legacy_version("billing", 1)?;  // observed from the log
let (patch, mut typed) = Registry::typing_patch(
    "billing-typing",
    &legacy,
    typed_signature,                       // the types you claim
    [(TypeName::start(), CodecRef::json()),
     (TypeName::new("charge_card.result"), CodecRef::json())].into(),
);
typed.legacy = true;  // positions still untyped (e.g. command args) keep the carve-out

Its dry run asks one question per execution: does every recorded payload parse? Failures are reported per event and codec. The patch is format-only, so it applies without a replay check and rolls back exactly. The other verbs in this guide require it first.

Pre-apply checklist

1. Sample values registered for every semantic type the patch touches, including the edge cases production recorded.
2. Every conversion has an inverse, or you have accepted that rollback through that step is not exact (the plan reports which steps).
3. Every new event type has a value source: backfill, placeholder, or blocked.
4. Backfills decline with a reason instead of guessing.
5. Dropped types: nothing surviving reads them; archived if rollback matters.
6. The dry run is clean, or every non-clean row has an owner.
7. The target version’s code is registered (needed by patches that change meaning).
8. Cross-service rollouts: the order is validated, rollback steps included.