N-way match, end to end

Time: 40 minutes

What you'll build: a three-way match that groups purchase-order lines, receipts, and invoices on a shared key, sums partial deliveries and partial invoices inside each group, classifies every group, keeps the exceptions in a table, investigates them with a bounded agent, pauses for a payment-release approval, and leaves a record.

This is the pipeline the shipped case CTR-11 runs. Build it yourself to understand every setting, or open CTR-11 in Process Studio and skip to the results.

Base URL used below: https://your-hyphen.example.com with X-Org-Id.


Prerequisites

  • An organization on a Hyphen engine. N-way matching is generally available; no feature flag.
  • Three datasets that share one key. In this guide: purchase-order lines, receipts, and invoices keyed on authorization_line_id.
  • A registered action or two for the agent to look things up with, for Step 6.

Step 1: Shape the legs

Each leg is a named set of records. The step accepts two to four.

json
"sets": [
  { "key": "purchase_order", "records": "@input.purchase_order_rows" },
  { "key": "receipt",        "records": "@input.receipt_rows" },
  { "key": "invoice",        "records": "@input.invoice_rows" }
]

records is an inline array or an @path reference to one: workflow input, a prior step's output, or a custom-table read. Every record needs the match-key fields and the fields you will compare.

Two rules about the fields you compare:

  • Use additive fields. Configured numeric fields are summed within each group before the tolerance check. Quantity and extended line amount add up across partial receipts and partial invoices; a repeated unit price does not.
  • Normalize before you match. Match keys compare exactly. Trim, case, and pad them upstream, or in a step before this one.
json
{ "authorization_line_id": "AUTH-1001-1", "quantity": 40, "line_amount": 4800.00, "date": "2026-08-02" }

Step 2: Pick the match keys

json
"matchOn": ["authorization_line_id"]

Every field in matchOn must match exactly across legs for records to land in the same group. A composite key such as ["contract_id", "line_number"] works the same way. A record whose key appears in only one leg is an orphan.


Step 3: Set the tolerances

json
"tolerances": {
  "line_amount": { "type": "currency", "value": 0.01, "pairs": { "purchase_order:invoice": 0.05 } },
  "quantity":    { "type": "abs", "value": 0 }
}
Setting Meaning
type: "abs" The summed values may differ by at most value in the field's own units
type: "currency" The same, for money; use it for amounts so the comparison is read as money in the record
value The allowed difference. Zero means the sums must be equal
pairs Override value for one pair of legs, written "<leg>:<leg>". The order in the string does not matter

A field with no tolerance entry is not compared. A group with every configured comparison inside tolerance is full; one outside is variance, and the discrepancy names the field and the legs.

Two more settings carry over from the two-way matcher:

  • dateWindowDays compares each record's date field across legs and allows that many days of drift.
  • fuzzyThreshold (0 to 100) allows near-equal text, as in the two-way matcher.

rules accepts the same condition objects a step filter uses (equal, greaterThan, and, and the rest), evaluated on the grouped records.


Step 4: Keep the exceptions

Create the table the step writes non-full groups and orphans into, keyed on run_id and match_key so a replay updates a row rather than adding one:

bash
curl -X POST https://your-hyphen.example.com/custom-tables \
  -H "X-Org-Id: acme-corp" \
  -H "Content-Type: application/json" \
  -d '{
    "name": "nway_match_exceptions",
    "key_fields": ["run_id", "match_key"],
    "fields": [
      { "name": "run_id", "type": "text", "required": true },
      { "name": "match_key", "type": "text", "required": true },
      { "name": "classification", "type": "text" },
      { "name": "missing_legs", "type": "text" },
      { "name": "discrepancy_fields", "type": "text" },
      { "name": "details", "type": "text" },
      { "name": "status", "type": "text" }
    ]
  }'

Then point the step at it:

json
"exceptionTable": "nway_match_exceptions"

The runtime upserts one row per non-full group and per orphan, and never writes a full group:

Column Contents
run_id The run that produced it
match_key The group's key; for an orphan, orphan:<leg>:<index>:<digest>
classification missing_leg, variance, or orphan
missing_legs Comma-separated leg keys that were absent
discrepancy_fields Comma-separated fields outside tolerance
details Bounded JSON: the legs present, the roll-ups, and each discrepancy
status pending_review when written; yours to advance

Every row also carries provenance naming the runtime and the run.


Step 5: Put the step together and run it

json
{
  "name": "three_way_match_before_payment",
  "definition": {
    "actions": [
      {
        "type": "nway_match",
        "properties": {
          "sets": [
            { "key": "purchase_order", "records": "@input.purchase_order_rows" },
            { "key": "receipt",        "records": "@input.receipt_rows" },
            { "key": "invoice",        "records": "@input.invoice_rows" }
          ],
          "matchOn": ["authorization_line_id"],
          "tolerances": {
            "line_amount": { "type": "currency", "value": 0.01 },
            "quantity":    { "type": "abs", "value": 0 }
          },
          "exceptionTable": "nway_match_exceptions"
        }
      }
    ]
  }
}

Run it with three small sets:

bash
curl -X POST https://your-hyphen.example.com/workflows/wfl-…/execute \
  -H "X-Org-Id: acme-corp" \
  -H "Content-Type: application/json" \
  -d '{ "input": {
    "purchase_order_rows": [
      { "authorization_line_id": "AUTH-1001-1", "quantity": 40, "line_amount": 4800.00 },
      { "authorization_line_id": "AUTH-1001-2", "quantity": 10, "line_amount": 1500.00 },
      { "authorization_line_id": "AUTH-1001-3", "quantity": 5,  "line_amount": 250.00 }
    ],
    "receipt_rows": [
      { "authorization_line_id": "AUTH-1001-1", "quantity": 25, "line_amount": 3000.00 },
      { "authorization_line_id": "AUTH-1001-1", "quantity": 15, "line_amount": 1800.00 },
      { "authorization_line_id": "AUTH-1001-2", "quantity": 10, "line_amount": 1500.00 }
    ],
    "invoice_rows": [
      { "authorization_line_id": "AUTH-1001-1", "quantity": 40, "line_amount": 4800.00 },
      { "authorization_line_id": "AUTH-1001-2", "quantity": 10, "line_amount": 1650.00 },
      { "authorization_line_id": "AUTH-9999-1", "quantity": 1,  "line_amount": 99.00 }
    ]
  } }'

Read the run status. The step wrote four context keys; they are fixed and not configurable:

Key What you get for the input above
@matchGroupCounts groups: 2, full: 1, missing_leg: 0, variance: 1, orphan: 2
@matchGroups AUTH-1001-1 is full: two partial receipts sum to the order and the invoice. AUTH-1001-2 is variance on line_amount
@matchOrphans The AUTH-1001-3 order line and the AUTH-9999-1 invoice, each present in one leg only
@matchGroupExceptions The three non-full results, the same rows now in nway_match_exceptions

A key that appears in one leg only is an orphan, not a group. missing_leg is for a key present in at least two legs but not all of them: add a receipt for AUTH-1001-3 and it becomes a missing_leg group with the invoice absent.

The full result carries schema_version: "hyphen.match-group.v1". Classification is deterministic: the same rows and the same tolerances always produce the same groups.


Step 6: Investigate the exceptions

Give one bounded agent a fixed read tool for the exception table. It explains the batch; it does not reclassify.

Register the read tool once:

json
{
  "action_name": "read_nway_match_exceptions",
  "kind": "custom-table",
  "description": "Read N-way match exceptions for the current run.",
  "properties": {
    "table": "nway_match_exceptions",
    "operation": "read",
    "where": { "run_id": "@__run_id" },
    "limit": 1000
  }
}
json
{
  "type": "loop",
  "filter": { "condition": { "greaterThan": [{ "length": "@matchGroupExceptions" }, 0] } },
  "properties": {
    "mode": "react",
    "objective": "Read the current run's N-way match exceptions with the supplied tool. Explain each exception for a procurement reviewer, cite its match key and the legs involved, and do not decide whether to pay.",
    "tools": [
      { "type": "action", "name": "read_nway_match_exceptions" }
    ],
    "max_iterations": 6,
    "result_key": "explanations"
  }
}

The tool is read-only and fixed to the current run's exception rows. The match evidence reaches the agent separately from model-authored context, and the runtime grounds its claims on that evidence: a discrepancy claim is allowed only when the latest counts contain a missing leg, a variance, or an orphan, and a clean-match claim only when every group is full and there are no orphans.


Step 7: Pause for the decision

json
{
  "type": "PbotApproval",
  "filter": { "condition": { "greaterThan": [{ "length": "@matchGroupExceptions" }, 0] } },
  "properties": {
    "comment": "Release payment for the full groups and decide the exceptions",
    "request_payload": {
      "counts": "@matchGroupCounts",
      "exceptions": "@matchGroupExceptions",
      "explanations": "@explanations"
    }
  }
}

The run pauses. The reviewer sees the counts, the exceptions, and the explanations in their task list. A rejected approval ends the run; the exception rows already written stay.


Step 8: Act and record

After approval, record the authorization with an idempotent upsert so a replay cannot double-book it. The table is created with "key_fields": ["run_id"], and the step selects the row by it:

json
{
  "type": "custom-table",
  "properties": {
    "table": "payment_release_authorizations",
    "operation": "upsert",
    "key_fields": ["run_id"],
    "keys": ["run_id"],
    "values": ["@__run_id"],
    "fields": {
      "run_id": "@__run_id",
      "full_groups": "@matchGroupCounts.full",
      "exceptions": "@matchGroupCounts.missing_leg",
      "approved": "@__approved",
      "reviewer_note": "@__comment"
    }
  }
}

The record of the run, at GET /runs/:runId/evidence, keeps the counts, each exception, the explanations, the decision with who made it, and the effect in order.


Large inputs, replays, and signals

  • Large inputs. Use doc: references for large datasets. The result contract and exception rows stay the same.
  • Replays. Because exception rows are keyed on run_id and match_key, running the same input again updates the same rows.
  • Webhooks. nway_matcher.started, nway_matcher.completed, nway_matcher.failed.

As an agent tool

Register the step as an action of kind nway_match and declare it as a tool. The stored sets, record sources, match keys, tolerances, and rules are authoritative; the model supplies only the record fields those sources declare and cannot change the configuration.

This suits a bounded batch the model can hold in context. For thousands of rows, run the match as a workflow step and give the agent the exceptions, or expose a workflow tool that reads the rows from a table.

json
{
  "action_name": "three_way_match",
  "kind": "nway_match",
  "description": "Match purchase-order, receipt, and invoice rows on the authorization line.",
  "properties": {
    "sets": [
      { "key": "purchase_order", "records": "@input.purchase_order_rows" },
      { "key": "receipt", "records": "@input.receipt_rows" },
      { "key": "invoice", "records": "@input.invoice_rows" }
    ],
    "matchOn": ["authorization_line_id"],
    "tolerances": { "line_amount": { "type": "currency", "value": 0.01 } }
  }
}

Where @input.purchase_order_rows comes from. In an action used as a tool, @input means the tool call, not the workflow input and not the objective. The engine reads each leg's path, takes the field name after @input., and makes it a required parameter of the tool. So this action is presented to the model as a tool with three array parameters, and a call from the model looks like this:

json
{
  "action": "three_way_match",
  "action_input": {
    "purchase_order_rows": [ { "authorization_line_id": "AUTH-1001-1", "quantity": 40, "line_amount": 4800 } ],
    "receipt_rows":        [ { "authorization_line_id": "AUTH-1001-1", "quantity": 40, "line_amount": 4800 } ],
    "invoice_rows":        [ { "authorization_line_id": "AUTH-1001-1", "quantity": 40, "line_amount": 4800 } ]
  }
}

The model fills those three arrays from what it can see, which is one of three places:

  1. The objective. Paste the rows into it, as JSON or as a table. This is the only way in for a standalone agent started with POST /agents/execute, whose body has no input field.
  2. A document. Reference an uploaded CSV in the objective with doc:doc_…; its rows are expanded in place and the model copies them into the call.
  3. An earlier tool result. A db or custom-table action the agent called first, whose rows it then passes on.

Whatever the model sends under other names, such as its own sets or tolerances, is discarded and logged.

Large legs inside an agent

Three documents of 5,000 rows each will not fit through the model, and they do not have to. Pass the references instead of the rows: the model calls the tool with "purchase_order_rows": "doc:doc_…" and the engine resolves each reference to its rows before the match runs. The rows never enter the model's context. What the model gets back is bounded:

  • the tool result is cut at 10,000 characters, keeping the counts and array lengths as a summary and the first part as a preview;
  • the counts also reach the model as trusted match evidence, separate from anything it wrote, and its claims are checked against them. A "clean match" claim with exceptions present is refused and the model is told the real counts.

Two things to know when writing the objective:

  • Name the documents by id, not by doc: reference. A doc:doc_… token written into the objective expands in place, so the whole document lands in the prompt. Three 5,000-row documents expanded that way exceed the run's input-token budget and the run stops as budget_exceeded before the first model call.
  • Tell the model to pass references. It will otherwise try to paste rows into the call, and its reply is capped per call, so a large paste is cut short and fails to parse.

Measured on a local engine: three 5,000-row CSVs passed by reference matched in one tool call, and the agent completed in nine seconds with 4,700 full groups, 100 variances, 200 missing legs, and 20 orphans. The same documents referenced with doc: in the objective stopped on the input budget in three seconds.

Budgets per run default to 100,000 input tokens, 50,000 output tokens, 150,000 in total, 100 model calls, and five minutes of wall time; an agent's config.llm_budget can raise or lower them. Documents extract up to 10,000 rows each.


The v2 options

The walkthrough above clears on one exact key. Settlement and statement data adds harder cases: rows with no reference, a payout sent twice, a posting and its reversal in the same file, and many identical transfers on one day. The v2 options handle each one without guessing. A row the matcher cannot place with certainty becomes an exception.

Every option is additive and opt-in. With none set, the step behaves as in Steps 1 to 5 and returns hyphen.match-group.v1. With any set, including mode: "records", the result carries schema_version: "hyphen.match-group.v2". A reader of v2 results should treat a class it does not recognize as an exception. In an action used as an agent tool, the options that change matching apply; packets and carryForward take effect only in a workflow step.

Situation Option What happens
A row has no reference, or a blank one keyPolicy The row is set aside for later passes instead of failing the step or grouping with every row that shares its blank key
A payout appears twice multiplicity duplicate or possible_duplicate, never full
One side often lacks the reference passes A later pass matches on other fields; candidates: "unique" makes an ambiguous candidate set ambiguous
A posting and its reversal sit in the same leg netting netted, cleared, instead of two orphans
Line numbers restart in every file identity Each row is identified by scope, leg, batch, and row
A person already paired two rows manualMatches The pairing applies on the next run; a changed row comes back as manual_conflict
Exceptions need review in bounded batches packets Packets of cases, each with an immutable evidence artifact, optionally grouped by pattern with suggested rules
An unmatched item may clear tomorrow carryForward Open items rejoin their legs on the next run
A file must prove its own totals mode: "totals" Signed sums, balance proofs, and side comparisons

A settlement example

A ledger and a settlement file for one day, amounts in minor units:

json
"ledger_rows": [
  { "ref": "PAY-1001", "direction": "debit", "amount_minor": 500000, "value_date": "2026-10-09", "account": "0123456789" },
  { "ref": "PAY-1002", "direction": "debit", "amount_minor": 75000,  "value_date": "2026-10-09", "account": "0111111111" },
  { "ref": "",         "direction": "debit", "amount_minor": 120000, "value_date": "2026-10-09", "account": "0222222222" },
  { "ref": "PAY-1003", "direction": "debit", "amount_minor": 30000,  "value_date": "2026-10-09", "account": "0333333333" },
  { "ref": "REV-1003", "reversal_of": "PAY-1003", "direction": "credit", "amount_minor": 30000, "value_date": "2026-10-09", "account": "0333333333" },
  { "ref": "",         "direction": "debit", "amount_minor": 10000,  "value_date": "2026-10-09", "account": "0444444444" },
  { "ref": "",         "direction": "debit", "amount_minor": 10000,  "value_date": "2026-10-09", "account": "0444444444" },
  { "direction": "debit", "amount_minor": 999, "value_date": "2026-10-09" }
],
"settlement_rows": [
  { "ref": "PAY-1001", "direction": "debit", "amount_minor": 500000, "value_date": "2026-10-09", "account": "0123456789" },
  { "ref": "PAY-1002", "direction": "debit", "amount_minor": 75000,  "value_date": "2026-10-09", "account": "0111111111" },
  { "ref": "PAY-1002", "direction": "debit", "amount_minor": 75000,  "value_date": "2026-10-09", "account": "0111111111" },
  { "ref": "STL-77",   "direction": "debit", "amount_minor": 120000, "value_date": "2026-10-09", "account": "0222222222" },
  { "ref": "STL-81",   "direction": "debit", "amount_minor": 10000,  "value_date": "2026-10-09", "account": "0444444444" },
  { "ref": "STL-82",   "direction": "debit", "amount_minor": 10000,  "value_date": "2026-10-09", "account": "0444444444" }
]

With v1 settings only ("matchOn": ["ref", "direction"] and the same tolerance), this input fails the step: the last ledger row has no ref. Drop that row and the v1 result has two groups (PAY-1001 full, PAY-1002 a variance on amount) and eight orphans, including both halves of the reversal.

With the v2 options:

json
{
  "type": "nway_match",
  "properties": {
    "sets": [
      { "key": "ledger",     "records": "@input.ledger_rows" },
      { "key": "settlement", "records": "@input.settlement_rows" }
    ],
    "keyPolicy": { "missing": "unkeyed", "blank": "unkeyed" },
    "multiplicity": { "expect": "one" },
    "netting": [
      { "name": "linked_reversals", "leg": "ledger",
        "link": { "from": "reversal_of", "to": "ref" },
        "opposite": "direction", "equal": ["amount_minor"] }
    ],
    "passes": [
      { "name": "reference", "matchOn": ["ref", "direction"] },
      { "name": "amount_date_account",
        "matchOn": ["direction", "amount_minor", "value_date", "account"],
        "candidates": "unique" }
    ],
    "tolerances": { "amount_minor": { "type": "abs", "value": 0 } }
  }
}
Rows Classification matched_by
PAY-1003 and its reversal REV-1003, both in the ledger netted linked_reversals
PAY-1001 full reference
PAY-1002, once in the ledger and twice in settlement duplicate reference
The blank-reference ledger row for 120000 to 0222222222, and STL-77 full amount_date_account
The two blank-reference ledger rows for 10000 to 0444444444, and STL-81 and STL-82 ambiguous, candidate_count: 4 amount_date_account
The ledger row with no ref field orphan, unkeyed

@matchGroupCounts is groups: 5, full: 2, netted: 1, duplicate: 1, ambiguous: 1, orphan: 1, unkeyed: 1, with the other classes at zero and by_pass: { linked_reversals: 1, reference: 2, amount_date_account: 2 }. Three cases reach @matchGroupExceptions: the duplicate, the ambiguous group, and the unkeyed orphan. Nothing was paired on a guess.

Key policy

json
"keyPolicy": { "missing": "unkeyed", "blank": "unkeyed" }
  • missing: "fail" (default) fails the step on a row without a match-key field and names the leg, the row index, and the field, as v1 does. "unkeyed" sets the row aside.
  • blank: "group" (default) groups blank keys by their value, as v1 does, so every row with an empty reference lands in one group. "unkeyed" sets rows with a null, empty, or whitespace-only key aside.

A row set aside by one pass is still offered to later passes, whose matchOn may not need the missing field. A set-aside row that no pass places is an orphan: unkeyed_orphans lists its position in orphans per leg, counts.unkeyed counts it, and its exception row has unkeyed: true.

Duplicates

json
"multiplicity": { "expect": "one", "legs": ["ledger"], "fingerprint": ["account", "amount_minor", "value_date"] }
  • expect accepts only "one", its default. legs defaults to every leg and must name configured legs.
  • A group in which a listed leg has more than one row is duplicate, whatever its sums. Without multiplicity, a payout sent twice on both sides sums equal and is full.
  • fingerprint catches repeats that carry different references. Rows of one listed leg that share every fingerprint value make each group holding them possible_duplicate: an exception, not a flag on a cleared group. Rows whose fingerprint fields are all null or empty are not compared.

Ordered passes

  • One to four passes, run in order. passes replaces the top-level matchOn, which is then optional.
  • A pass name uses lowercase letters, digits, and underscores, starts with a letter, has at most 40 characters, is unique, and is not netting.
  • A pass groups only the rows earlier passes left. A key found in one leg only hands its rows to the next pass; whatever the last pass leaves is an orphan.
  • A pass may set its own tolerances and dateWindowDays; otherwise it uses the top-level values. fuzzyThreshold and rules apply to every pass.
  • candidates: "unique": if any leg of a group has more than one row, the group is ambiguous and nothing in it is paired. candidate_count is the number of rows in the group, and the exception row lists up to 20 of them in candidate_ids. Use it on any pass that matches on amounts and dates, where ten identical transfers on one day are ten candidates, not ten matches.
  • Every group records matched_by and pass. Without passes, the single pass is named exact. counts.by_pass counts groups per pass, per netting rule, and for manual when pairings are set.

Within-leg netting

json
"netting": [
  { "name": "linked_reversals", "leg": "ledger", "link": { "from": "reversal_of", "to": "ref" }, "opposite": "direction", "equal": ["amount_minor"] },
  { "name": "same_cycle", "leg": "ledger", "pairOn": ["account", "amount_minor", "cycle"], "opposite": "direction" }
]
  • Up to four rules, applied in order before any pass, each inside one leg. A row netted by one rule is not offered to the next.
  • link: a row nets with the row whose to value equals its from value, and only when exactly one such row exists.
  • pairOn (instead of link; requires opposite): two rows net when they are the only two rows in the leg with these values.
  • opposite: the two rows must hold different, non-blank values in this field, such as debit and credit.
  • equal: the two rows must hold the same values in these fields. A partial reversal does not net; both rows stay in matching.
  • A netted pair forms a group of class netted, which clears, with matched_by set to the rule name (default netting_1, netting_2, and so on) and pass: "netting".

Netting stays inside one leg, so it does not hide an original that also reached the other leg: that row has nothing left to pair with and is reported as an orphan.

Source-row identity

json
"identity": { "scope": "@input.account_id", "batch": "statement_id", "row": "source_row_id" }
  • batch and row name the fields that identify a row's delivery and its place in it, such as a statement number and a line number. Line numbers restart on every statement, so row alone is not unique; scope, leg, batch, and row together are.
  • scope is optional: a literal label or an @path resolved when the step runs, such as the account. Open items, claims, and decision history are read per scope.
  • Every row needs non-blank batch and row values, and an identity may appear only once per leg; otherwise the step fails. Positions in a file are never used as identity.
  • With identity set, source_row_ids on exception rows lists leg:batch:row identities, and details.source_rows carries each row's identity and content fingerprint (sha256). Without it, source_row_ids uses each row's own source_row_id field when present, else a content digest, which identical rows share.
  • manualMatches, carryForward, packets.skipClaimed, and packets.history require it.

Approved pairings

json
"manualMatches": { "table": "match_pairings" }

With table, the step reads every active pairing of its identity scope from that custom table, page by page in a stable order, and hands all of them to the matcher. The table needs scope, manual_match_id, members, approval_id, and status (NWAY_MANUAL_MATCHES_COLUMNS_MISSING otherwise). More than 10,000 active pairings in one scope fails the step with NWAY_MANUAL_MATCHES_LIMIT rather than leaving some out, because a pairing left out would let the matcher contradict a person's decision. Prefer table whenever pairings live in a custom table.

"manualMatches": { "records": "@approvedManualMatches" } is the alternative: an array or an @path you supply, applied as given. A step takes records or table, not both. Each record whose status is active or absent is applied; any other status, such as superseded, is ignored. An active record without these fields fails the step:

Field Contents
manual_match_id The pairing's id
approval_id The approval that decided it
members Two or more rows, each { "leg", "source_batch", "source_row_id", "sha256" }, as an array or as JSON text
  • Pairings apply first, before netting and passes. When every member row is present and unchanged, the rows form one group classified by the normal comparisons (full, variance, or missing_leg), with matched_by: "manual:<approval_id>", pass: "manual", manual_match_id, and approval_id.
  • sha256 is the row's content fingerprint at approval time, as recorded in details.source_rows and in packet evidence. When a member row is absent from this run (row_missing), has different content (row_changed), or already belongs to an earlier pairing in the list (row_reused), the pairing is not applied: its present rows form one manual_conflict group with that conflict_reason. A changed row is never re-paired on an old decision.
  • Either way, the pairing's rows are not offered to netting or passes. A pairing with no row in this run belongs to other data and is skipped.

Exception packets

json
"packets": { "size": 10 }

Packets split the run's exceptions (every group that did not clear, and every orphan) into sets of up to size cases, 1 to 20, default 10. Size them so one bounded agent run can investigate a packet and one approval can decide it. groupBy: "class" (the default) keeps one class per packet, "pattern" one pattern per packet, and "none" mixes classes.

Each packet stores an immutable evidence artifact, schema hyphen.nway-evidence-packet.v1: the run, the scope, the checksums of the input sources, and every case with its classification, missing legs, discrepancy fields, details, and complete source rows (leg, identity when set, sha256, and the full record). The exception table stays compact, and a reviewer or agent reads full rows from the artifact rather than from a context preview. packet_id is derived from the scope, the packet's partition, its case keys, and the input checksums, so a retry or replay of the same input produces the same packet ids.

The packet index goes to @matchPackets:

Field Contents
packet_id pkt_ followed by 24 hex characters
classification The packet's class; null with groupBy: "none"
case_count, case_keys The cases, by their exception-row match_key
evidence artifact_id, version, sha256, doc_reference (doc:<artifact_id>@<version>), and the full reference
run_id The run that built the packet
claim_rows With identity: one row per source row, { scope, leg, source_batch, source_row_id, decision_ref, claim_version }, with decision_ref set to the packet id and claim_version to 1
case_claims With identity: the rows nobody has claimed, each { case_key, scope, leg, source_batch, source_row_id }
claim_moves With identity and skipClaimed: the rows already claimed, each with the decision_ref and claim_version that hold the claim and next_claim_version
supersessions With identity: for each manual_conflict case, { case_key, manual_match_id, approval_id } naming the approved pairing that no longer holds and the approval that decided it
pattern_key, pattern_description Pattern packets
scope Pattern packets, with identity
history With packets.history: { approvals, dispositions, suggested_disposition }

A loop over @matchPackets can start one review per packet and hand its agent the packet's evidence.doc_reference. With exceptionTable set, each packaged case's row also records its packet_id, the artifact's id, version, and checksum, and its evidence_offset in the artifact.

Skipping decided cases. "skipClaimed": { "table": "match_claims" } reads that claims table for the scope and leaves out of packets every case whose every source row is already claimed. The table needs scope, leg, source_batch, and source_row_id columns; inserting a packet's claim_rows as they are also needs decision_ref and claim_version. A skipped case is still an exception, in @matchGroupExceptions and in the exception table; it is only not packaged for another review. A case with any unclaimed row is packaged, and a manual_conflict case is always packaged: its rows are claimed by the pairing that no longer holds, and it is exactly the case that needs a new decision. Claims are read per source batch, up to 10,000 per batch. Requires identity.

Deciding only what was proposed. claim_rows claims every row of a packet. To claim only the cases a review actually decides, insert from case_claims and move claim_moves in one transaction, each filtered to the decided case keys, for example the proposed_case_ids an agent with per-case results returns. A move is a compare-and-swap on the holding decision_ref and claim_version, so an earlier decision is superseded without a second claim, and two decisions racing for one row produce one success and one conflict. Rows of undecided cases stay unclaimed and come back in the next run's packets.

Retiring a pairing that no longer holds. Deciding a manual_conflict case should retire its old pairing in the same transaction. Update your pairings table from supersessions, filtered to the decided case keys, with where on the manual_match_id and the scope, and expect the pairing to be active with the listed approval_id. The pairing to retire comes from the match itself, so a decision can only retire the pairing reported for its case, and only while the approval that holds its rows still does.

Pattern packets and suggested rules.

json
"packets": {
  "groupBy": "pattern",
  "patternOn": ["direction", "type_code"],
  "history": { "table": "match_decisions", "minApprovals": 5, "exclude": ["investigation_required"] }
}
  • A case's pattern is its class, the legs present, the missing legs, the fields that differ, and the values of up to five patternOn fields. Cases that differ only in references, amounts, and dates share a pattern, across runs. pattern_key is pat_ followed by 16 hex characters, and pattern_description reads, for example, orphan on statement, missing mirror, direction debit, type_code NCHG.
  • With exceptionTable set, the table also needs a pattern_key column.
  • history names a decisions table with scope, pattern_key, disposition, and status columns; without them the step fails with NWAY_PACKET_HISTORY_COLUMNS_MISSING. For each pattern in the run, the step counts rows with the run's scope, that pattern_key, and status: "approved", ignoring blank dispositions and those in exclude (up to 20). Up to 5,000 rows are read per pattern. Requires identity.
  • @matchRuleSuggestions is written whenever history is set. It holds one entry per pattern in this run whose counted approvals reach minApprovals (2 to 1000, default 5) and all chose one disposition: pattern_key, pattern, pattern_description, disposition, approvals, open_cases, and packet_ids. Mixed decisions never become a suggestion, however many there are.
  • Suggestions are never applied automatically. Recording them, and deciding whether one becomes a rule, is up to your process.

Writing the decisions table is the review workflow's job: one row per approved case, with the packet's scope and pattern_key, the chosen disposition, and status: "approved". Put dispositions that only make sense for one case, such as a manual pairing, in exclude so they never count toward a rule.

Open items across runs

json
"identity": { "scope": "@input.account_id", "batch": "statement_id", "row": "source_row_id" },
"carryForward": { "table": "open_items" }

Before matching, the step reads the scope's open items (status: "open") from the table and returns each to its original leg, so yesterday's unmatched statement line can pair with today's ledger entry. A carried row keeps its original identity; if the same identity arrives again today, today's copy is used. After matching:

  • every source row in a group that did not clear, and every orphan, is upserted as open, with its classification, first_seen_at (kept from the first run that saw it), last_seen_at, age_days, last_run_id, and the full row in record;
  • every carried row that cleared (full or netted) is set to closed.

counts.carried is the number of carried rows that rejoined, and @matchCarryForward is { "table", "open", "closed" }. The table is the only state; nothing holds open items between runs.

The table must declare scope, leg, source_batch, source_row_id, record, status, classification, first_seen_at, last_seen_at, age_days, and last_run_id, or the step fails with NWAY_CARRY_FORWARD_COLUMNS_MISSING. Items are upserted by the first four, so make those the key fields. Any other column whose name matches a row field receives that field's value, which lets later steps read and sum open items without parsing record:

json
{
  "name": "open_items",
  "key_fields": ["scope", "leg", "source_batch", "source_row_id"],
  "fields": [
    { "name": "scope", "type": "text" },
    { "name": "leg", "type": "text" },
    { "name": "source_batch", "type": "text" },
    { "name": "source_row_id", "type": "text" },
    { "name": "record", "type": "text" },
    { "name": "status", "type": "text" },
    { "name": "classification", "type": "text" },
    { "name": "first_seen_at", "type": "timestamptz" },
    { "name": "last_seen_at", "type": "timestamptz" },
    { "name": "age_days", "type": "integer" },
    { "name": "last_run_id", "type": "text" },
    { "name": "account_id", "type": "text" },
    { "name": "currency", "type": "text" },
    { "name": "direction", "type": "text" },
    { "name": "amount_minor", "type": "numeric" }
  ]
}

A scope with more than 10,000 open items fails the step with NWAY_CARRY_FORWARD_LIMIT; resolve or archive items first. An open item that cannot rejoin the match, because its record is not a readable object or its leg is not one of the step's sets, fails the step with NWAY_CARRY_FORWARD_INVALID, naming up to 20 such items; correct or close them and run again. Requires identity.

Totals mode

Totals mode compares signed sums per group instead of pairing rows. Groups form on matchOn as usual, and each leg's rows in a group are summed with a sign.

json
{
  "type": "nway_match",
  "properties": {
    "mode": "totals",
    "compare": "none",
    "sets": [
      { "key": "statement", "records": "@input.statement_rows" },
      { "key": "mirror",    "records": "@input.mirror_rows" }
    ],
    "matchOn": ["account_id", "currency"],
    "totals": { "amount_minor": { "signBy": "direction", "positive": "credit" } },
    "balances": { "field": "row_type", "opening": "opening_balance", "closing": "closing_balance", "required": true },
    "tolerances": { "amount_minor": { "type": "abs", "value": 0 } }
  }
}
Property Meaning
totals Exactly one amount field. A row adds its amount when its signBy value equals positive exactly, and subtracts it otherwise
balances Optional. Rows whose field equals opening or closing are balances, not movements, and each leg gets a proof: opening plus net movements equals closing, within the field's tolerance value. With "required": true, every set must hold exactly one opening and one closing row in each group
compare "pairs" (default) compares sides; "none" runs only the proofs, and then one set is enough
compareOn "net" (default) compares net movements; "position" compares ending positions: a leg's closing balance, or its opening plus net when it has no closing row
side on a set Sets with the same side are summed before comparison. Without it, each set is its own side. In totals mode, tolerance pairs name sides

Each group carries totals per leg (opening, closing, credits, debits, net, invalid_rows, proof, position) and side_totals (null with compare: "none"); matched_by and pass are totals, and the result carries mode: "totals". Classification:

  • missing_leg: a side has no rows in the group (compare: "pairs" only).
  • variance: a proof fails, a row is invalid, a required balance is missing or repeated, or two sides differ by more than the tolerance. Each cause is a discrepancy: balance_proof (with the leg, opening, net, and closing), balance_rows (with the leg and its opening and closing row counts), invalid_rows (with the leg and count), or the amount field (with compare_on and each side's total).
  • full: otherwise.

Rules that keep totals honest:

  • A row whose amount is not a number, or whose signBy value is blank, is counted in invalid_rows rather than summed.
  • A leg with movements and only one balance row fails its proof. A leg with no balance rows has no proof (proof: null), and neither does a set that sent no rows, unless balances.required is set. Set it for a statement self-proof, so a missing file, missing balance rows, or a balance split across several rows never proves.
  • A row missing a matchOn field fails the step. Totals mode has no key policy and no orphans, and it does not accept keyPolicy, multiplicity, passes, netting, manualMatches, or carryForward.

Three uses:

Self-proof. The step above, given a statement with an opening balance of 1,000,000, a credit of 250,000, a debit of 40,000, and a closing balance of 1,210,000, reports totals.statement as opening: 1000000, credits: 250000, debits: 40000, net: 210000, closing: 1210000, proof: true. A mirror whose closing row says 1,249,000 after an opening of 1,000,000 and one credit of 250,000 makes the group variance with a balance_proof discrepancy for mirror.

Day totals. Two legs with compare: "pairs" and matchOn on the cycle or the value date. A day whose net movements agree is full, one that differs is variance with both sides' totals, and a day present in one source only is missing_leg. A mismatch here says a file is incomplete before a single exception is investigated.

Close proof. Balance per bank plus items only in the books equals balance per books plus items only on the statement:

json
"mode": "totals",
"compareOn": "position",
"sets": [
  { "key": "statement_closing",    "side": "bank",  "records": "@input.statement_rows" },
  { "key": "mirror_only_items",    "side": "bank",  "records": "@openMirrorItems" },
  { "key": "mirror_closing",       "side": "books", "records": "@input.mirror_rows" },
  { "key": "statement_only_items", "side": "books", "records": "@openStatementItems" }
]

With the statement above closing at 1,210,000, a mirror closing at 1,250,000, and one open statement-only debit of 40,000, side_totals is bank: 1210000, books: 1210000 and the group is full. Open items have no balance rows, so each open-item leg contributes its net movements.

v2 exception rows

For a v2 result, each exception row keeps the v1 columns from Step 4 and adds:

Column Type Contents
matched_by text The pass, netting rule, or manual:<approval_id>; empty for an orphan
candidate_count integer For ambiguous, the number of rows in the group; otherwise 0
candidate_ids text For ambiguous, up to 20 source-row identities, comma separated
unkeyed boolean true for an orphan the key policy set aside
source_row_ids text Up to 50 source-row identities in the case, comma separated
packet_id text With packets
evidence_artifact_id text With packets: the artifact holding the case
evidence_version integer With packets
evidence_sha256 text With packets
evidence_offset integer With packets: the case's position in the artifact
pattern_key text With groupBy: "pattern"

details adds the pass, the true source-row count, the first 50 source rows with their fingerprints, the scope, and, for a manual pairing, its id, approval, and conflict reason. @matchGroupExceptions carries the same rows without the packet columns.

The table must declare the first five columns for any v2 result, the packet columns when packets is set, and pattern_key for pattern packets. Otherwise the step fails with NWAY_EXCEPTION_TABLE_V2_COLUMNS_MISSING, listing the missing columns, before writing anything, so a v1 table is never written in a shape it cannot hold. Create the table with every column:

json
{
  "name": "settlement_cases",
  "key_fields": ["run_id", "match_key"],
  "fields": [
    { "name": "run_id", "type": "text", "required": true },
    { "name": "match_key", "type": "text", "required": true },
    { "name": "classification", "type": "text" },
    { "name": "missing_legs", "type": "text" },
    { "name": "discrepancy_fields", "type": "text" },
    { "name": "details", "type": "text" },
    { "name": "status", "type": "text" },
    { "name": "matched_by", "type": "text" },
    { "name": "candidate_count", "type": "integer" },
    { "name": "candidate_ids", "type": "text" },
    { "name": "unkeyed", "type": "boolean" },
    { "name": "source_row_ids", "type": "text" },
    { "name": "packet_id", "type": "text" },
    { "name": "evidence_artifact_id", "type": "text" },
    { "name": "evidence_version", "type": "integer" },
    { "name": "evidence_sha256", "type": "text" },
    { "name": "evidence_offset", "type": "integer" },
    { "name": "pattern_key", "type": "text" }
  ]
}

Statement reconciliation: the transformed data contract

Reconciling a correspondent bank's statement for a nostro account against your mirror ledger uses the options above together. Hyphen does not parse MT940, MT950, or camt.053 files. Your conversion step turns the statement and the ledger extract into the shape below; the matcher then rejects or flags what it can check. The same shape suits any statement-against-ledger reconciliation.

File rules

Rule Value
Format CSV (UTF-8, header row, comma separated) or JSON (an array of flat objects). Both files in one run use the same format
One file covers One account, one currency, one statement
Amounts amount_minor: a non-negative integer in minor units (cents, for example), digits only, no separators, no sign. The sign comes from direction
Direction direction: credit or debit, lowercase
Perspective Both files use the statement account's perspective: credit means funds into the account. A mirror ledger books the opposite way round, so flip mirror entries before delivery. Getting this wrong makes matching entries look like exceptions on both sides
Dates YYYY-MM-DD
Currency ISO 4217 alphabetic code
Identifiers References, account numbers, bank codes, and status codes are strings
Empty optional fields Omitted in JSON, an empty cell in CSV
Size Within the document extraction limit, 10,000 rows per document by default

Match keys compare by type, so a reference read as a number never equals the same reference read as a string. In JSON, quote every identifier. In CSV, a cell that looks like a number is read as a number unless that would change it; a digit string with a leading zero, or an integer too long to hold exactly, stays a string. Keeping both files in one format means the same value is always read the same way.

Row types

Every row has row_type: opening_balance, entry, or closing_balance. A file has exactly one opening and one closing balance row. The balance rows carry amount_minor and direction, where credit is a credit balance.

Fields

Field Required Statement source (MT940 / camt.053) Mirror ledger source
row_type Yes :60F:, :61:, :62F: / Bal and Ntry Opening, posting, and closing balance
account_id Yes :25: / Acct/Id The mirror account
currency Yes Currency of :60F: / Amt@Ccy Account currency
statement_id Yes :28C: / Stmt/Id The statement being reconciled against
statement_date Yes Date of :62F: / Stmt/CreDtTm date Same as the statement
value_date Entries :61: value date / ValDt Value date
entry_date No :61: entry date / BookgDt Posting date
direction Yes :61: mark (C credit, D debit, RC debit, RD credit) / CdtDbtInd Flipped to the statement account's perspective
amount_minor Yes :61: amount / Amt Amount
transaction_reference Entries, may be empty :61: account owner reference / EndToEndId Your payment reference
servicer_reference No :61: servicer reference / AcctSvcrRef Not used
reversal_of No The original reference, when the statement carries one with RC/RD or RvslInd The reversed entry's reference
type_code No :61: transaction type / BkTxCd Ledger transaction code
narrative No :86: / AddtlNtryInf Narration
source_row_id Yes Line or entry sequence Ledger entry id

What the matcher checks

  • Required fields. A row missing a field the step groups on fails the step, and the error names the leg, the row index, and the field. Totals mode always works this way; records mode does under the default key policy. Run the self-proof first so account_id and currency are checked on every row.
  • Identity. With identity set, a row without statement_id or source_row_id fails the step, and so does the same identity twice in one file.
  • Balance integrity. Totals mode with balances (with "required": true) and compare: "none" proves each file: exactly one opening and one closing row, and opening plus net movements equals closing. A missing file, missing or repeated balance rows, a file that does not prove, and a row with a non-numeric amount or a blank direction all make the group variance. Gate the match on the proof so nothing is matched against a broken statement.

The matcher does not check the file rules, the perspective, or the spelling of direction. In totals mode only the exact positive value adds, so Credit or C counts as a debit. These are the conversion step's to get right.

Putting it together

A daily statement run, in order:

  1. Prove both files with the totals-mode self-proof above. Record its counts, for example with a custom-table write, because every nway_match step overwrites the fixed context keys. Gate the steps that follow on the recorded result.
  2. Match in one step, with identity, open items, pairings read from their table, netting, passes, duplicates, and packets:
json
{
  "type": "nway_match",
  "properties": {
    "sets": [
      { "key": "statement", "records": "@input.statement_rows" },
      { "key": "mirror",    "records": "@input.mirror_rows" }
    ],
    "identity": { "scope": "@input.account_id", "batch": "statement_id", "row": "source_row_id" },
    "balances": { "field": "row_type", "opening": "opening_balance", "closing": "closing_balance" },
    "carryForward": { "table": "open_items" },
    "manualMatches": { "table": "statement_pairings" },
    "keyPolicy": { "missing": "unkeyed", "blank": "unkeyed" },
    "netting": [
      { "name": "statement_reversals", "leg": "statement", "link": { "from": "reversal_of", "to": "transaction_reference" }, "opposite": "direction", "equal": ["amount_minor", "currency"] },
      { "name": "mirror_reversals", "leg": "mirror", "link": { "from": "reversal_of", "to": "transaction_reference" }, "opposite": "direction", "equal": ["amount_minor", "currency"] }
    ],
    "passes": [
      { "name": "reference", "matchOn": ["transaction_reference", "direction", "currency", "account_id"] },
      { "name": "amount_value_date", "matchOn": ["direction", "currency", "account_id", "amount_minor", "value_date"], "candidates": "unique" }
    ],
    "multiplicity": { "expect": "one" },
    "tolerances": { "amount_minor": { "type": "abs", "value": 0 } },
    "packets": {
      "size": 10,
      "groupBy": "pattern",
      "patternOn": ["direction", "type_code"],
      "skipClaimed": { "table": "statement_claims" },
      "history": { "table": "statement_decisions" }
    },
    "exceptionTable": "statement_cases"
  }
}
  1. Review packets with a loop over @matchPackets, one review per packet, that writes decisions, claims, and pairings back to their tables for the cases it decides.
  2. Close the statement. Read the open items of each leg (status: "open") and run the close proof above. Close only when every close-proof group is full: a missing_leg there has no variance but does not prove the close. A custom-table read returns at most 1,000 rows by default, so confirm it covers every open item.

Balance rows are left out of pairing because balances is set, so the same input rows feed the proof, the match, and the close proof.


Pitfalls

  • Unit price instead of extended amount. Partial deliveries then look like variances. Compare quantity and extended amount.
  • Keys that almost match. AUTH-1001-1 and auth-1001-1 are two groups. Normalize first.
  • Expecting left and right. There are no outputMatched, outputUnmatchedLeft, or outputUnmatchedRight settings; a group has no stable sides. Read the four fixed keys.
  • More than four legs. Split the match, or fold two sources into one leg upstream.
  • A missing exception table. Without exceptionTable, the exceptions exist only in the run context.
  • Two nway_match steps instead of passes. The second step overwrites the first's context keys, and a large result leaves only a preview of the orphans in context. Put the passes in one step.
  • A v1 exception table under v2 options. Add the v2 columns first, or the step fails with NWAY_EXCEPTION_TABLE_V2_COLUMNS_MISSING.
  • Reading a v2 class as cleared. Only full and netted clear. ambiguous, duplicate, possible_duplicate, and manual_conflict are exceptions, and so is any class your reader does not recognize.
  • Direction spelled differently across files. In totals mode only the exact positive value adds, and a direction in matchOn must match exactly.
  • An identifier read as a number in one file and a string in the other. Match keys compare by type. Deliver both files in one format.

What You Built

  • three legs grouped on one key, with partial receipts and invoices summed inside each group
  • per-field tolerances, with a per-pair override
  • an exception table that survives replays
  • a bounded agent that explains exceptions and cannot decide them
  • an approval that pauses the run, and an idempotent record after it
  • the same pipeline as CTR-11, which you can now run from Process Studio instead

Verified on a local engine: an agent given the rows above in its objective called the tool once with the three arrays and reported groups: 2, full: 1, variance: 1, orphan: 2.

→ Next: Invoice reconciliation, the two-way version