N-way match, end to end
Time: 40 minutes
What you'll build: a three-way match that groups purchase-order lines, receipts, and invoices on a shared key, sums partial deliveries and partial invoices inside each group, classifies every group, keeps the exceptions in a table, investigates them with a bounded agent, pauses for a payment-release approval, and leaves a record.
This is the pipeline the shipped case CTR-11 runs. Build it yourself to understand every setting, or open CTR-11 in Process Studio and skip to the results.
Base URL used below: https://your-hyphen.example.com with X-Org-Id.
Prerequisites
- An organization on a Hyphen engine. N-way matching is generally available; no feature flag.
- Three datasets that share one key. In this guide: purchase-order lines, receipts, and invoices keyed on
authorization_line_id. - A registered action or two for the agent to look things up with, for Step 6.
Step 1: Shape the legs
Each leg is a named set of records. The step accepts two to four.
"sets": [
{ "key": "purchase_order", "records": "@input.purchase_order_rows" },
{ "key": "receipt", "records": "@input.receipt_rows" },
{ "key": "invoice", "records": "@input.invoice_rows" }
]
records is an inline array or an @path reference to one: workflow input, a prior step's output, or a custom-table read. Every record needs the match-key fields and the fields you will compare.
Two rules about the fields you compare:
- Use additive fields. Configured numeric fields are summed within each group before the tolerance check. Quantity and extended line amount add up across partial receipts and partial invoices; a repeated unit price does not.
- Normalize before you match. Match keys compare exactly. Trim, case, and pad them upstream, or in a step before this one.
{ "authorization_line_id": "AUTH-1001-1", "quantity": 40, "line_amount": 4800.00, "date": "2026-08-02" }
Step 2: Pick the match keys
"matchOn": ["authorization_line_id"]
Every field in matchOn must match exactly across legs for records to land in the same group. A composite key such as ["contract_id", "line_number"] works the same way. A record whose key appears in only one leg is an orphan.
Step 3: Set the tolerances
"tolerances": {
"line_amount": { "type": "currency", "value": 0.01, "pairs": { "purchase_order:invoice": 0.05 } },
"quantity": { "type": "abs", "value": 0 }
}
| Setting | Meaning |
|---|---|
type: "abs" |
The summed values may differ by at most value in the field's own units |
type: "currency" |
The same, for money; use it for amounts so the comparison is read as money in the record |
value |
The allowed difference. Zero means the sums must be equal |
pairs |
Override value for one pair of legs, written "<leg>:<leg>". The order in the string does not matter |
A field with no tolerance entry is not compared. A group with every configured comparison inside tolerance is full; one outside is variance, and the discrepancy names the field and the legs.
Two more settings carry over from the two-way matcher:
dateWindowDayscompares each record'sdatefield across legs and allows that many days of drift.fuzzyThreshold(0 to 100) allows near-equal text, as in the two-way matcher.
rules accepts the same condition objects a step filter uses (equal, greaterThan, and, and the rest), evaluated on the grouped records.
Step 4: Keep the exceptions
Create the table the step writes non-full groups and orphans into, keyed on run_id and match_key so a replay updates a row rather than adding one:
curl -X POST https://your-hyphen.example.com/custom-tables \
-H "X-Org-Id: acme-corp" \
-H "Content-Type: application/json" \
-d '{
"name": "nway_match_exceptions",
"key_fields": ["run_id", "match_key"],
"fields": [
{ "name": "run_id", "type": "text", "required": true },
{ "name": "match_key", "type": "text", "required": true },
{ "name": "classification", "type": "text" },
{ "name": "missing_legs", "type": "text" },
{ "name": "discrepancy_fields", "type": "text" },
{ "name": "details", "type": "text" },
{ "name": "status", "type": "text" }
]
}'
Then point the step at it:
"exceptionTable": "nway_match_exceptions"
The runtime upserts one row per non-full group and per orphan, and never writes a full group:
| Column | Contents |
|---|---|
run_id |
The run that produced it |
match_key |
The group's key; for an orphan, orphan:<leg>:<index>:<digest> |
classification |
missing_leg, variance, or orphan |
missing_legs |
Comma-separated leg keys that were absent |
discrepancy_fields |
Comma-separated fields outside tolerance |
details |
Bounded JSON: the legs present, the roll-ups, and each discrepancy |
status |
pending_review when written; yours to advance |
Every row also carries provenance naming the runtime and the run.
Step 5: Put the step together and run it
{
"name": "three_way_match_before_payment",
"definition": {
"actions": [
{
"type": "nway_match",
"properties": {
"sets": [
{ "key": "purchase_order", "records": "@input.purchase_order_rows" },
{ "key": "receipt", "records": "@input.receipt_rows" },
{ "key": "invoice", "records": "@input.invoice_rows" }
],
"matchOn": ["authorization_line_id"],
"tolerances": {
"line_amount": { "type": "currency", "value": 0.01 },
"quantity": { "type": "abs", "value": 0 }
},
"exceptionTable": "nway_match_exceptions"
}
}
]
}
}
Run it with three small sets:
curl -X POST https://your-hyphen.example.com/workflows/wfl-…/execute \
-H "X-Org-Id: acme-corp" \
-H "Content-Type: application/json" \
-d '{ "input": {
"purchase_order_rows": [
{ "authorization_line_id": "AUTH-1001-1", "quantity": 40, "line_amount": 4800.00 },
{ "authorization_line_id": "AUTH-1001-2", "quantity": 10, "line_amount": 1500.00 },
{ "authorization_line_id": "AUTH-1001-3", "quantity": 5, "line_amount": 250.00 }
],
"receipt_rows": [
{ "authorization_line_id": "AUTH-1001-1", "quantity": 25, "line_amount": 3000.00 },
{ "authorization_line_id": "AUTH-1001-1", "quantity": 15, "line_amount": 1800.00 },
{ "authorization_line_id": "AUTH-1001-2", "quantity": 10, "line_amount": 1500.00 }
],
"invoice_rows": [
{ "authorization_line_id": "AUTH-1001-1", "quantity": 40, "line_amount": 4800.00 },
{ "authorization_line_id": "AUTH-1001-2", "quantity": 10, "line_amount": 1650.00 },
{ "authorization_line_id": "AUTH-9999-1", "quantity": 1, "line_amount": 99.00 }
]
} }'
Read the run status. The step wrote four context keys; they are fixed and not configurable:
| Key | What you get for the input above |
|---|---|
@matchGroupCounts |
groups: 2, full: 1, missing_leg: 0, variance: 1, orphan: 2 |
@matchGroups |
AUTH-1001-1 is full: two partial receipts sum to the order and the invoice. AUTH-1001-2 is variance on line_amount |
@matchOrphans |
The AUTH-1001-3 order line and the AUTH-9999-1 invoice, each present in one leg only |
@matchGroupExceptions |
The three non-full results, the same rows now in nway_match_exceptions |
A key that appears in one leg only is an orphan, not a group. missing_leg is for a key present in at least two legs but not all of them: add a receipt for AUTH-1001-3 and it becomes a missing_leg group with the invoice absent.
The full result carries schema_version: "hyphen.match-group.v1". Classification is deterministic: the same rows and the same tolerances always produce the same groups.
Step 6: Investigate the exceptions
Give one bounded agent a fixed read tool for the exception table. It explains the batch; it does not reclassify.
Register the read tool once:
{
"action_name": "read_nway_match_exceptions",
"kind": "custom-table",
"description": "Read N-way match exceptions for the current run.",
"properties": {
"table": "nway_match_exceptions",
"operation": "read",
"where": { "run_id": "@__run_id" },
"limit": 1000
}
}
{
"type": "loop",
"filter": { "condition": { "greaterThan": [{ "length": "@matchGroupExceptions" }, 0] } },
"properties": {
"mode": "react",
"objective": "Read the current run's N-way match exceptions with the supplied tool. Explain each exception for a procurement reviewer, cite its match key and the legs involved, and do not decide whether to pay.",
"tools": [
{ "type": "action", "name": "read_nway_match_exceptions" }
],
"max_iterations": 6,
"result_key": "explanations"
}
}
The tool is read-only and fixed to the current run's exception rows. The match evidence reaches the agent separately from model-authored context, and the runtime grounds its claims on that evidence: a discrepancy claim is allowed only when the latest counts contain a missing leg, a variance, or an orphan, and a clean-match claim only when every group is full and there are no orphans.
Step 7: Pause for the decision
{
"type": "PbotApproval",
"filter": { "condition": { "greaterThan": [{ "length": "@matchGroupExceptions" }, 0] } },
"properties": {
"comment": "Release payment for the full groups and decide the exceptions",
"request_payload": {
"counts": "@matchGroupCounts",
"exceptions": "@matchGroupExceptions",
"explanations": "@explanations"
}
}
}
The run pauses. The reviewer sees the counts, the exceptions, and the explanations in their task list. A rejected approval ends the run; the exception rows already written stay.
Step 8: Act and record
After approval, record the authorization with an idempotent upsert so a replay cannot double-book it. The table is created with "key_fields": ["run_id"], and the step selects the row by it:
{
"type": "custom-table",
"properties": {
"table": "payment_release_authorizations",
"operation": "upsert",
"key_fields": ["run_id"],
"keys": ["run_id"],
"values": ["@__run_id"],
"fields": {
"run_id": "@__run_id",
"full_groups": "@matchGroupCounts.full",
"exceptions": "@matchGroupCounts.missing_leg",
"approved": "@__approved",
"reviewer_note": "@__comment"
}
}
}
The record of the run, at GET /runs/:runId/evidence, keeps the counts, each exception, the explanations, the decision with who made it, and the effect in order.
Large inputs, replays, and signals
- Large inputs. Use
doc:references for large datasets. The result contract and exception rows stay the same. - Replays. Because exception rows are keyed on
run_idandmatch_key, running the same input again updates the same rows. - Webhooks.
nway_matcher.started,nway_matcher.completed,nway_matcher.failed.
As an agent tool
Register the step as an action of kind nway_match and declare it as a tool. The stored sets, record sources, match keys, tolerances, and rules are authoritative; the model supplies only the record fields those sources declare and cannot change the configuration.
This suits a bounded batch the model can hold in context. For thousands of rows, run the match as a workflow step and give the agent the exceptions, or expose a workflow tool that reads the rows from a table.
{
"action_name": "three_way_match",
"kind": "nway_match",
"description": "Match purchase-order, receipt, and invoice rows on the authorization line.",
"properties": {
"sets": [
{ "key": "purchase_order", "records": "@input.purchase_order_rows" },
{ "key": "receipt", "records": "@input.receipt_rows" },
{ "key": "invoice", "records": "@input.invoice_rows" }
],
"matchOn": ["authorization_line_id"],
"tolerances": { "line_amount": { "type": "currency", "value": 0.01 } }
}
}
Where @input.purchase_order_rows comes from. In an action used as a tool, @input means the tool call, not the workflow input and not the objective. The engine reads each leg's path, takes the field name after @input., and makes it a required parameter of the tool. So this action is presented to the model as a tool with three array parameters, and a call from the model looks like this:
{
"action": "three_way_match",
"action_input": {
"purchase_order_rows": [ { "authorization_line_id": "AUTH-1001-1", "quantity": 40, "line_amount": 4800 } ],
"receipt_rows": [ { "authorization_line_id": "AUTH-1001-1", "quantity": 40, "line_amount": 4800 } ],
"invoice_rows": [ { "authorization_line_id": "AUTH-1001-1", "quantity": 40, "line_amount": 4800 } ]
}
}
The model fills those three arrays from what it can see, which is one of three places:
- The objective. Paste the rows into it, as JSON or as a table. This is the only way in for a standalone agent started with
POST /agents/execute, whose body has no input field. - A document. Reference an uploaded CSV in the objective with
doc:doc_…; its rows are expanded in place and the model copies them into the call. - An earlier tool result. A
dborcustom-tableaction the agent called first, whose rows it then passes on.
Whatever the model sends under other names, such as its own sets or tolerances, is discarded and logged.
Large legs inside an agent
Three documents of 5,000 rows each will not fit through the model, and they do not have to. Pass the references instead of the rows: the model calls the tool with "purchase_order_rows": "doc:doc_…" and the engine resolves each reference to its rows before the match runs. The rows never enter the model's context. What the model gets back is bounded:
- the tool result is cut at 10,000 characters, keeping the counts and array lengths as a summary and the first part as a preview;
- the counts also reach the model as trusted match evidence, separate from anything it wrote, and its claims are checked against them. A "clean match" claim with exceptions present is refused and the model is told the real counts.
Two things to know when writing the objective:
- Name the documents by id, not by
doc:reference. Adoc:doc_…token written into the objective expands in place, so the whole document lands in the prompt. Three 5,000-row documents expanded that way exceed the run's input-token budget and the run stops asbudget_exceededbefore the first model call. - Tell the model to pass references. It will otherwise try to paste rows into the call, and its reply is capped per call, so a large paste is cut short and fails to parse.
Measured on a local engine: three 5,000-row CSVs passed by reference matched in one tool call, and the agent completed in nine seconds with 4,700 full groups, 100 variances, 200 missing legs, and 20 orphans. The same documents referenced with doc: in the objective stopped on the input budget in three seconds.
Budgets per run default to 100,000 input tokens, 50,000 output tokens, 150,000 in total, 100 model calls, and five minutes of wall time; an agent's config.llm_budget can raise or lower them. Documents extract up to 10,000 rows each.
The v2 options
The walkthrough above clears on one exact key. Settlement and statement data adds harder cases: rows with no reference, a payout sent twice, a posting and its reversal in the same file, and many identical transfers on one day. The v2 options handle each one without guessing. A row the matcher cannot place with certainty becomes an exception.
Every option is additive and opt-in. With none set, the step behaves as in Steps 1 to 5 and returns hyphen.match-group.v1. With any set, including mode: "records", the result carries schema_version: "hyphen.match-group.v2". A reader of v2 results should treat a class it does not recognize as an exception. In an action used as an agent tool, the options that change matching apply; packets and carryForward take effect only in a workflow step.
| Situation | Option | What happens |
|---|---|---|
| A row has no reference, or a blank one | keyPolicy |
The row is set aside for later passes instead of failing the step or grouping with every row that shares its blank key |
| A payout appears twice | multiplicity |
duplicate or possible_duplicate, never full |
| One side often lacks the reference | passes |
A later pass matches on other fields; candidates: "unique" makes an ambiguous candidate set ambiguous |
| A posting and its reversal sit in the same leg | netting |
netted, cleared, instead of two orphans |
| Line numbers restart in every file | identity |
Each row is identified by scope, leg, batch, and row |
| A person already paired two rows | manualMatches |
The pairing applies on the next run; a changed row comes back as manual_conflict |
| Exceptions need review in bounded batches | packets |
Packets of cases, each with an immutable evidence artifact, optionally grouped by pattern with suggested rules |
| An unmatched item may clear tomorrow | carryForward |
Open items rejoin their legs on the next run |
| A file must prove its own totals | mode: "totals" |
Signed sums, balance proofs, and side comparisons |
A settlement example
A ledger and a settlement file for one day, amounts in minor units:
"ledger_rows": [
{ "ref": "PAY-1001", "direction": "debit", "amount_minor": 500000, "value_date": "2026-10-09", "account": "0123456789" },
{ "ref": "PAY-1002", "direction": "debit", "amount_minor": 75000, "value_date": "2026-10-09", "account": "0111111111" },
{ "ref": "", "direction": "debit", "amount_minor": 120000, "value_date": "2026-10-09", "account": "0222222222" },
{ "ref": "PAY-1003", "direction": "debit", "amount_minor": 30000, "value_date": "2026-10-09", "account": "0333333333" },
{ "ref": "REV-1003", "reversal_of": "PAY-1003", "direction": "credit", "amount_minor": 30000, "value_date": "2026-10-09", "account": "0333333333" },
{ "ref": "", "direction": "debit", "amount_minor": 10000, "value_date": "2026-10-09", "account": "0444444444" },
{ "ref": "", "direction": "debit", "amount_minor": 10000, "value_date": "2026-10-09", "account": "0444444444" },
{ "direction": "debit", "amount_minor": 999, "value_date": "2026-10-09" }
],
"settlement_rows": [
{ "ref": "PAY-1001", "direction": "debit", "amount_minor": 500000, "value_date": "2026-10-09", "account": "0123456789" },
{ "ref": "PAY-1002", "direction": "debit", "amount_minor": 75000, "value_date": "2026-10-09", "account": "0111111111" },
{ "ref": "PAY-1002", "direction": "debit", "amount_minor": 75000, "value_date": "2026-10-09", "account": "0111111111" },
{ "ref": "STL-77", "direction": "debit", "amount_minor": 120000, "value_date": "2026-10-09", "account": "0222222222" },
{ "ref": "STL-81", "direction": "debit", "amount_minor": 10000, "value_date": "2026-10-09", "account": "0444444444" },
{ "ref": "STL-82", "direction": "debit", "amount_minor": 10000, "value_date": "2026-10-09", "account": "0444444444" }
]
With v1 settings only ("matchOn": ["ref", "direction"] and the same tolerance), this input fails the step: the last ledger row has no ref. Drop that row and the v1 result has two groups (PAY-1001 full, PAY-1002 a variance on amount) and eight orphans, including both halves of the reversal.
With the v2 options:
{
"type": "nway_match",
"properties": {
"sets": [
{ "key": "ledger", "records": "@input.ledger_rows" },
{ "key": "settlement", "records": "@input.settlement_rows" }
],
"keyPolicy": { "missing": "unkeyed", "blank": "unkeyed" },
"multiplicity": { "expect": "one" },
"netting": [
{ "name": "linked_reversals", "leg": "ledger",
"link": { "from": "reversal_of", "to": "ref" },
"opposite": "direction", "equal": ["amount_minor"] }
],
"passes": [
{ "name": "reference", "matchOn": ["ref", "direction"] },
{ "name": "amount_date_account",
"matchOn": ["direction", "amount_minor", "value_date", "account"],
"candidates": "unique" }
],
"tolerances": { "amount_minor": { "type": "abs", "value": 0 } }
}
}
| Rows | Classification | matched_by |
|---|---|---|
PAY-1003 and its reversal REV-1003, both in the ledger |
netted |
linked_reversals |
PAY-1001 |
full |
reference |
PAY-1002, once in the ledger and twice in settlement |
duplicate |
reference |
The blank-reference ledger row for 120000 to 0222222222, and STL-77 |
full |
amount_date_account |
The two blank-reference ledger rows for 10000 to 0444444444, and STL-81 and STL-82 |
ambiguous, candidate_count: 4 |
amount_date_account |
The ledger row with no ref field |
orphan, unkeyed |
@matchGroupCounts is groups: 5, full: 2, netted: 1, duplicate: 1, ambiguous: 1, orphan: 1, unkeyed: 1, with the other classes at zero and by_pass: { linked_reversals: 1, reference: 2, amount_date_account: 2 }. Three cases reach @matchGroupExceptions: the duplicate, the ambiguous group, and the unkeyed orphan. Nothing was paired on a guess.
Key policy
"keyPolicy": { "missing": "unkeyed", "blank": "unkeyed" }
missing:"fail"(default) fails the step on a row without a match-key field and names the leg, the row index, and the field, as v1 does."unkeyed"sets the row aside.blank:"group"(default) groups blank keys by their value, as v1 does, so every row with an empty reference lands in one group."unkeyed"sets rows with anull, empty, or whitespace-only key aside.
A row set aside by one pass is still offered to later passes, whose matchOn may not need the missing field. A set-aside row that no pass places is an orphan: unkeyed_orphans lists its position in orphans per leg, counts.unkeyed counts it, and its exception row has unkeyed: true.
Duplicates
"multiplicity": { "expect": "one", "legs": ["ledger"], "fingerprint": ["account", "amount_minor", "value_date"] }
expectaccepts only"one", its default.legsdefaults to every leg and must name configured legs.- A group in which a listed leg has more than one row is
duplicate, whatever its sums. Withoutmultiplicity, a payout sent twice on both sides sums equal and isfull. fingerprintcatches repeats that carry different references. Rows of one listed leg that share every fingerprint value make each group holding thempossible_duplicate: an exception, not a flag on a cleared group. Rows whose fingerprint fields are allnullor empty are not compared.
Ordered passes
- One to four passes, run in order.
passesreplaces the top-levelmatchOn, which is then optional. - A pass name uses lowercase letters, digits, and underscores, starts with a letter, has at most 40 characters, is unique, and is not
netting. - A pass groups only the rows earlier passes left. A key found in one leg only hands its rows to the next pass; whatever the last pass leaves is an orphan.
- A pass may set its own
tolerancesanddateWindowDays; otherwise it uses the top-level values.fuzzyThresholdandrulesapply to every pass. candidates: "unique": if any leg of a group has more than one row, the group isambiguousand nothing in it is paired.candidate_countis the number of rows in the group, and the exception row lists up to 20 of them incandidate_ids. Use it on any pass that matches on amounts and dates, where ten identical transfers on one day are ten candidates, not ten matches.- Every group records
matched_byandpass. Withoutpasses, the single pass is namedexact.counts.by_passcounts groups per pass, per netting rule, and formanualwhen pairings are set.
Within-leg netting
"netting": [
{ "name": "linked_reversals", "leg": "ledger", "link": { "from": "reversal_of", "to": "ref" }, "opposite": "direction", "equal": ["amount_minor"] },
{ "name": "same_cycle", "leg": "ledger", "pairOn": ["account", "amount_minor", "cycle"], "opposite": "direction" }
]
- Up to four rules, applied in order before any pass, each inside one
leg. A row netted by one rule is not offered to the next. link: a row nets with the row whosetovalue equals itsfromvalue, and only when exactly one such row exists.pairOn(instead oflink; requiresopposite): two rows net when they are the only two rows in the leg with these values.opposite: the two rows must hold different, non-blank values in this field, such asdebitandcredit.equal: the two rows must hold the same values in these fields. A partial reversal does not net; both rows stay in matching.- A netted pair forms a group of class
netted, which clears, withmatched_byset to the rule name (defaultnetting_1,netting_2, and so on) andpass: "netting".
Netting stays inside one leg, so it does not hide an original that also reached the other leg: that row has nothing left to pair with and is reported as an orphan.
Source-row identity
"identity": { "scope": "@input.account_id", "batch": "statement_id", "row": "source_row_id" }
batchandrowname the fields that identify a row's delivery and its place in it, such as a statement number and a line number. Line numbers restart on every statement, sorowalone is not unique; scope, leg, batch, and row together are.scopeis optional: a literal label or an@pathresolved when the step runs, such as the account. Open items, claims, and decision history are read per scope.- Every row needs non-blank
batchandrowvalues, and an identity may appear only once per leg; otherwise the step fails. Positions in a file are never used as identity. - With identity set,
source_row_idson exception rows listsleg:batch:rowidentities, anddetails.source_rowscarries each row's identity and content fingerprint (sha256). Without it,source_row_idsuses each row's ownsource_row_idfield when present, else a content digest, which identical rows share. manualMatches,carryForward,packets.skipClaimed, andpackets.historyrequire it.
Approved pairings
"manualMatches": { "table": "match_pairings" }
With table, the step reads every active pairing of its identity scope from that custom table, page by page in a stable order, and hands all of them to the matcher. The table needs scope, manual_match_id, members, approval_id, and status (NWAY_MANUAL_MATCHES_COLUMNS_MISSING otherwise). More than 10,000 active pairings in one scope fails the step with NWAY_MANUAL_MATCHES_LIMIT rather than leaving some out, because a pairing left out would let the matcher contradict a person's decision. Prefer table whenever pairings live in a custom table.
"manualMatches": { "records": "@approvedManualMatches" } is the alternative: an array or an @path you supply, applied as given. A step takes records or table, not both. Each record whose status is active or absent is applied; any other status, such as superseded, is ignored. An active record without these fields fails the step:
| Field | Contents |
|---|---|
manual_match_id |
The pairing's id |
approval_id |
The approval that decided it |
members |
Two or more rows, each { "leg", "source_batch", "source_row_id", "sha256" }, as an array or as JSON text |
- Pairings apply first, before netting and passes. When every member row is present and unchanged, the rows form one group classified by the normal comparisons (
full,variance, ormissing_leg), withmatched_by: "manual:<approval_id>",pass: "manual",manual_match_id, andapproval_id. sha256is the row's content fingerprint at approval time, as recorded indetails.source_rowsand in packet evidence. When a member row is absent from this run (row_missing), has different content (row_changed), or already belongs to an earlier pairing in the list (row_reused), the pairing is not applied: its present rows form onemanual_conflictgroup with thatconflict_reason. A changed row is never re-paired on an old decision.- Either way, the pairing's rows are not offered to netting or passes. A pairing with no row in this run belongs to other data and is skipped.
Exception packets
"packets": { "size": 10 }
Packets split the run's exceptions (every group that did not clear, and every orphan) into sets of up to size cases, 1 to 20, default 10. Size them so one bounded agent run can investigate a packet and one approval can decide it. groupBy: "class" (the default) keeps one class per packet, "pattern" one pattern per packet, and "none" mixes classes.
Each packet stores an immutable evidence artifact, schema hyphen.nway-evidence-packet.v1: the run, the scope, the checksums of the input sources, and every case with its classification, missing legs, discrepancy fields, details, and complete source rows (leg, identity when set, sha256, and the full record). The exception table stays compact, and a reviewer or agent reads full rows from the artifact rather than from a context preview. packet_id is derived from the scope, the packet's partition, its case keys, and the input checksums, so a retry or replay of the same input produces the same packet ids.
The packet index goes to @matchPackets:
| Field | Contents |
|---|---|
packet_id |
pkt_ followed by 24 hex characters |
classification |
The packet's class; null with groupBy: "none" |
case_count, case_keys |
The cases, by their exception-row match_key |
evidence |
artifact_id, version, sha256, doc_reference (doc:<artifact_id>@<version>), and the full reference |
run_id |
The run that built the packet |
claim_rows |
With identity: one row per source row, { scope, leg, source_batch, source_row_id, decision_ref, claim_version }, with decision_ref set to the packet id and claim_version to 1 |
case_claims |
With identity: the rows nobody has claimed, each { case_key, scope, leg, source_batch, source_row_id } |
claim_moves |
With identity and skipClaimed: the rows already claimed, each with the decision_ref and claim_version that hold the claim and next_claim_version |
supersessions |
With identity: for each manual_conflict case, { case_key, manual_match_id, approval_id } naming the approved pairing that no longer holds and the approval that decided it |
pattern_key, pattern_description |
Pattern packets |
scope |
Pattern packets, with identity |
history |
With packets.history: { approvals, dispositions, suggested_disposition } |
A loop over @matchPackets can start one review per packet and hand its agent the packet's evidence.doc_reference. With exceptionTable set, each packaged case's row also records its packet_id, the artifact's id, version, and checksum, and its evidence_offset in the artifact.
Skipping decided cases. "skipClaimed": { "table": "match_claims" } reads that claims table for the scope and leaves out of packets every case whose every source row is already claimed. The table needs scope, leg, source_batch, and source_row_id columns; inserting a packet's claim_rows as they are also needs decision_ref and claim_version. A skipped case is still an exception, in @matchGroupExceptions and in the exception table; it is only not packaged for another review. A case with any unclaimed row is packaged, and a manual_conflict case is always packaged: its rows are claimed by the pairing that no longer holds, and it is exactly the case that needs a new decision. Claims are read per source batch, up to 10,000 per batch. Requires identity.
Deciding only what was proposed. claim_rows claims every row of a packet. To claim only the cases a review actually decides, insert from case_claims and move claim_moves in one transaction, each filtered to the decided case keys, for example the proposed_case_ids an agent with per-case results returns. A move is a compare-and-swap on the holding decision_ref and claim_version, so an earlier decision is superseded without a second claim, and two decisions racing for one row produce one success and one conflict. Rows of undecided cases stay unclaimed and come back in the next run's packets.
Retiring a pairing that no longer holds. Deciding a manual_conflict case should retire its old pairing in the same transaction. Update your pairings table from supersessions, filtered to the decided case keys, with where on the manual_match_id and the scope, and expect the pairing to be active with the listed approval_id. The pairing to retire comes from the match itself, so a decision can only retire the pairing reported for its case, and only while the approval that holds its rows still does.
Pattern packets and suggested rules.
"packets": {
"groupBy": "pattern",
"patternOn": ["direction", "type_code"],
"history": { "table": "match_decisions", "minApprovals": 5, "exclude": ["investigation_required"] }
}
- A case's pattern is its class, the legs present, the missing legs, the fields that differ, and the values of up to five
patternOnfields. Cases that differ only in references, amounts, and dates share a pattern, across runs.pattern_keyispat_followed by 16 hex characters, andpattern_descriptionreads, for example,orphan on statement, missing mirror, direction debit, type_code NCHG. - With
exceptionTableset, the table also needs apattern_keycolumn. historynames a decisions table withscope,pattern_key,disposition, andstatuscolumns; without them the step fails withNWAY_PACKET_HISTORY_COLUMNS_MISSING. For each pattern in the run, the step counts rows with the run's scope, thatpattern_key, andstatus: "approved", ignoring blank dispositions and those inexclude(up to 20). Up to 5,000 rows are read per pattern. Requiresidentity.@matchRuleSuggestionsis written wheneverhistoryis set. It holds one entry per pattern in this run whose counted approvals reachminApprovals(2 to 1000, default 5) and all chose one disposition:pattern_key,pattern,pattern_description,disposition,approvals,open_cases, andpacket_ids. Mixed decisions never become a suggestion, however many there are.- Suggestions are never applied automatically. Recording them, and deciding whether one becomes a rule, is up to your process.
Writing the decisions table is the review workflow's job: one row per approved case, with the packet's scope and pattern_key, the chosen disposition, and status: "approved". Put dispositions that only make sense for one case, such as a manual pairing, in exclude so they never count toward a rule.
Open items across runs
"identity": { "scope": "@input.account_id", "batch": "statement_id", "row": "source_row_id" },
"carryForward": { "table": "open_items" }
Before matching, the step reads the scope's open items (status: "open") from the table and returns each to its original leg, so yesterday's unmatched statement line can pair with today's ledger entry. A carried row keeps its original identity; if the same identity arrives again today, today's copy is used. After matching:
- every source row in a group that did not clear, and every orphan, is upserted as
open, with itsclassification,first_seen_at(kept from the first run that saw it),last_seen_at,age_days,last_run_id, and the full row inrecord; - every carried row that cleared (
fullornetted) is set toclosed.
counts.carried is the number of carried rows that rejoined, and @matchCarryForward is { "table", "open", "closed" }. The table is the only state; nothing holds open items between runs.
The table must declare scope, leg, source_batch, source_row_id, record, status, classification, first_seen_at, last_seen_at, age_days, and last_run_id, or the step fails with NWAY_CARRY_FORWARD_COLUMNS_MISSING. Items are upserted by the first four, so make those the key fields. Any other column whose name matches a row field receives that field's value, which lets later steps read and sum open items without parsing record:
{
"name": "open_items",
"key_fields": ["scope", "leg", "source_batch", "source_row_id"],
"fields": [
{ "name": "scope", "type": "text" },
{ "name": "leg", "type": "text" },
{ "name": "source_batch", "type": "text" },
{ "name": "source_row_id", "type": "text" },
{ "name": "record", "type": "text" },
{ "name": "status", "type": "text" },
{ "name": "classification", "type": "text" },
{ "name": "first_seen_at", "type": "timestamptz" },
{ "name": "last_seen_at", "type": "timestamptz" },
{ "name": "age_days", "type": "integer" },
{ "name": "last_run_id", "type": "text" },
{ "name": "account_id", "type": "text" },
{ "name": "currency", "type": "text" },
{ "name": "direction", "type": "text" },
{ "name": "amount_minor", "type": "numeric" }
]
}
A scope with more than 10,000 open items fails the step with NWAY_CARRY_FORWARD_LIMIT; resolve or archive items first. An open item that cannot rejoin the match, because its record is not a readable object or its leg is not one of the step's sets, fails the step with NWAY_CARRY_FORWARD_INVALID, naming up to 20 such items; correct or close them and run again. Requires identity.
Totals mode
Totals mode compares signed sums per group instead of pairing rows. Groups form on matchOn as usual, and each leg's rows in a group are summed with a sign.
{
"type": "nway_match",
"properties": {
"mode": "totals",
"compare": "none",
"sets": [
{ "key": "statement", "records": "@input.statement_rows" },
{ "key": "mirror", "records": "@input.mirror_rows" }
],
"matchOn": ["account_id", "currency"],
"totals": { "amount_minor": { "signBy": "direction", "positive": "credit" } },
"balances": { "field": "row_type", "opening": "opening_balance", "closing": "closing_balance", "required": true },
"tolerances": { "amount_minor": { "type": "abs", "value": 0 } }
}
}
| Property | Meaning |
|---|---|
totals |
Exactly one amount field. A row adds its amount when its signBy value equals positive exactly, and subtracts it otherwise |
balances |
Optional. Rows whose field equals opening or closing are balances, not movements, and each leg gets a proof: opening plus net movements equals closing, within the field's tolerance value. With "required": true, every set must hold exactly one opening and one closing row in each group |
compare |
"pairs" (default) compares sides; "none" runs only the proofs, and then one set is enough |
compareOn |
"net" (default) compares net movements; "position" compares ending positions: a leg's closing balance, or its opening plus net when it has no closing row |
side on a set |
Sets with the same side are summed before comparison. Without it, each set is its own side. In totals mode, tolerance pairs name sides |
Each group carries totals per leg (opening, closing, credits, debits, net, invalid_rows, proof, position) and side_totals (null with compare: "none"); matched_by and pass are totals, and the result carries mode: "totals". Classification:
missing_leg: a side has no rows in the group (compare: "pairs"only).variance: a proof fails, a row is invalid, a required balance is missing or repeated, or two sides differ by more than the tolerance. Each cause is a discrepancy:balance_proof(with the leg, opening, net, and closing),balance_rows(with the leg and its opening and closing row counts),invalid_rows(with the leg and count), or the amount field (withcompare_onand each side's total).full: otherwise.
Rules that keep totals honest:
- A row whose amount is not a number, or whose
signByvalue is blank, is counted ininvalid_rowsrather than summed. - A leg with movements and only one balance row fails its proof. A leg with no balance rows has no proof (
proof: null), and neither does a set that sent no rows, unlessbalances.requiredis set. Set it for a statement self-proof, so a missing file, missing balance rows, or a balance split across several rows never proves. - A row missing a
matchOnfield fails the step. Totals mode has no key policy and no orphans, and it does not acceptkeyPolicy,multiplicity,passes,netting,manualMatches, orcarryForward.
Three uses:
Self-proof. The step above, given a statement with an opening balance of 1,000,000, a credit of 250,000, a debit of 40,000, and a closing balance of 1,210,000, reports totals.statement as opening: 1000000, credits: 250000, debits: 40000, net: 210000, closing: 1210000, proof: true. A mirror whose closing row says 1,249,000 after an opening of 1,000,000 and one credit of 250,000 makes the group variance with a balance_proof discrepancy for mirror.
Day totals. Two legs with compare: "pairs" and matchOn on the cycle or the value date. A day whose net movements agree is full, one that differs is variance with both sides' totals, and a day present in one source only is missing_leg. A mismatch here says a file is incomplete before a single exception is investigated.
Close proof. Balance per bank plus items only in the books equals balance per books plus items only on the statement:
"mode": "totals",
"compareOn": "position",
"sets": [
{ "key": "statement_closing", "side": "bank", "records": "@input.statement_rows" },
{ "key": "mirror_only_items", "side": "bank", "records": "@openMirrorItems" },
{ "key": "mirror_closing", "side": "books", "records": "@input.mirror_rows" },
{ "key": "statement_only_items", "side": "books", "records": "@openStatementItems" }
]
With the statement above closing at 1,210,000, a mirror closing at 1,250,000, and one open statement-only debit of 40,000, side_totals is bank: 1210000, books: 1210000 and the group is full. Open items have no balance rows, so each open-item leg contributes its net movements.
v2 exception rows
For a v2 result, each exception row keeps the v1 columns from Step 4 and adds:
| Column | Type | Contents |
|---|---|---|
matched_by |
text |
The pass, netting rule, or manual:<approval_id>; empty for an orphan |
candidate_count |
integer |
For ambiguous, the number of rows in the group; otherwise 0 |
candidate_ids |
text |
For ambiguous, up to 20 source-row identities, comma separated |
unkeyed |
boolean |
true for an orphan the key policy set aside |
source_row_ids |
text |
Up to 50 source-row identities in the case, comma separated |
packet_id |
text |
With packets |
evidence_artifact_id |
text |
With packets: the artifact holding the case |
evidence_version |
integer |
With packets |
evidence_sha256 |
text |
With packets |
evidence_offset |
integer |
With packets: the case's position in the artifact |
pattern_key |
text |
With groupBy: "pattern" |
details adds the pass, the true source-row count, the first 50 source rows with their fingerprints, the scope, and, for a manual pairing, its id, approval, and conflict reason. @matchGroupExceptions carries the same rows without the packet columns.
The table must declare the first five columns for any v2 result, the packet columns when packets is set, and pattern_key for pattern packets. Otherwise the step fails with NWAY_EXCEPTION_TABLE_V2_COLUMNS_MISSING, listing the missing columns, before writing anything, so a v1 table is never written in a shape it cannot hold. Create the table with every column:
{
"name": "settlement_cases",
"key_fields": ["run_id", "match_key"],
"fields": [
{ "name": "run_id", "type": "text", "required": true },
{ "name": "match_key", "type": "text", "required": true },
{ "name": "classification", "type": "text" },
{ "name": "missing_legs", "type": "text" },
{ "name": "discrepancy_fields", "type": "text" },
{ "name": "details", "type": "text" },
{ "name": "status", "type": "text" },
{ "name": "matched_by", "type": "text" },
{ "name": "candidate_count", "type": "integer" },
{ "name": "candidate_ids", "type": "text" },
{ "name": "unkeyed", "type": "boolean" },
{ "name": "source_row_ids", "type": "text" },
{ "name": "packet_id", "type": "text" },
{ "name": "evidence_artifact_id", "type": "text" },
{ "name": "evidence_version", "type": "integer" },
{ "name": "evidence_sha256", "type": "text" },
{ "name": "evidence_offset", "type": "integer" },
{ "name": "pattern_key", "type": "text" }
]
}
Statement reconciliation: the transformed data contract
Reconciling a correspondent bank's statement for a nostro account against your mirror ledger uses the options above together. Hyphen does not parse MT940, MT950, or camt.053 files. Your conversion step turns the statement and the ledger extract into the shape below; the matcher then rejects or flags what it can check. The same shape suits any statement-against-ledger reconciliation.
File rules
| Rule | Value |
|---|---|
| Format | CSV (UTF-8, header row, comma separated) or JSON (an array of flat objects). Both files in one run use the same format |
| One file covers | One account, one currency, one statement |
| Amounts | amount_minor: a non-negative integer in minor units (cents, for example), digits only, no separators, no sign. The sign comes from direction |
| Direction | direction: credit or debit, lowercase |
| Perspective | Both files use the statement account's perspective: credit means funds into the account. A mirror ledger books the opposite way round, so flip mirror entries before delivery. Getting this wrong makes matching entries look like exceptions on both sides |
| Dates | YYYY-MM-DD |
| Currency | ISO 4217 alphabetic code |
| Identifiers | References, account numbers, bank codes, and status codes are strings |
| Empty optional fields | Omitted in JSON, an empty cell in CSV |
| Size | Within the document extraction limit, 10,000 rows per document by default |
Match keys compare by type, so a reference read as a number never equals the same reference read as a string. In JSON, quote every identifier. In CSV, a cell that looks like a number is read as a number unless that would change it; a digit string with a leading zero, or an integer too long to hold exactly, stays a string. Keeping both files in one format means the same value is always read the same way.
Row types
Every row has row_type: opening_balance, entry, or closing_balance. A file has exactly one opening and one closing balance row. The balance rows carry amount_minor and direction, where credit is a credit balance.
Fields
| Field | Required | Statement source (MT940 / camt.053) | Mirror ledger source |
|---|---|---|---|
row_type |
Yes | :60F:, :61:, :62F: / Bal and Ntry |
Opening, posting, and closing balance |
account_id |
Yes | :25: / Acct/Id |
The mirror account |
currency |
Yes | Currency of :60F: / Amt@Ccy |
Account currency |
statement_id |
Yes | :28C: / Stmt/Id |
The statement being reconciled against |
statement_date |
Yes | Date of :62F: / Stmt/CreDtTm date |
Same as the statement |
value_date |
Entries | :61: value date / ValDt |
Value date |
entry_date |
No | :61: entry date / BookgDt |
Posting date |
direction |
Yes | :61: mark (C credit, D debit, RC debit, RD credit) / CdtDbtInd |
Flipped to the statement account's perspective |
amount_minor |
Yes | :61: amount / Amt |
Amount |
transaction_reference |
Entries, may be empty | :61: account owner reference / EndToEndId |
Your payment reference |
servicer_reference |
No | :61: servicer reference / AcctSvcrRef |
Not used |
reversal_of |
No | The original reference, when the statement carries one with RC/RD or RvslInd |
The reversed entry's reference |
type_code |
No | :61: transaction type / BkTxCd |
Ledger transaction code |
narrative |
No | :86: / AddtlNtryInf |
Narration |
source_row_id |
Yes | Line or entry sequence | Ledger entry id |
What the matcher checks
- Required fields. A row missing a field the step groups on fails the step, and the error names the leg, the row index, and the field. Totals mode always works this way; records mode does under the default key policy. Run the self-proof first so
account_idandcurrencyare checked on every row. - Identity. With
identityset, a row withoutstatement_idorsource_row_idfails the step, and so does the same identity twice in one file. - Balance integrity. Totals mode with
balances(with"required": true) andcompare: "none"proves each file: exactly one opening and one closing row, and opening plus net movements equals closing. A missing file, missing or repeated balance rows, a file that does not prove, and a row with a non-numeric amount or a blank direction all make the groupvariance. Gate the match on the proof so nothing is matched against a broken statement.
The matcher does not check the file rules, the perspective, or the spelling of direction. In totals mode only the exact positive value adds, so Credit or C counts as a debit. These are the conversion step's to get right.
Putting it together
A daily statement run, in order:
- Prove both files with the totals-mode self-proof above. Record its counts, for example with a custom-table write, because every
nway_matchstep overwrites the fixed context keys. Gate the steps that follow on the recorded result. - Match in one step, with identity, open items, pairings read from their table, netting, passes, duplicates, and packets:
{
"type": "nway_match",
"properties": {
"sets": [
{ "key": "statement", "records": "@input.statement_rows" },
{ "key": "mirror", "records": "@input.mirror_rows" }
],
"identity": { "scope": "@input.account_id", "batch": "statement_id", "row": "source_row_id" },
"balances": { "field": "row_type", "opening": "opening_balance", "closing": "closing_balance" },
"carryForward": { "table": "open_items" },
"manualMatches": { "table": "statement_pairings" },
"keyPolicy": { "missing": "unkeyed", "blank": "unkeyed" },
"netting": [
{ "name": "statement_reversals", "leg": "statement", "link": { "from": "reversal_of", "to": "transaction_reference" }, "opposite": "direction", "equal": ["amount_minor", "currency"] },
{ "name": "mirror_reversals", "leg": "mirror", "link": { "from": "reversal_of", "to": "transaction_reference" }, "opposite": "direction", "equal": ["amount_minor", "currency"] }
],
"passes": [
{ "name": "reference", "matchOn": ["transaction_reference", "direction", "currency", "account_id"] },
{ "name": "amount_value_date", "matchOn": ["direction", "currency", "account_id", "amount_minor", "value_date"], "candidates": "unique" }
],
"multiplicity": { "expect": "one" },
"tolerances": { "amount_minor": { "type": "abs", "value": 0 } },
"packets": {
"size": 10,
"groupBy": "pattern",
"patternOn": ["direction", "type_code"],
"skipClaimed": { "table": "statement_claims" },
"history": { "table": "statement_decisions" }
},
"exceptionTable": "statement_cases"
}
}
- Review packets with a loop over
@matchPackets, one review per packet, that writes decisions, claims, and pairings back to their tables for the cases it decides. - Close the statement. Read the open items of each leg (
status: "open") and run the close proof above. Close only when every close-proof group isfull: amissing_legthere has no variance but does not prove the close. A custom-table read returns at most 1,000 rows by default, so confirm it covers every open item.
Balance rows are left out of pairing because balances is set, so the same input rows feed the proof, the match, and the close proof.
Pitfalls
- Unit price instead of extended amount. Partial deliveries then look like variances. Compare quantity and extended amount.
- Keys that almost match.
AUTH-1001-1andauth-1001-1are two groups. Normalize first. - Expecting left and right. There are no
outputMatched,outputUnmatchedLeft, oroutputUnmatchedRightsettings; a group has no stable sides. Read the four fixed keys. - More than four legs. Split the match, or fold two sources into one leg upstream.
- A missing exception table. Without
exceptionTable, the exceptions exist only in the run context. - Two
nway_matchsteps instead of passes. The second step overwrites the first's context keys, and a large result leaves only a preview of the orphans in context. Put the passes in one step. - A v1 exception table under v2 options. Add the v2 columns first, or the step fails with
NWAY_EXCEPTION_TABLE_V2_COLUMNS_MISSING. - Reading a v2 class as cleared. Only
fullandnettedclear.ambiguous,duplicate,possible_duplicate, andmanual_conflictare exceptions, and so is any class your reader does not recognize. - Direction spelled differently across files. In totals mode only the exact
positivevalue adds, and adirectioninmatchOnmust match exactly. - An identifier read as a number in one file and a string in the other. Match keys compare by type. Deliver both files in one format.
What You Built
- three legs grouped on one key, with partial receipts and invoices summed inside each group
- per-field tolerances, with a per-pair override
- an exception table that survives replays
- a bounded agent that explains exceptions and cannot decide them
- an approval that pauses the run, and an idempotent record after it
- the same pipeline as CTR-11, which you can now run from Process Studio instead
Verified on a local engine: an agent given the rows above in its objective called the tool once with the three arrays and reported groups: 2, full: 1, variance: 1, orphan: 2.
→ Next: Invoice reconciliation, the two-way version