provenance is the product

Why every model output on a deal team needs a citation trail back to the filing.

The first thing a deal team does with an unfamiliar output is check it. Not skim it — check it. Someone opens the model’s summary of a target’s revenue concentration, then opens the filing, finds the segment note, and reads one against the other line by line. That takes roughly as long as writing the summary would have. The work moved; it didn’t disappear.

The fix sits upstream of the model. Provenance — where a figure came from, what it passed through, who touched it on the way — is a property you build in at ingestion or a property you never have. A citation attached to an answer after the fact produces a plausible link to a document that may or may not contain the number. A trail built as the document is ingested produces something a reviewer can open, land on, and sign against. One is a footnote. The other is the product.

On a live deal the agent’s output is not the deliverable. The credit memo is, and the memo carries a name — a person who will sit in front of a committee and be asked where the leverage figure came from. If the honest answer is that a model produced it from a corpus, the analyst rebuilds the number by hand before the meeting. The rebuild is the real cost, not the model call.

We watched this on a credit desk we worked with. The summarization was good and adoption was flat. Analysts used the tool to orient themselves and then did the work again in the source documents, because nothing let them jump from a sentence in the output to the paragraph that produced it. Time saved on drafting was spent twice on verification.

The failure isn’t accuracy. A system that is right ninety-five percent of the time and traceable beats one that is right ninety-nine percent and opaque, because the traceable system tells you which five percent to look at. Opacity forces uniform review — every line gets the same suspicion — and uniform review is the thing the desk was trying to stop doing.

A citation trail is not a document name. It has to record the source document under a stable identity rather than a filename; the version, because data rooms get re-uploaded and filings get amended; the location inside it, down to the page, table, or clause; the extraction step, including whether the value came from text, a table parser, or an OCR pass over a scanned exhibit; every transform applied afterward, including unit conversions, currency translation, and pro-forma adjustments; and the person who reviewed it, when, and what they were shown at the time.

Transforms are where most trails quietly break. A number is pulled correctly from a Q3 filing, converted from thousands to millions, annualized, then adjusted for a disposal — and the output shows the final figure beside a link to the filing, where that figure does not appear. The reviewer opens the link, can’t find the number, and stops trusting the system. The link was real; the chain between it and the answer was missing, so it read as a fabrication.

Review belongs in the record as an event, not a status. Knowing a covenant check was approved is worth much less than knowing a named person approved this extraction from this version of the agreement at a particular hour. When the agreement is amended, the first record tells you nothing and the second tells you exactly which approvals to invalidate.

So we treat provenance as an ingestion problem. A document entering the context graph gets a stable identity, a version, and a hash; every chunk keeps a pointer to its coordinates in the original; every derived value keeps a reference to the chunk and to the transform that produced it. None of it is added later because none of it can be reconstructed later — once a number is separated from its origin, the origin can only be guessed at.

The constraint has a cost and it lands early. It slows ingestion, it forces decisions about document identity nobody wants to make in week one, and it rules out the fastest route to a demo. What it buys is that review shifts from verification to judgment. Nobody is checking whether the number is the number; they are arguing about whether the assumption behind it survives the last three deals, which is the argument worth a managing director’s afternoon.

It also holds up outside the desk. When an examiner or an internal auditor asks how a figure in a submitted memo was produced, the answer is a trail rather than a description of a methodology. That difference is most of what regulated institutions mean when they call a system auditable — and it is why provenance isn’t a feature attached to the thing we build. It is the thing we build.