./research / verifiable-audit-layer
A verifiable audit layer for AI.
A tamper-evident audit layer for model inference and agent execution: canonical serialization, domain-separated hashing, DSSE signatures, a per-run Merkle DAG, and a cross-run Merkle Mountain Range that proves no run was deleted. Every primitive was chosen from a head-to-head benchmark against the real alternatives, with the raw data committed. This note is the design and the experiments behind it.
Threat model
We assume an adversary who controls storage and can rewrite records after the fact, including the operator who produced them. The goal is that any edit to a sealed run is detectable, that the record stays verifiable for a retention period measured in years, and that verification exposes no private inputs. Two properties are kept distinct throughout: integrity of a single run, and completeness of the set of runs. A per-run receipt gives the first and says nothing about the second, so both are engineered separately.
Every choice below was made the same way. We do not assert that a primitive is best; we benchmark it against the credible alternatives on the same machine and commit the raw data, so any figure here reproduces in one command. Timings are the fastest of several runs after discarding warmup, with mean and standard deviation retained so an unstable measurement is flagged rather than hidden. Numbers are from a single AMD64 host on CPython 3.14 and are meaningful as ratios, not absolutes.
Experiments and outcomes
Each milestone was a head-to-head test against the real alternatives, not a single measurement. The overview is below; the detailed results for each are in the sections that follow.
| Milestone | Tested against | Outcome |
|---|---|---|
| Canonical form | naive JSON, CBOR | JCS, byte-identical, ~3x cost |
| Hashing | SHA-256, BLAKE3, keccak-256 | keccak native 17x, same bytes |
| Native core | pip backends vs Rust | 6x seal, byte-identical root |
| Signature | ECDSA, BLS, ML-DSA | Ed25519, 64 B, deterministic |
| Post-quantum | ML-DSA 44 / 65 / 87 | staged, verify on par, 38x size |
| Per-run trail | 8 forgery attacks, overhead | all caught, ~121 us / event |
| Cross-run log | plain Merkle rebuild | MMR, flat append, 483x |
Canonical serialization
A capsule must hash to the same bytes in our Python core, our TypeScript client, and an auditor's own tool, or a valid record fails to verify. We use RFC 8785 (JSON Canonicalization Scheme), which fixes number formatting, key ordering by UTF-16 code unit, and rejection of non-finite values. We tested it against naive sorted-key JSON and against CBOR.
| Approach | Python vs Node | Cost |
|---|---|---|
| Naive sorted JSON | diverges on Unicode and numbers | baseline |
| RFC 8785 JCS | byte-identical | about 3x serialize |
| CBOR | binary, needs a decoder to read | compact |
Naive JSON diverges between runtimes on Unicode ordering and number formatting, so it silently breaks cross-language verification. JCS is byte-identical in both, at roughly 3x the serialization time, paid once per hash and negligible against the hashing and network work around it. We keep exactly one implementation of it. When our native module shipped its own JSON serializer, we removed it, because a second canonicalizer that agrees on ordinary inputs and diverges on the edge cases an adversary can choose is worse than none.
Hashing and the native core
Leaves are hashed with BLAKE3 and Merkle nodes with keccak-256, both with RFC 6962 domain separation (distinct prefixes for leaves and interior nodes) to close the second-preimage class where an interior node is presented as a leaf. keccak-256 is not SHA3-256; the two share a permutation but differ in one padding byte, and the on-chain verifier speaks keccak, so a silent substitution would make every root fail on-chain. The hashing layer therefore computes the requested algorithm or raises, never a lookalike, and records the algorithm in each capsule so a verifier reproduces the exact function rather than inferring it from what is installed locally.
| Primitive, 64 B input | Throughput | Note |
|---|---|---|
| SHA-256 (stdlib) | 1.50M ops/s | baseline, always present |
| BLAKE3 (pip) | 1.44M ops/s | on par with SHA-256 |
| keccak-256 (pip) | 99k ops/s | 15x slower, the bottleneck |
| keccak-256 (Rust core) | 1.69M ops/s | 17x faster than pip |
keccak has no standard-library implementation, which is what made the native module worth building. The gap is widest at event-sized inputs because pycryptodome's cost there is per-call object allocation rather than hashing, exactly the size an audit trail hashes most; end to end, the native core cuts the seal of a 1,000-event run from 78 to 491 seals per second, about 6x. Crucially, enabling the accelerator cannot change the result: sealing the same run with and without the crate yields a bit-identical root, a conformance suite checks both paths against the algorithm authors' published vectors, and CI builds one stable-ABI (abi3) wheel and verifies that single artifact on every Python from 3.10 to 3.14. Installing it changes speed and nothing else.
Signatures for a decade-long record
A signature on an audit record has to hold for as long as the record can be disputed, and AI logging obligations now run toward a decade. If the scheme breaks while a record is still in its retention window, an adversary can forge and backdate it, so the whole archive fails at once. We benchmarked Ed25519 against ECDSA, BLS, and the NIST post-quantum family ML-DSA, on throughput and sizes.
| Scheme | Verify (ops/s) | Signature size |
|---|---|---|
| ECDSA P-256 | 16,887 | ~72 B |
| ML-DSA-44 (post-quantum) | 10,725 | 2,420 B |
| Ed25519 | 10,375 | 64 B |
| BLS12-381 | reference impl only | 96 B, aggregates |
Two results overturned assumptions we had carried in, and we keep both on the record. Ed25519 is not our fastest to verify: ECDSA P-256 measured about 1.6x faster, on optimized library assembly. Ed25519 stays the default on signature size and on determinism (RFC 8032), which removes the ECDSA nonce-reuse failure that leaks a private key while signatures still verify. And post-quantum signatures are not slow to verify: ML-DSA-44 edged out Ed25519, its cost being signing (about 13x slower) and size (about 38x the bytes). Since a trail is signed once and verified for years, the expensive side is the side we do least. BLS was rejected on a requirement, not a number: aggregation collapses many signatures into 96 bytes, but an aggregate verifies all-or-nothing and needs every key and message, which is incompatible with disclosing one branch of a decision without revealing the rest.
The reframing that matters: signatures are not confidential, so harvest-now-decrypt-later does not apply, and the real threat is retroactive forgery inside the retention window. Anchoring a root to a public ledger proves when a record existed independently of the signature scheme, so a broken key costs the ability to prove who sealed a record but not when or what. Anchoring is therefore on the critical path and ML-DSA is staged behind it, with the DSSE envelope kept algorithm-agile and the algorithm recorded per capsule.
The per-run audit trail
Each run is an append-only, hash-chained sequence of events (LLM calls, tool calls, retrievals, sub-agent spawns, human approvals, guardrail decisions), with a Merkle root over all of them and a DSSE signature over the root. Only hashes of inputs and outputs are stored, never the payloads, so the trail is privacy-preserving by default. The verifier does not merely return valid or invalid; it names the guarantee that broke.
| Operation (102-event run) | Cost | Scaling |
|---|---|---|
| Record one event | ~121 us | flat, under 0.01% of a step |
| Seal, build the root | 0.15 ms | linear in events |
| Verify, recompute all leaves | 9.2 ms | linear, about 60x seal |
Recording is under a hundredth of a percent of a typical agent step, which is dominated by the model round-trip, so overhead is not a reason to sample and the trail stays complete rather than statistical. Verification is far heavier than sealing because it recomputes every leaf from content while sealing only builds the tree over hashes already present. That asymmetry is correct for an audit trail, where recording is constant and verification is rare, and it marks bulk re-verification as where the native core and parallelism pay off next. To make the guarantees concrete, a demo attacks a sealed run eight ways, each rejected with a specific reason:
| Attack | Rejected as |
|---|---|
| Rewrite a model output | leaf-hash mismatch |
| Delete or insert an event | event-count mismatch |
| Reorder events | chain break |
| Truncate the run | chain-head mismatch |
| Downgrade the hash algorithm | leaf-hash mismatch |
| Present under the wrong key | signature does not verify |
The cross-run transparency log
A valid per-run capsule cannot prove the set of runs is complete. An operator who drops an inconvenient run holds a collection of individually perfect capsules and a history with a hole in it. We close that with a single append-only log of capsule roots supporting inclusion proofs (a run is in the log) and consistency proofs (the log at an earlier size is an exact prefix of the log now). Consistency catches censorship: a rebuilt log with a run removed is internally well-formed and every surviving capsule still verifies, but it fails a consistency check against the previously published root, and the same check rejects a backdated insertion. This works only against a root committed before the tampering, so that root must live somewhere the operator cannot revise, such as a public anchor.
We use a Merkle Mountain Range rather than a plain Merkle tree because an append only adds nodes and never rewrites one, which lets the log live on write-once or object storage and keeps old inclusion proofs valid as the log grows. That property is near-optimal, not merely convenient: ePrint 2025/234 proves any succinct append-only commitment must induce a superlinear number of witness updates, and a close MMR variant essentially meets the bound.
| Log size | MMR build | Plain rebuild |
|---|---|---|
| 500 | 1.3 ms | 158 ms (120x) |
| 1,000 | 2.6 ms | 618 ms (237x) |
| 2,000 | 5.2 ms | 2,491 ms (483x) |
Appending stays flat as the log grows (about 390k appends per second at both 100 and 10,000 entries) while rebuilding a plain tree falls from 8,176 to 84 roots per second across the same range, the quadratic-versus-linearithmic signature. Storage settles at 2.00 stored hashes per entry and inclusion proofs grow only from 7 to 17 steps between 100 and 100,000 entries. Where we lose: our consistency proofs are O(log^2 n), carrying one path per peak of the old log rather than the achievable O(log n), a few kilobytes at 100,000 entries and a deliberate trade for a construction a reviewer can check by reading.
Limits we do not cross
Hashing gives tamper evidence, not tamper proofing. An adversary who rebuilds a run and recomputes its root produces a self-consistent record, because a Merkle root proves the events match the root, not that the root is the one that was published. The signature catches that, and the public anchor proves the time. The transparency log rests on the same assumption in the large: it detects a deleted run only relative to a root committed before the deletion. We state these boundaries in the product and its demos, and a test asserts the forgery demo keeps showing the attack we do not stop, so the honest case cannot quietly disappear.
Reproducibility
Every figure here comes from a committed benchmark that reruns in one command, alongside the tests, the cross-language and cross-implementation conformance suites, and the CI matrix that exercises Python 3.10 to 3.14 with and without the native core. The methodology, the raw JSON, and the decision notes (including the dimensions where each choice loses) are in the repository, because a layer that makes AI auditable has to be auditable itself.