About
One engineer, one question, one checkable answer.
I wanted a public, checkable answer to "what does a correct clinical document pipeline look like?" Arbiter AI is that answer, with the evaluation harness that shows where it holds and where it does not yet.
Who built it
Yoonbo Cho. I designed, built and evaluated every part of it: the pipeline, the schema and its isolation policies, the eval harness, and the post-mortems on what the harness caught.
The code, the tests and the eval harness are open source. Apache-2.0 licensed.
Why
The hard part of putting language models in front of medical records is not getting them to answer. It is getting them to answer only when they can prove it, never with information they should not have seen, and being able to show both. Most write-ups on this stop at the diagram. I wanted the code, the tests, the numbers, and the bugs.
What it is not
- Not a product. There is nothing to buy and no plan to sell.
- Not a hosted service. It runs where you run it.
- Not a place for real records. Every record it ships is synthetic, and no Business Associate Agreement is offered. If you want to run it against real data, run it in your own environment under your own compliance program.
- Not certified. There is no HIPAA certification, and no SOC 2 or HITRUST report is claimed.
The security model says exactly what is in the code and what is not. The docs show how to run it and reproduce the numbers.