Arbiter AI

Security model

Five invariants, each enforced in code and checked by a test.

This is a design-and-verification document, not a compliance page. For each property: what the code does to make it hold, and what would fail if it stopped holding.

Invariants

  1. Nothing reaches a model until the state is de-identified

    The ingest and query graphs carry a deidentified flag that only the de-identification node sets. The annotate, chunk_embed and generate nodes assert it before doing anything, so a refactor that reorders the graph fails loudly instead of leaking quietly. The LangGraph checkpointer is off by default because the state before that node holds the raw text and the plaintext map.

    Enforced by assert state["deidentified"] in worker/graph.py (annotate, chunk_embed, generate); Presidio and the added recognizers in worker/deid.py; CHECKPOINTER=none
    Tested by tests/test_deid.py (real Presidio, thirty-plus cases: labelled MRNs, phones with extensions, address lines, titles, frequencies kept, calendar dates removed, round trip through restore); deid_recall and deid_recall_strict in eval/run_eval.py

  2. The PHI map never enters the vector store

    Chunks hold placeholder text and an embedding of placeholder text. The map from placeholder to value goes to a separate table, phi_tokens, encrypted inside Postgres with pgp_sym_encrypt under a per-tenant key. Decrypting with another tenant's key returns "Wrong key or corrupt data". Only the re-identification step calls pgp_sym_decrypt, and only for the tokens the answer uses.

    Enforced in worker/store.py (chunks are written from the de-identified text; the map only ever through pgp_sym_encrypt) and worker/reid.py
    Tested by deid_recall_strict, which scans every stored chunk for every component of every injected identifier (1.000 on the 20-record set); tests/test_reid.py::test_restore_uses_the_right_document_and_only_needed_tokens

  3. Retrieval is hard-filtered on patient

    Every retrieval query carries WHERE tenant_id=%s AND patient_id=%s in its SQL, on top of the row-level-security policy on chunks. The patient is resolved inside the tenant, so an external id that belongs to another tenant resolves to nothing, and a question about a patient with nothing indexed takes the no_chunks branch without a model call.

    Enforced in worker/retrieval.py::hybrid; the tenant_isolation policy on every PHI table in db/init.sql
    Tested by tests/test_retrieval.py::test_query_is_hard_filtered_by_tenant_and_patient, which checks the SQL and the bind parameters; cross_patient_leaks in eval/run_eval.py, which checks every answer for every other patient's injected identifiers, 0 in every run

  4. Tenant identity comes from the credential, never a client-supplied field

    An API key is stored as an HMAC-SHA256 under a pepper kept outside the database, and the lookup returns the tenant. Request bodies have no tenant field to send. Every transaction runs set_config('app.tenant_id', …, true), and the application connects as app_rw, a plain role the row-level-security policies apply to. Key lookups are cached for 60 seconds, so revocation lands within a minute.

    Enforced in gateway/auth.py; per-transaction app.tenant_id in gateway/main.py and worker/store.py; app_rw with its password set by db/02_app_role.sh
    Tested by tests/test_auth.py (hashing, pepper rotation, revoked and tampered keys, rate limits); the cross-tenant probes below; eval/run_eval.py refuses to run as a superuser or a BYPASSRLS role, so its leak count cannot be flattered by the same mistake twice

  5. The audit log is append-only

    Every ingest, query and fact read writes an audit_log row with the credential, the action, the patient and a detail object. The application role holds INSERT and SELECT on that table and nothing else, so there is no code path that could edit history.

    Enforced by REVOKE UPDATE, DELETE ON audit_log FROM app_rw in db/init.sql (the blanket grant above it had included UPDATE; that was the bug)
    Tested by running UPDATE and DELETE on audit_log as app_rw: permission denied

The superuser finding

The gateway and the worker originally connected to Postgres as the database owner, a superuser. Row-level security does not apply to a superuser, so every policy in the schema was inert. The README said other tenants' jobs return 404 via RLS, and that was false: the isolation the code described did not exist. It surfaced during the first end-to-end bring-up, not in the unit tests, which fake the pool.

The fix is small. The application connects as app_rw, a plain login role the policies apply to. Its password comes from the environment through db/02_app_role.sh, because Postgres runs init .sql files without environment substitution and .sh files with it, which also removed the hard-coded password from init.sql. The role gets DELETE on chunks and phi_tokens, which re-ingest needs, and loses UPDATE and DELETE on audit_log.

Then the probes, run against the live stack and kept: key B gets 404 on tenant A's job; an app_rw session with app.tenant_id set to A sees zero of B's rows; a cross-tenant INSERT while set to A violates the policy. And eval/run_eval.py now refuses to run as a superuser or a BYPASSRLS role.

What I took from it: isolation should not depend on application code being bug-free, and it also should not depend on a connection string being right. The probes are what make the second half true.

This is a reference implementation maintained by one engineer. It is not a hosted service, does not process real PHI, and no Business Associate Agreement is offered. If you want to run it against real records, run it in your own environment under your own compliance program.

Also in the code

  • Scoped credentials: ingest, query, admin. A key without the scope gets 403.
  • Models are local: Ollama on Apple Silicon or vLLM on NVIDIA. No cloud model endpoint is configured anywhere, and cloud tracing is off (LANGCHAIN_TRACING_V2=false).
  • medspacy's sentence splitter logs document text at debug level by default; that logging is disabled at import, so de-identified text never reaches container logs.
  • storage_uri is confined to DATA_ROOT after symlink resolution; a /etc/passwd probe fails the job cleanly.
  • Idempotent jobs: a retried POST with the same Idempotency-Key maps to the same job.

Not in the reference implementation

  • TLS between the internal services. The public site has it; the compose stack does not.
  • A KMS-issued key per tenant. Today the key that encrypts the PHI map, the job request and result, and the patient identifier is derived from TENANT_KEK and the tenant id.
  • A written risk analysis, workforce training, incident response: the things a compliance program adds around the software, which the software cannot supply.
  • Real data, ever. TODO.md in the repository lists what would have to be true first.