How the metadata policy judge actually works.
Two different machines guard your calls. A deterministic policy engine runs on your machine, before the model call. The judge grades afterwards, and it never sees your text. Here is exactly what it receives.
First machine: the policy engine, before the model call
Enforcement is local and deterministic. When you run in block or redact mode, the SDK's own policy engine runs inside your process before the model call: PHI/PII sweeps, secret detection, and known prompt-injection patterns plus a local statistical classifier (measured, not guaranteed). It blocks or redacts before the request leaves. No hosted AI is involved in that decision, no network is required for it, and its verdict is sealed into the hash chain.
Second machine: the judge, after capture
Once a captured record lands in the ledger, an optional judge (Google Gemini, OpenAI, Qwen on Alibaba Cloud, or a combination of them) grades it after the fact. Its instruction sheet is short, and we can quote it, because it is in our own source: the judge receives no content, only structural metadata about the exchange itself and two facts about your workspace:
- the name of the commitment algorithm, never the commitments themselves: the hashes of prompt and response stay in the ledger, and no identifier of the record, your client, the request or the session is sent,
- an integer token count,
- your policy tag and the event type,
- optionally a
pii_signalslist naming the pattern types the local engine detected, - the model and provider you called, how many responses you asked for, the names of any tools you offered, and the size of any gap in your client’s sequence numbers, and
- the policy settings you configured, with a seven-day count of your workspace’s breaches and graded records.
It never receives the prompt or response text. There is nothing to send: the SDK threw the raw text away on your machine, and there is no raw-text column anywhere in our schema. The judge's instructions also tell it to treat every field value as untrusted data, never as instructions, so a hostile prompt cannot smuggle orders to its own reviewer.
One judge receives one thing more, and only when it asks. The Qwen judge can look up how your own reviewers previously ruled on the same policy tag, and the answer it gets back is a set of aggregate counts: how many interactions carrying that tag were escalated, and how many a reviewer then cleared, confirmed as a breach, or judged a policy gap, together with the length of the window counted. Whole numbers and the tag, never a reviewer's name, note or reason, and never any prompt or response text. Those counts are about your workspace's review history rather than about this exchange, and because they are sent back to the judge to grade with, they leave us. See Privacy Policy Section 8 for where they go. Gemini and OpenAI have no such lookup and never receive it.
What the verdict is, and is not
The judge returns a breach flag, a short reason, and a 0 to 100 risk score. That verdict is advisory: a review signal for your team, not a certification, and deliberately not part of the hash chain. The chain seals the deterministic facts of capture and enforcement, which exist before any judge has run. You can tune how conservatively ambiguity is graded per workspace: minimise false positives, surface every plausible concern, or the balanced default.
- Honest states: clean, flagged, pending, failed, and unavailable remain visible rather than being invented.
- Linked evidence: the verdict is linked to the captured audit record for investigation and export.
- Visible operations: the desktop app and dashboard surface record and grading status as it arrives.
Which judge, and whose key pays for it
You pick the model that grades your workspace: Google Gemini, OpenAI, Qwen on Alibaba Cloud, or a combination of them (each record graded once per judge you pick; a breach found by any of them wins). Qwen is the only one we give tools to, and there are two of them. It can hand an interaction to a human reviewer inside your own workspace instead of grading it. That escalation is recorded with you and goes nowhere else. And it can ask for the prior-review counts described above, whose answer is sent back to it. The two differ in exactly that: where the answer goes. You also pick whose provider key pays for those model calls, and that part depends on your plan:
- Pro and Free bring their own key. Paste a Gemini, OpenAI or Qwen key into the dashboard and the judge grades on it. Those model calls are billed by Google, OpenAI or Alibaba Cloud to your account, at your rates, not by us.
- Premium chooses either. Keep using your own key, or switch to Foxy's managed keys and let us cover the model calls. It is a toggle in the dashboard, not a migration.
To say the quiet part out loud: audit-event credits are not model tokens. Your plan buys capture, chaining, anchoring, exports and verification. Unless you are on Premium using our managed keys, the AI judge runs on your provider account and appears on your provider's bill.
A key you store is encrypted before it is written down, decrypted in memory only for the moment a verdict is produced, and never returned by our API, written into your audit chain, or included in an export. The dashboard can only tell you that a key is set, never show it back to you.
If no key is configured and you are not on managed keys, nothing is faked: those records fall back to deterministic metadata checks and are labelled as such, so an unavailable judge reads as an honest unknown rather than a clean pass.
See the judge catch a breach.
Book a demo and we'll run a policy-violating call and watch the flag fire in real time.
Book a demo
Foxy Audit