Foxy Audit
Book a demo
← The manual

Questions and answers

Short answers to the questions people ask most, with a link to the chapter that says more.

Skip to the chapter
All chapters

What you'll learn

  • what Foxy Audit sees, and what it never sees;
  • whether the SDK can slow down or break your program;
  • what the evidence is good for, and what it is not.

Does Foxy Audit see my prompts?

No. Your prompts and answers never go to Foxy Audit; only hashes, counts and short labels do. The SDK turns your prompt and your model's answer into hashes on your own machine, and drops the text. See The wire format, which lists every field.

Does my model provider see my prompts?

Yes. Your own code sends the prompt to your model provider, the same way it did before. The SDK does not change where your prompts go. In block mode it can stop a risky prompt before it is sent.

Do I need an account to try it?

No. With no API key, the SDK runs its checks and makes its hashes on your machine, and sends nothing. Your first 10 minutes starts that way.

Will it slow down my program?

A little. Making the hashes is quick. In observe mode, the default, the prompt is not checked before the call at all. In block or redact mode, the prompt is checked before the call. That takes milliseconds for a normal prompt, and longer for a very long one. Configuration has measured times.

Sending happens in the background, not inside your call. Saving each record to the spool file does happen on your thread. It normally takes milliseconds, and longer while the file is being cleaned or another program has it locked. With audit_required=True, each call also waits for its record to be confirmed.

Can the SDK break my program?

Only in these ways:

  • block mode stops a risky prompt with a FoxyPolicyBlocked error;
  • response_scan="block" stops an answer the scan flags with a FoxyResponseBlocked error;
  • audit_required=True raises an AuditRequiredError when a record cannot be confirmed;
  • with an API key set, making the client raises an error when the spool file's folder cannot be written to.

One of these can be turned on without your code: by default the SDK reads your workspace's enforcement setting, and a workspace set to block blocks risky prompts even where your code says observe. Set FOXY_ORG_POLICY=off to keep the choice in your code. See Modes.

Any other problem while recording, such as a full disk or a dead network, is logged and swallowed. Your function still returns its answer.

What happens if Foxy Audit is down?

Your program keeps working. Records wait in the spool on your disk, and they are sent when the backend is back. Up to 100,000 records can wait for each key and address. Past that, new records are refused, not saved, and a warning is logged. See The spool and delivery.

Does it work with my model library?

Yes. The SDK wraps your own function, so it works with any library that function uses. It finds the prompt among your function's arguments: by name, such as prompt= or messages=, or as the first positional argument that is a string. The decorator and the client lists the names. For the answer check, it understands the answer shapes of the common model libraries, and plain text.

Does it work with async code and streamed answers?

Yes. The decorator works with async def functions, and with functions that stream their answer in pieces. See The decorator and the client.

Which Python versions does it support?

Python 3.9 to 3.13.

Does this make my company compliant?

No. Foxy Audit gives you evidence: a record of what your AI did, in which any change shows. An auditor can weigh that evidence. It is not a certificate, and it does not replace your own review.

Does it catch every risky prompt?

No. Its checks are measured, not guaranteed. The rule catalogue gives the measured results, and it lists what the checks cannot see.

Can anyone change the records?

Not without it showing, as long as you keep a copy of the chain's head. Each record is linked to the one before it by a hash. If one record is changed and nothing else is, the links after it stop matching, and the free verifier shows where. You can run the verifier yourself, without relying on Foxy Audit.

The links use no secret. Someone who can rewrite the whole ledger could rebuild every link after a change, and then the export checks out on its own. What gives that away is a head you saved earlier: the rebuilt chain no longer has it. So save the head the verifier prints, and compare it later. See Keep the head.

What if I lose my API key?

Make a new one in the dashboard, under Access, and set it in your programs. Records that were already sent with the old key are still in your ledger. To prove one of them with foxy explain, you need the key its hashes were made with. Since version 1.18.0 that is, by default, the SDK's own key file, so a new API key changes nothing. A record made before 1.18.0 with no key set was made with the old API key. See Configuration.

Records still waiting in the spool under the lost key can never be sent: sending them, or moving them to your new key, needs the old key. They stay in the spool, and foxy doctor counts them in a notice. A key that was revoked, not lost, is different: foxy spool rekey can move its records. See If your API key changes.

What if I lose the commitment key file?

You can no longer prove which text those records' hashes came from. The records are still in your ledger, and the chain still checks out, because it never needs the key. The SDK makes a new key the next time your program starts, and new records use it. So back the file up, and keep the copy as safe as your API key. In a container, keep the key in a secret store and set FOXY_COMMITMENT_KEY.

Is the SDK open source?

The SDK is under the MIT licence. The verifier comes with each export, under its own licence. That licence lets you run it, read it, and pass it on with the export, for example to your auditor.

How do I remove the SDK?

Three steps, in this order:

  1. Run foxy spool status, and check that nothing is waiting. If records are waiting, send them first with foxy spool drain.
  2. Run python -m pip uninstall foxy-audit. The command goes with the package, so step 1 comes first.
  3. Delete the ~/.foxy-audit folder. Uninstalling leaves it. It holds the spool file, and org_policy.json, a copy of your workspace's enforcement setting.

The SDK limits who can open the spool file, usually to your own account, so delete the folder as the same account. The spool and delivery explains who can read it.

What is the fox on my desktop?

An optional desktop app that reacts to your records as they happen. The SDK sends it small signals on your own machine. The SDK works the same without it. Download it, for Windows or Linux, from the desktop app page.