Start here

What you'll learn

• what Foxy Audit is for;
• what an SDK is, and what this one does;
• what goes to Foxy Audit, and what never does;
• where to go next.

What Foxy Audit is for

Many companies now use an AI model to help with their work. In health care, banking and law, people often have to prove what their AI did. An auditor may ask: what did the program send to the model, and when? Was any private data sent?

Foxy Audit helps you keep that proof. It keeps a record each time your program asks an AI model something.

The record does not hold the words your program sent. It does not hold the words that came back, either. It holds a hash of them. A hash is like a fingerprint. It is a short string of letters and numbers, worked out from the text. The same text always gives the same hash. You cannot turn a hash back into the text.

Later, you can prove that a record matches a prompt you still have. You work out the hash again and compare the two. If they match, the record is about that prompt.

Where each record goes

The records go into a ledger. A ledger is a list that only grows. Each new record is linked to the record before it, like links in a chain. A free tool called the verifier works out every link again and tells you where the first one fails.

The links use no secret. So someone who could rewrite the whole stored ledger could also work out new links. That is why you keep the last link, the head, each time you check. A later change to an old record then shows against the head you kept, or against an anchor, where your workspace has one: a copy of the head on a public blockchain.

1. Your programcalls an AI model, the way it does today.
2. The SDKmakes hashes on your machine, and can check the prompt there.
3. Hashes, counts and labelsare all that Foxy Audit receives.
4. The ledgerlinks each record to the one before it.
5. The verifierworks out the links again, on your own machine.

What an SDK is

SDK stands for software development kit. It is a package of code that you add to your own program. You install it once. Then your program can use it.

The Foxy Audit SDK is a Python package called foxy-audit. You install it with pip, the tool Python uses to add packages.

You use it by adding one line above the function in your program that calls the AI model. That line is called a decorator. After that, every call to the function is recorded. In block or redact mode, the prompt is also checked before the call. Your function still works the way it did before.

What the SDK does

• It records each call to your model: a hash of the prompt, a hash of the answer, and a few labels and counts.
• In block or redact mode, it checks each prompt on your machine first. It looks for API keys, private keys and access tokens, for attempts to trick the model, and, under the hipaa and gdpr tags, for personal data.
• It can stop a risky prompt before it reaches the model. It can also blank out the risky part and send the rest.
• It checks the answer that comes back from the model, and records what it finds.
• It makes each hash with a key that Foxy Audit never sees. The SDK makes that key for you and keeps it in a file on your machine. Back the file up.
• With an API key set, it saves each record in a small file on your disk first. Then it sends the record to the ledger in the background.
• If the network is down, the record waits in that file. It is sent when the network comes back, even after a restart. The file holds up to 100,000 waiting records for each key and address. After that, new records are refused, not saved.

What the SDK does not do

• It does not send your prompts or answers to Foxy Audit. Your model provider still gets the prompt, because your own program sends it there.
• It does not catch every risky prompt. Its checks are measured, not guaranteed. The rule catalogue says what they catch and what they miss.
• It does not make your company compliant. It gives you evidence that you can show to an auditor. It is not a certificate.
• It does not change the answer your function returns. It passes it on as it is, or, if you ask it to, stops a flagged one.

Each call takes a little longer. The save to the file happens on your own thread. It normally takes milliseconds, and longer while the file is being cleaned or another program has it locked. In redact mode your function gets the cleaned prompt.

How to use this manual

If you are new, read Your first 10 minutes next. It gets a small program working, first with no account and then connected to Foxy Audit.

After that, read the lessons in Part 3 when you need them. Each one covers one feature. If a word is new to you, look it up in the glossary.