The decorator and the client

What you'll learn

• how to make a client and decorate your model call;
• which of your function's arguments the SDK treats as the prompt;
• how to use the SDK with async functions and with streamed answers;
• how to get a receipt for each call;
• what to do when you cannot use a decorator.

Step 1: Make a client

A client holds your settings. Make one when your program starts, and keep it for as long as the program runs.

from foxy_audit import FoxyClient

foxy = FoxyClient()

With no arguments, the client reads its settings from your environment, such as FOXY_API_KEY and FOXY_BACKEND_URL. You can also pass settings in code, for example FoxyClient(mode="block"). Configuration lists every setting.

A missing key or a network problem does not stop the client being made. With no key, the SDK still hashes each call and gives you a receipt. It saves nothing to disk and sends nothing over the network. It does send a small signal to an optional desktop app on your own machine (127.0.0.1), unless you pass desktop_ping=False.

With a key set, making the client opens the spool file on your disk. If that file's folder cannot be written to, the client raises an error at once, so you see the problem when your program starts.

With a key set, making the client also reads your commitment key, or makes it the first time. The key is kept in commitment.key, next to the spool, and Foxy Audit never sees it. When the SDK makes it, it logs a warning with the file's path. Back that file up. To keep it somewhere else, pass commitment_key_path=. If that path cannot be used, the client raises an error at once.

Step 2: Decorate your model call

Find the function in your program that calls your AI model. Put the decorator on the line above it:

from foxy_audit import FoxyClient

foxy = FoxyClient()


@foxy.audit(policy="default", agent="my-model")
def ask_model(prompt):
    return your_model.generate(prompt)   # your existing code, unchanged

Here your_model.generate stands for whatever your code does today to call the model. The decorator takes four arguments, and all of them are optional:

The dashboard's Systems page lists your systems by name, but it does not show their ids. To see them, ask the backend with your API key:

import os

import requests

answer = requests.get("https://app.foxyaudit.tech/v1/systems",
                      headers={"Authorization": "Bearer " + os.environ["FOXY_API_KEY"]},
                      timeout=10)
print(answer.json())

Each system in the answer has an id. Copy it exactly as it is printed.

Step 3: Call your function as before

You call your function the same way as before. The SDK does its work around the call:

1. Before the call, it finds the prompt among your function's arguments. In block or redact mode, it checks the prompt now.
2. It calls your function. In block mode, a risky prompt stops the call here, and your function is not called. In redact mode, your function gets the cleaned prompt.
3. After the call, it checks the answer and records what it finds. The answer is stopped only if you set the answer check to block. See Response scanning.
4. It makes a hash of the prompt and of the answer. With an API key, it saves a record on your disk.
5. A background thread sends the record to Foxy Audit.

Your code is not the only thing that picks the mode. By default the SDK also reads your workspace's enforcement setting, and a workspace set to block blocks risky prompts even where your code says observe. Modes explains it, and how to turn it off.

If your function raises an error, the SDK records the call as an exception and then raises the same error again. Your own error handling keeps working.

Where the SDK looks for the prompt

The SDK looks at your function's arguments in this order and uses the first one it finds:

1. a keyword argument called prompt, user_prompt, message, messages, contents, text, input or query;
2. if none of those was passed, the first positional argument that is a string.

A prompt passed under any other keyword name is not found. The SDK then checks and hashes an empty prompt. So if your function takes the prompt as, say, question=, rename the argument, or pass the prompt as the first positional argument.

A prompt can be a string, or a list of messages like [{"role": "user", "content": "..."}]. If you pass a list of messages, pass it by keyword, for example ask_model(messages=my_messages). A list passed without a name is not found either.

Step 4: Get a receipt for each call

If you want to know what was recorded, give the client an on_event function. The SDK calls it after each call, with a receipt:

from foxy_audit import FoxyClient


def on_event(receipt):
    print(receipt["event_id"], receipt["event_type"], receipt["submitted"])


foxy = FoxyClient(on_event=on_event)

A receipt is a plain Python dictionary with thirteen keys. Here is a real one, from a call with no API key. Only the event_id changes from call to call. The same prompt and answer give the same two hashes, unless you turned on salts.

{
  "event_id": "<a new id for each call>",
  "event_type": "interaction",
  "policy_tag": "hipaa",
  "decision": null,
  "policy_rules": null,
  "blocked_reason": null,
  "ruleset_version": null,
  "ruleset_hash": null,
  "commitment_alg": "sha256-legacy",
  "prompt_hash": "<64 letters and digits>",
  "response_hash": "<64 letters and digits>",
  "pii_signals": [],
  "submitted": false
}

• event_id is the record's id in the ledger. Keep it if you might need to prove this call later with foxy explain.
• submitted is true when the record was saved and queued for sending. It does not mean the backend has received it yet.
• decision and policy_rules are null when no check ran before the call, as in observe mode. In block or redact mode, a clean prompt gives "allowed" and an empty list.
• pii_signals is always a list. An empty one means no personal data was found.

Four things to know about on_event:

• It must be a normal function. An async def function, or something that is not a function, is refused with a TypeError when you set it.
• If it raises an error, the SDK ignores the error and notes it in its debug log, so your model call still works.
• In an async program, it runs on a helper thread, not on your event loop.
• One kind of call gives no receipt: with audit_required=True, a call whose confirmation did not arrive in time. It raises AuditRequiredError, and its record is sent later all the same. To account for every record, compare with an export, not with your receipts.

Step 5: Use the shortcut for small scripts

If your program has one client, you can skip making it yourself. The module-level decorator makes one for you from your environment:

from foxy_audit import audit


@audit(policy="default")
def summarize(text):
    return your_model.generate("Summarize: " + text)

This shortcut takes policy, agent and mode. It has no system_id argument: set FOXY_SYSTEM_ID in the environment instead, or make a FoxyClient.

Step 6: Use it with async code and streamed answers

The decorator works with four kinds of function, and it notices which kind yours is:

An async function is decorated the same way:

@foxy.audit(policy="default")
async def ask_model(prompt):
    return await your_model.generate_async(prompt)

So is a function that streams its answer in pieces:

@foxy.audit(policy="default")
def stream_answer(prompt):
    for piece in your_model.stream(prompt):
        yield piece

For a stream, the record is written when the stream ends. It is recorded as a stream, or as redacted in redact mode when the prompt was cleaned. If your code stops reading early, for example with break, the stream never ends, and the call is recorded as an exception.

Step 7: Record a call without a decorator

Sometimes you cannot wrap a function. The call may happen inside a library you do not control. In that case, record it yourself with log_interaction:

from foxy_audit import FoxyClient

foxy = FoxyClient()
prompt = "What is the capital of France?"
answer = "Paris."
foxy.log_interaction(prompt, answer, policy="default", agent="my-model")

This makes the same kind of record as the decorator, but it does not check the prompt first. It runs after the call, so it cannot stop a risky prompt. Use the decorator when you can.

Step 8: Check a prompt without calling a model

check asks the SDK what it thinks of a prompt. It needs no key and no network. It returns only labels, never your text:

from foxy_audit import check

result = check("My social security number is 123-45-6789", policy="hipaa")
print(result.triggered, result.rules)

What you should see:

True ['phi.ssn_pattern']

FoxyClient has the same method, foxy.check(prompt, policy), if you already have a client.

If it goes wrong