Modes: observe, block and redact

What you'll learn

• what each of the three modes does to a real call;
• how to pick a mode, and where to set it;
• what happens when blanking out a finding is not possible;
• how your workspace can make the SDK stricter, and how to turn that off.

The three modes

observe is the default because it never stops a call. It runs no checks on the prompt before the call, so its records cannot tell you what block would stop. They do carry two kinds of finding: labels for the personal data found in the prompt and the answer, such as email, and the ids of anything the answer check flagged.

To see what block would stop before you turn it on, ask the SDK directly: run foxy check on your prompts, or call check() in your code. Or run a test copy of your program in block mode.

Step 1: Choose where to set the mode

You can set the mode in three places. The most specific one wins.

1. For one function, on its decorator: @foxy.audit(policy="hipaa", mode="block").
2. For the whole client, in code: FoxyClient(mode="block").
3. For the whole program, in the environment: FOXY_MODE=block.

A value the SDK does not know, such as a typing mistake, falls back to observe. On a decorator it also logs a warning. A mistake never turns blocking on by surprise.

Step 2: Try block mode

This is the program from Your first 10 minutes, cut down to the blocked call:

from foxy_audit import FoxyClient, FoxyPolicyBlocked

foxy = FoxyClient(mode="block")


@foxy.audit(policy="hipaa")
def ask_model(prompt):
    return "You said: " + prompt


try:
    ask_model("My social security number is 123-45-6789")
except FoxyPolicyBlocked as error:
    print("Blocked:", error)

What you should see:

Blocked: Foxy Audit blocked a prompt under policy 'hipaa' (reason: phi). The wrapped function was not called.

The message names the policy tag and the reason. It never repeats your prompt, so it is safe to write to a log.

In your own program, decide what to do when a prompt is blocked. You might show the user a polite message, or ask them to remove the private data and try again.

If you also use audit_required=True, a blocked call can raise AuditRequiredError instead, when the record of the block was not confirmed in time. The prompt was still stopped. Catch both:

from foxy_audit import AuditRequiredError, FoxyPolicyBlocked

try:
    ask_model("My social security number is 123-45-6789")
except (FoxyPolicyBlocked, AuditRequiredError) as error:
    print("No answer:", type(error).__name__)

Step 3: Try redact mode

In redact mode the SDK blanks out what it found, and sends the rest to your model:

from foxy_audit import FoxyClient

foxy = FoxyClient(mode="redact")


@foxy.audit(policy="gdpr")
def ask_model(prompt):
    return "The model received: " + prompt


print(ask_model("Write a thank-you note to jane@example.com for her order."))

What you should see:

The model received: Write a thank-you note to [REDACTED:email] for her order.

Your function got the prompt with the email address blanked out. The record says redacted, and it lists the rule that fired.

When blanking out does not work

Some findings cannot be blanked out. The SDK checks the cleaned prompt a second time. If a finding still fires on it, the SDK blocks the call instead of sending it. It would rather stop the call than record a redaction that did not work.

This happens when:

• the finding came from the classifier, which scores the whole prompt and has no single part to remove;
• the finding came from the optional deep personal-data check, which has no exact span to replace;
• the risky value is in a part of a structured prompt that is not text.

When this happens, the error message names the rules that were still found, so you can see why a redact call was blocked.

The markers come from a fixed list, such as [REDACTED:email] and [REDACTED:ssn]. A marker you type yourself does not hide anything from the second check.

What each mode records

Every call makes a record, whatever the mode. This table shows what the record says:

A function that streams its answer is recorded as stream in place of interaction.

For a blocked call, the record still has a hash of the prompt. The answer's hash is the hash of an empty answer, because your function never ran. For an exception, the hash is of the prompt as you passed it, before any redaction.

When your workspace sets the mode

An admin can set an enforcement level for the whole workspace, in the dashboard under Settings. With an API key set, the SDK reads it from Foxy Audit by default, every five minutes, in the background. The read never happens on your call's path.

The rule is simple to state: your workspace can make the SDK stricter, but never looser.

So if your code asks for observe and the workspace says redact, the SDK stays in observe. Redaction changes what your model sees, and a workspace should not do that to code that asked only to watch.

When the workspace caused a block, the error message says so:

This block came from your Foxy Audit workspace policy (sdk_enforcement=block), not from this code's own mode. Change it in Settings, or set FOXY_ORG_POLICY=off to ignore workspace policy in this deployment.

To stop the SDK from reading the workspace setting, set FOXY_ORG_POLICY=off. This can only make the SDK equal to or looser than the workspace, never stricter than your code. It is useful on a machine with no internet, or when you want your code to be the only place the mode is set.

If it goes wrong