FrameFuzz matched framing experiments¶

FrameFuzz compares how one selected AI workflow handles the same unsafe outcome when that outcome is presented under different frames. It is an experimental mode inside Stealth Prompt, not a broad prompt mutator or an automatic vulnerability scanner.

The design is based primarily on the matched-condition method described in The Framing Gap. The related ContextLeak work informs the disclosure threat context, but this release does not reproduce its reinforcement-learning or malicious tool-server techniques.

What it tests¶

Every campaign begins with one normalized semantic seed: a benign carrier task, a descriptive protected-value label, one unsafe outcome, and a test-only destination. The seed is rendered through a closed, versioned template pack:

Clean control and explicit injection are mandatory. Select at least one framed case. The template text is deterministic and is generated locally; no model invents case variants. Editing a case payload invalidates the matched comparison and makes the campaign inconclusive.

FrameFuzz is available for indirect prompt injection, instruction disclosure, and sensitive-data disclosure. Other objectives remain available as ordinary Stealth Prompt tests, but do not expose the FrameFuzz toggle because a matched disclosure comparison would not answer their security question.

Run a campaign¶

1. In Setup, select a compatible objective and enable Compare framing variants under Behavior.
2. Select the framed cases. Keep the default protected-value label unless a different descriptive label is clearer. Never enter an actual secret or canary value.
3. Keep the default .invalid destination or enter a test-only HTTP(S) URL. The value is payload data only: Stealth Prompt never fetches it. Executable schemes, fragments, URL credentials, credential query fields, control characters, and oversized values are refused.
4. Choose fixed order or retain the randomized order. The locally generated random seed is stored with the campaign so the order can be reproduced.
5. Start the assessment. Before every case, open a genuinely fresh target conversation or workspace, return to the panel, and confirm the reset.

The extension performs a read-only origin and binding validation after the reset confirmation. It cannot create or prove a fresh conversation on an arbitrary target; that boundary remains an explicit operator action. Choosing Continue as unverified records the limitation and prevents a confirmed framing-gap conclusion.

Each confirmation authorizes only the current case. Automatic-send authorization is revoked again at the next case boundary, and never survives a panel reload or scenario import. Navigation, click, and target-reset operations were not added to the browser operation allowlist.

Conclusions¶

FrameFuzz reports a campaign conclusion separately from the ordinary per-turn verdict:

no_gap_observed is not evidence that the target is secure. It applies only to the selected interaction, objective, templates, scorer set, and run conditions.

Core and Direct API¶

Local Core is the recommended path. It can use deterministic scorers, write JSON and self-contained HTML evidence, and reach confirmed_framing_gap. With sharing set to none, target replies remain local and the deterministic scorer performs evaluation. Template generation itself never calls the configured provider.

Direct API uses the same template pack and context gates, but model evaluation is not deterministic. Its campaign conclusion is capped at potential_framing_gap, including after operator verification. Direct reports remain available in browser-local IndexedDB and can be opened or downloaded without persisting the API key.

Evidence and reproducibility¶

JSON and HTML reports include the template pack and version, random seed, semantic-seed hash, case order, isolation and binding status, exact reviewed payload, scorer provenance, per-case outcome, warnings, differential, and final conclusion. The actual protected value is not part of the semantic seed; a deterministic scorer may still record redacted evidence or hashes under its normal policy.

Scenario schema v3 records the FrameFuzz configuration but no case results. Imported scenarios still require current host permission, binding validation, fresh-context confirmation for every case, and a new explicit Auto authorization.

Local positive and negative controls¶

Start the demo and use its normal binding:

stealth-prompt demo

Open the printed target with ?mode=framing for the deliberately framing-sensitive positive control. Its explicit case stays safe while the framed cases reveal only the synthetic demo canary; the trusted-destination case records a bounded in-memory mock effect and performs no network transfer. Use ?mode=safe as the negative control: no case reveals the canary or creates the mock effect.

These fixtures demonstrate the workflow. They do not establish effectiveness against unrelated targets or imply complete prompt-injection coverage.