Generated by Grok
Generated by Grok

Introduction

Hi everyone!

As AI agents become increasingly embedded in business workflows, automation, customer support, DevOps, document analysis, and even internal operations, security testing is quickly becoming a necessary part of AppSec and pentesting.

I’ve tested a wide range of AI-powered systems in my work — simple LLMs, multi-step agents, and even agent frameworks wrapped in business logic and guardrails. And like many security engineers, I realized something important:

Manual AI agent testing takes forever.

  • You rewrite the same intent twenty different ways.
  • You try different personalities.
  • You run through endless payload variations.
  • Guardrails behave unpredictably.
  • You might or might not hit a working Prompt Injection.

And even if you do, reproducing and reusing it can be surprisingly painful.

One day, I asked myself:

Can I automate AI agent pentesting… using another AI?

Turns out — yes. And that’s how Stealth Promptwas born.

This article is the story of how the idea formed, how the tool works, and what I plan next.

🔍 The Problem: Manual AI Pentesting Doesn’t Scale

If you’ve ever tried to break an AI assistant, you already know:

  • endless reformulations
  • different tones and personas
  • long message chains
  • inconsistent guardrails
  • secondary LLM “proxies” altering behavior
  • and hours of trial-and-error

Even when protection uses a defense LLM, you still need to measure the effectiveness of those guardrails.

This is slow. This is tedious. And this is something a machine should do better than a human.

So I designed a simple concept: AI attacker ↔ browser ↔ AI agent

Where the attacker is itself an LLM.

That was the seed of Stealth Prompt.

💡 The Idea Behind Stealth Prompt

I wanted Stealth Prompt to act like a real pentester:

  • generate payloads
  • send them through a real browser
  • imitate human interaction
  • observe the agent’s behavior
  • extract information
  • detect sensitive output
  • store successful attack chains
  • retry and iterate intelligently

In other words:

AI testing AI, through the browser, using AI to evaluate the results.

Stealth Prompt is a small but powerful tool that automates the most boring and repetitive parts of prompt injection testing.

🛠 Architecture of Stealth Prompt

1. Real Browser Interaction (Selenium + Chromium)

I didn’t want to start from API testing. Why?

Because UI-based testing:

  • reflects real user behavior
  • triggers actual guardrails
  • avoids token/session headaches
  • exposes UX-level logic
  • matches how many commercial AI agents operate

So Stealth Promptuses:

  • Selenium
  • Chromiumin remote debug mode
  • no login automation — you authenticate once manually

You open the browser → log in → run Stealth Prompt → and it begins interacting with the AI agent in real time.

2. YAML-Based Configuration

Every AI agent UI is different:

  • input field selectors
  • send button selectors
  • message containers

All of this is defined in .yaml files.

You can:

  • use a default config, or
  • create specific configs for different agents or challenges

This makes Stealth Prompt highly flexible.

3. Payload Generation (LLM-Based)

Stealth Prompt uses:

  • OpenAI API(fast and inexpensive, perfect for this use case)
  • or local models viaOllama

On my hardware, Ollama is slower, so I used OpenAI’s mini model.

The LLM tries to:

  • bypass guardrails
  • trigger unintended behaviors
  • extract capabilities or system instructions
  • retrieve sensitive data

The objective is at least Information Disclosure, ideally something deeper.

4. Automatic Detection of “Interesting Output”

If Stealth Prompt sees something that looks like:

  • API keys
  • credentials
  • system prompts
  • secrets
  • structured sensitive information

…it asks the user for confirmation in the console.

If confirmed — the entire chain is saved as a successful payload path.

Next time you run the tool, it can reuse this chain without regeneration.

5. Logging and Replay

Every run produces separate logs and output files. You can:

  • analyze previous sessions
  • replay chains
  • compare behaviors
  • track improvements over time

This is especially useful for guardrail regression testing.

🎥 Demo (Gandalf Challenge)

Stealth Prompt successfully identifies the early Gandalf keys and maneuvers around the guardrails fairly easily.

📌 Test Environment

My first test target was Gandalf by LakeraAI— ideal for this type of automated testing:

  • multiple guardrail layers
  • progressive difficulty
  • predictable format
  • measurable success criteria

Next, I aim to test Stealth Prompt on:

The more diverse the testing grounds → the more universal the tool becomes.

⚙ What I Want to Improve Next

1. Better Success Chain Database

Right now, Stealth Prompt saves successful payload chains, but:

  • chain reuse logic can improve
  • some chains need scoring
  • there are thousands of public PI payloads I want to integrate

This will make detection faster and more reliable.

2. Improved Conversational Mode

Stealth Prompt already supports:

AI Pentester ↔ AI Assistant dialogue simulation

But I want it to be:

  • more goal-driven
  • persona-aware
  • targeted based on agent type
  • more adaptive

For example:

  • receptionist bot → attempt to extract API endpoints or user info
  • document analyzer → attempt to leak file content
  • knowledge assistant → attempt to bypass restrictions on context

This will make Stealth Prompt smarter and more effective.

3. Externalizing Prompts

All system prompts should live in separate files for fast iteration.

4. API Mode

Not all AI agents have a UI — some are API-only.

Stealth Prompt will eventually support:

  • UI mode
  • API mode
  • hybrid mode

Configurable in YAML.

5. Web Dashboard

In the future, I want to build a small dashboard for:

  • visualizing results
  • tracking statistics
  • managing configs
  • browsing successful chains

This will make Stealth Prompt feel like a real pentesting toolkit.

💸 Cost Analysis: It’s Cheap

Here’s the surprising part:

Stealth Prompt’s total cost after several days of testing and settings → about $0.05

OpenAI API Usage Statistics
OpenAI API Usage Statistics

That’s basically free. The time savings alone justify the entire approach.

📦 GitHub Repository

The project is available here:

👉 https://github.com/whoishacked/stealth_prompt

Contributions, ideas, and improvements are welcome.

🎯 Final Thoughts

In a short period of time, I built a tool that:

  • automates AI agent pentesting
  • interacts through the actual browser
  • generates intelligent payloads
  • evaluates responses using LLMs
  • stores and reuses successful attack chains
  • costs almost nothing
  • and makes AI testing a lot more fun

Most importantly, I gained valuable experience and a pet project I genuinely want to continue developing.

More updates about Stealth Prompt coming soon!

Stay tuned.