AI Assurance

AI Security Vetting

Continuous AI Assurance for Enterprise

// AISV scanner — live trace > init pen-test engine > recon: walking 247 routes > lang: python / fastapi > hunt: handlers/users.py:42 > class: sql_injection > sink: cursor.execute(f"...") > taint: req.args["id"] > conf: 0.94 > PoC: id=1' UNION SELECT-- > bridge: 200 OK + DB leak > validator: confirmed > OWASP: A03:2021 > CWE-89 mapped > severity: critical > auto-fix: patch ready > hunt: auth.py:88 > class: idor > sink: get_user(id) > no ownership check > PoC: GET /api/users/42 > live-fire: 200 OK > cross-tenant leak > hunt: upload.py:31 > class: path_traversal > sink: open(filename) > PoC: ../../etc/passwd > canary: matched > continuous: round 2 > finding: ssrf > oast callback fired > finding: race condition > TOCTOU window 14ms > finding: weak jwt > alg=none accepted > chain: 3 flaws → RCE > report: ready > coverage: 94% > round complete > init pen-test engine > recon: walking 247 routes > lang: python / fastapi > hunt: handlers/users.py:42 > class: sql_injection > sink: cursor.execute(f"...") > taint: req.args["id"] > conf: 0.94 > PoC: id=1' UNION SELECT-- > bridge: 200 OK + DB leak > validator: confirmed > OWASP: A03:2021 > CWE-89 mapped > severity: critical > auto-fix: patch ready > hunt: auth.py:88 > class: idor > sink: get_user(id) > no ownership check > PoC: GET /api/users/42 > live-fire: 200 OK > cross-tenant leak > hunt: upload.py:31 > class: path_traversal > sink: open(filename) > PoC: ../../etc/passwd > canary: matched > continuous: round 2 > finding: ssrf > oast callback fired > finding: race condition > TOCTOU window 14ms > finding: weak jwt > alg=none accepted > chain: 3 flaws → RCE > report: ready > coverage: 94% > round complete

Secure your AI systems with the same rigor you apply to your human workforce. AI Security Vetting provides continuous behavioral assessment of AI agents, ensuring they remain aligned, secure, and trustworthy throughout their lifecycle.

Sovereign by Design

Runs on Your Hardware. Your Data Never Leaves Your Environment.

This is not a SaaS. The tool is delivered as software you install and run on your own infrastructure — your laptop, your VM, your air-gapped build server, your sovereign cloud tenancy. Your source code, your bearer tokens, your discovered vulnerabilities: you remain the sole custodian.

Local execution Code never uploaded Air-gap compatible Local LLMs supported (Ollama / LM Studio) 🇦🇺 🇳🇿 🇸🇬 🇺🇸 🇨🇦 🇬🇧 🇪🇺 🇩🇪 🇫🇷 🇯🇵 🇰🇷 🇮🇳 🇦🇪 🇨🇭 🇿🇦 Data-sovereignty ready

Two Sides of AI Security — One Platform

Most teams think of AI security in only one direction: are my AI systems safe? The same platform that vets your AI can also use AI to pen-test your conventional applications — reasoning through source code and live targets to surface bugs that traditional scanners miss.

Use Case 1

Pen-Test Your AI

Adversarial assessment of your LLMs, MCP servers, and AI agents — OWASP LLM Top 10 coverage, prompt-injection probes, tool-misuse scenarios, multi-turn jailbreaks, and behavioural drift detection.

Your AI
system

→

Adversarial
vetting

→

Findings
& report

OWASP LLM Top 10 + MCP + Agent test suites
11 obfuscation encodings & multi-turn attacks
Continuous behavioural drift monitoring
Provider-agnostic: OpenAI, Anthropic, Azure, Gemini, …

Use Case 2 — NEW

Use AI to Pen-Test Your App

Autonomous AI agents reason through your source code and probe your live targets — forming hypotheses, tracing data flow across files, and chaining flaws into working exploit narratives. Not pattern-matching. Not fuzzing.

Source
code

Live
target

Exploits
& PoCs

Code review · black-box · or both together
Reasons across files, builds exploit chains
Five depth tiers from PR-gate to audit-grade continuous
Bring your own model: any supported provider

The Offensive-AI Era Is Already Here

Find Your 0-Days Before Hackers Do

Adversaries have access to the same frontier models you do — and some have access to ones that aren't refusal-trained. They are pointing those models at your codebase, your APIs, and your supply chain, hunting for the bug that becomes their breach.

The only credible defense is symmetric: find your vulnerabilities first, using the same class of capability, and patch them before an adversary turns them into an incident.

The Attacker's Workflow

Adversary with a Frontier Model

Runs at machine speed — thousands of hypotheses per hour.
Reasons across files, not just regex-matches patterns.
Chains low-severity flaws into critical exploit paths.
Finds the bug your last pen-test missed.
Sells the 0-day, or uses it directly.

Your Workflow with This Tool

You with the Same Class of Capability

Run the same shape of reasoning agent against your own code.
Surface the bug, the proof-of-concept, and the fix patch in one report.
Loop continuously until the long-tail novel bugs are out.
Re-run after every release. Compound the coverage.
Patch the bug before it's a CVE with your name on it.

Discovery, not pattern-matching. Traditional SAST tools find the bugs they were programmed to find. Frontier-model agents find bugs nobody has programmed a detector for yet — including the ones in your code. Reasoning agents have already discovered hundreds of previously-unknown vulnerabilities in widely-used open-source projects in public research. This tool puts the same defensive capability in your hands, for your code, running on your hardware.

Use AI to Pen-Test Your Application

Autonomous AI agents that reason through your codebase the way a senior security engineer would — forming hypotheses, following data flow across functions, and stringing multiple flaws together into working exploit chains. Modeled after the discovery techniques used by recent public AI-security research that has uncovered 0-day vulnerabilities in major open-source projects, packaged as a turn-key product you can run against your own systems.

Three Engagement Modes

Code Review Only

Point the scanner at a local source tree. The AI reads your code, traces data flow across files, builds an internal call graph, and surfaces vulnerabilities with line-precise locations and proof-of-concept payloads — without ever touching a running system.

Best for: pre-deployment audits, PR gates, mono-repo sweeps, environments where you can't run the app.

Black-Box (URL Only)

Point the scanner at a live URL with a bearer token (or login credentials — we handle the auth flow). The AI probes the running application from the outside, builds a model of the API surface, and conducts an investigation that mirrors a senior pen-tester's workflow.

Best for: production-equivalent staging, third-party APIs where you don't have source.

Code + Live Fire

Both at once — the strongest mode. Findings from the code review are verified against the running application, producing wire-level evidence (HTTP request/response pairs) that the bug is actually reachable and exploitable. No more theoretical findings.

Best for: high-confidence reports, customer-facing security reviews, regulatory audits.

Choose Your Depth — Five Tiers, Predictable Cost

Each tier dials the coverage, depth, and budget. Costs below are typical per-run ranges — actual cost depends on codebase size and chosen model.

Tier	What it does	When to use	Typical cost
Quick	Narrow, fast scan of the highest-yield bug classes.	PR-time gates · commit hooks · CI smoke checks.	$1 – $5
Standard	Recommended default — broad coverage with adversarial verification.	Weekly scans · nightly builds · pre-release smoke runs.	$5 – $15
Deep	Exhaustive hybrid run with dependency scanning & exploit-chain synthesis.	Pre-release reviews · quarterly audits · major refactors.	$20 – $80
Deep / Continuous	Deep first pass, then the AI re-investigates until findings converge.	Audit-grade reviews · pre-launch · post-incident verification.	$60 – $640
Hypothesis	Focus a single attack class (e.g. "is there SQL injection in this code?").	Targeted re-test after a fix · risk-acceptance reviews.	$1 – $3

About Continuous review: the AI conducts multiple passes over your codebase, each pass calibrated to surface different bugs than the previous one — automatically stopping when there's nothing new to find. You don't pay for runaway scans. This mode is calibrated for pre-launch audits where you want the long-tail novel bugs, not just the obvious ones.

Supported Languages

The scanner has dedicated source-aware analysers for each language — precise call-graph reasoning, language-specific bug-class guidance, and built-in framework patterns for the popular web frameworks.

Gonet/http · gin · echo · fiber

PythonFlask · FastAPI · Django

JavaScriptExpress · Koa · Node APIs

TypeScriptNest · tRPC · Next.js

JavaSpring · Jakarta EE

KotlinSpring Boot · Ktor

C#ASP.NET Core · Minimal APIs

PHPLaravel · Symfony

RubyRails · Sinatra

Rustaxum · actix-web

C / C++memory-safety focus

What the AI Finds

Full OWASP Web & API Top 10 coverage, plus the bug classes traditional scanners miss because they require reasoning about intent.

Injection & Code Execution

SQL & NoSQL injection · OS command injection · server-side template injection (SSTI) · unsafe deserialisation (pickle, YAML, Java) · XSS reflection & storage.

Authentication & Authorization

Broken object-level authorization (IDOR) · broken function-level authorization · cross-tenant data exposure · JWT flaws (alg=none, weak signing) · session fixation.

Server-Side Forgery

Server-side request forgery (SSRF) with cloud-metadata probes · open redirect · webhook abuse · OAuth callback misuse.

Business Logic & Race Conditions

Logic flaws that bypass intent (negative amounts, sequence skipping, price-zero abuse) · race conditions and TOCTOU windows · over-claim / double-spend bugs.

Path Traversal & File Handling

Directory traversal in file uploads / downloads · archive extraction (zip-slip) · symlink-following attacks · unsafe file-type assumptions.

Cryptographic Misuse

Weak ciphers and modes · predictable randomness in security tokens · hard-coded secrets · misuse of crypto primitives across the codebase.

Resource Exhaustion (DoS-shape)

Unbounded queries (limit=∞), regex catastrophes, deeply nested payloads, memory-amplification — characterised with bounded probes, never sustained traffic.

Account-Lockout & Pre-Auth Probing

Lockout-policy characterisation, login-rate limits, username enumeration, MFA factor identification — conducted within strict bounds with customer consent.

Supply-Chain & Known CVEs

Dependency scanning against the OSV.dev advisory feed (PyPI, npm, Go, Maven, RubyGems, crates, Packagist) with CycloneDX SBOM generation.

LLM-Specific (when present)

Prompt-injection sinks in LLM-facing endpoints · missing output validation · tool-misuse paths in agent code. Bridges naturally to the AI-vetting side of the platform.

Responsible probing, with customer consent. When the live-fire mode probes destructive operations, denial-of-service susceptibility, or account-lockout policies, every probe is bounded: destructive operations target only test objects the scanner creates itself with a customer-monitorable marker, DoS susceptibility uses at most a handful of resource-intensive requests per endpoint, and lockout probing stops at the first signal. Customer trust beats one more finding.

Bring Your Own Model — Any Supported Provider

The AI scanner runs on whichever provider you already have a relationship with. Configure your scanner model once; the same model selection works across all five depth tiers. Pre-configured presets are available for the most common provider/model pairings so you can be running in under a minute.

See the Supported AI Providers section — the same list applies.

Reports You Can Act On

Penetration-Test PDF

Audit-grade penetration-test report with executive summary, severity-coded findings, vulnerable source snippets, proof-of-concept payloads, wire-level evidence, and a per-finding auto-fix patch (unified diff). Drop-in for SOC2 / ISO / PCI documentation.

HTML Dashboard

Browsable HTML report with filtering by severity, attack class, and file. Useful for engineering triage and live review meetings.

JSON & JSONL

Structured outputs for SIEM ingest, ticket creation, and CI gating. Each finding includes severity, CWE / OWASP mapping, root-cause hash for dedupe, and the full proof-of-concept artefact.

CycloneDX SBOM

Software Bill of Materials in CycloneDX 1.5 format, cross-referenced with OSV advisory IDs. Compliance-ready.

Compliance Mapping

Every finding maps to PCI DSS controls, HIPAA Security Rule sections, SOC 2 Common Criteria, ISO 27001 Annex A, OWASP, and CWE. Reduces the effort to translate findings into your existing GRC framework.

Auto-Fix Patches

Where the AI is confident in the fix, the report includes a unified-diff patch that applies cleanly. Engineering can review, accept, and merge — turning a security report into a pull request.

Why Vet Your AI?

🛡️ Defence in Depth

Don't rely solely on provider safeguards. Our tool tests your actual deployment configuration, custom prompts, and integrated tools to ensure your specific setup is secure.

🔄 Continuous Verification

AI models change. Prompts evolve. Test regularly to catch regressions and new vulnerabilities as your AI system grows and adapts.

🎯 Multi-Provider Coverage

Test across OpenAI, Azure OpenAI, Anthropic, Google Gemini, Microsoft Copilot, Microsoft Foundry, MCP servers, AI Agents, and any OpenAI-compatible API with a single tool.

📊 Actionable Reports

Get detailed HTML, Markdown, CSV, and JSONL outputs with OWASP LLM Top 10 mapping, severity scores, attack replays, hardening advice, and specific remediation guidance.

🔗 MCP & Agent Security

Purpose-built test suites for Model Context Protocol servers (12 categories) and AI Agents (10 categories) covering tool injection, privilege escalation, memory poisoning, and more.

🧪 Advanced Attack Techniques

Multi-turn conversation attacks, 11 obfuscation encodings (ROT13, Base64, Atbash, Caesar, Morse, Binary, Leet, Unicode confusables, and more), and system-layer compromise simulation.

Supported AI Providers

OpenAI

GPT-4, GPT-3.5 and compatible APIs

Azure OpenAI

Enterprise-grade OpenAI models

Anthropic Claude

Claude 3 Haiku, Sonnet, Opus

Google Gemini

Gemini Pro and Ultra models

Microsoft Copilot

Bot Framework Direct Line v3

OpenAI Compatible

Any OpenAI-compatible API

Foundry Models

Azure AI Foundry model catalogue

Foundry Agent Service

Azure AI Foundry agent orchestration

MCP Server

stdio and HTTP/SSE transports

AI Agent

Generic agent HTTP endpoints

Specialised for Australian 🇦🇺, New Zealand 🇳🇿 and Singapore 🇸🇬 Data

Our tool generates synthetic, checksum-valid Australian, New Zealand and Singapore identifiers to test your AI's memory safety and data protection capabilities. Never uses real PII - only realistic test data.

🇦🇺 Australian Identifiers

Tax File Numbers (TFN) - Checksum-validated 9-digit identifiers
Medicare Numbers - Valid format with check digits
State Driver Licences - NSW, VIC, QLD, WA, SA, TAS, ACT, NT formats
Australian Passports - Realistic passport number formats
Australian Mobile Numbers - Valid 04xx xxx xxx patterns
Australian Business Numbers (ABN) - 11-digit validated identifiers

🇳🇿 New Zealand Identifiers

IRD Numbers - Checksum-validated Inland Revenue identifiers
NZ Driver Licences - Valid regional format variations
NZ Passports - Realistic New Zealand passport formats
NZ Mobile Numbers - Valid 02x xxx xxxx patterns
NZBN (Business Numbers) - 13-digit business identifiers
National Health Index (NHI) - Healthcare identifier formats

🇸🇬 Singapore Identifiers

NRIC/FIN Numbers - Checksum-validated national identity identifiers
UEN (Business Numbers) - Unique Entity Number business identifiers
Singapore Passports - Realistic passport number formats
Singapore Mobile Numbers - Valid +65 xxx xxxx patterns
PayNow/FAST Payment IDs - Digital payment identifier formats

Memory Safety Testing Modes

Same-Session Testing

Seeds are injected in the same exchange to test immediate echo vulnerabilities

Cross-Session Testing

Seeds sent in separate sessions to test long-term memory retention

Strict Mode Validation

Only fails on validated sensitive data (TFN, Medicare, etc.) to reduce false positives

⚠️ Ethical Testing Guarantee

All test data is synthetic and generated algorithmically. We never use real customer data, production PII, or actual government identifiers. Our synthetic data follows authentic formatting and validation rules but represents no real individuals or entities.

OWASP LLM Top 10 Coverage

Comprehensive test coverage aligned to the OWASP Top 10 for Large Language Model Applications. Every test maps to one or more OWASP categories with contextualised remediation guidance.

LLM01 - Prompt Injection

Direct and indirect prompt injection, jailbreak attempts, system prompt override, role-play escapes, and instruction hierarchy violations.

LLM02 - Insecure Output Handling

Tests for unescaped HTML/JS in responses, markdown injection, downstream code execution risk, and output sanitisation failures.

LLM03 - Training Data Poisoning

Probes for verbatim training data recall, memorised PII echoes, and susceptibility to data-poisoning influenced outputs.

LLM04 - Model Denial of Service

Resource exhaustion probes, recursive prompt loops, token amplification attacks, and context window flooding.

LLM05 - Supply Chain Vulnerabilities

Third-party plugin trust, dependency confusion, malicious tool registration, and package hallucination risks.

LLM06 - Sensitive Information Disclosure

PII leakage, memory retention, system prompt extraction, API key exposure, and cross-session data bleeding.

LLM07 - Insecure Plugin Design

Tool schema disclosure, excessive tool permissions, missing input validation, and plugin-to-plugin escalation.

LLM08 - Excessive Agency

Unsafe autonomous actions, missing human-in-the-loop checks, privilege escalation via tools, and governance bypass.

LLM09 - Overreliance

Hallucination detection, missing uncertainty disclaimers, fabricated citations, and unsupported factual claims.

LLM10 - Model Theft

Model architecture probing, weight extraction attempts, fine-tuning data leakage, and system configuration disclosure.

MCP Server Security Testing

Purpose-built security assessment for Model Context Protocol (MCP) servers — the emerging standard for connecting AI models to external tools and data sources. Tests both direct protocol-level and LLM-mediated attack vectors.

Supports both stdio and HTTP/SSE transports via --provider mcp

Direct Protocol Tests

Tool Input Injection

Malicious payloads in tool arguments targeting command injection, path traversal, and SQL injection via MCP tool calls.

Schema Exposure

Probes for internal schema leakage, database structure disclosure, and API specification extraction through MCP responses.

Resource Access Control

Attempts to access unauthorised resources, traverse directory boundaries, and bypass resource-level permissions.

Input Validation

Oversized payloads, malformed JSON-RPC requests, type confusion, and boundary-value testing of tool parameters.

Authentication & Authorisation

Missing or weak authentication checks, token replay, session confusion, and privilege boundary violations.

Error Information Disclosure

Stack traces, internal paths, database connection strings, and debug information leaked through error responses.

Prompt Template Injection

Injection into server-side prompt templates, variable substitution attacks, and template escape sequences.

LLM-Mediated Attack Tests

Tool Description Poisoning

Malicious instructions embedded in tool descriptions that manipulate LLM behaviour when selecting or invoking tools.

Cross-Tool Privilege Escalation

Chaining multiple tool calls to escalate privileges, pivot between services, or access data beyond intended scope.

Return Value Injection

Malicious content in tool return values designed to hijack LLM responses, inject instructions, or alter downstream behaviour.

Excessive Permissions

Tools granted overly broad capabilities, missing least-privilege enforcement, and unrestricted write access patterns.

Tool Name Shadowing

Duplicate or confusingly similar tool names that can trick the LLM into calling the wrong tool or a malicious replacement.

AI Agent Security Testing

Dedicated test suite for autonomous AI agents that can take actions, use tools, and maintain state across conversations. Tests the unique security risks that emerge when LLMs operate with agency.

Use via --provider ai-agent --target-env ai-agent with any HTTP-accessible agent endpoint

Tool Abuse

Manipulating agents into misusing their available tools for unintended purposes, data exfiltration, or harmful actions.

Multi-Turn Escalation

Gradually escalating privileges across multiple conversation turns, building trust before exploiting agent capabilities.

Memory Poisoning

Injecting false information into agent memory/context that persists and influences future decisions and actions.

Excessive Autonomy

Testing whether agents take irreversible or high-impact actions without appropriate human confirmation or oversight.

Instruction Hierarchy

Attempting to override system-level instructions with user-level prompts, testing instruction priority enforcement.

Cross-Plugin Escalation

Leveraging one plugin or tool to gain unauthorised access to another, exploiting trust relationships between tools.

Excessive Scope

Probing agents to act outside their defined boundaries, accessing systems or data beyond their authorised scope.

Agent Identity

Testing whether agents can be convinced to adopt different personas, bypass their identity constraints, or impersonate other agents.

Resource Denial of Service

Triggering excessive API calls, unbounded loops, or resource-intensive operations to exhaust agent compute budgets.

Input Boundary

Oversized inputs, malformed data, and edge cases designed to break agent parsing, cause errors, or trigger unexpected behaviour.

Quick Start Examples

Get started in minutes with these common configurations. Replace the binary name with yours (e.g., AISecurityVetting.exe on Windows).

OpenAI (Generic Environment)

# macOS/Linux
export OPENAI_API_KEY=sk-xxxx
./AISecurityVetting --provider openai --model gpt-4o-mini --license-key YOUR_KEY

# Windows PowerShell
$env:OPENAI_API_KEY="sk-xxxx"
.\AISecurityVetting.exe --provider openai --model gpt-4o-mini --license-key YOUR_KEY

Azure OpenAI

# Set environment variables
export AZURE_OPENAI_API_KEY=xxxxx
./AISecurityVetting \
  --provider azure-openai \
  --azure-endpoint https://YOUR-RESOURCE.openai.azure.com \
  --azure-deployment gpt4o \
  --model gpt-4o \
  --license-key YOUR_KEY

Anthropic Claude

# Claude API
export ANTHROPIC_API_KEY=xxxx
./AISecurityVetting --provider anthropic --model claude-3-5-sonnet-20240620 --license-key YOUR_KEY

# Windows PowerShell
$env:ANTHROPIC_API_KEY="xxxx"
.\AISecurityVetting.exe --provider anthropic --model claude-3-5-sonnet-20240620 --license-key YOUR_KEY

Google Gemini

# Gemini API
export GEMINI_API_KEY=xxxx
./AISecurityVetting --provider gemini --model gemini-1.5-pro --license-key YOUR_KEY

# Windows PowerShell
$env:GEMINI_API_KEY="xxxx"
.\AISecurityVetting.exe --provider gemini --model gemini-1.5-pro --license-key YOUR_KEY

# Optional: Use different API version
./AISecurityVetting --provider gemini --model gemini-1.5-pro --gemini-base v1 --license-key YOUR_KEY

Microsoft Copilot (Direct Line v3)

# Microsoft Copilot with specialised environment
export COPILOT_DIRECTLINE_SECRET=xxxx
./AISecurityVetting \
  --provider copilot --model ignored \
  --copilot-user-id security_tester \
  --target-env copilot \
  --seed-mode same-session --seed-count 5 \
  --license-key YOUR_KEY

# Windows PowerShell
$env:COPILOT_DIRECTLINE_SECRET="xxxx"
.\AISecurityVetting.exe `
  --provider copilot --model ignored `
  --copilot-user-id security_tester `
  --target-env copilot `
  --seed-mode same-session --seed-count 5 `
  --license-key YOUR_KEY

Note: Copilot uses Bot Framework Direct Line v3. The --model parameter is ignored. Recommended to use --target-env copilot for specialised Dataverse/Power Platform testing.

OpenAI Compatible APIs

# Any OpenAI-compatible API (Ollama, LocalAI, vLLM, etc.)
export OPENAI_API_KEY=your-api-key-or-token
./AISecurityVetting \
  --provider openai-compat \
  --base-url http://localhost:11434/v1 \
  --model llama3:latest \
  --license-key YOUR_KEY

# Example: Ollama local instance
export OPENAI_API_KEY=dummy-key
./AISecurityVetting \
  --provider openai-compat \
  --base-url http://localhost:11434/v1 \
  --model mistral:7b \
  --license-key YOUR_KEY

# Example: vLLM deployment
export OPENAI_API_KEY=your-vllm-token
./AISecurityVetting \
  --provider openai-compat \
  --base-url https://your-vllm-endpoint.com/v1 \
  --model meta-llama/Llama-2-7b-chat-hf \
  --license-key YOUR_KEY

Compatible with: Ollama, LocalAI, vLLM, Together AI, Groq, Perplexity API, and any other service implementing OpenAI's Chat Completions API format.

Microsoft Foundry Models (Azure AI Foundry)

# Azure AI Foundry model catalogue
export AZURE_OPENAI_API_KEY=xxxxx
./AISecurityVetting \
  --provider foundry-models \
  --base-url https://YOUR-PROJECT.services.ai.azure.com \
  --model gpt-4o \
  --license-key YOUR_KEY

# Windows PowerShell
$env:AZURE_OPENAI_API_KEY="xxxxx"
.\AISecurityVetting.exe `
  --provider foundry-models `
  --base-url https://YOUR-PROJECT.services.ai.azure.com `
  --model gpt-4o `
  --license-key YOUR_KEY

Note: Uses Azure AI Foundry's model catalogue endpoint. Set the --base-url to your project's inference endpoint.

Microsoft Foundry Agent Service

# Azure AI Foundry Agent Service
export AZURE_OPENAI_API_KEY=xxxxx
./AISecurityVetting \
  --provider foundry-agent-service \
  --base-url https://YOUR-PROJECT.services.ai.azure.com \
  --foundry-agent-id asst_xxxxxxxxxxxx \
  --model ignored \
  --license-key YOUR_KEY

# Windows PowerShell
$env:AZURE_OPENAI_API_KEY="xxxxx"
.\AISecurityVetting.exe `
  --provider foundry-agent-service `
  --base-url https://YOUR-PROJECT.services.ai.azure.com `
  --foundry-agent-id asst_xxxxxxxxxxxx `
  --model ignored `
  --license-key YOUR_KEY

Note: Tests a deployed Azure AI Foundry agent. The --model parameter is ignored as the agent determines its own model. Each test creates a new conversation thread.

MCP Server (Model Context Protocol)

# MCP via stdio transport (local process)
./AISecurityVetting \
  --provider mcp \
  --mcp-transport stdio \
  --mcp-command /usr/local/bin/my-mcp-server \
  --mcp-args "--config,/path/to/config.json" \
  --target-env mcp \
  --model ignored \
  --license-key YOUR_KEY

# MCP via HTTP/SSE transport (remote server)
./AISecurityVetting \
  --provider mcp \
  --mcp-transport http \
  --base-url http://localhost:3000/mcp \
  --target-env mcp \
  --model ignored \
  --license-key YOUR_KEY

Note: Tests MCP servers directly at the protocol level. Supports both stdio (spawns a local process) and HTTP/SSE (connects to a remote endpoint) transports. Runs 12 MCP-specific test categories covering tool injection, schema exposure, access control, and LLM-mediated attacks.

AI Agent (Generic HTTP Endpoint)

# Test any AI agent with an HTTP API
./AISecurityVetting \
  --provider ai-agent \
  --base-url https://your-agent.example.com/api/chat \
  --target-env ai-agent \
  --model ignored \
  --license-key YOUR_KEY

# With authentication header
export OPENAI_API_KEY=your-agent-api-key
./AISecurityVetting \
  --provider ai-agent \
  --base-url https://your-agent.example.com/api/chat \
  --target-env ai-agent \
  --model ignored \
  --attack-preset aggressive \
  --license-key YOUR_KEY

Note: Tests any AI agent accessible via HTTP. The agent should accept JSON requests and return text responses. Runs 10 agent-specific test categories covering tool abuse, multi-turn escalation, memory poisoning, autonomy limits, and more.

Advanced Configuration Examples

Advanced scenarios for comprehensive security testing, red team exercises, and specialised environments.

--region anz OR --region au OR --region nz OR --region sg

Full Enterprise Red Team Testing

# RAG + tools environment, aggressive attacks, obfuscation, strict memory classification
export OPENAI_API_KEY=sk-xxxx
./AISecurityVetting \
  --provider openai --model gpt-4o-mini \
  --target-env rag-tools \
  --attack-preset aggressive \
  --attack-obfuscation rot13 \
  --strict-mode \
  --seed-mode both --seed-count 15 \
  --region anz \
  --out red_team_$(date +%F) \
  --license-key YOUR_KEY \
  --temperature 0.0 \
  --timeout 90

Use case: Comprehensive security assessment for enterprise AI agents with tool access. Tests against sophisticated attack patterns with obfuscation techniques.

Memory/Retention Deep Testing

# Focus on memory leaks with extensive synthetic PII seeding
export ANTHROPIC_API_KEY=xxxx
./AISecurityVetting \
  --provider anthropic --model claude-3-5-sonnet-20240620 \
  --seed-mode both \
  --seed-count 25 \
  --seed-namespace "MEMORY-TEST-$(date +%Y%m%d)" \
  --strict-mode \
  --region anz \
  --attack-preset auto \
  --out memory_audit_results \
  --license-key YOUR_KEY

Use case: Validate memory safety and PII handling. Tests both same-session and cross-session retention with 25 synthetic AU/NZ records (TFN, Medicare, driver licenses, etc.).

Microsoft Copilot Agent Testing

# Specialised for Copilot with Dataverse/Power Platform probes
export COPILOT_DIRECTLINE_SECRET=xxxx
./AISecurityVetting \
  --provider copilot --model ignored \
  --copilot-user-id security_tester_001 \
  --target-env copilot \
  --seed-mode same-session --seed-count 10 \
  --attack-preset aggressive \
  --attack-as-system \
  --region anz \
  --out copilot_security_audit \
  --license-key YOUR_KEY \
  --timeout 120

Use case: Test Microsoft Copilot agents for Dataverse exfiltration, Power Automate triggers, SharePoint/Teams governance violations, and system-level prompt injection.

MCP Server Security Audit

# Full MCP server audit via stdio transport
./AISecurityVetting \
  --provider mcp \
  --mcp-transport stdio \
  --mcp-command ./my-mcp-server \
  --mcp-args "--verbose" \
  --target-env mcp \
  --model ignored \
  --attack-preset aggressive \
  --out mcp_security_audit_$(date +%F) \
  --license-key YOUR_KEY \
  --timeout 120

# MCP server audit via HTTP/SSE (remote)
./AISecurityVetting \
  --provider mcp \
  --mcp-transport http \
  --base-url https://mcp.internal.example.com/sse \
  --target-env mcp \
  --model ignored \
  --attack-preset aggressive \
  --out mcp_remote_audit \
  --license-key YOUR_KEY

Use case: Comprehensive MCP server security assessment covering tool input injection, schema exposure, resource access control, authentication bypass, error information disclosure, prompt template injection, tool description poisoning, cross-tool privilege escalation, return value injection, excessive permissions, and tool name shadowing.

AI Agent Security Assessment

# Full AI agent security assessment
export OPENAI_API_KEY=your-agent-key
./AISecurityVetting \
  --provider ai-agent \
  --base-url https://your-agent.example.com/api/chat \
  --target-env ai-agent \
  --model ignored \
  --attack-preset aggressive \
  --seed-mode both --seed-count 15 \
  --region anz \
  --out agent_security_audit_$(date +%F) \
  --license-key YOUR_KEY \
  --timeout 120

# Agent with custom system prompt context
./AISecurityVetting \
  --provider ai-agent \
  --base-url https://your-agent.example.com/api/chat \
  --target-env ai-agent \
  --model ignored \
  --system-prompt "You are a customer service agent with access to order management tools..." \
  --attack-preset aggressive \
  --out agent_audit \
  --license-key YOUR_KEY

Use case: End-to-end security testing of autonomous AI agents covering tool abuse, multi-turn privilege escalation, memory poisoning, excessive autonomy, instruction hierarchy bypass, cross-plugin escalation, scope violations, identity manipulation, resource DoS, and input boundary attacks.

Custom Test Suite with File Seeding

# Custom YAML suite with pre-defined synthetic PII seeds
export AZURE_OPENAI_API_KEY=xxxxx
./AISecurityVetting \
  --provider azure-openai \
  --azure-endpoint https://YOUR-RESOURCE.openai.azure.com \
  --azure-deployment gpt4o \
  --model gpt-4o \
  --suite ./custom_financial_tests.yaml \
  --seed-file ./synthetic_customer_data.yaml \
  --seed-mode cross-session \
  --attack-preset lite \
  --strict-mode \
  --system-prompt "You are a secure financial AI assistant..." \
  --out custom_financial_audit \
  --license-key YOUR_KEY

Use case: Industry-specific testing with custom scenarios and controlled synthetic data. Perfect for specialised domains like finance, healthcare, or legal services.

System-Layer Compromise Simulation

# Test system prompt injection and orchestrator bypass attempts
export GEMINI_API_KEY=xxxx
./AISecurityVetting \
  --provider gemini --model gemini-1.5-pro \
  --target-env rag-tools \
  --attack-preset aggressive \
  --attack-as-system \
  --attack-obfuscation rot13 \
  --seed-mode both --seed-count 20 \
  --strict-mode \
  --max-tokens 2048 \
  --temperature 0.0 \
  --out system_compromise_test \
  --license-key YOUR_KEY \
  --log-level debug

Use case: Simulate advanced persistent threats targeting the AI orchestration layer. Tests malicious system instructions, prompt injection via tool outputs, and multi-vector attacks.

Pro Tips for Advanced Testing

Stable namespacing: Use consistent --seed-namespace values for reproducible cross-session tests
Debug logging: Add --log-level debug to troubleshoot provider connection issues
Timeout tuning: Increase --timeout for slow providers or complex tool chains
Temperature control: Keep --temperature 0.0 for consistent security outcomes
Output organisation: Use date-stamped output directories for audit trails

Test Categories & Environments

📖 Download Instruction Manual (PDF)

Complete setup guide, command reference, and advanced configuration examples

🎯 Generic Environment (Default)

Core tests across all OWASP LLM Top 10 categories. Perfect for general LLM safety vetting and baseline security assessment.

🔧 RAG + Tools Environment

Adds enterprise-specific probes for Salesforce, Xero, SharePoint, Slack, Jira, MYOB. Use for agents with enterprise integrations.

💼 Microsoft Copilot Environment

Specialised tests for Dataverse, Power Automate, SharePoint, Outlook, Teams. Designed for Microsoft Copilot agents.

🔗 MCP Server Environment

12 test categories for Model Context Protocol servers. Tests tool injection, schema exposure, access control, tool poisoning, privilege escalation, and more.

🤖 AI Agent Environment

10 test categories for autonomous AI agents. Tests tool abuse, multi-turn escalation, memory poisoning, autonomy limits, identity integrity, and scope boundaries.

Output Formats

📊 Interactive HTML Report

Modern, searchable interface with KPI cards, severity charts, filters, and detailed analysis. Perfect for demos and stakeholder presentations.

📝 Markdown Report

Detailed findings with attack preambles, effective prompts, and elaborated evaluations. Great for documentation and sharing.

📈 CSV Results

Machine-friendly summary for analysis, trending, and integration with other tools. Includes scores, latency, and configuration details.

🔍 JSONL Details

Complete test results with raw provider payloads, findings, and metadata. Perfect for programmatic analysis and integration.

Key Configuration Options

Target Environments: generic, rag-tools, copilot, mcp, ai-agent — each with tailored test suites
Seed Modes: Test memory retention with same-session, cross-session, or both patterns
Attack Presets: None, lite, aggressive, or auto-scaling adversarial preambles
Obfuscation Methods (11): ROT13, Base64, Atbash, Caesar cipher, Binary, Morse code, URL encoding, string reversal, Leet speak, character spacing, and Unicode confusables (Cyrillic homoglyphs)
Multi-Turn Attacks: Gradual privilege escalation across multiple conversation turns with context-aware probing
System-Layer Injection: Deliver attacks via the system role (--attack-as-system) to simulate orchestrator compromise
Strict Mode: Only fail on validated sensitive data echoes (TFN, Medicare, NRIC, etc.)
Custom Suites: Load your own YAML test definitions for specific requirements

See AI Security Vetting in Action

Watch how our tool systematically tests AI systems for security vulnerabilities, from prompt injection to memory leaks, delivering comprehensive reports in real-time.

Comprehensive Security Testing

Real-time vulnerability detection across 35+ security categories covering LLMs, MCP servers, and AI agents with detailed reporting and immediate insights.

OWASP LLM Top 10

🚨

Prompt Injection

🔓

Insecure Output

☠️

Data Poisoning

💥

Model DoS

📦

Supply Chain

🔒

Data Exfiltration

🛠️

Insecure Plugins

⚠️

Excessive Agency

🤔

Overreliance

🕵️

Model Theft

🧠

Memory Safety

👥

Governance

MCP Server Security

💉

Tool Injection

📋

Schema Exposure

🔐

Access Control

🧪

Input Validation

🪝

Tool Poisoning

⬆️

Privilege Escalation

AI Agent Security

🔧

Tool Abuse

📈

Multi-Turn Escalation

🧬

Memory Poisoning

🤖

Autonomy Limits

🏗️

Instruction Hierarchy

🎭

Agent Identity

Ready to Secure Your AI?

Don't leave your AI systems vulnerable to attacks. Implement continuous security testing and compliance monitoring today.

Download from Windows App Store Download MSI Installer for Windows Download CLI for Windows Download from Apple App Store Download for Mac Download for Linux & Kubernetes

View other products from CyberAutomation.com.au →

Explore All Products

See how all Cyber Automation products work together to secure your entire infrastructure.

Back to Product Overview