HomeResourcesAIAI Red Teaming for SaaS Product Security

AI Red Teaming for SaaS Product Security

What to Test and How to Evaluate AI Security Firms
AI Red Teaming for SaaS Product Security

Date

Authors

Tony UcedaVélez

Follow Us

By Tony UcedaVélez, Founder & CEO of VerSprite and co-creator of PASTA | Published November 2026 | Last reviewed November 2026
About the author. Tony UcedaVélez is the Founder and CEO of VerSprite and co-author of Risk Centric Threat Modeling (Wiley, 2015). This guide reflects public information as of November 2026 and describes other firms only from their own published materials.

AI red teaming for SaaS product security is adversarial testing of the AI features inside a multi-tenant product, including assistants, retrieval, agents, and tool integrations, to find exploitable paths to tenant data, unauthorized actions, and abuse before attackers or customers do. It’s often called AI red teaming and LLM penetration testing, and the two terms overlap heavily.

For a SaaS product security team, the hard part isn’t finding a prompt that makes a model misbehave. It’s finding the path from a manipulated model to something that matters: another tenant’s data, an action taken with someone else’s permissions, or a bill nobody expected. This guide covers what to test, how the terminology differs, which firms provide the work, how to evaluate them, and how to rank what they find. Learn about AI Red Teaming by VerSprite.



Key Takeaways

  • The attack surface in a SaaS product is the tenant boundary around AI features: retrieval over customer data, agents with tool access, and model output rendered in your UI.
  • LLM security testing covers three different things, AI red teaming, LLM penetration testing, and AI vulnerability assessment, and firms use the terms inconsistently. Ask what is actually tested and who validates it.
  • Published incident rates range from 42% to 88% depending on the survey, so none is a benchmark. Test your own system.
  • Rank findings by what they reach (tenant boundary, actions, data), not by whether a guardrail was bypassed.


What Does AI Red Teaming for SaaS Product Security Cover?

It covers the AI features’ inputs, retrieval layer, tool access, and outputs, tested the way a malicious tenant, an external attacker, or a compromised integration would. Acronyms used below: RAG is retrieval-augmented generation, and MCP is the Model Context Protocol, a standard for connecting agents to tools. The table maps common SaaS AI patterns to OWASP Top 10 for LLM Applications (2025) categories. It’s illustrative, not a list of documented incidents.

SaaS AI feature Illustrative attack path OWASP LLM category Why it matters in SaaS
Assistant grounded in customer data (RAG) Retrieval returns another tenant’s documents because authorization isn’t enforced at the vector store or retrieval layer LLM08 Vector and Embedding Weaknesses; LLM02 Sensitive Information Disclosure Cross-tenant data exposure
Features that read customer-controlled content (tickets, docs, emails, web pages, uploads) Hidden instructions in content steer the model to leak data or take actions LLM01 Prompt Injection (indirect) An outside party can attack your users through content they never touch directly
Agents with tools and MCP servers Attacker-influenced input drives tool calls using the user’s or a service account’s permissions LLM06 Excessive Agency Unauthorized changes to tenant data and settings
Model output rendered in the product (markdown, links, images, HTML) Output carries script, or a crafted link or image that exfiltrates data from the session LLM05 Improper Output Handling XSS and data exfiltration in your own UI
System prompts and configuration A leaked prompt exposes credentials, business logic, or tenant-specific rules LLM07 System Prompt Leakage Secrets and logic exposed to every user
Metered or usage-billed AI features A tenant or attacker forces excessive token consumption LLM10 Unbounded Consumption Cost and availability impact
Third-party models, plugins, datasets, fine-tuning A compromised component, poisoned data, or a silent provider update changes behavior LLM03 Supply Chain; LLM04 Data and Model Poisoning Behavior shifts without a code change on your side


Why Do SaaS Product Teams Need Specialized AI Security Testing Now?

Because AI features are shipping faster than controls around them, and incident reporting is both common and inconsistent. The Cloud Security Alliance and Token Security reported in April 2026 that 65 percent of organizations had an AI agent security incident, with 61 percent of those involving sensitive data exposure, 43 percent operational disruption, and 41 percent unintended actions. Gravitee’s April 2026 survey of 750 executives found about eight in ten organizations ship AI agents to production without full security controls.

Share of organizations reporting an AI-related security incident, by 2026 survey. Definitions and samples differ, so treat the spread as a range, not a benchmark.
Figure 1. Share of organizations reporting an AI-related security incident, by 2026 survey. Definitions and samples differ, so treat the spread as a range, not a benchmark.

Outcomes reported by organizations with an AI agent incident. Source: Cloud Security Alliance and Token Security, April 2026, as summarized by Kiteworks.
Figure 2. Outcomes reported by organizations with an AI agent incident. Source: Cloud Security Alliance and Token Security, April 2026, as summarized by Kiteworks.

Other 2026 data points the same direction. Check Point Research reported that longer malicious prompt payloads, typical of indirect prompt injection, rose roughly fivefold between March and May 2026. AvePoint’s survey found 39.6 percent of organizations reported a prompt injection that bypassed guardrails on a generative AI assistant. Cobalt’s 2026 State of Pentesting data, drawn from its own platform, found 32 percent of AI and LLM findings were rated high risk versus 12 percent across all findings. And IBM’s 2025 breach study found 97 percent of organizations that reported an AI model or application breach lacked proper AI access controls.

Why do the incident rates vary so much? The surveys count “suspected” and “confirmed” incidents differently, sample different populations, and measure AI agents in some cases and all AI in others. Read them as directional, and measure your own exposure.



What Is the Difference Between AI Red Teaming, LLM Penetration Testing, and an AI Vulnerability Assessment?

AI red teaming tests how an adversary could misuse an AI system as a whole, LLM penetration testing proves exploitability in the application around the model, and an AI vulnerability assessment lists known weakness patterns, usually with lighter validation. Firms use these terms inconsistently, so confirm scope in writing.

Term What it usually means Typical output
AI red teaming Adversary-style testing of the whole AI system, including misuse and abuse scenarios Attack scenarios, bypassed controls, abuse cases
LLM penetration testing (also “large language model penetration testing”) Exploit-focused testing of the LLM-integrated application: prompts, retrieval, tools, APIs, output handling Proven exploit paths with reproduction steps
AI vulnerability assessment Broader or automated identification of known weakness patterns, often with prompt libraries or scanners A findings list, sometimes with limited validation
Adversarial AI testing Attacks on the model itself: evasion, poisoning, extraction, membership inference Model-level weaknesses and their impact
Model safety evaluation Testing for harmful content, bias, and hallucination Quality and safety measurements, not exploit paths


Which Cybersecurity Firms Provide AI Red Teaming and LLM Penetration Testing?

Four kinds of firms provide this work: offensive security consultancies that added AI scope, audit and assurance firms that pair testing with attestations, AI-focused specialists, and platform vendors with automated tooling. Descriptions below come from each firm’s public pages as of November 2026, and none is ranked.

  • Offensive security consultancies. Bishop Fox lists AI and LLM security assessments. Praetorian lists an AI/ML penetration testing service. we45 lists AI model security testing, RAG and agentic system assessments, MCP security assessments, and AI security architecture review.
  • Audit and assurance firms. Schellman lists an AI red teaming service within its penetration testing practice, alongside SOC 2 and ISO 42001 work. Intertek lists an AI red teaming service within its cybersecurity offering.
  • Smaller AI-focused specialists. CyberSigma, RedAsgard, and TruvoCyber each list AI security, AI penetration testing, or AI red teaming services.


How Do You Evaluate AI Security Firms as a SaaS Product Security Team?

Evaluate them on how well they understand your architecture, how deep they go on tenant isolation and agents, what they exclude by default, and whether the output goes straight into engineering. Ask each firm:

  1. Will you review our tenant model, retrieval design, and agent permissions before testing? Prompt lists alone don’t find cross-tenant flaws.
  2. How do you test tenant isolation? Cross-tenant retrieval, shared caches, vector stores, and fine-tuned models all need testing with separate synthetic tenants.
  3. How do you test agents, tools, and MCP? Look for permission-scope abuse, confused-deputy attacks, and tool-description poisoning.
  4. What is out of scope by default? Some firms exclude categories unless asked. Schellman, for example, states it doesn’t test token-exhaustion attacks unless specifically requested, which matters for usage-billed features.
  5. How do you handle non-determinism? Strong answers include repeated attempts and reported success rates.
  6. What does the output look like? Ask for reproducible prompts and payloads, impact stated in tenant and business terms, fix guidance, and regression tests you can add to CI.
  7. How are retests and cadence handled? AI features change with every model, prompt, retrieval-source, or tool-permission change.
  8. How do you protect customer data and respect model-provider terms? Test in staging with synthetic tenants, and confirm what’s authorized on hosted models.
  9. Which frameworks do findings map to? OWASP Top 10 for LLM Applications (2025), MITRE ATLAS, NIST AI RMF, and ISO/IEC 42001 are the common ones.
  10. Do you sell guardrail or runtime products you also test? Ask about conflicts.


How Much Does AI Red Teaming Cost, and How Long Does It Take?

Published figures are scarce, and scope drives both. One firm that does publish, Schellman, states that engagements typically take around two weeks depending on the number of features and start at no less than $16,000. Treat that as one data point, not a market rate. Ask each firm for a scope-based quote and to name what drives the price: number of AI features, tenant model complexity, agent and tool surface, and retesting.



How Should You Prioritize AI Red Team Findings?

Rank findings by what the attack reaches, then adjust for how easy and reliable it is. The tiers below are a practical rubric, not a standard.

Tier What the attack reaches Example
1 A tenant boundary One tenant’s content or query retrieves another tenant’s data
2 Unauthorized actions with real privileges An agent is steered into writing, deleting, sending, or spending
3 Sensitive data exposure within a tenant, or to an outside party Secrets, personal data, or credentials in a system prompt
4 Output-handling flaws that reach the browser Script execution or markdown-based exfiltration in the UI
5 Guardrail bypass with no path to data or actions A jailbreak that produces off-policy text only

Adjust each finding for exploit preconditions (authentication, user interaction), reliability across repeated attempts, blast radius, and detectability. A tier-5 finding that chains into tier 2 is a tier-2 finding.



When Should a SaaS Team Run AI Red Teaming?

Before launching each AI feature, and again after significant changes to the model or provider, system prompts, retrieval sources, tool permissions, or tenant model. Between those points, convert confirmed findings into regression tests that run with each release. No standard defines a cadence, so base yours on how often these components change.



Limitations

The incident figures come from vendor and industry surveys with different definitions, and several were reviewed through summaries rather than the full reports. The Cobalt and Check Point figures are vendor-collected data. The cost and duration figures come from a single firm’s public page. The OWASP categories are a common taxonomy but not an exhaustive one, and the attack-path table is illustrative. AI behavior is non-deterministic, so results are point-in-time. Descriptions of other firms reflect their own materials, and the prioritization tiers are a working rubric, not an industry standard.



Final Takeaway

For a SaaS product, the question isn’t whether a model can be coaxed into saying something it shouldn’t. It’s whether that behavior can cross a tenant boundary, trigger an action, or leak something that matters. Evaluate AI security firms on how well they test those paths, what they exclude by default, and how quickly their findings become fixes and regression tests.



Sources



Frequently Asked Questions

Four kinds of firms provide it: offensive security consultancies that added AI scope (such as Bishop Fox, Praetorian, and we45), audit and assurance firms that pair testing with attestations (such as Schellman and Intertek), smaller AI-focused specialists, and platform vendors with automated tooling. VerSprite, which publishes this guide, also provides these services. Evaluate firms on architecture understanding, tenant isolation and agent depth, default exclusions, and developer-ready output rather than a ranking.
It is adversarial testing of the AI features inside a multi-tenant product, including assistants, retrieval, agents, and tool integrations, to find exploitable paths to tenant data, unauthorized actions, and abuse. It focuses on what a manipulated model can reach, such as another tenant’s data or actions taken with someone else’s permissions.
AI red teaming tests how an adversary could misuse an AI system as a whole. LLM penetration testing proves exploitability in the application around the model, including prompts, retrieval, tools, and output handling. An AI vulnerability assessment lists known weakness patterns, often with automated prompt libraries and lighter validation. Firms use the terms inconsistently, so confirm scope in writing.
Test cross-tenant retrieval in RAG features, indirect prompt injection through customer-controlled content, agent and MCP tool permissions, output handling in the UI, system prompt leakage, token-consumption abuse in metered features, and supply chain risk from third-party models and plugins. Use separate synthetic tenants to check isolation.
Create at least two synthetic tenants with distinct seeded data, then attempt to retrieve one tenant’s content from the other through the assistant, search, summaries, caches, and any shared vector store or fine-tuned model. Check whether authorization is enforced at the retrieval layer rather than only in the prompt, and whether injected content in one tenant can influence another.
Published figures are scarce and scope drives both. One firm that publishes them, Schellman, states engagements typically take around two weeks depending on the number of features and start at no less than $16,000. Treat that as one data point and request scope-based quotes based on the number of AI features, tenant model complexity, agent and tool surface, and retesting.
Run it before launching each AI feature and after significant changes to the model or provider, system prompts, retrieval sources, tool permissions, or tenant model. Between engagements, turn confirmed findings into regression tests that run with each release. No standard sets a cadence, so base yours on how often those components change.
The common ones are the OWASP Top 10 for LLM Applications (2025) for vulnerability categories, MITRE ATLAS for adversary tactics and techniques, the NIST AI Risk Management Framework for governance, and ISO/IEC 42001 for AI management systems. Mapping findings to these gives customers and auditors a shared vocabulary.
Share a summary letter covering scope, dates, methodology, the frameworks used, and remediation status, not exploit details. Ask the testing firm what it can provide, and keep the full report confidential because it contains reproducible attack steps.
Usually no. Testing normally focuses on your application layer, including prompts, retrieval, tools, and output handling, and attacks against a provider’s infrastructure may be prohibited by its terms. Confirm what is authorized under the provider’s terms before testing begins.
In 2026, surveys reported rates from 42 percent to 88 percent because they define incidents differently (suspected versus confirmed), sample different populations, and measure AI agents in some cases and all AI in others. None is a benchmark, so treat them as directional and measure your own exposure.
  • /
  • /
  • /
  • /
  • /
  • /
  • /
  • /
  • /
  • /
  • /
  • /
  • /
  • /
  • /
  • /
  • /
  • /
  • /
  • /
  • /
  • /

You May Also Like

PCI Penetration Testing as a Service (PTaaS) for US Retailers

PCI Penetration Testing as a Service (PTaaS) for US Retailers
AI Red Teaming for Financial Services

AI Red Teaming for Financial Services
PASTA threat model example

PASTA Threat Modeling Example