Our AI Security Testing and AI Penetration Testing Approach
Kroll has developed an AI security testing methodology that aligns with the OWASP Top 10 for LLM applications.
The OWASP LLM Top 10 is a new standard that will also continue to evolve and mature with LLM security; it serves as a baseline for our coverage. Importantly, our approach goes beyond ensuring coverage of the OWASP LLM Top 10 categories to help clients identify and understand the risks presented by LLM systems in the context of their applications and business. Our AI penetration testing approach has the following components:
Dynamic LLM Testing
- Our consultants interact with the LLM using adversarial prompts to discover system behavior and identify vulnerabilities.
Cloud Configuration Review
- Cloud configuration reviews cover all cloud components in scope with additional validation LLM system and data components.
LLM Developer Survey
- Kroll clients a complete developer survey to provide key background information about the model, training data and process and system components. This helps our consultants to more accurately evaluate system components that are not accessible from a pentester perspective. This survey initiates a dialog between our consultants and clients, allowing us to maintain open communication at every stage of the AI security testing or AI penetration test, especially around LLM security issues.
Application Penetration Testing
- All LLM security assessments will be conducted as part of a web application penetration test. This provides a comprehensive assessment of the application and ensures that application vulnerabilities in non-LLM components do not impact LLM systems.
