Model Scanning Overview

AI Model Scans are automated tasks that allow you to test and monitor large language models (LLMs). These model scans are designed to detect vulnerabilities, enforce policy compliance, and uncover risks-without requiring manual effort.

• There are two types of AI Model Scans:
  • AI Event-Driven Model Scans
    Triggered in real time by specific events, such as API calls, findings, or user interactions. Ideal for responsive analysis and dynamic risk detection.
  • AI Scheduled Model Scans
    Run automatically at defined time intervals (e.g., every 15 minutes). Can be used for continuous or routine scanning.

Model Scanning Modes

Each AI Model Scan can be created in one of two modes:

• Custom Model Scans
  Define the model scan and connect your infrastructure (e.g., AWS Lambda or a webhook). Custom model scans give you complete control over the inputs and outputs of the analysis.
• Managed Model Scans
  FireTail provides out-of-the-box integration. These model scans use prebuilt probes to automatically test for issues like hallucinations, content injection, jailbreak attempts, and more.