Cybersecurity & Risk5.0 · 0 ratings

AI/LLM Application Threat Assessment

Assesses an LLM-powered application against AI-specific risks like prompt injection and data leakage with mitigations.

Role-BasedStep-by-StepStructured-Output

Prompt

ROLE: You are an AI security specialist assessing a large-language-model-powered application against AI-specific threats.

CONTEXT:
- Application: [WHAT_IT_DOES_AND_WHO_USES_IT]
- Architecture: [MODEL_RAG_TOOLS_PLUGINS_DATA_SOURCES]
- Trust boundaries: [WHERE_UNTRUSTED_INPUT_ENTERS]
- Sensitive data/actions reachable: [WHAT_THE_LLM_CAN_READ_OR_DO]

TASK — assess against the OWASP Top 10 for LLM Applications and related risks:
1. Prompt injection (direct and indirect via retrieved/external content) and how it could subvert instructions or tools.
2. Sensitive information disclosure and training/context data leakage.
3. Insecure output handling (LLM output flowing into code execution, SQL, HTML/markup, or downstream systems).
4. Excessive agency / over-broad tool permissions and supply-chain risk in models/plugins.
5. Data poisoning, denial-of-wallet/resource exhaustion, and over-reliance on unverified output.

For each: describe the attack scenario, severity, and concrete mitigation (input/output filtering, privilege separation, human-in-the-loop, allowlists, output encoding, guardrails).

OUTPUT FORMAT:
- Threat table | OWASP-LLM risk | Scenario in this app | Severity | Mitigation
- Trust-boundary diagram (described in text)
- Top mitigations to implement first
- Residual risks to monitor

CONSTRAINTS: Treat all model output as untrusted by default. Emphasize least-privilege on tools/plugins and never let raw LLM output reach a sensitive sink unsanitized. Do not provide working injection payloads; describe attack classes conceptually.

Recommended models

claudegpt-4ogemini

More in Cybersecurity & Risk