Secure AI Deployment Checklist

Validate and sanitize all user inputs before passing them to the LLM

Why it matters: Unsanitized input is the primary vector for direct prompt injection attacks that can override system instructions.

Implement strict separation between system prompts and user-supplied content

Why it matters: Clear boundaries prevent attackers from escaping the user context and manipulating system-level instructions.

Apply output filtering to detect and block prompt injection attempts in responses

Why it matters: Even when injection succeeds, output filtering acts as a second line of defense to prevent harmful content from reaching users.

Use least-privilege access for LLM tool calls and API integrations

Why it matters: Limiting what the LLM can do reduces the blast radius if a prompt injection attack succeeds.

Scrub PII and secrets from training data and retrieval-augmented generation (RAG) sources

Why it matters: Data present in training or retrieval sources can be extracted through targeted queries.

Implement output guardrails that detect and redact sensitive data patterns (SSNs, API keys, etc.)

Why it matters: Automated redaction catches leaks that manual review would miss, especially at scale.

Apply role-based access controls to data sources connected to the LLM

Why it matters: Without access controls, the LLM may surface data that the requesting user should not be authorized to see.

Log and monitor LLM outputs for anomalous data disclosure patterns

Why it matters: Monitoring provides visibility into potential leaks and supports incident response and forensic analysis.

Vet and inventory all third-party models, plugins, and data sources

Why it matters: You cannot secure what you do not know about. An inventory is the foundation of supply chain risk management.

Verify model integrity with checksums and signatures before deployment

Why it matters: Tampered models can contain backdoors or biased behaviors that are invisible without integrity checks.

Pin model versions and maintain a rollback plan for third-party dependencies

Why it matters: Unpinned versions can silently introduce regressions or vulnerabilities through automatic updates.

Conduct security reviews of third-party plugin code and API integrations

Why it matters: Plugins execute with the LLM's privileges. A vulnerable plugin is a direct attack vector.

Validate training data provenance and apply data quality checks before ingestion

Why it matters: Poisoned training data is difficult to detect after the fact. Prevention at ingestion is far more effective.

Implement anomaly detection on training pipelines to catch data manipulation

Why it matters: Statistical anomalies in training data often signal poisoning attempts before they affect model behavior.

Use adversarial testing to probe for planted backdoors or biased outputs

Why it matters: Targeted red-teaming can surface hidden behaviors that standard evaluation benchmarks miss.

Restrict and audit access to training infrastructure and fine-tuning pipelines

Why it matters: Insider threats and compromised credentials are common vectors for model poisoning attacks.

Treat all LLM outputs as untrusted and apply context-appropriate encoding before rendering

Why it matters: LLM outputs can contain executable code or markup. Treating them as trusted input enables injection attacks.

Validate LLM-generated code, SQL, or API calls against an allowlist before execution

Why it matters: Unrestricted code execution from LLM output is equivalent to a remote code execution vulnerability.

Implement Content Security Policy (CSP) headers for web applications using LLM output

Why it matters: CSP provides defense-in-depth against XSS even when output sanitization has gaps.

Sandbox environments where LLM-generated code is executed

Why it matters: Sandboxing limits the damage if malicious code slips through validation, preventing lateral movement.

Apply least-privilege principles to all LLM tool integrations and API access

Why it matters: Overprivileged LLM agents can perform destructive actions if manipulated. Least privilege limits the blast radius.

Require human-in-the-loop approval for high-impact actions (data deletion, financial transactions, etc.)

Why it matters: Human oversight on critical actions prevents irreversible damage from hallucinated or manipulated decisions.

Limit the number and scope of tools and plugins available to the LLM

Why it matters: Each additional tool increases the attack surface. Only expose what is genuinely needed.

Implement rate limiting and action budgets for autonomous LLM operations

Why it matters: Rate limits prevent runaway agents from causing widespread damage in a short time window.

Avoid embedding secrets, API keys, or sensitive business logic directly in system prompts

Why it matters: System prompts should be assumed extractable. Any secrets in them will eventually be exposed.

Implement prompt leakage detection that flags responses containing system prompt fragments

Why it matters: Automated detection catches leaks in real time, enabling rapid response before damage spreads.

Externalize sensitive configuration to server-side logic rather than relying on prompt instructions

Why it matters: Server-side enforcement cannot be bypassed through prompt manipulation, unlike prompt-based rules.

Regularly test prompts with extraction techniques to identify leakage vulnerabilities

Why it matters: Proactive testing finds weaknesses before attackers do. Prompt robustness degrades as models evolve.

Apply access controls to vector database collections so users only retrieve authorized content

Why it matters: Without access controls, RAG systems can surface documents the user has no business seeing.

Validate and sanitize documents before embedding them into the vector store

Why it matters: Malicious content in embeddings can poison retrieval results and influence LLM outputs at scale.

Monitor for embedding inversion attacks that attempt to reconstruct source documents

Why it matters: Embeddings are not fully opaque. Sophisticated attacks can recover sensitive source material from them.

Implement retrieval-augmented generation (RAG) to ground responses in verified data sources

Why it matters: Grounding reduces hallucinations by anchoring outputs to factual, curated knowledge bases.

Add confidence indicators and source citations to LLM-generated outputs

Why it matters: Users need signals to judge reliability. Unsourced AI output is often trusted uncritically.

Establish human review workflows for LLM outputs used in high-stakes decisions

Why it matters: Automated fact-checking is imperfect. Human review remains essential for consequential outputs.

Clearly label AI-generated content so consumers know its provenance

Why it matters: Transparency about AI authorship enables appropriate skepticism and accountability.

Set token limits and timeouts on all LLM API calls

Why it matters: Unbounded requests can exhaust compute budgets and degrade service for legitimate users.

Implement per-user and per-session rate limiting for LLM endpoints

Why it matters: Rate limits prevent individual actors from monopolizing resources or running up costs.

Monitor and alert on anomalous usage patterns (token spikes, unusual query volumes)

Why it matters: Early detection of abuse patterns enables response before costs spiral or availability degrades.

Set hard spending caps and budget alerts for LLM API consumption

Why it matters: Without financial guardrails, a single attack or bug can generate unbounded costs.