AI Guardrails: Secure Generative AI and AI Agents

About this course

This course focuses on the implementation of artificial intelligence.

Imagine your AI assistant informing a customer that their water damage claim is valid. It isn't. In another instance, a chat leaked an address to an incorrect recipient, a refund was processed without proper authorization, and the assistant assured an exception that was nonexistent. Each scenario began with a helpful AI lacking essential guardrails.

You will learn to design, construct, and validate the safeguards necessary to keep generative AI applications and AI agents in check. This course goes beyond just defining prompt injection; it also teaches you where each safeguard should be implemented in a real-world application, what each control evaluates, the consequences of its failure, and how to demonstrate its effectiveness to those managing the associated risks.

One comprehensive case study provides a complete view: HarborPoint Services, a home services company, is launching a customer service assistant that addresses policy inquiries, accesses accounts, schedules visits, and processes refunds. Each lesson builds upon the previous one, leading from application mapping to a defensible release decision.

Here is what you will cover:

• Mapping before implementing controls: track a request through retrieval, generation, validation, and execution to identify trust boundaries.
• Translating policy into actionable rules: convert "ensure safety" into a decision table to guide real controls, including exceptions and human oversight.
• Countering hostile inputs: manage both direct and indirect prompt injection, which includes concealed instructions in documents and images, recognizing that screening alone is insufficient.
• Handling sensitive data and retrieval: restrict the model’s visibility, enforce permissions prior to retrieval, and maintain a reliable knowledge base.
• Distinguishing between plausible and validated answers, and enforcing business parameters beyond the model’s understanding.
• Managing AI agents: limit access to tools, credentials, connectors, and MCP servers, establish meaningful approval gates, and impose spending restrictions.
• Establishing a multi-layered architecture for guardrails, ensuring safe failure, and aligning them with the OWASP Top 10 for LLM Applications, the OWASP Top 10 for Agentic Applications, and MITRE ATLAS.
• Conducting testing: develop evaluation sets, assess missed violations and unnecessary blocks, red-team the assistant, and perform regression testing following changes.
• Managing operations: oversee decisions made in production, investigate control failures, contain issues, restore functionality, and designate responsibility.

Engage through hands-on experiences. Each lesson includes pause-and-decide exercises, and the capstone project involves correcting a flawed release configuration and presenting a comprehensive guardrail strategy alongside its evidence, constraints, and responsible parties. Practical labs utilize synthetic data and provided code within Jupyter or Google Colab, requiring no AI models, API keys, subscriptions, or specialized hardware.

This course is geared towards defenders. It does not serve as an introduction to a single product or a compilation of attack strategies. Instead, it prioritizes controls that are enforced by the application, maintained through honest measurements over time, ensuring that the assistant remains beneficial to genuine customers while safety measures function effectively.

Real-life, publicly documented incidents are included where relevant, all backed by verification. By course completion, you will be equipped to evaluate any AI assistant or agent, clearly identifying what keeps it under control and what remains unaddressed.