Strategies for Testing AI-Based Systems

Day 1: Foundations of AI QA and Testing GenAI Solutions

Morning Session: The New Paradigm & The Golden Set

1. Introduction: The QA Shift from Deterministic to Probabilistic

• The Core Problem: Moving from Assert.AreEqual(expected, actual) to evaluate range-based valid outputs.
• The "Oracle" Problem: Why traditional QA lacks a single source of truth for AI responses.
• Defining the System Under Test:
  • GenAI Solutions: RAG chatbots and content generators.
  • Agentic AI: Systems that reason, plan, and execute actions via tools.
• Scope of Testing: Why we focus on "Inference" (Solution-level) rather than "Training" (Model-level).

2. The "Golden Dataset": The New Regression Suite

• Concept: Creating a curated set of high-quality inputs and ideal outcomes to serve as a baseline.
• Why it is Critical: Detecting regression when prompts or underlying models change.
• Creation Strategy:
  • Sourcing real production data vs. synthetic data generation.
  • Labeling data: "Good" (Pass) vs. "Bad" (Fail) examples.
• Hands-on Lab: Building a "Golden Test Set"

Afternoon Session: Testing RAG & LLM Integration

3. Evaluation Metrics & Methodologies (Beyond Accuracy)

• Key Quality Metrics:
  • Faithfulness/Groundedness: Is the answer supported by the context?.
  • Relevance: Did it answer the user's specific query?.
  • Coherence & Toxicity: Readability and safety checks.
• Evaluation Techniques:
  • Heuristics: Keyword matching and similarity thresholds (Exact match vs. Semantic similarity).
  • LLM-as-a-Judge: Using a strong model (e.g., GPT-4) to grade the response of the system under test.

4. Deep Dive: Testing Retrieval-Augmented Generation (RAG)

• Testing the Retrieval Component:
  • Context Precision vs. Context Recall: Did the system find the right document chunk?.
  • Technique: Verifying document IDs in the Golden Dataset.
• Testing the Generation Component:
  • Prompt Engineering Tests: A/B testing prompt templates (Zero-shot vs. Few-shot).
  • Hallucination Detection: Identifying when the model "makes things up" outside of provided context.
• Hands-on Lab: Testing retrieval accuracy and faithfulness checks

Day 2: Advanced Testing – Agents, Robustness, and Security

Morning Session: Testing Agentic AI & Integration

5. The Challenge of Agentic AI

• What makes Agents different? Statefulness, memory, and complex execution paths.
• Testing Tool Use:
  • Verifying the agent selects the correct API/Tool for the job.
  • Parameter Verification: Did the agent extract the correct variables (e.g., origin='JFK', dest='SFO')?.
• Discuss Agentic flow of: Thought -> Plan -> Action -> Observation.

6. Trajectory Analysis & Multi-Agent Systems Trajectory

• Testing: Evaluating how the agent arrived at the answer, not just the final result.
  • Bad Trajectory: Inefficient loops or asking redundant questions.
  • Good Trajectory: Efficient planning and execution.
• Multi-Agent Considerations:
  • Testing handoffs between agents (Routing).
  • System stability and infinite loops in agent conversations.
• Hands-on Lab: Testing Agents. Analyzing logs to determine if the agent took the most efficient path ​

Afternoon Session:
NFRs, Security, and Operations

7. Robustness, Error Handling, and Non-Functional Requirements

• Robustness Testing:
  • API Failures: How does the AI behave when a tool (e.g., Flight Search API) returns a 500 error? Does it degrade gracefully?
  • Boundary Conditions: Testing out-of-distribution inputs and edge cases.
• Non-Functional Testing:
  • Latency: Measuring "Time to First Token" vs. Total Generation Time.
  • Cost: Tracking token consumption per query to prevent budget overrun.

8. Security, Fairness, and "Red Teaming"

• Adversarial Testing (Red Teaming):
  • Prompt Injection: Attempts to hijack the system instructions (e.g., "Ignore previous instructions").
  • PII Leakage: Ensuring the model does not reveal sensitive data from the knowledge base.
• Fairness & Ethics:
  • Testing for bias in responses (e.g., gender, race, or socio-economic bias). Implementing safety guardrails and output filters.
• Hands-on Lab: Performing a "Red Team" attack

9. Automation & MLOps for Testers

• Continuous Testing in CI/CD:
  • Automating the Golden Set execution in the build pipeline.
  • Tools Overview: Introduction to frameworks like Deepchecks, LangSmith, and prompt evaluation tools.
• Production Monitoring:
  • Shift-Right Testing: Monitoring for "Drift" (answers getting worse over time) and user feedback loops.

10. Wrap-up & Retro: Discussion on the future of AI testing (Self-healing systems, Formal verification).