AI Con USA 2025 - AI Infrastructure
Tuesday, June 10
Clean Your Filthy RAGS! Optimizing, Accelerating, and Evaluating RAG Applications
NewRetrieval-Augmented Generation (RAG) applications are becoming essential for companies, combining AI with real-time data retrieval to enhance customer experiences. While Large Language Models (LLMs) handle general conversation well, they struggle with domain-specific, up-to-date information, often producing inaccurate or unhelpful responses. This workshop will empower participants with the necessary skills to optimize RAG applications using existing best practices. Justin will walk through integrating RAGAS, a framework designed to evaluate, monitor, and fine-tune the performance of RAG...
Common Software, Data, and AI Architectures and the Ways They Fail
NewThis tutorial will examine the complexity that lives in software systems, data ingestion workflows, MLOps pipelines, and artificial intelligence systems. This session blends together cloud architecture, quality assurance, risk management, and security mindsets as Matt Eland explores how modern systems are structured, the problems their complexity helps us solve, and the ways these systems can break - or be broken. The session will alternate between interactive lectures with practical illustrations and group exercises around case studies as you explore how existing systems can fail and what...
A Quality Engineering Introduction to AI and Machine Learning
Although there are several controversies and misunderstandings surrounding AI and machine learning, one thing is apparent — people have quality concerns about the safety, reliability, and trustworthiness of these types of systems. Not only are ML-based systems shrouded in mystery due to their largely black-box nature, they also tend to be unpredictable since they can adapt and learn new things at runtime. Validating ML systems is challenging and requires a cross-section of knowledge, skills, and experience from areas such as mathematics, data science, software engineering, cyber-security,...
AI Deep Dive: Exploring AWS Using Real-World Scenarios
Deepen your AI and machine learning expertise using AWS in an Immersive, hands-on workshop. You’ll use real-world AI challenges while leveraging AWS services like Amazon SageMaker, Bedrock, and Lambda to build and optimize AI-driven solutions. As the session unfolds, new constraints and data anomalies will emerge, mirroring the complexities of real-world AI/ML implementation. Gain insight into how AI solutions perform under evolving conditions, learning to adapt, optimize, and troubleshoot unexpected challenges. Learn the importance of collaboration, strategic thinking, problem-solving,...
Wednesday, June 11
Best Practices for Using AI for Structured Data Extraction
Structured data extraction, or data tagging, is one of the easiest and impactful applications of modern AI. Before the wide availability of pre-trained AI models, the process of “understanding” unstructured data either required constructing complex heuristic logic or investing in a machine learning team who could train models in-house. Now, cheap and powerful tagging machines are an API call away, redefining what is possible for how we can understand our data. In this talk, I'll share how we’ve used AI at Yahoo News to improve our content understanding pipelines. Yahoo was a pioneer in...
The Impact of AI on Developer Productivity
Generative AI tools hold promise to increase human productivity. In the world of software development, GitHub Copilot was one of the first practical applications of the use of generative AI to support developer productivity. However, measuring software productivity is non-trivial. For example, developer productivity gains is more than just producing code faster. If these artifacts don’t meet quality standards or themselves bring cost efficiency challenges then there may not be much overall improvement. To truly understand the benefits and challenges of AI-powered copilots requires real-...
Building Reliable AI Agent Flows
AI agents are revolutionizing how we interact with software, but their reliability remains a critical challenge. In this talk, Jason Arbon explores the key principles and techniques for designing AI agent flows that are predictable, testable, and robust. Attendees will learn how to structure AI-driven interactions to minimize failure points, leverage validation mechanisms, and integrate automated testing strategies to ensure smooth execution. Drawing from real-world applications and lessons learned in AI-driven software testing, this session will provide practical insights for engineers,...
Test Machina: Demystifying AI-Driven Testing Agents
Software vendors and practitioners are using artificial intelligence (AI) and machine learning (ML) to create a new wave of test automation tools. Such tools leverage autonomous and intelligent agents to explore, model, reason and learn about a software product. But how do these testing agents really work? Is this technology any good? And can we really trust them to validate software? Tariq King will introduce you to the world of agentic AI and discuss its benefits, challenges and other limitations. Learn how AI test agents use AI/ML technologies to mimic human testing activities such as...
Powering Complex Solutions with Agentic Systems
This session explores the revolutionary potential of agentic systems—autonomous agents equipped with diverse tools to tackle multifaceted tasks effectively. By integrating tools, agentic workflows enable intelligent, adaptive solutions to complex challenges. Apurva will start with an introduction to agentic systems, highlighting their ability to utilize various tools dynamically to achieve goals with minimal human intervention. Through real-world case studies, she will demonstrate how tools like APIs, databases, and external services are orchestrated within these systems to simulate...
Securing the Foundations of AI: Addressing the Past to Safeguard the Future
AI’s future hinges on an ecosystem built on decades of technical debt, fragmented tools, and opaque processes; creating vulnerabilities that threaten the reliability and security of modern applications. In this talk, Peter will examine how the legacy of open-source numerical computing and software supply chains is influencing AI’s trajectory. Drawing from over a decade of leadership in the Python and scientific computing communities, Peter will share strategies for tackling these challenges: improving transparency in data and dependencies, building curated software stacks, addressing...
Operationalizing Disruptive Technologies: A Strategic Framework for Harnessing the Power of GenAI
The advent of disruptive technologies, particularly in artificial intelligence, has ushered in a new era of possibilities and challenges. This talk proposes a comprehensive framework for operationalizing disruptive technologies, specifically focusing on the transformative potential of Generative Artificial Intelligence (GenAI). As organizations grapple with integrating GenAI into their operations, there is a pressing need for a structured approach that addresses technical, ethical, and organizational considerations. Mary will delve into a strategic framework designed to guide businesses in...
Thursday, June 12
AI at Scale: Balancing innovation, governance, and risk in large organizations
As AI reshapes industries, large organizations must scale innovation while upholding governance, security, and ethical responsibility. Deploying AI at scale isn’t just a technical challenge—it’s a strategic balancing act between agility and compliance, risk and reward. Andreas Bohman, CIO at the University of Washington, will discuss strategies to drive AI-powered innovation without compromising regulatory obligations, operational effectiveness, or public trust. He will share governance strategies that enable innovation rather than restrict it. He’ll also talk about addressing critical...
RAG at Scale: Building Production-Ready GenAI Solutions
This session serves as a deep dive into the strategies and best practices for data scientists aiming to build, fine-tune, and scale Retrieval Augmented Generation (RAG) based Generative AI (GenAI) applications. The core objectives for data scientists in the AI development cycle center around providing highly relevant results for end users and maintaining cost-effectiveness to support the sustainable growth of their products. RAG is a pivotal method that enables GenAI applications to operate effectively on proprietary data. At the heart of RAG are robust retrieval systems, which are crucial...