RAG Has Evolved - Enhance Your RAG Pipeline with These Concepts

The majority of businesses today can set up a fundamental RAG pipeline that effectively handles most use cases. However, this basic setup eventually reaches its limitations in terms of functionality and accuracy, hindering further advancements. Matt Payne aims to detail the necessary pipeline components for building advanced RAG pipelines. For each component, he will explain the what, when, why, and how and provide real-world examples. Key areas of focus include leveraging tools and function calling, which enables you to create a systematic approach to using knowledge from multiple sources and enhance simple VectorDB/retrieval/semantic similarity workflows. Query decomposition is a critical tool for handling complex queries, especially multi-step or multi-part queries. It allows you to break down the main query into sub-queries and search based on those, enabling you to capture context that fits sub-topics within the query. He will review various chunking strategies beyond simple size-based chunking. Matt will also provide examples and supporting research to illustrate how these three concepts integrate with the fundamental RAG pipeline. This will clarify the audience's comprehension of RAG's operation and enhance their understanding of how these elements interact within the existing RAG framework.