Agentic retrieval in Azure AI Search

Note

Azure AI Search is available through the Azure portal, REST APIs, and Azure SDKs.

Important

Features, capabilities, or properties marked (preview) aren't covered by a service-level agreement, aren't recommended for production workloads, and might change or be constrained before they become generally available. The Azure AI Search preview terms apply to all preview functionality, whether it's standalone or part of a generally available feature.

In Azure AI Search, agentic retrieval is a multi-query pipeline designed for complex questions posed by users or agents in chat and copilot apps. It's intended for retrieval-augmented generation (RAG) patterns and agent-to-agent workflows.

Here's what agentic retrieval does:

  • Can use LLM-based query planning (preview) to break down a complex query into smaller, focused subqueries for better coverage across proprietary and external content. Query planning can use chat history for additional context.

  • Runs subqueries in parallel. Each subquery is semantically reranked to promote the most relevant matches.

  • Combines the best results into a unified response that an LLM can use to generate grounded answers.

  • Can return source references and an activity log alongside the merged content, so you can use just the grounding data or pass it to an LLM for a full answer.

This high-performance pipeline helps you generate high-quality grounding data or answers for your chat application, with the ability to answer complex questions quickly.

Why use agentic retrieval?

Use agentic retrieval when you want to provide agents and apps with the most relevant content for answering harder questions, drawing on chat context, your proprietary content, and external sources.

Agentic retrieval adds latency compared to a single-query pipeline, but it handles query complexity that a single query can't. For example, it can handle:

  • Questions with multiple asks, such as "find me a hotel near the beach, with airport transportation, and that's within walking distance of vegetarian restaurants."

  • Questions that depend on earlier context in the conversation.

  • Queries that benefit from rewriting, using synonym maps and LLM-generated paraphrasing to expand coverage across your content.

  • Spelling mistakes.

Diagram of a complex query showing how agentic retrieval handles implied context and an intentional typo.

Architecture and workflow

The agentic retrieval process works as follows:

  1. Workflow initiation: Your application calls a knowledge base with a retrieve action that provides a query and conversation history.

  2. Query planning: At low and medium retrieval reasoning effort, the knowledge base sends your query and conversation history to an LLM, which generates focused subqueries. At minimal effort, this step is skipped and queries are issued directly to knowledge sources. Reasoning effort defaults to low and is configured on the knowledge base.

  3. Query execution: The knowledge base sends the subqueries to your knowledge sources. All subqueries run simultaneously and can be keyword, vector, or hybrid search. Each subquery undergoes semantic reranking to find the most relevant matches. References are extracted and retained for citation purposes.

  4. Result synthesis: The system combines all results into a unified response. Merged content is always returned. Source references and an execution activity log are optional.

Diagram of agentic retrieval workflow using an example query.

Components

For all agentic retrieval scenarios, a knowledge base and at least one knowledge source are required. Other components are optional and depend on your configuration.

Component Service Role
Knowledge base Azure AI Search Orchestrates the pipeline, managing knowledge sources and query parameters.
Knowledge source Azure AI Search Defines the content used in the pipeline. Can be indexed (backed by a search index on your service) or remote (content retrieved at query time from an external platform).
Search index Azure AI Search Stores searchable content (text and vectors) with a semantic configuration. Determines which query types run and which optimizations apply. Required for indexed knowledge sources only.
Semantic ranker Azure AI Search Used internally by the agentic retrieval pipeline to rerank results for relevance (L2 reranking).

Integration requirements

Your application drives the pipeline by calling the knowledge base and handling the response. The pipeline returns grounding data that you can pass to an LLM for answer generation or use directly in your conversation interface.

How to get started

To create an agentic retrieval solution, you can use the Azure portal, REST APIs, or an Azure SDK package that provides the functionality.

  • Last updated on