Set the retrieval reasoning effort

Note

This feature is currently in public preview. This preview is provided without a service-level agreement and isn't recommended for production workloads. Certain features might not be supported or might have constrained capabilities. For more information, see Supplemental Terms of Use for Azure Previews.

In agentic retrieval, you can specify the level of large language model (LLM) processing for query planning and answer formulation. Use the retrievalReasoningEffort property to determine LLM processing levels. You can set this property in a knowledge base or on a retrieve request.

Levels of reasoning effort include:

Level Effort
minimal No LLM processing.
low Runs a single pass of LLM-based query planning and knowledge source selection. This is the default.
medium Adds deeper search and an enhanced retrieval stack to agentic retrieval to maximize completeness.

Prerequisites

Set retrievalReasoningEffort in a knowledge base

To establish the default behavior, set the property in the knowledge base.

  1. Use Create or Update Knowledge Base to set the retrievalReasoningEffort.

  2. Add the retrievalReasoningEffort property. The following JSON shows the syntax. For more information about knowledge bases, see Create a knowledge base.

    "retrievalReasoningEffort": { /* no other parameters when effort is minimal */
    "kind": "low"
}

Set retrievalReasoningEffort in a retrieve request

To override the default on a query-by-query basis, set the property in the retrieve request.

  1. Modify a retrieve action to override the knowledge base retrievalReasoningEffort default.

  2. Add the retrievalReasoningEffort property. A retrieve request might look similar to the following example.

    {
    "messages": [ /* trimmed for brevity */  ],
    "retrievalReasoningEffort": { "kind": "low" },
    "outputMode": "answerSynthesis",
    "maxRuntimeInSeconds": 30,
    "maxOutputSize": 6000
}

Choose a retrieval reasoning effort

Level Description Recommendation Limits
minimal Disables LLM-based query planning to deliver the lowest cost and latency for agentic retrieval. It issues direct text and vector searches across the knowledge sources listed in the knowledge base, and returns the best-matching passages. Because all knowledge sources are always searched and no query expansion is performed, behavior is predictable and easy to control. It also means the alwaysQueryKnowledgeSource property on a retrieve request is ignored. Use "minimal" for migrations from the Search API or when you want to manage query planning yourself. outputMode must be set to extractiveData.
Answer synthesis and web knowledge aren't supported.
low The default mode of agentic retrieval, running a single pass of LLM-based query planning and knowledge source selection. The agentic retrieval engine generates subqueries and fans them out to the selected knowledge sources, then merges the results. You can enable answer synthesis to produce a grounded natural-language response with inline citations. Use "low" when you want a balance between minimal latency and deeper processing. 5,000 answer tokens.
Maximum three subqueries from a maximum of three knowledge sources.
Maximum of 50 documents for semantic ranking, and 10 documents if the semantic ranker uses L3 classification.

Related content

  • Last updated on