Create a hybrid query in Azure AI Search

Article
11/29/2024

Hybrid search combines text (keyword) and vector queries in a single search request. All subqueries in the request execute in parallel. The results are merged and reordered by new search scores, using Reciprocal Rank Fusion (RRF) to return a unified result set.

In many cases, per benchmark tests, hybrid queries with semantic ranking return the most relevant results.

Prerequisites

A search index containing searchable vector and nonvector fields. We recommend the Import and vectorize data wizard to create an index quickly. Otherwise, see Create an index and Add vector fields to a search index.
(Optional) If you want built-in text-to-vector conversion of a query string, create and assign a vectorizer to vector fields in the search index.

Choose an API or tool

Search Explorer in the Azure portal (supports both stable and preview API search syntax) has a JSON view that lets you paste in a hybrid request.
2024-07-01 stable version or a recent preview API version if you're using preview features like maxTextRecallSize and countAndFacetMode(preview).

For readability, we use REST examples to explain how the APIs work. You can use a REST client like Visual Studio Code with the REST extension to build hybrid queries. For more information, see Quickstart: Vector search using REST APIs.
Newer stable or beta packages of the Azure SDKs (see change logs for SDK feature support).

Set up a hybrid query in Search Explorer

In Search Explorer, make sure the API version is 2024-07-01 or a newer preview API version.
Under View, select JSON view so that you can paste in a vector query.

Replace the default query template with a hybrid query, such as the "Run a hybrid query" example starting on line 539 in the vector quickstart. For brevity, the vector is truncated in this article.

A hybrid query has a text query specified in search, and a vector query specified under vectorQueries.vector.

The text query and vector query can be equivalent or divergent, but it's common for them to share the same intent.

{
    "count": true,
    "search": "historic hotel walk to restaurants and shopping",
    "select": "HotelId, HotelName, Category, Tags, Description",
    "top": 7,
    "vectorQueries": [
        {
            "vector": [0.01944167, 0.0040178085, -0.007816401 ... <remaining values omitted> ], 
            "k": 7,
            "fields": "DescriptionVector",
            "kind": "vector",
            "exhaustive": true
        }
    ]
}

Select Search.

Tip

Search results are easier to read if you hide the vectors. In Query Options, turn on Hide vector values in search results.

Hybrid query request (REST API)

A hybrid query combines text search and vector search, where the search parameter takes a query string and vectorQueries.vector takes the vector query. The search engine runs full text and vector queries in parallel. The union of all matches is evaluated for relevance using Reciprocal Rank Fusion (RRF) and a single result set is returned in the response.

Results are returned in plain text, including vectors in fields marked as retrievable. Because numeric vectors aren't useful in search results, choose other fields in the index as a proxy for the vector match. For example, if an index has "descriptionVector" and "descriptionText" fields, the query can match on "descriptionVector" but the search result can show "descriptionText". Use the select parameter to specify only human-readable fields in the results.

The following example shows a hybrid query configuration.

POST https://{{search-service-name}}.search.azure.cn/indexes/{{index-name}}/docs/search?api-version=2024-07-01
Content-Type: application/json
api-key: {{admin-api-key}}
{
    "vectorQueries": [
        {
            "vector": [
                -0.009154141,
                0.018708462,
                . . . 
                -0.02178128,
                -0.00086512347
            ],
            "fields": "DescriptionVector",
            "kind": "vector",
            "exhaustive": true,
            "k": 10
        },
        {
            "vector": [
                -0.009154141,
                0.018708462,
                . . . 
                -0.02178128,
                -0.00086512347
            ],
            "fields": "DescriptionVector",
            "kind": "vector",
            "exhaustive": true,
            "k": 10
        }
    ],
    "search": "historic hotel walk to restaurants and shopping",
    "select": "HotelName, Description, Address/City",
    "top": 10
}

Key points:

The vector query string is specified through the vectorQueries.vector property. The query executes against the "DescriptionVector" field. Set kind to "vector" to indicate the query type. Optionally, set exhaustive to true to query the full contents of the vector field.
Keyword search is specified through search property. It executes in parallel with the vector query.
k determines how many nearest neighbor matches are returned from the vector query and provided to the RRF ranker.
top determines how many matches are returned in the response all-up. In this example, the response includes 10 results, assuming there are at least 10 matches in the merged results.

Hybrid search with filter

This example adds a filter, which is applied to the filterable nonvector fields of the search index.

POST https://{{search-service-name}}.search.azure.cn/indexes/{{index-name}}/docs/search?api-version=2024-07-01
Content-Type: application/json
api-key: {{admin-api-key}}
{
    "vectorQueries": [
        {
            "vector": [
                -0.009154141,
                0.018708462,
                . . . 
                -0.02178128,
                -0.00086512347
            ],
            "fields": "DescriptionVector",
            "kind": "vector",
            "k": 10
        }
    ],
    "search": "historic hotel walk to restaurants and shopping",
    "vectorFilterMode": "postFilter",
    "filter": "ParkingIncluded",
    "top": "10"
}

Key points:

Filters are applied to the content of filterable fields. In this example, the ParkingIncluded field is a boolean and it's marked as filterable in the index schema.
In hybrid queries, filters can be applied before query execution to reduce the query surface, or after query execution to trim results. "preFilter" is the default. To use postFilter, set the filter processing mode as shown in this example.
When you postfilter query results, the number of results might be less than top-n.

Configure a query response

When you're setting up the hybrid query, think about the response structure. The response is a flattened rowset. Parameters on the query determine which fields are in each row and how many rows are in the response. The search engine ranks the matching documents and returns the most relevant results.

Fields in a response

Search results are composed of retrievable fields from your search index. A result is either:

All retrievable fields (a REST API default).
Fields explicitly listed in a "select" parameter on the query.

The examples in this article used a "select" statement to specify text (nonvector) fields in the response.

Note

Vectors aren't reverse engineered into human readable text, so avoid returning them in the response. Instead, choose nonvector fields that are representative of the search document. For example, if the query targets a "DescriptionVector" field, return an equivalent text field if you have one ("Description") in the response.

Number of results

A query might match to any number of documents, as many as all of them if the search criteria are weak (for example "search=*" for a null query). Because it's seldom practical to return unbounded results, you should specify a maximum for the overall response:

"top": n results for keyword-only queries (no vector)
"k": n results for vector-only queries
"top": n results for hybrid queries (with or without semantic) that include a "search" parameter

Both "k" and "top" are optional. Unspecified, the default number of results in a response is 50. You can set "top" and "skip" to page through more results or change the default.

Note

If you're using hybrid search in 2024-05-01-preview API, you can control the number of results from the keyword query using maxTextRecallSize. Combine this with a setting for "k" to control the representation from each search subsystem (keyword and vector).

Semantic ranker results

Note

The semantic ranker can take up to 50 results.

If you're using semantic ranker in 2024-05-01-preview API, it's a best practice to set "k" and "maxTextRecallSize" to sum to at least 50 total. You can then restrict the results returned to the user with the "top" parameter.

If you're using semantic ranker in previous APIs do the following:

if doing keyword-only search (no vector) set "top" to 50
if doing hybrid search set "k" to 50, to ensure that the semantic ranker gets at least 50 results.

Ranking

Multiple sets are created for hybrid queries. Ranking of results is computed by Reciprocal Rank Fusion (RRF).

In this section, compare the responses between single vector search and simple hybrid search for the top result. The different ranking algorithms, HNSW's similarity metric and RRF is this case, produce scores that have different magnitudes. This behavior is by design. RRF scores can appear quite low, even with a high similarity match. Lower scores are a characteristic of the RRF algorithm. In a hybrid query with RRF, more of the reciprocal of the ranked documents are included in the results, given the relatively smaller score of the RRF ranked documents, as opposed to pure vector search.

Single Vector Search: @search.score for results ordered by cosine similarity (default vector similarity distance function).

{
    "@search.score": 0.8399121,
    "HotelId": "49",
    "HotelName": "Swirling Currents Hotel",
    "Description": "Spacious rooms, glamorous suites and residences, rooftop pool, walking access to shopping, dining, entertainment and the city center.",
    "Category": "Luxury",
    "Address": {
    "City": "Arlington"
    }
}

Hybrid Search: @search.score for hybrid results ranked using Reciprocal Rank Fusion.

{
    "@search.score": 0.032786883413791656,
    "HotelId": "49",
    "HotelName": "Swirling Currents Hotel",
    "Description": "Spacious rooms, glamorous suites and residences, rooftop pool, walking access to shopping, dining, entertainment and the city center.",
    "Category": "Luxury",
    "Address": {
    "City": "Arlington"
    }
}

Next steps

As a next step, we recommend reviewing the demo code for Python, C# or JavaScript.

Create a hybrid query in Azure AI Search

Prerequisites

Choose an API or tool

Set up a hybrid query in Search Explorer

Hybrid query request (REST API)

Hybrid search with filter

Configure a query response

Fields in a response

Number of results

Semantic ranker results

Ranking

Next steps

Additional resources