Create a blob knowledge source from Azure Blob Storage and ADLS Gen2

Note

This agentic retrieval feature is generally available in the 2026-04-01 REST API version via programmatic access. The Azure portal continue to provide preview-only access to all agentic retrieval features.

If you choose to use a preview REST API version, you can access capabilities that aren't yet generally available for this feature. Preview features are provided without a service-level agreement and aren't recommended for production workloads.

Use a blob knowledge source to index and query Azure blob content in an agentic retrieval pipeline. Knowledge sources are created independently, referenced in a knowledge base, and used as grounding data when an agent or chatbot calls a retrieve action at query time.

When you create a blob knowledge source, you specify an external data source, models, and properties to automatically generate the following Azure AI Search objects:

  • A data source that represents a blob container.
  • A skillset that chunks and optionally vectorizes multimodal content from the container.
  • An index that stores enriched content and meets the criteria for agentic retrieval.
  • An indexer that uses the previous objects to drive the indexing and enrichment pipeline.

Note

If user access is specified at the document (blob) level in Azure Storage, a knowledge source can carry permission metadata forward to indexed content in Azure AI Search. For more information, see ADLS Gen2 permission metadata or Blob RBAC scopes.

Usage support

Azure portal .NET SDK Python SDK Java SDK JavaScript SDK REST API
✔️ ✔️ ✔️ ✔️ ✔️ ✔️

Prerequisites

Check for existing knowledge sources

A knowledge source is a top-level, reusable object. Knowing about existing knowledge sources is helpful for either reuse or naming new objects.

Run the following code to list knowledge sources by name and type.

// List knowledge sources by name and type
using Azure.Search.Documents.Indexes;

var indexClient = new SearchIndexClient(new Uri(searchEndpoint), credential);
var knowledgeSources = indexClient.GetKnowledgeSourcesAsync();

Console.WriteLine("Knowledge Sources:");

await foreach (var ks in knowledgeSources)
{
    Console.WriteLine($"  Name: {ks.Name}, Type: {ks.GetType().Name}");
}

You can also return a single knowledge source by name to review its JSON definition.

using Azure.Search.Documents.Indexes;
using System.Text.Json;

var indexClient = new SearchIndexClient(new Uri(searchEndpoint), credential);

// Specify the knowledge source name to retrieve
string ksNameToGet = "earth-knowledge-source";

// Get its definition
var knowledgeSourceResponse = await indexClient.GetKnowledgeSourceAsync(ksNameToGet);
var ks = knowledgeSourceResponse.Value;

// Serialize to JSON for display
var jsonOptions = new JsonSerializerOptions 
{ 
    WriteIndented = true,
    DefaultIgnoreCondition = System.Text.Json.Serialization.JsonIgnoreCondition.Never
};
Console.WriteLine(JsonSerializer.Serialize(ks, ks.GetType(), jsonOptions));

A knowledge source is a top-level, reusable object. Knowing about existing knowledge sources is helpful for either reuse or naming new objects.

Run the following code to list knowledge sources by name and type.

# List knowledge sources by name and type
import requests
import json

endpoint = "{search_url}/knowledgesources"
params = {"api-version": "2025-11-01-preview", "$select": "name, kind"}
headers = {"api-key": "{api_key}"}

response = requests.get(endpoint, params = params, headers = headers)
print(json.dumps(response.json(), indent = 2))

You can also return a single knowledge source by name to review its JSON definition.

# Get a knowledge source definition
import requests
import json

endpoint = "{search_url}/knowledgesources/{knowledge_source_name}"
params = {"api-version": "2025-11-01-preview"}
headers = {"api-key": "{api_key}"}

response = requests.get(endpoint, params = params, headers = headers)
print(json.dumps(response.json(), indent = 2))

A knowledge source is a top-level, reusable object. Knowing about existing knowledge sources is helpful for either reuse or naming new objects.

Use Knowledge Sources - Get (REST API) to list knowledge sources by name and type.

### List knowledge sources by name and type
GET {{search-url}}/knowledgesources?api-version=2025-11-01-preview&$select=name,kind
api-key: {{api-key}}

You can also return a single knowledge source by name to review its JSON definition.

### Get a knowledge source definition
GET {{search-url}}/knowledgesources/{{knowledge-source-name}}?api-version=2025-11-01-preview
api-key: {{api-key}}

Note

Document-level permissions enforcement using ingestionPermissionOptions requires the 2025-11-01-preview API version. 2026-04-01 doesn't support this feature.

Source-specific properties

For both the 2025-11-01-preview and 2026-04-01 API versions, you can pass the following properties to create a blob knowledge source.

Name Description Type Editable Required
Name The name of the knowledge source, which must be unique within the knowledge sources collection and follow the naming guidelines for objects in Azure AI Search. String No Yes
Description A description of the knowledge source. String Yes No
EncryptionKey A customer-managed key to encrypt sensitive information in both the knowledge source and the generated objects. Object Yes No
AzureBlobParameters Parameters specific to blob knowledge sources: ConnectionString, ContainerName, FolderPath, and IsADLSGen2. Object No
ConnectionString A key-based connection string or, if you're using a managed identity, the resource ID. String No Yes
ContainerName The name of the blob storage container. String No Yes
FolderPath A folder within the container. String No No
IsADLSGen2 The default is False. Set to True if you're using an ADLS Gen2 storage account. Boolean No No
Name Description Type Editable Required
name The name of the knowledge source, which must be unique within the knowledge sources collection and follow the naming guidelines for objects in Azure AI Search. String No Yes
description A description of the knowledge source. String Yes No
encryption_key A customer-managed key to encrypt sensitive information in both the knowledge source and the generated objects. Object Yes No
azure_blob_parameters Parameters specific to blob knowledge sources: connection_string, container_name, folder_path, and is_adls_gen2. Object No
connection_string A key-based connection string or, if you're using a managed identity, the resource ID. String No Yes
container_name The name of the blob storage container. String No Yes
folder_path A folder within the container. String No No
is_adls_gen2 The default is False. Set to True if you're using an ADLS Gen2 storage account. Boolean No No
Name Description Type Editable Required
name The name of the knowledge source, which must be unique within the knowledge sources collection and follow the naming guidelines for objects in Azure AI Search. String No Yes
kind The kind of knowledge source, which is azureBlob in this case. String No Yes
description A description of the knowledge source. String Yes No
encryptionKey A customer-managed key to encrypt sensitive information in both the knowledge source and the generated objects. Object Yes No
azureBlobParameters Parameters specific to blob knowledge sources: connectionString, containerName, folderPath, and isADLSGen2. Object No
connectionString A key-based connection string or, if you're using a managed identity, the resource ID. String No Yes
containerName The name of the blob storage container. String No Yes
folderPath A folder within the container. String No No
isADLSGen2 The default is false. Set to true if you're using an ADLS Gen2 storage account. Boolean No No

Check ingestion status

Run the following code to monitor ingestion progress and health, including the knowledge source kind and detailed indexing errors for knowledge sources that generate an indexer pipeline and populate a search index.

using Azure.Search.Documents.Indexes;
using System.Text.Json;

var indexClient = new SearchIndexClient(new Uri(searchEndpoint), new AzureKeyCredential(apiKey));

// Get knowledge source ingestion status
var statusResponse = await indexClient.GetKnowledgeSourceStatusAsync(knowledgeSourceName);
var status = statusResponse.Value;

// Serialize to JSON for display
var json = JsonSerializer.Serialize(status, new JsonSerializerOptions { WriteIndented = true });
Console.WriteLine(json);

Reference: SearchIndexClient

# Check knowledge source ingestion status
from azure.core.credentials import AzureKeyCredential
from azure.search.documents.indexes import SearchIndexClient
import json

index_client = SearchIndexClient(endpoint="search_url", credential=AzureKeyCredential("api_key"))

status = index_client.get_knowledge_source_status("knowledge_source_name")
print(json.dumps(status.as_dict(), indent=2))

Reference: SearchIndexClient

### Check knowledge source ingestion status
GET {{search-url}}/knowledgesources/{{knowledge-source-name}}/status?api-version={{api-version}}
api-key: {{api-key}}
Content-Type: application/json 

Reference: Knowledge Sources - Get Status

A response for a request that includes ingestion parameters and is actively ingesting content might look like the following example.

{
  "kind": "azureBlob",
  "synchronizationStatus": "active",
  "synchronizationInterval": "1d",
  "currentSynchronizationState": {
    "startTime": "2026-04-10T19:30:00Z",
    "itemUpdatesProcessed": 1100,
    "itemsUpdatesFailed": 100,
    "itemsSkipped": 1100,
    "errors": [
      {
        "key": "Item id 1",
        "docURL": "https://contoso.blob.core.chinacloudapi.cn/contracts/2024/Q4/doc-00023.csv",
        "statusCode": 400,
        "componentName": "DocumentExtraction.AzureBlob.MyDataSource",
        "errorMessage": "Could not read the value of column 'foo' at index '0'.",
        "details": "The file could not be parsed.",
        "documentationLink": "https://go.microsoft.com/fwlink/?linkid=2049388"
      }
    ]
  },
  "lastSynchronizationState": {
    "status": "partialSuccess",
    "startTime": "2026-04-09T19:30:00Z",
    "endTime": "2026-04-09T19:40:01Z",
    "itemUpdatesProcessed": 1100,
    "itemsUpdatesFailed": 100,
    "itemsSkipped": 1100,
    "errors": null
  },
  "statistics": {
    "totalSynchronizations": 25,
    "averageSynchronizationDuration": "00:15:20",
    "averageItemsProcessedPerSynchronization": 500
  }
}

Note

The kind property and currentSynchronizationState.errors[] array with document-level error details are available starting with the 2026-04-01 API version. For earlier API versions, these fields aren't returned. The lastSynchronizationState.status field is also new in 2026-04-01.

Review the created objects

When you create a blob knowledge source, your search service also creates an indexer, index, skillset, and data source. We don't recommend that you edit these objects, as introducing an error or incompatibility can break the pipeline.

After you create a knowledge source, the response lists the created objects. These objects are created according to a fixed template, and their names are based on the name of the knowledge source. You can't change the object names.

We recommend using the Azure portal to validate output creation. The workflow is:

  1. Check the indexer for success or failure messages. Connection or quota errors appear here.
  2. Check the index for searchable content. Use Search Explorer to run queries.
  3. Check the skillset to learn how your content is chunked and optionally vectorized.
  4. Check the data source for connection details. Our example uses API keys for simplicity, but you can use Microsoft Entra ID for authentication and role-based access control for authorization.

Assign to a knowledge base

If you're satisfied with the knowledge source, continue to the next step: specify the knowledge source in a knowledge base.

After the knowledge base is configured, use the retrieve action to query the knowledge source.

Tip

To enforce document-level permissions, set ingestionPermissionOptions when you create this knowledge source, and then include the user's access token in the retrieve request. For more information, see Enforce permissions at query time.

Delete a knowledge source

Before you can delete a knowledge source, you must delete any knowledge base that references it or update the knowledge base definition to remove the reference. For knowledge sources that generate an index and indexer pipeline, all generated objects are also deleted. However, if you used an existing index to create a knowledge source, your index isn't deleted.

If you try to delete a knowledge source that's in use, the action fails and returns a list of affected knowledge bases.

To delete a knowledge source:

  1. Get a list of all knowledge bases on your search service.

    using Azure.Search.Documents.Indexes;
    
    var indexClient = new SearchIndexClient(new Uri(searchEndpoint), credential);
    var knowledgeBases = indexClient.GetKnowledgeBasesAsync();
    
    Console.WriteLine("Knowledge Bases:");
    
    await foreach (var kb in knowledgeBases)
    {
        Console.WriteLine($"  - {kb.Name}");
    }
    

    Reference: SearchIndexClient

    An example response might look like the following:

     Knowledge Bases:
       - earth-knowledge-base
       - hotels-sample-knowledge-base
       - my-demo-knowledge-base
    
  2. Get an individual knowledge base definition to check for knowledge source references.

    using Azure.Search.Documents.Indexes;
    using System.Text.Json;
    
    var indexClient = new SearchIndexClient(new Uri(searchEndpoint), credential);
    
    // Specify the knowledge base name to retrieve
    string kbNameToGet = "earth-knowledge-base";
    
    // Get a specific knowledge base definition
    var knowledgeBaseResponse = await indexClient.GetKnowledgeBaseAsync(kbNameToGet);
    var kb = knowledgeBaseResponse.Value;
    
    // Serialize to JSON for display
    string json = JsonSerializer.Serialize(kb, new JsonSerializerOptions { WriteIndented = true });
    Console.WriteLine(json);
    

    Reference: SearchIndexClient

    An example response might look like the following:

     {
       "Name": "earth-knowledge-base",
       "KnowledgeSources": [
         {
           "Name": "earth-knowledge-source"
         }
       ],
       "Models": [
         {}
       ],
       "RetrievalReasoningEffort": {},
       "OutputMode": {},
       "ETag": "\u00220x8DE278629D782B3\u0022",
       "EncryptionKey": null,
       "Description": null,
       "RetrievalInstructions": null,
       "AnswerInstructions": null
     }
    
  3. Either delete the knowledge base or, if you have multiple knowledge sources, update the knowledge base to remove the source. This example shows deletion.

    using Azure.Search.Documents.Indexes;
    var indexClient = new SearchIndexClient(new Uri(searchEndpoint), credential);
    
    await indexClient.DeleteKnowledgeBaseAsync(knowledgeBaseName);
    System.Console.WriteLine($"Knowledge base '{knowledgeBaseName}' deleted successfully.");
    

    Reference: SearchIndexClient

  4. Delete the knowledge source.

    await indexClient.DeleteKnowledgeSourceAsync(knowledgeSourceName);
    System.Console.WriteLine($"Knowledge source '{knowledgeSourceName}' deleted successfully.");
    

    Reference: SearchIndexClient

  1. Get a list of all knowledge bases on your search service.

    # Get knowledge bases
    from azure.core.credentials import AzureKeyCredential
    from azure.search.documents.indexes import SearchIndexClient
    
    index_client = SearchIndexClient(endpoint = "search_url", credential = AzureKeyCredential("api_key"))
    
    print("Knowledge Bases:")
    for kb in index_client.list_knowledge_bases():
        print(f"  - {kb.name}")
    

    Reference: SearchIndexClient

    An example response might look like the following:

     {
         "@odata.context": "https://my-search-service.search.azure.cn/$metadata#knowledgebases(name)",
         "value": [
         {
             "name": "my-kb"
         },
         {
             "name": "my-kb-2"
         }
         ]
     }
    
  2. Get an individual knowledge base definition to check for knowledge source references.

    # Get a knowledge base definition
    from azure.core.credentials import AzureKeyCredential
    from azure.search.documents.indexes import SearchIndexClient
    
    index_client = SearchIndexClient(endpoint = "search_url", credential = AzureKeyCredential("api_key"))
    kb = index_client.get_knowledge_base("knowledge_base_name")
    print(kb)
    

    Reference: SearchIndexClient

    An example response might look like the following:

     {
       "name": "my-kb",
       "description": null,
       "retrievalInstructions": null,
       "answerInstructions": null,
       "outputMode": null,
       "knowledgeSources": [
         {
           "name": "my-blob-ks",
         }
       ],
       "models": [],
       "encryptionKey": null,
       "retrievalReasoningEffort": {
         "kind": "low"
       }
     }
    
  3. Either delete the knowledge base or, if you have multiple knowledge sources, update the knowledge base to remove the source. This example shows deletion.

    # Delete a knowledge base
    from azure.core.credentials import AzureKeyCredential 
    from azure.search.documents.indexes import SearchIndexClient
    
    index_client = SearchIndexClient(endpoint = "search_url", credential = AzureKeyCredential("api_key"))
    index_client.delete_knowledge_base("knowledge_base_name")
    print(f"Knowledge base deleted successfully.")
    

    Reference: SearchIndexClient

  4. Delete the knowledge source.

    # Delete a knowledge source
    from azure.core.credentials import AzureKeyCredential 
    from azure.search.documents.indexes import SearchIndexClient
    
    index_client = SearchIndexClient(endpoint = "search_url", credential = AzureKeyCredential("api_key"))
    index_client.delete_knowledge_source("knowledge_source_name")
    print(f"Knowledge source deleted successfully.")
    

    Reference: SearchIndexClient

  1. Get a list of all knowledge bases on your search service.

    ### Get knowledge bases
    GET {{search-url}}/knowledgebases?api-version={{api-version}}&$select=name
    api-key: {{api-key}}
    

    Reference: Knowledge Bases - List

    An example response might look like the following:

     {
         "@odata.context": "https://my-search-service.search.azure.cn/$metadata#knowledgebases(name)",
         "value": [
         {
             "name": "my-kb"
         },
         {
             "name": "my-kb-2"
         }
         ]
     }
    
  2. Get an individual knowledge base definition to check for knowledge source references.

    ### Get a knowledge base definition
    GET {{search-url}}/knowledgebases/{{knowledge-base-name}}?api-version={{api-version}}
    api-key: {{api-key}}
    

    Reference: Knowledge Bases - Get

    An example response might look like the following:

     {
       "name": "my-kb",
       "description": null,
       "retrievalInstructions": null,
       "answerInstructions": null,
       "outputMode": null,
       "knowledgeSources": [
         {
           "name": "my-blob-ks",
         }
       ],
       "models": [],
       "encryptionKey": null,
       "retrievalReasoningEffort": {
         "kind": "low"
       }
     }
    
  3. Either delete the knowledge base or, if you have multiple knowledge sources, update the knowledge base to remove the source. This example shows deletion.

    ### Delete a knowledge base
    DELETE {{search-url}}/knowledgebases/{{knowledge-base-name}}?api-version={{api-version}}
    api-key: {{api-key}}
    

    Reference: Knowledge Bases - Delete

  4. Delete the knowledge source.

    ### Delete a knowledge source
    DELETE {{search-url}}/knowledgesources/{{knowledge-source-name}}?api-version={{api-version}}
    api-key: {{api-key}}
    

    Reference: Knowledge Sources - Delete