Monitor Kubernetes Online Endpoint inference server logs

  • Article

APPLIES TO: Azure CLI ml extension v2 (current) Python SDK azure-ai-ml v2 (current)

To diagnose online issues and monitor Azure Machine Learning model inference server metrics, we usually need to collect model inference server logs.

AKS cluster

In AKS cluster, you can use the built-in ability to collect container logs. Follow the steps to collect inference server logs in AKS:

  1. Go to the AKS portal and select Logs tab

    Diagram illustrating how to configure Azure monitor in AKS.

  2. Click Configure Monitoring to enable Azure Monitor for your AKS. In the Advanced Settings section, you can specify an existing Log Analytics or create a new one for collecting logs.

    Diagram illustrating how to configure container insight in AKS monitor.

  3. After about 1 hour for it to take effect, you can query inference server logs from AKS or Log Analytics portal.

    Example of query run in AKS monitor.

  4. Query example:

        let starttime = ago(1d);
    ContainerLogV2
    | where TimeGenerated > starttime
    | where PodName has "blue-sklearn-mnist"
    | where ContainerName has "inference-server"
    | project TimeGenerated, PodNamespace, PodName, ContainerName, LogMessage
    | limit 100