启用和查看 Azure Kubernetes 服务 (AKS) 中 Kubernetes 主节点的日志Enable and review Kubernetes master node logs in Azure Kubernetes Service (AKS)

使用 Azure Kubernetes 服务 (AKS),可以提供 kube-apiserverkube-controller-manager 等主组件作为托管服务。With Azure Kubernetes Service (AKS), the master components such as the kube-apiserver and kube-controller-manager are provided as a managed service. 创建和管理运行 kubelet 与容器运行时的节点,并通过托管的 Kubernetes API 服务器部署应用程序。You create and manage the nodes that run the kubelet and container runtime, and deploy your applications through the managed Kubernetes API server. 为帮助排查应用程序和服务问题,可能需要查看这些主组件生成的日志。To help troubleshoot your application and services, you may need to view the logs generated by these master components. 本文介绍如何使用 Azure Monitor 日志从 Kubernetes 主组件启用和查询日志。This article shows you how to use Azure Monitor logs to enable and query the logs from the Kubernetes master components.

准备阶段Before you begin

本文要求在 Azure 帐户中运行一个现有的 AKS 群集。This article requires an existing AKS cluster running in your Azure account. 如果还没有 AKS 群集,请使用 Azure CLIAzure 门户创建一个。If you do not already have an AKS cluster, create one using the Azure CLI or Azure portal. Azure Monitor 日志适用于支持 RBAC 和不支持 RBAC 的 AKS 群集。Azure Monitor logs works with both RBAC and non-RBAC enabled AKS clusters.

启用诊断日志Enable diagnostics logs

为帮助收集和审查来自多个源的数据,Azure Monitor 日志提供了查询语言和分析引擎,该引擎可提供环境的见解。To help collect and review data from multiple sources, Azure Monitor logs provides a query language and analytics engine that provides insights to your environment. 工作区用于整理和分析数据,并可与 Application Insights 和安全中心等其他 Azure 服务集成。A workspace is used to collate and analyze the data, and can integrate with other Azure services such as Application Insights and Security Center. 若要使用不同的平台分析日志,可以选择将诊断日志发送到 Azure 存储帐户或事件中心。To use a different platform to analyze the logs, you can instead choose to send diagnostic logs to an Azure storage account or event hub. 有关详细信息,请参阅什么是 Azure Monitor 日志?For more information, see What is Azure Monitor logs?.

Azure Monitor 日志是在 Azure 门户中启用和管理的。Azure Monitor logs are enabled and managed in the Azure portal. 若要为 AKS 群集中的 Kubernetes 主组件启用日志收集,请在 Web 浏览器中打开 Azure 门户并完成以下步骤:To enable log collection for the Kubernetes master components in your AKS cluster, open the Azure portal in a web browser and complete the following steps:

  1. 选择 AKS 群集的资源组,例如 MC_myResourceGroup_myAKSCluster_chinaeast2 。Select the resource group for your AKS cluster, such as MC_myResourceGroup_myAKSCluster_chinaeast2.

  2. 在左侧选择“诊断设置”。 On the left-hand side, choose Diagnostic settings.

  3. 选择资源组和 AKS 群集(例如,“myResourceGroup” 和“myAKSCluster” ),然后选择“添加诊断设置” 。Select the resource group and your AKS cluster, such as myResourceGroup and myAKSCluster, then choose to Add diagnostic setting.

  4. 输入名称(例如 myAKSClusterLogs ),然后选择“发送到 Log Analytics”选项。 Enter a name, such as myAKSClusterLogs, then select the option to Send to Log Analytics.

  5. 选择现有工作区或者创建新的工作区。Select an existing workspace or create a new one. 如果创建工作区,请提供工作区名称、资源组和位置。If you create a workspace, provide a workspace name, a resource group, and a location.

  6. 在可用日志列表中,选择要启用的日志。In the list of available logs, select the logs you wish to enable. 常见日志包括 kube-apiserver 、kube-controller-manager 和 kube-scheduler 。Common logs include the kube-apiserver, kube-controller-manager, and kube-scheduler. 你可以启用其他日志,例如 kube-audit 。You can enable additional logs, such as kube-audit. 启用 Log Analytics 工作区后,可以返回并更改收集的日志。You can return and change the collected logs once Log Analytics workspaces are enabled.

  7. 准备就绪后,选择“保存”以启用收集选定日志。 When ready, select Save to enable collection of the selected logs.

以下示例门户屏幕截图显示了“诊断设置”窗口: The following example portal screenshot shows the Diagnostics settings window:

为 AKS 群集的 Azure Monitor 日志启用 Log Analytics 工作区

在 AKS 群集上计划测试 podSchedule a test pod on the AKS cluster

若要生成某些日志,请在 AKS 群集中创建新的 pod。To generate some logs, create a new pod in your AKS cluster. 以下示例 YAML 清单可用于创建一个基本的 NGINX 实例。The following example YAML manifest can be used to create a basic NGINX instance. 在所选的编辑器中创建名为 nginx.yaml 的文件,并在其中粘贴以下内容:Create a file named nginx.yaml in an editor of your choice and paste the following content:

apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
  - name: mypod
    image: dockerhub.azk8s.cn/library/nginx:1.15.5
    resources:
      requests:
        cpu: 100m
        memory: 128Mi
      limits:
        cpu: 250m
        memory: 256Mi
    ports:
    - containerPort: 80

使用 kubectl create 命令创建 Pod 并指定 YAML 文件,如以下示例所示:Create the pod with the kubectl create command and specify your YAML file, as shown in the following example:

$ kubectl create -f nginx.yaml

pod/nginx created

查看收集的日志View collected logs

可能需要等待几分钟,诊断日志才会启用并显示在 Log Analytics 工作区中。It may take a few minutes for the diagnostics logs to be enabled and appear in the Log Analytics workspace. 在 Azure 门户中,选择 Log Analytics 工作区的资源组(例如 myResourceGroup),然后选择 Log Analytics 资源(例如 myAKSLogs)。In the Azure portal, select the resource group for your Log Analytics workspace, such as myResourceGroup, then choose your log analytics resource, such as myAKSLogs.

选择 AKS 群集的 Log Analytics 工作区

在左侧选择“日志”。 On the left-hand side, choose Logs. 若要查看 kube-apiserver,请在文本框中输入以下查询:To view the kube-apiserver, enter the following query in the text box:

AzureDiagnostics
| where Category == "kube-apiserver"
| project log_s

可能返回了 API 服务器的多个日志。Many logs are likely returned for the API server. 若要缩小查询范围,以便查看上一步骤中创建的 NGINX pod 的相关日志,请额外添加一个 where 语句来搜索 pod/nginx,如以下示例查询中所示:To scope down the query to view the logs about the NGINX pod created in the previous step, add an additional where statement to search for pods/nginx as shown in the following example query:

AzureDiagnostics
| where Category == "kube-apiserver"
| where log_s contains "pods/nginx"
| project log_s

此时会显示 NGINX pod 的特定日志,如以下示例屏幕截图中所示:The specific logs for your NGINX pod are displayed, as shown in the following example screenshot:

示例 NGINX pod 的 Log Analytics 查询结果

若要查看其他日志,可将针对 Category 名称的查询更新为 kube-controller-managerkube-scheduler,具体取决于启用的其他日志。To view additional logs, you can update the query for the Category name to kube-controller-manager or kube-scheduler, depending on what additional logs you enable. 然后,可以使用附加的 where 语句来具体化要查找的事件。Additional where statements can then be used to refine the events you are looking for.

有关如何查询和筛选日志数据的详细信息,请参阅查看或分析使用 Log Analytics 日志搜索收集的数据For more information on how to query and filter your log data, see View or analyze data collected with log analytics log search.

日志事件架构Log event schema

为帮助分析日志数据,下表详细说明了用于每个事件的架构:To help analyze the log data, the following table details the schema used for each event:

字段名称Field name 说明Description
resourceIdresourceId 生成日志的 Azure 资源Azure resource that produced the log
timetime 上传日志的时间戳Timestamp of when the log was uploaded
categorycategory 生成日志的容器/组件的名称Name of container/component generating the log
operationNameoperationName Always Microsoft.ContainerService/managedClusters/diagnosticLogs/ReadAlways Microsoft.ContainerService/managedClusters/diagnosticLogs/Read
properties.logproperties.log 来自组件的日志的完整文本Full text of the log from the component
properties.streamproperties.stream stderrstdoutstderr or stdout
properties.podproperties.pod 日志的来源 pod 名称Pod name that the log came from
properties.containerIDproperties.containerID 此日志的来源 Docker 容器的 IDID of the docker container this log came from

日志角色Log Roles

角色Role 说明Description
aksServiceaksService 审核日志中控制平面操作的显示名称(来自 hcpService)The display name in audit log for the control plane operation (from the hcpService)
masterclientmasterclient 审核日志中 MasterClientCertificate(通过 az aks get-credentials 获得的证书)的显示名称The display name in audit log for MasterClientCertificate, the certificate you get from az aks get-credentials
nodeclientnodeclient 代理节点使用的 ClientCertificate 的显示名称The display name for ClientCertificate, which is used by agent nodes

后续步骤Next steps

本文已介绍如何启用和查看 AKS 群集中 Kubernetes 主组件的日志。In this article, you learned how to enable and review the logs for the Kubernetes master components in your AKS cluster. 若要进一步进行监视和故障排除,还可以查看 Kubelet 日志启用 SSH 节点访问To monitor and troubleshoot further, you can also view the Kubelet logs and enable SSH node access.