启用和查看 Azure Kubernetes 服务 (AKS) 中 Kubernetes 主节点的日志Enable and review Kubernetes master node logs in Azure Kubernetes Service (AKS)

使用 Azure Kubernetes 服务 (AKS),可以提供 kube-apiserverkube-controller-manager 等主组件作为托管服务。With Azure Kubernetes Service (AKS), the master components such as the kube-apiserver and kube-controller-manager are provided as a managed service. 创建和管理运行 kubelet 与容器运行时的节点,并通过托管的 Kubernetes API 服务器部署应用程序。You create and manage the nodes that run the kubelet and container runtime, and deploy your applications through the managed Kubernetes API server. 为帮助排查应用程序和服务问题,可能需要查看这些主组件生成的日志。To help troubleshoot your application and services, you may need to view the logs generated by these master components. 本文介绍如何使用 Azure Monitor 日志从 Kubernetes 主组件启用和查询日志。This article shows you how to use Azure Monitor logs to enable and query the logs from the Kubernetes master components.

准备阶段Before you begin

本文要求在 Azure 帐户中运行一个现有的 AKS 群集。This article requires an existing AKS cluster running in your Azure account. 如果还没有 AKS 群集,请使用 Azure CLIAzure 门户创建一个。If you do not already have an AKS cluster, create one using the Azure CLI or Azure portal. Azure Monitor 日志适用于支持 RBAC 和不支持 RBAC 的 AKS 群集。Azure Monitor logs works with both RBAC and non-RBAC enabled AKS clusters.

启用资源日志Enable resource logs

为帮助收集和审查来自多个源的数据,Azure Monitor 日志提供了查询语言和分析引擎,该引擎可提供环境的见解。To help collect and review data from multiple sources, Azure Monitor logs provides a query language and analytics engine that provides insights to your environment. 工作区用于整理和分析数据,并可与 Application Insights 和安全中心等其他 Azure 服务集成。A workspace is used to collate and analyze the data, and can integrate with other Azure services such as Application Insights and Security Center. 若要使用不同的平台分析日志,可以选择将资源日志发送到 Azure 存储帐户或事件中心。To use a different platform to analyze the logs, you can instead choose to send resource logs to an Azure storage account or event hub. 有关详细信息,请参阅什么是 Azure Monitor 日志?For more information, see What is Azure Monitor logs?.

Azure Monitor 日志是在 Azure 门户中启用和管理的。Azure Monitor logs are enabled and managed in the Azure portal. 若要为 AKS 群集中的 Kubernetes 主组件启用日志收集,请在 Web 浏览器中打开 Azure 门户并完成以下步骤:To enable log collection for the Kubernetes master components in your AKS cluster, open the Azure portal in a web browser and complete the following steps:

  1. 选择 AKS 群集的资源组,例如 MC_myResourceGroup_myAKSCluster_chinaeast2。Select the resource group for your AKS cluster, such as MC_myResourceGroup_myAKSCluster_chinaeast2 .

  2. 在左侧选择“诊断设置”。On the left-hand side, choose Diagnostic settings .

  3. 选择资源组和 AKS 群集(例如,“myResourceGroup”和“myAKSCluster”),然后选择“添加诊断设置”。Select the resource group and your AKS cluster, such as myResourceGroup and myAKSCluster , then choose to Add diagnostic setting .

  4. 输入名称(例如 myAKSClusterLogs),然后选择“发送到 Log Analytics”选项。Enter a name, such as myAKSClusterLogs , then select the option to Send to Log Analytics .

  5. 选择现有工作区或者创建新的工作区。Select an existing workspace or create a new one. 如果创建工作区,请提供工作区名称、资源组和位置。If you create a workspace, provide a workspace name, a resource group, and a location.

  6. 在可用日志列表中,选择要启用的日志。In the list of available logs, select the logs you wish to enable. 对于本示例,请启用 kube-audit 和 kube-audit-admin 日志 。For this example, enable the kube-audit and kube-audit-admin logs. 常见日志包括 kube-apiserver、kube-controller-manager 和 kube-scheduler。Common logs include the kube-apiserver , kube-controller-manager , and kube-scheduler . 启用 Log Analytics 工作区后,可以返回并更改收集的日志。You can return and change the collected logs once Log Analytics workspaces are enabled.

  7. 准备就绪后,选择“保存”以启用收集选定日志。When ready, select Save to enable collection of the selected logs.

日志类别Log categories

除了 Kubernetes 编写的条目,项目的审核日志还包含 来自 AKS 的条目。In addition to entries written by Kubernetes, your project's audit logs also have entries from AKS.

审核日志记录为两种类别:kube-audit-admin 和 kube-audit 。Audit logs are recorded into two categories, kube-audit-admin and kube-audit . kube-audit 类别包含每个审核事件的所有审核日志数据,包括 get、list、create、update、delete、patch 和 post 。The kube-audit category contains all audit log data for every audit event, including get , list , create , update , delete , patch , and post .

kube-audit-admin 类别是 kube-audit 日志类别的子集 。The kube-audit-admin category is a subset of the kube-audit log category. kube-audit-admin 通过从日志中排除 get 和 list 审核事件,大大减少了日志数量 。kube-audit-admin reduces the number of logs significantly by excluding the get and list audit events from the log.

在 AKS 群集上计划测试 podSchedule a test pod on the AKS cluster

若要生成某些日志,请在 AKS 群集中创建新的 pod。To generate some logs, create a new pod in your AKS cluster. 以下示例 YAML 清单可用于创建一个基本的 NGINX 实例。The following example YAML manifest can be used to create a basic NGINX instance. 在所选的编辑器中创建名为 nginx.yaml 的文件,并在其中粘贴以下内容:Create a file named nginx.yaml in an editor of your choice and paste the following content:

apiVersion: v1
kind: Pod
metadata:
  name: nginx
spec:
  containers:
  - name: mypod
    image: dockerhub.azk8s.cn/library/nginx:1.15.5
    resources:
      requests:
        cpu: 100m
        memory: 128Mi
      limits:
        cpu: 250m
        memory: 256Mi
    ports:
    - containerPort: 80

使用 kubectl create 命令创建 Pod 并指定 YAML 文件,如以下示例所示:Create the pod with the kubectl create command and specify your YAML file, as shown in the following example:

$ kubectl create -f nginx.yaml

pod/nginx created

查看收集的日志View collected logs

可能需要等待几分钟,诊断日志才会启用并显示。It may take a few minutes for the diagnostics logs to be enabled and appear.

备注

如果需要将所有审核日志数据用于实现合规性或其他目的,请收集这些数据并将其存储在成本较低的存储(例如 blob 存储)中。If you need all audit log data for compliance or other purposes, collect and store it in inexpensive storage such as blob storage. 使用 kube-audit-admin 日志类别收集和保存有意义的审核日志数据集,以便进行监视和发出警报。Use the kube-audit-admin log category to collect and save a meaningful set of audit log data for monitoring and alerting purposes.

在 Azure 门户中导航到 AKS 群集,然后选择左侧的“日志”。In the Azure portal, navigate to your AKS cluster, and select Logs on the left-hand side. 关闭“示例查询”窗口(如果出现了此窗口)。Close the Example Queries window if it appears.

在左侧选择“日志”。On the left-hand side, choose Logs . 若要查看 kube-audit 日志,请在文本框中输入以下查询:To view the kube-audit logs, enter the following query in the text box:

AzureDiagnostics
| where Category == "kube-audit"
| project log_s

可能会返回多个日志。Many logs are likely returned. 若要缩小查询范围,以便查看上一步骤中创建的 NGINX pod 的相关日志,请额外添加一个 where 语句来搜索 nginx,如以下示例查询所示:To scope down the query to view the logs about the NGINX pod created in the previous step, add an additional where statement to search for nginx as shown in the following example query:

AzureDiagnostics
| where Category == "kube-audit"
| where log_s contains "nginx"
| project log_s

若要查看 kube-audit-admin 日志,请在文本框中输入以下查询:To view the kube-audit-admin logs, enter the following query in the text box:

AzureDiagnostics
| where Category == "kube-audit-admin"
| project log_s

在本例中,查询显示了 kube-audit-admin 中的所有创建作业。返回的结果可能很多,若要缩小查询范围,以便查看上一步创建的 NGINX Pod 的相关日志,请另外添加一个 where 语句来搜索 nginx,如以下示例查询所示 。In this example, the query shows all create jobs in kube-audit-admin . There are likely many results returned, to scope down the query to view the logs about the NGINX pod created in the previous step, add an additional where statement to search for nginx as shown in the following example query.

AzureDiagnostics
| where Category == "kube-audit-admin"
| where log_s contains "nginx"
| project log_s

有关如何查询和筛选日志数据的详细信息,请参阅查看或分析使用 Log Analytics 日志搜索收集的数据For more information on how to query and filter your log data, see View or analyze data collected with log analytics log search.

日志事件架构Log event schema

AKS 记录以下事件:AKS logs the following events:

日志角色Log Roles

角色Role 说明Description
aksServiceaksService 审核日志中控制平面操作的显示名称(来自 hcpService)The display name in audit log for the control plane operation (from the hcpService)
masterclientmasterclient 审核日志中 MasterClientCertificate(通过 az aks get-credentials 获得的证书)的显示名称The display name in audit log for MasterClientCertificate, the certificate you get from az aks get-credentials
nodeclientnodeclient 代理节点使用的 ClientCertificate 的显示名称The display name for ClientCertificate, which is used by agent nodes

后续步骤Next steps

本文已介绍如何启用和查看 AKS 群集中 Kubernetes 主组件的日志。In this article, you learned how to enable and review the logs for the Kubernetes master components in your AKS cluster. 若要进一步进行监视和故障排除,还可以查看 Kubelet 日志启用 SSH 节点访问To monitor and troubleshoot further, you can also view the Kubelet logs and enable SSH node access.