在 Azure VM 上安装 Elastic StackInstall the Elastic Stack on an Azure VM

本文逐步讲解如何在 Azure 中的 Ubuntu VM 上部署 ElasticsearchLogstashKibanaThis article walks you through how to deploy Elasticsearch, Logstash, and Kibana, on an Ubuntu VM in Azure. 要查看实际操作中的 Elastic Stack,可选择连接到 Kibana 并处理某些示例日志记录数据。To see the Elastic Stack in action, you can optionally connect to Kibana and work with some sample logging data.

本教程介绍如何执行下列操作:In this tutorial you learn how to:

  • 在 Azure 资源组中创建 Ubuntu VMCreate an Ubuntu VM in an Azure resource group
  • 在 VM 上安装 Elasticsearch、Logstash 和 KibanaInstall Elasticsearch, Logstash, and Kibana on the VM
  • 使用 Logstash 将示例数据发送到 ElasticsearchSend sample data to Elasticsearch with Logstash
  • 在 Kibana 控制台中打开端口并处理数据Open ports and work with data in the Kibana console

此部署适用于使用 Elastic Stack 进行基本开发的情况。This deployment is suitable for basic development with the Elastic Stack. 有关 Elastic 堆栈的详细信息,包括针对生产环境的建议,请参阅 Elastic 文档Azure 体系结构中心For more on the Elastic Stack, including recommendations for a production environment, see the Elastic documentation and the Azure Architecture Center.

Note

在 Azure China 中使用 Azure CLI 2.0 之前,请首先运行 az cloud set -n AzureChinaCloud 更改云环境。Before you can use Azure CLI 2.0 in Azure China, please run az cloud set -n AzureChinaCloud first to change the cloud environment. 如果要切换回全局 Azure,请再次运行 az cloud set -n AzureCloudIf you want to switch back to Global Azure, run az cloud set -n AzureCloud again.

如果选择在本地安装并使用 CLI,本教程要求运行 Azure CLI 2.0.4 或更高版本。If you choose to install and use the CLI locally, this tutorial requires that you are running the Azure CLI version 2.0.4 or later. 运行 az --version 即可查找版本。Run az --version to find the version. 如果需要进行安装或升级,请参阅安装 Azure CLIIf you need to install or upgrade, see Install Azure CLI.

创建资源组Create a resource group

使用 az group create 命令创建资源组。Create a resource group with the az group create command. Azure 资源组是在其中部署和管理 Azure 资源的逻辑容器。An Azure resource group is a logical container into which Azure resources are deployed and managed.

以下示例在“chinaeast”位置创建名为“myResourceGroup”的资源组。The following example creates a resource group named myResourceGroup in the chinaeast location.

az group create --name myResourceGroup --location chinaeast

创建虚拟机Create a virtual machine

使用 az vm create 命令创建 VM。Create a VM with the az vm create command.

下面的示例创建一个名为 myVM 的 VM,并且在默认密钥位置中不存在 SSH 密钥时创建这些密钥。The following example creates a VM named myVM and creates SSH keys if they do not already exist in a default key location. 若要使用特定的一组密钥,请使用 --ssh-key-value 选项。To use a specific set of keys, use the --ssh-key-value option.

az vm create \
    --resource-group myResourceGroup \
    --name myVM \
    --image UbuntuLTS \
    --admin-username azureuser \
    --generate-ssh-keys

创建 VM 后,Azure CLI 显示类似于以下示例的信息。When the VM has been created, the Azure CLI shows information similar to the following example. 记下 publicIpAddressTake note of the publicIpAddress. 此地址用于访问 VM。This address is used to access the VM.

{
  "fqdns": "",
  "id": "/subscriptions/<subscription ID>/resourceGroups/myResourceGroup/providers/Microsoft.Compute/virtualMachines/myVM",
  "location": "chinaeast",
  "macAddress": "00-0D-3A-23-9A-49",
  "powerState": "VM running",
  "privateIpAddress": "10.0.0.4",
  "publicIpAddress": "40.68.254.142",
  "resourceGroup": "myResourceGroup"
}

通过 SSH 连接到 VMSSH into your VM

如果还不知道 VM 的公共 IP 地址,请运行 az network public-ip list 命令:If you don't already know the public IP address of your VM, run the az network public-ip list command:

az network public-ip list --resource-group myResourceGroup --query [].ipAddress

使用以下命令创建与虚拟机的 SSH 会话。Use the following command to create an SSH session with the virtual machine. 替换为虚拟机的相应公共 IP 地址。Substitute the correct public IP address of your virtual machine. 在此示例中,IP 地址为 40.68.254.142In this example, the IP address is 40.68.254.142.

ssh azureuser@40.68.254.142

安装 Elastic StackInstall the Elastic Stack

导入 Elasticsearch 签名密钥并更新 APT 源列表,以包括 Elastic 包存储库:Import the Elasticsearch signing key and update your APT sources list to include the Elastic package repository:

wget -qO - https://artifacts.elastic.co/GPG-KEY-elasticsearch | sudo apt-key add -
echo "deb https://artifacts.elastic.co/packages/5.x/apt stable main" | sudo tee -a /etc/apt/sources.list.d/elastic-5.x.list

在 VM 上安装 Java Virtual 并配置 JAVA_HOME 变量 - 这是运行 Elastic Stack 组件所必需的。Install the Java Virtual on the VM and configure the JAVA_HOME variable-this is necessary for the Elastic Stack components to run.

sudo apt update && sudo apt install openjdk-8-jre-headless
export JAVA_HOME=/usr/lib/jvm/java-8-openjdk-amd64

运行以下命令更新 Ubuntu 包源并安装 Elasticsearch、Kibana 和 Logstash。Run the following commands to update Ubuntu package sources and install Elasticsearch, Kibana, and Logstash.

sudo apt update && sudo apt install elasticsearch kibana logstash   

Note

Elastic 文档中详述了安装说明,包括目录布局和初始配置Detailed installation instructions, including directory layouts and initial configuration, are maintained in Elastic's documentation

启动 ElasticsearchStart Elasticsearch

使用以下命令启动 VM 上的 Elasticsearch:Start Elasticsearch on your VM with the following command:

sudo systemctl start elasticsearch.service

此命令不会产生输出,因此请使用 curl 命令验证该 Elasticsearch 是否在 VM 上运行:This command produces no output, so verify that Elasticsearch is running on the VM with this curl command:

sudo curl -XGET 'localhost:9200/'

如果 Elasticsearch 在运行,则会看到如下所示的输出:If Elasticsearch is running, you see output like the following:

{
  "name" : "w6Z4NwR",
  "cluster_name" : "elasticsearch",
  "cluster_uuid" : "SDzCajBoSK2EkXmHvJVaDQ",
  "version" : {
    "number" : "5.6.3",
    "build_hash" : "1a2f265",
    "build_date" : "2017-10-06T20:33:39.012Z",
    "build_snapshot" : false,
    "lucene_version" : "6.6.1"
  },
  "tagline" : "You Know, for Search"
}

启动 Logstash 并将数据添加到 ElasticsearchStart Logstash and add data to Elasticsearch

使用以下命令启动 Logstash:Start Logstash with the following command:

sudo systemctl start logstash.service

在交互模式下测试 Logstash,确保它正确运行:Test Logstash in interactive mode to make sure it's working correctly:

sudo /usr/share/logstash/bin/logstash -e 'input { stdin { } } output { stdout {} }'

这是基本 logstash 管道,它将标准输入回响到标准输出。This is a basic logstash pipeline that echoes standard input to standard output.

The stdin plugin is now waiting for input:
hello azure
2017-10-11T20:01:08.904Z myVM hello azure

设置 Logstash,将内核消息从此 VM 转发到 Elasticsearch。Set up Logstash to forward the kernel messages from this VM to Elasticsearch. 在名为 vm-syslog-logstash.conf 的空目录中新建文件,并粘贴到以下 Logstash 配置中:Create a new file in an empty directory called vm-syslog-logstash.conf and paste in the following Logstash configuration:

input {
    stdin {
        type => "stdin-type"
    }

    file {
        type => "syslog"
        path => [ "/var/log/*.log", "/var/log/*/*.log", "/var/log/messages", "/var/log/syslog" ]
        start_position => "beginning"
    }
}

output {

    stdout {
        codec => rubydebug
    }
    elasticsearch {
        hosts  => "localhost:9200"
    }
}

测试此配置,然后将 syslog 数据发送到 Elasticsearch:Test this configuration and send the syslog data to Elasticsearch:

sudo /usr/share/logstash/bin/logstash -f vm-syslog-logstash.conf

将 syslog 项发送到 Elasticsearch 时回响的终端中会显示这些 syslog 项。You see the syslog entries in your terminal echoed as they are sent to Elasticsearch. 发送一些数据后,请使用 CTRL+C 退出 Logstash。Use CTRL+C to exit out of Logstash once you've sent some data.

启动 Kibana 并在 Elasticsearch 中直观显示数据Start Kibana and visualize the data in Elasticsearch

编辑 /etc/kibana/kibana.yml 并更改 Kibana 侦听的 IP 地址,以便通过 Web 浏览器对其进行访问。Edit /etc/kibana/kibana.yml and change the IP address Kibana listens on so you can access it from your web browser.

server.host:"0.0.0.0"

使用以下命令启动 Kibana:Start Kibana with the following command:

sudo systemctl start kibana.service

通过 Azure CLI 打开端口 5601,允许远程访问 Kibana 控制台:Open port 5601 from the Azure CLI to allow remote access to the Kibana console:

az vm open-port --port 5601 --resource-group myResourceGroup --name myVM

打开 Kibana 控制台并选择“创建”,根据之前发送给 Elasticsearch 的 syslog 数据生成默认索引。Open up the Kibana console and select Create to generate a default index based on the syslog data you sent to Elasticsearch earlier.

浏览 Kibana 中的 Syslog 事件

在 Kibana 控制台上选择“发现”,搜索、浏览和筛选 syslog 事件。Select Discover on the Kibana console to search, browse, and filter through the syslog events.

浏览 Kibana 中的 Syslog 事件

后续步骤Next steps

本教程已将 Elastic Stack 部署到 Azure 中的开发 VM。In this tutorial, you deployed the Elastic Stack into a development VM in Azure. 你已了解如何:You learned how to:

  • 在 Azure 资源组中创建 Ubuntu VMCreate an Ubuntu VM in an Azure resource group
  • 在 VM 上安装 Elasticsearch、Logstash 和 KibanaInstall Elasticsearch, Logstash, and Kibana on the VM
  • 通过 Logstash 将示例数据发送到 ElasticsearchSend sample data to Elasticsearch from Logstash
  • 在 Kibana 控制台中打开端口并处理数据Open ports and work with data in the Kibana console