在 Azure HDInsight 中控制网络流量Control network traffic in Azure HDInsight

可以使用以下方法控制 Azure 虚拟网络中的网络流量:Network traffic in an Azure Virtual Networks can be controlled using the following methods:

  • 网络安全组 (NSG):用于筛选网络的入站和出站流量。Network security groups (NSG) allow you to filter inbound and outbound traffic to the network. 有关详细信息,请参阅使用网络安全组筛选网络流量文档。For more information, see the Filter network traffic with network security groups document.

  • 网络虚拟设备 (NVA) 只能用于出站流量。Network virtual appliances (NVA) can be used with outbound traffic only. NVA 可复制设备(如防火墙和路由器)的功能。NVAs replicate the functionality of devices such as firewalls and routers. 有关详细信息,请参阅网络设备文档。For more information, see the Network Appliances document.

作为托管服务,HDInsight 需要对 HDInsight 运行状况和管理服务具有不受限制的访问权限,以处理从 VNET 传入和传出的流量。As a managed service, HDInsight requires unrestricted access to the HDInsight health and management services both for incoming and outgoing traffic from the VNET. 使用 NSG 时,必须确保这些服务仍然可以与 HDInsight 群集进行通信。When using NSGs, you must ensure that these services can still communicate with HDInsight cluster.

在 Azure 自定义 VNET 中创建的 HDInsight 实体示意图

使用网络安全组的 HDInsightHDInsight with network security groups

如果计划使用网络安全组来控制网络流量,请在安装 HDInsight 之前执行以下操作:If you plan on using network security groups to control network traffic, perform the following actions before installing HDInsight:

  1. 确定计划用于 HDInsight 的 Azure 区域。Identify the Azure region that you plan to use for HDInsight.

  2. 确定 HDInsight 需要的用于你所在区域的服务标记。Identify the service tags required by HDInsight for your region. 有关详细信息,请参阅 Azure HDInsight 的网络安全组 (NSG) 服务标记For more information, see Network security group (NSG) service tags for Azure HDInsight.

  3. 为计划将 HDInsight 安装到其中的子网创建或修改网络安全组。Create or modify the network security groups for the subnet that you plan to install HDInsight into.

    • 网络安全组:在端口 443 上允许来自 IP 地址的入站流量。 Network security groups: allow inbound traffic on port 443 from the IP addresses. 这将确保 HDInsight 管理服务可以从虚拟网络外部访问群集。This will ensure that HDInsight management services can reach the cluster from outside the virtual network. 对于启用了 Kafka REST 代理的群集,还要在端口 9400 上允许入站流量 。For Kafka REST proxy enabled clusters, allow inbound traffic on port 9400 as well. 这将确保 Kafka REST 代理服务器可访问。This will ensure that Kafka REST proxy server is reachable.

有关网络安全组的详细信息,请参阅网络安全组概述For more information on network security groups, see the overview of network security groups.

控制 HDInsight 群集的出站流量Controlling outbound traffic from HDInsight clusters

有关控制 HDInsight 群集的出站流量的详细信息,请参阅配置 Azure HDInsight 群集的出站网络流量限制For more information on controlling outbound traffic from HDInsight clusters, see Configure outbound network traffic restriction for Azure HDInsight clusters.

到本地的强制隧道Forced tunneling to on-premises

强制隧道是用户定义的路由配置,其中来自子网的所有流量都强制流向特定网络或位置,例如你的本地网络或防火墙。Forced tunneling is a user-defined routing configuration where all traffic from a subnet is forced to a specific network or location, such as your on-premises network or Firewall. 因为数据传输量大且可能影响性能,所以在将所有数据传输回本地时,建议不要使用强制隧道__。Forced tunneling of all data transfer back to on-premise is not recommended due to large volumes of data transfer and potential performance impact.

有兴趣设置强制隧道的客户应使用自定义元存储,并设置从群集子网或本地网络到这些自定义元存储的适当连接。Customers who are interested to setup forced tunneling, should use custom metastores and setup the appropriate connectivity from the cluster subnet or on-premise network to these custom metastores.

若要查看有关使用 Azure 防火墙设置 UDR 的示例,请参阅配置 Azure HDInsight 群集的出站网络流量限制To see an example of the UDR setup with Azure Firewall, see Configure outbound network traffic restriction for Azure HDInsight clusters.

需要的 IP 地址Required IP addresses

如果使用网络安全组或用户定义的路由来控制流量,请参阅 HDInsight 管理 IP 地址If you use network security groups or user-defined routes to control traffic, see HDInsight management IP addresses.

必需端口Required ports

如果计划使用防火墙并在特定端口上从外部访问群集,则需要允许你的方案所需的那些端口上的流量。If you plan on using a firewall and access the cluster from outside on certain ports, you might need to allow traffic on those ports needed for your scenario. 默认情况下,只要允许上一部分中介绍的 Azure 管理流量在端口 443 上到达群集,就不需要特地将端口列入允许列表。By default, no special whitelisting of ports is needed as long as the Azure management traffic explained in the previous section is allowed to reach cluster on port 443.

对于特定服务的端口列表,请参阅 HDInsight 上的 Apache Hadoop 服务所用的端口文档。For a list of ports for specific services, see the Ports used by Apache Hadoop services on HDInsight document.

有关虚拟设备防火墙规则的详细信息,请参阅虚拟设备方案文档。For more information on firewall rules for virtual appliances, see the virtual appliance scenario document.

后续步骤Next steps