Service Fabric networking patterns
You can integrate your Azure Service Fabric cluster with other Azure networking features. In this article, we show you how to create clusters that use the following features:
- Existing virtual network or subnet
- Static public IP address
- Internal-only load balancer
- Internal and external load balancer
Service Fabric runs in a standard virtual machine scale set. Any functionality that you can use in a virtual machine scale set, you can use with a Service Fabric cluster. The networking sections of the Azure Resource Manager templates for virtual machine scale sets and Service Fabric are identical. After you deploy to an existing virtual network, it's easy to incorporate other networking features, like Azure ExpressRoute, Azure VPN Gateway, a network security group, and virtual network peering.
Allowing the Service Fabric resource provider to query your cluster
Service Fabric is unique from other networking features in one aspect. The Azure portal internally uses the Service Fabric resource provider to call to a cluster to get information about nodes and applications. The Service Fabric resource provider requires publicly accessible inbound access to the HTTP gateway port (port 19080, by default) on the management endpoint. Service Fabric Explorer uses the management endpoint to manage your cluster. The Service Fabric resource provider also uses this port to query information about your cluster, to display in the Azure portal.
If port 19080 is not accessible from the Service Fabric resource provider, a message like Nodes Not Found appears in the portal, and your node and application list appears empty. If you want to see your cluster in the Azure portal, your load balancer must expose a public IP address, and your network security group must allow incoming port 19080 traffic. If your setup does not meet these requirements, the Azure portal does not display the status of your cluster.
Note
We recommend that you use the Azure Az PowerShell module to interact with Azure. To get started, see Install Azure PowerShell. To learn how to migrate to the Az PowerShell module, see Migrate Azure PowerShell from AzureRM to Az.
Templates
All Service Fabric templates are in GitHub. You should be able to deploy the templates as-is by using the following PowerShell commands. If you are deploying the existing Azure Virtual Network template or the static public IP template, first read the Initial setup section of this article.
Initial setup
Existing virtual network
In the following example, we start with an existing virtual network named ExistingRG-vnet, in the ExistingRG resource group. The subnet is named default. These default resources are created when you use the Azure portal to create a standard virtual machine (VM). You could create the virtual network and subnet without creating the VM, but the main goal of adding a cluster to an existing virtual network is to provide network connectivity to other VMs. Creating the VM gives a good example of how an existing virtual network typically is used. If your Service Fabric cluster uses only an internal load balancer, without a public IP address, you can use the VM and its public IP as a secure jump box.
Static public IP address
A static public IP address generally is a dedicated resource that's managed separately from the VM or VMs it's assigned to. It's provisioned in a dedicated networking resource group (as opposed to in the Service Fabric cluster resource group itself). Create a static public IP address named staticIP1 in the same ExistingRG resource group, either in the Azure portal or by using PowerShell:
PS C:\Users\user> New-AzPublicIpAddress -Name staticIP1 -ResourceGroupName ExistingRG -Location chinanorth -AllocationMethod Static -DomainNameLabel sfnetworking
Name : staticIP1
ResourceGroupName : ExistingRG
Location : chinanorth
Id : /subscriptions/1237f4d2-3dce-1236-ad95-123f764e7123/resourceGroups/ExistingRG/providers/Microsoft.Network/publicIPAddresses/staticIP1
Etag : W/"fc8b0c77-1f84-455d-9930-0404ebba1b64"
ResourceGuid : 77c26c06-c0ae-496c-9231-b1a114e08824
ProvisioningState : Succeeded
Tags :
PublicIpAllocationMethod : Static
IpAddress : 40.83.182.110
PublicIpAddressVersion : IPv4
IdleTimeoutInMinutes : 4
IpConfiguration : null
DnsSettings : {
"DomainNameLabel": "sfnetworking",
"Fqdn": "sfnetworking.chinanorth.cloudapp.chinacloudapi.cn"
}
Service Fabric template
In the examples in this article, we use the Service Fabric template.json. You can use the standard portal wizard to download the template from the portal before you create a cluster. You also can use one of the sample templates, like the secure five-node Service Fabric cluster.
Existing virtual network or subnet
Change the subnet parameter to the name of the existing subnet, and then add two new parameters to reference the existing virtual network:
"subnet0Name": { "type": "string", "defaultValue": "default" }, "existingVNetRGName": { "type": "string", "defaultValue": "ExistingRG" }, "existingVNetName": { "type": "string", "defaultValue": "ExistingRG-vnet" }, /* "subnet0Name": { "type": "string", "defaultValue": "Subnet-0" }, "subnet0Prefix": { "type": "string", "defaultValue": "10.0.0.0/24" },*/You also can comment out the parameter with the name "virtualNetworkName" so that it won't prompt you to enter the virtual network name twice in the cluster deployment blade in the Azure portal.
Comment out
nicPrefixOverrideattribute ofMicrosoft.Compute/virtualMachineScaleSets, because you are using existing subnet and you have disabled this variable in step 1./*"nicPrefixOverride": "[parameters('subnet0Prefix')]",*/Change the
vnetIDvariable to point to the existing virtual network:/*old "vnetID": "[resourceId('Microsoft.Network/virtualNetworks',parameters('virtualNetworkName'))]",*/ "vnetID": "[concat('/subscriptions/', subscription().subscriptionId, '/resourceGroups/', parameters('existingVNetRGName'), '/providers/Microsoft.Network/virtualNetworks/', parameters('existingVNetName'))]",Remove
Microsoft.Network/virtualNetworksfrom your resources, so Azure does not create a new virtual network:/*{ "apiVersion": "[variables('vNetApiVersion')]", "type": "Microsoft.Network/virtualNetworks", "name": "[parameters('virtualNetworkName')]", "location": "[parameters('computeLocation')]", "properties": { "addressSpace": { "addressPrefixes": [ "[parameters('addressPrefix')]" ] }, "subnets": [ { "name": "[parameters('subnet0Name')]", "properties": { "addressPrefix": "[parameters('subnet0Prefix')]" } } ] }, "tags": { "resourceType": "Service Fabric", "clusterName": "[parameters('clusterName')]" } },*/Comment out the virtual network from the
dependsOnattribute ofMicrosoft.Compute/virtualMachineScaleSets, so you don't depend on creating a new virtual network:"apiVersion": "[variables('vmssApiVersion')]", "type": "Microsoft.Computer/virtualMachineScaleSets", "name": "[parameters('vmNodeType0Name')]", "location": "[parameters('computeLocation')]", "dependsOn": [ /*"[concat('Microsoft.Network/virtualNetworks/', parameters('virtualNetworkName'))]", */ "[Concat('Microsoft.Storage/storageAccounts/', variables('uniqueStringArray0')[0])]",Deploy the template:
New-AzResourceGroup -Name sfnetworkingexistingvnet -Location chinanorth New-AzResourceGroupDeployment -Name deployment -ResourceGroupName sfnetworkingexistingvnet -TemplateFile C:\SFSamples\Final\template\_existingvnet.jsonAfter deployment, your virtual network should include the new scale set VMs. The virtual machine scale set node type should show the existing virtual network and subnet. You also can use Remote Desktop Protocol (RDP) to access the VM that was already in the virtual network, and to ping the new scale set VMs:
C:>\Users\users>ping 10.0.0.5 -n 1 C:>\Users\users>ping NOde1000000 -n 1
For another example, see one that is not specific to Service Fabric.
Static public IP address
Add parameters for the name of the existing static IP resource group, name, and fully qualified domain name (FQDN):
"existingStaticIPResourceGroup": { "type": "string" }, "existingStaticIPName": { "type": "string" }, "existingStaticIPDnsFQDN": { "type": "string" }Remove the
dnsNameparameter. (The static IP address already has one.)/* "dnsName": { "type": "string" }, */Add a variable to reference the existing static IP address:
"existingStaticIP": "[concat('/subscriptions/', subscription().subscriptionId, '/resourceGroups/', parameters('existingStaticIPResourceGroup'), '/providers/Microsoft.Network/publicIPAddresses/', parameters('existingStaticIPName'))]",Remove
Microsoft.Network/publicIPAddressesfrom your resources, so Azure does not create a new IP address:/* { "apiVersion": "[variables('publicIPApiVersion')]", "type": "Microsoft.Network/publicIPAddresses", "name": "[concat(parameters('lbIPName'),)'-', '0')]", "location": "[parameters('computeLocation')]", "properties": { "dnsSettings": { "domainNameLabel": "[parameters('dnsName')]" }, "publicIPAllocationMethod": "Dynamic" }, "tags": { "resourceType": "Service Fabric", "clusterName": "[parameters('clusterName')]" } }, */Comment out the IP address from the
dependsOnattribute ofMicrosoft.Network/loadBalancers, so you don't depend on creating a new IP address:"apiVersion": "[variables('lbIPApiVersion')]", "type": "Microsoft.Network/loadBalancers", "name": "[concat('LB', '-', parameters('clusterName'), '-', parameters('vmNodeType0Name'))]", "location": "[parameters('computeLocation')]", /* "dependsOn": [ "[concat('Microsoft.Network/publicIPAddresses/', concat(parameters('lbIPName'), '-', '0'))]" ], */ "properties": {In the
Microsoft.Network/loadBalancersresource, change thepublicIPAddresselement offrontendIPConfigurationsto reference the existing static IP address instead of a newly created one:"frontendIPConfigurations": [ { "name": "LoadBalancerIPConfig", "properties": { "publicIPAddress": { /*"id": "[resourceId('Microsoft.Network/publicIPAddresses',concat(parameters('lbIPName'),'-','0'))]"*/ "id": "[variables('existingStaticIP')]" } } } ],In the
Microsoft.ServiceFabric/clustersresource, changemanagementEndpointto the DNS FQDN of the static IP address. If you are using a secure cluster, make sure you change http:// to https://. (Note that this step applies only to Service Fabric clusters. If you are using a virtual machine scale set, skip this step.)"fabricSettings": [], /*"managementEndpoint": "[concat('http://',reference(concat(parameters('lbIPName'),'-','0')).dnsSettings.fqdn,':',parameters('nt0fabricHttpGatewayPort'))]",*/ "managementEndpoint": "[concat('http://',parameters('existingStaticIPDnsFQDN'),':',parameters('nt0fabricHttpGatewayPort'))]",Deploy the template:
New-AzResourceGroup -Name sfnetworkingstaticip -Location chinanorth $staticip = Get-AzPublicIpAddress -Name staticIP1 -ResourceGroupName ExistingRG $staticip New-AzResourceGroupDeployment -Name deployment -ResourceGroupName sfnetworkingstaticip -TemplateFile C:\SFSamples\Final\template\_staticip.json -existingStaticIPResourceGroup $staticip.ResourceGroupName -existingStaticIPName $staticip.Name -existingStaticIPDnsFQDN $staticip.DnsSettings.Fqdn
After deployment, you can see that your load balancer is bound to the public static IP address from the other resource group. The Service Fabric client connection endpoint and Service Fabric Explorer endpoint point to the DNS FQDN of the static IP address.
Internal-only load balancer
This scenario replaces the external load balancer in the default Service Fabric template with an internal-only load balancer. See earlier in the article for implications for the Azure portal and for the Service Fabric resource provider.
Remove the
dnsNameparameter. (It's not needed.)/* "dnsName": { "type": "string" }, */Optionally, if you use a static allocation method, you can add a static IP address parameter. If you use a dynamic allocation method, you do not need to do this step.
"internalLBAddress": { "type": "string", "defaultValue": "10.0.0.250" }Remove
Microsoft.Network/publicIPAddressesfrom your resources, so Azure does not create a new IP address:/* { "apiVersion": "[variables('publicIPApiVersion')]", "type": "Microsoft.Network/publicIPAddresses", "name": "[concat(parameters('lbIPName'),)'-', '0')]", "location": "[parameters('computeLocation')]", "properties": { "dnsSettings": { "domainNameLabel": "[parameters('dnsName')]" }, "publicIPAllocationMethod": "Dynamic" }, "tags": { "resourceType": "Service Fabric", "clusterName": "[parameters('clusterName')]" } }, */Remove the IP address
dependsOnattribute ofMicrosoft.Network/loadBalancers, so you don't depend on creating a new IP address. Add the virtual networkdependsOnattribute because the load balancer now depends on the subnet from the virtual network:"apiVersion": "[variables('lbApiVersion')]", "type": "Microsoft.Network/loadBalancers", "name": "[concat('LB','-', parameters('clusterName'),'-',parameters('vmNodeType0Name'))]", "location": "[parameters('computeLocation')]", "dependsOn": [ /*"[concat('Microsoft.Network/publicIPAddresses/',concat(parameters('lbIPName'),'-','0'))]"*/ "[concat('Microsoft.Network/virtualNetworks/',parameters('virtualNetworkName'))]" ],Change the load balancer's
frontendIPConfigurationssetting from using apublicIPAddress, to using a subnet andprivateIPAddress.privateIPAddressuses a predefined static internal IP address. To use a dynamic IP address, remove theprivateIPAddresselement, and then changeprivateIPAllocationMethodto Dynamic."frontendIPConfigurations": [ { "name": "LoadBalancerIPConfig", "properties": { /* "publicIPAddress": { "id": "[resourceId('Microsoft.Network/publicIPAddresses',concat(parameters('lbIPName'),'-','0'))]" } */ "subnet" :{ "id": "[variables('subnet0Ref')]" }, "privateIPAddress": "[parameters('internalLBAddress')]", "privateIPAllocationMethod": "Static" } } ],In the
Microsoft.ServiceFabric/clustersresource, changemanagementEndpointto point to the internal load balancer address. If you use a secure cluster, make sure you change http:// to https://. (Note that this step applies only to Service Fabric clusters. If you are using a virtual machine scale set, skip this step.)"fabricSettings": [], /*"managementEndpoint": "[concat('http://',reference(concat(parameters('lbIPName'),'-','0')).dnsSettings.fqdn,':',parameters('nt0fabricHttpGatewayPort'))]",*/ "managementEndpoint": "[concat('http://',reference(variables('lbID0')).frontEndIPConfigurations[0].properties.privateIPAddress,':',parameters('nt0fabricHttpGatewayPort'))]",Deploy the template:
New-AzResourceGroup -Name sfnetworkinginternallb -Location chinanorth New-AzResourceGroupDeployment -Name deployment -ResourceGroupName sfnetworkinginternallb -TemplateFile C:\SFSamples\Final\template\_internalonlyLB.json
After deployment, your load balancer uses the private static 10.0.0.250 IP address. If you have another machine in that same virtual network, you can go to the internal Service Fabric Explorer endpoint. Note that it connects to one of the nodes behind the load balancer.
Internal and external load balancer
In this scenario, you start with the existing single-node type external load balancer, and add an internal load balancer for the same node type. A back-end port attached to a back-end address pool can be assigned only to a single load balancer. Choose which load balancer will have your application ports, and which load balancer will have your management endpoints (ports 19000 and 19080). If you put the management endpoints on the internal load balancer, keep in mind the Service Fabric resource provider restrictions discussed earlier in the article. In the example we use, the management endpoints remain on the external load balancer. You also add a port 80 application port, and place it on the internal load balancer.
In a two-node-type cluster, one node type is on the external load balancer. The other node type is on the internal load balancer. To use a two-node-type cluster, in the portal-created two-node-type template (which comes with two load balancers), switch the second load balancer to an internal load balancer. For more information, see the Internal-only load balancer section.
Add the static internal load balancer IP address parameter. (For notes related to using a dynamic IP address, see earlier sections of this article.)
"internalLBAddress": { "type": "string", "defaultValue": "10.0.0.250" }Add an application port 80 parameter.
To add internal versions of the existing networking variables, copy and paste them, and add "-Int" to the name:
/* Add internal load balancer networking variables */ "lbID0-Int": "[resourceId('Microsoft.Network/loadBalancers', concat('LB','-', parameters('clusterName'),'-',parameters('vmNodeType0Name'), '-Internal'))]", "lbIPConfig0-Int": "[concat(variables('lbID0-Int'),'/frontendIPConfigurations/LoadBalancerIPConfig')]", "lbPoolID0-Int": "[concat(variables('lbID0-Int'),'/backendAddressPools/LoadBalancerBEAddressPool')]", "lbProbeID0-Int": "[concat(variables('lbID0-Int'),'/probes/FabricGatewayProbe')]", "lbHttpProbeID0-Int": "[concat(variables('lbID0-Int'),'/probes/FabricHttpGatewayProbe')]", "lbNatPoolID0-Int": "[concat(variables('lbID0-Int'),'/inboundNatPools/LoadBalancerBEAddressNatPool')]", /* Internal load balancer networking variables end */If you start with the portal-generated template that uses application port 80, the default portal template adds AppPort1 (port 80) on the external load balancer. In this case, remove AppPort1 from the external load balancer
loadBalancingRulesand probes, so you can add it to the internal load balancer:"loadBalancingRules": [ { "name": "LBHttpRule", "properties":{ "backendAddressPool": { "id": "[variables('lbPoolID0')]" }, "backendPort": "[parameters('nt0fabricHttpGatewayPort')]", "enableFloatingIP": "false", "frontendIPConfiguration": { "id": "[variables('lbIPConfig0')]" }, "frontendPort": "[parameters('nt0fabricHttpGatewayPort')]", "idleTimeoutInMinutes": "5", "probe": { "id": "[variables('lbHttpProbeID0')]" }, "protocol": "tcp" } } /* Remove AppPort1 from the external load balancer. { "name": "AppPortLBRule1", "properties": { "backendAddressPool": { "id": "[variables('lbPoolID0')]" }, "backendPort": "[parameters('loadBalancedAppPort1')]", "enableFloatingIP": "false", "frontendIPConfiguration": { "id": "[variables('lbIPConfig0')]" }, "frontendPort": "[parameters('loadBalancedAppPort1')]", "idleTimeoutInMinutes": "5", "probe": { "id": "[concate(variables('lbID0'), '/probes/AppPortProbe1')]" }, "protocol": "tcp" } }*/ ], "probes": [ { "name": "FabricGatewayProbe", "properties": { "intervalInSeconds": 5, "numberOfProbes": 2, "port": "[parameters('nt0fabricTcpGatewayPort')]", "protocol": "tcp" } }, { "name": "FabricHttpGatewayProbe", "properties": { "intervalInSeconds": 5, "numberOfProbes": 2, "port": "[parameters('nt0fabricHttpGatewayPort')]", "protocol": "tcp" } } /* Remove AppPort1 from the external load balancer. { "name": "AppPortProbe1", "properties": { "intervalInSeconds": 5, "numberOfProbes": 2, "port": "[parameters('loadBalancedAppPort1')]", "protocol": "tcp" } } */ ], "inboundNatPools": [Add a second
Microsoft.Network/loadBalancersresource. It looks similar to the internal load balancer created in the Internal-only load balancer section, but it uses the "-Int" load balancer variables, and implements only the application port 80. This also removesinboundNatPools, to keep RDP endpoints on the public load balancer. If you want RDP on the internal load balancer, moveinboundNatPoolsfrom the external load balancer to this internal load balancer:/* Add a second load balancer, configured with a static privateIPAddress and the "-Int" load balancer variables. */ { "apiVersion": "[variables('lbApiVersion')]", "type": "Microsoft.Network/loadBalancers", /* Add "-Internal" to the name. */ "name": "[concat('LB','-', parameters('clusterName'),'-',parameters('vmNodeType0Name'), '-Internal')]", "location": "[parameters('computeLocation')]", "dependsOn": [ /* Remove public IP dependsOn, add vnet dependsOn "[concat('Microsoft.Network/publicIPAddresses/',concat(parameters('lbIPName'),'-','0'))]" */ "[concat('Microsoft.Network/virtualNetworks/',parameters('virtualNetworkName'))]" ], "properties": { "frontendIPConfigurations": [ { "name": "LoadBalancerIPConfig", "properties": { /* Switch from Public to Private IP address */ "publicIPAddress": { "id": "[resourceId('Microsoft.Network/publicIPAddresses',concat(parameters('lbIPName'),'-','0'))]" } */ "subnet" :{ "id": "[variables('subnet0Ref')]" }, "privateIPAddress": "[parameters('internalLBAddress')]", "privateIPAllocationMethod": "Static" } } ], "backendAddressPools": [ { "name": "LoadBalancerBEAddressPool", "properties": {} } ], "loadBalancingRules": [ /* Add the AppPort rule. Be sure to reference the "-Int" versions of backendAddressPool, frontendIPConfiguration, and the probe variables. */ { "name": "AppPortLBRule1", "properties": { "backendAddressPool": { "id": "[variables('lbPoolID0-Int')]" }, "backendPort": "[parameters('loadBalancedAppPort1')]", "enableFloatingIP": "false", "frontendIPConfiguration": { "id": "[variables('lbIPConfig0-Int')]" }, "frontendPort": "[parameters('loadBalancedAppPort1')]", "idleTimeoutInMinutes": "5", "probe": { "id": "[concat(variables('lbID0-Int'),'/probes/AppPortProbe1')]" }, "protocol": "tcp" } } ], "probes": [ /* Add the probe for the app port. */ { "name": "AppPortProbe1", "properties": { "intervalInSeconds": 5, "numberOfProbes": 2, "port": "[parameters('loadBalancedAppPort1')]", "protocol": "tcp" } } ], "inboundNatPools": [ ] }, "tags": { "resourceType": "Service Fabric", "clusterName": "[parameters('clusterName')]" } },In
networkProfilefor theMicrosoft.Compute/virtualMachineScaleSetsresource, add the internal back-end address pool:"loadBalancerBackendAddressPools": [ { "id": "[variables('lbPoolID0')]" }, { /* Add internal BE pool */ "id": "[variables('lbPoolID0-Int')]" } ],Deploy the template:
New-AzResourceGroup -Name sfnetworkinginternalexternallb -Location chinanorth New-AzResourceGroupDeployment -Name deployment -ResourceGroupName sfnetworkinginternalexternallb -TemplateFile C:\SFSamples\Final\template\_internalexternalLB.json
After deployment, you can see two load balancers in the resource group. If you browse the load balancers, you can see the public IP address and management endpoints (ports 19000 and 19080) assigned to the public IP address. You also can see the static internal IP address and application endpoint (port 80) assigned to the internal load balancer. Both load balancers use the same virtual machine scale set back-end pool.
Notes for production workloads
The above GitHub templates are designed to work with the default SKU for Azure Standard Load Balancer (SLB), the Standard SKU. Any Service Fabric cluster using the Standard SKU for SLB needs to ensure that each node type has a rule allowing outbound traffic on port 443. This is necessary to complete cluster setup, and any deployment without such a rule will fail. In the above example of an "internal only" load balancer, an additional external load balancer must be added to the template with a rule allowing outbound traffic for port 443.