在 Azure 中添加或删除 Service Fabric 群集的证书Add or remove certificates for a Service Fabric cluster in Azure

建议先了解 Service Fabric 使用 X.509 证书的方式,并熟悉群集安全性应用场景It is recommended that you familiarize yourself with how Service Fabric uses X.509 certificates and be familiar with the Cluster security scenarios. 在继续下一步之前,必须先了解群集证书的定义和用途。You must understand what a cluster certificate is and what is used for, before you proceed further.

Azure Service Fabrics SDK 的默认证书加载行为是部署和使用过期日期最远的已定义证书,而不管其主要或次要配置定义如何。Azure Service Fabrics SDK's default certificate load behavior is to deploy and use the defined certificate with expiry date furthest into the future; regardless of their primary or secondary configuration definition. 回退到经典行为是非推荐的高级操作,需要在 Fabric.Code 配置内将“UseSecondaryIfNewer”设置参数的值设置为 false。Falling back to the classic behavior is a not recommended advanced action, and requires setting the "UseSecondaryIfNewer" setting parameter value to false within your Fabric.Code configuration.

在创建群集期间配置证书安全性时,Service Fabric 允许指定两个群集证书(主要证书和辅助证书)以及客户端证书。Service fabric lets you specify two cluster certificates, a primary and a secondary, when you configure certificate security during cluster creation, in addition to client certificates. 请参阅通过门户创建 Azure 群集通过 Azure Resource Manager 创建 Azure 群集,了解在创建时进行相关设置的详细信息。Refer to creating an azure cluster via portal or creating an azure cluster via Azure Resource Manager for details on setting them up at create time. 如果在创建时只指定了一个群集证书,该证书会用作主证书。If you specify only one cluster certificate at create time, then that is used as the primary certificate. 在创建群集后,可以添加一个新证书作为辅助证书。After cluster creation, you can add a new certificate as a secondary.

备注

对于安全群集,始终至少需要部署一个有效的(未吊销或过期)群集证书(主证书或辅助证书),否则,群集无法正常运行。For a secure cluster, you will always need at least one valid (not revoked and not expired) cluster certificate (primary or secondary) deployed (if not, the cluster stops functioning). 在所有有效证书过期前的 90 天,系统针对节点生成警告跟踪和警告运行状况事件。90 days before all valid certificates reach expiration, the system generates a warning trace and also a warning health event on the node. Service Fabric 当前不会针对此文发送电子邮件或其他任何通知。There is currently no email or any other notification that Service Fabric sends out on this article.

备注

本文进行了更新,以便使用新的 Azure PowerShell Az 模块。This article has been updated to use the new Azure PowerShell Az module. 你仍然可以使用 AzureRM 模块,至少在 2020 年 12 月之前,它将继续接收 bug 修补程序。You can still use the AzureRM module, which will continue to receive bug fixes until at least December 2020. 若要详细了解新的 Az 模块和 AzureRM 兼容性,请参阅新 Azure Powershell Az 模块简介To learn more about the new Az module and AzureRM compatibility, see Introducing the new Azure PowerShell Az module. 有关 Az 模块安装说明,请参阅安装 Azure PowerShellFor Az module installation instructions, see Install Azure PowerShell.

使用门户添加辅助群集证书Add a secondary cluster certificate using the portal

无法通过 Azure 门户使用 Azure powershell 添加辅助群集证书。Secondary cluster certificate cannot be added through the Azure portal, use Azure powershell. 本文档稍后将概述该过程。The process is outlined later in this document.

使用门户删除群集证书Remove a cluster certificate using the portal

对安全群集,始终需要至少一个有效(未撤销且未过期)证书。For a secure cluster, you will always need at least one valid (not revoked and not expired) certificate. 将使用具有最远过期日期的已部署证书,并且删除该证书会导致群集停止运行;请确保仅删除过期的证书或最快过期的未使用证书。The certificate deployed with the furthest into the future expiring date will be in use, and removing it will make your cluster stop functioning; ensure to only remove the certificate that is expired, or an unused certificate that expires the soonest.

若要删除未使用的群集安全证书,请导航到“安全性”部分,然后在该未使用证书的上下文菜单中选择“删除”选项。To remove an unused cluster security certificate, Navigate to the Security section and select the 'Delete' option from the context menu on the unused certificate.

若要删除标记为“主要”的证书,则需部署一个过期日期比该主要证书更远的辅助证书,从而启用自动变换行为;并在自动变换完成后该删除主要证书。If your intent is to remove the certificate that is marked primary, then you will need to deploy a secondary certificate with an expiring date further into the future than the primary certificate, enabling the auto rollover behavior; delete the primary certificate after the auto rollover has completed.

使用 Resource Manager Powershell 添加辅助证书Add a secondary certificate using Resource Manager Powershell

提示

现在有一种更好、更简单的方法来使用 Add-AzServiceFabricClusterCertificate cmdlet 添加辅助证书。There is now a better and easier way to add a secondary certificate using the Add-AzServiceFabricClusterCertificate cmdlet. 无需执行本部分中的其余步骤。You don't need to follow the rest of the steps in this section. 此外,使用 Add-AzServiceFabricClusterCertificate cmdlet 时,不需要使用最初用来创建和部署群集的模板。Also, you do not need the template originally used to create and deploy the cluster when using the Add-AzServiceFabricClusterCertificate cmdlet.

执行这些步骤的前提是,熟悉资源管理器的工作原理,并已使用资源管理器模板至少部署了一个 Service Fabric 群集,同时已准备好在设置此群集时使用的模板。These steps assume that you are familiar with how Resource Manager works and have deployed at least one Service Fabric cluster using a Resource Manager template, and have the template that you used to set up the cluster handy. 此外,还有一个前提就是,可以熟练使用 JSON。It is also assumed that you are comfortable using JSON.

备注

如需可参考或入手的示例模板和参数,请从此 git-repo 下载。If you are looking for a sample template and parameters that you can use to follow along or as a starting point, then download it from this git-repo.

备注

必须修改从 GitHub 存储库“Azure-Samples”下载或引用的模板,使之适应 Azure 中国云环境。Templates you downloaded or referenced from the GitHub Repo "Azure-Samples" must be modified in order to fit in the Azure China Cloud Environment. 例如,替换某些终结点(将“blob.core.windows.net”替换为“blob.core.chinacloudapi.cn”,将“cloudapp.azure.com”替换为“chinacloudapp.cn”);必要时更改某些不受支持的位置、VM 映像、VM 大小、SKU 以及资源提供程序的 API 版本。For example, replace some endpoints -- "blob.core.windows.net" by "blob.core.chinacloudapi.cn", "cloudapp.azure.com" by "chinacloudapp.cn"; change some unsupported Location, VM images, VM sizes, SKU and resource-provider's API Version when necessary.

编辑 Resource Manager 模板Edit your Resource Manager template

为了便于参考,示例 5-VM-1-NodeTypes-Secure_Step2.JSON 包含我们要进行的所有编辑。For ease of following along, sample 5-VM-1-NodeTypes-Secure_Step2.JSON contains all the edits we will be making. 该示例位于 git-repothe sample is available at git-repo.

请确保执行所有步骤Make sure to follow all the steps

  1. 打开用于部署群集的 Resource Manager 模板。Open up the Resource Manager template you used to deploy you Cluster. (如果已从上述存储库下载此示例,则使用 5-VM-1-NodeTypes-Secure_Step1.JSON 部署安全群集,并打开该模板)。(If you have downloaded the sample from the preceding repo, then use 5-VM-1-NodeTypes-Secure_Step1.JSON to deploy a secure cluster and then open up that template).

  2. 向模板的参数部分添加两个新参数“secCertificateThumbprint”和“secCertificateUrlValue”,类型为“string”。Add two new parameters "secCertificateThumbprint" and "secCertificateUrlValue" of type "string" to the parameter section of your template. 可复制以下代码片段,并将其添加到该模板。You can copy the following code snippet and add it to the template. 根据具体的模板源,可能已经存在这些定义,如果是这样,请转至下一步。Depending on the source of your template, you may already have these defined, if so move to the next step.

    "secCertificateThumbprint": {
      "type": "string",
      "metadata": {
        "description": "Certificate Thumbprint"
      }
    },
    "secCertificateUrlValue": {
      "type": "string",
      "metadata": {
        "description": "Refers to the location URL in your key vault where the certificate was uploaded, it is should be in the format of https://<name of the vault>.vault.azure.cn:443/secrets/<exact location>"
      }
    },
    
  3. Microsoft.ServiceFabric/clusters 资源进行更改 - 在模板中找到“Microsoft.ServiceFabric/clusters”资源定义。Make changes to the Microsoft.ServiceFabric/clusters resource - Locate the "Microsoft.ServiceFabric/clusters" resource definition in your template. 在该定义的属性下,找到“Certificate”JSON 标记,如以下 JSON 代码片段所示:Under properties of that definition, you will find "Certificate" JSON tag, which should look something like the following JSON snippet:

    "properties": {
    "certificate": {
      "thumbprint": "[parameters('certificateThumbprint')]",
      "x509StoreName": "[parameters('certificateStoreValue')]"
    }
    

    添加新标记“thumbprintSecondary”并为其指定值“[parameters('secCertificateThumbprint')]”。Add a new tag "thumbprintSecondary" and give it a value "[parameters('secCertificateThumbprint')]".

    资源定义现在应如下所示(根据具体的模板源,有时与下面的代码片段不完全相同)。So now the resource definition should look like the following (depending on your source of the template, it may not be exactly like the snippet below).

    "properties": {
    "certificate": {
      "thumbprint": "[parameters('certificateThumbprint')]",
      "thumbprintSecondary": "[parameters('secCertificateThumbprint')]",
      "x509StoreName": "[parameters('certificateStoreValue')]"
    }
    

    如果要滚动更新证书,请将新证书指定为主要证书,并将当前的主要证书移为辅助证书。If you want to roll over the cert, then specify the new cert as primary and moving the current primary as secondary. 这样就可以通过一个部署步骤,将当前主要证书滚动更新为新证书。This results in the rollover of your current primary certificate to the new certificate in one deployment step.

    "properties": {
    "certificate": {
      "thumbprint": "[parameters('secCertificateThumbprint')]",
      "thumbprintSecondary": "[parameters('certificateThumbprint')]",
      "x509StoreName": "[parameters('certificateStoreValue')]"
    }
    
  4. 所有 Microsoft.Compute/virtualMachineScaleSets 资源定义进行更改 - 查找 Microsoft.Compute/virtualMachineScaleSets 资源定义。Make changes to all the Microsoft.Compute/virtualMachineScaleSets resource definitions - Locate the Microsoft.Compute/virtualMachineScaleSets resource definition. 在“virtualMachineProfile”下,滚动到“publisher”:“Microsoft.Azure.ServiceFabric”。Scroll to the "publisher": "Microsoft.Azure.ServiceFabric", under "virtualMachineProfile".

    在 Service Fabric 发布服务器设置中,应看到类似如下的内容。In the Service Fabric publisher settings, you should see something like this.

    Json_Pub_Setting1

    向其中添加新的证书项Add the new cert entries to it

    "certificateSecondary": {
        "thumbprint": "[parameters('secCertificateThumbprint')]",
        "x509StoreName": "[parameters('certificateStoreValue')]"
        }
      },
    
    

    属性现在应如下所示The properties should now look like this

    Json_Pub_Setting2

    如果要滚动更新证书,请将新证书指定为主要证书,并将当前的主要证书移为辅助证书。If you want to roll over the cert, then specify the new cert as primary and moving the current primary as secondary. 这样就可以通过一个部署步骤,将当前证书滚动更新为新证书。This results in the rollover of your current certificate to the new certificate in one deployment step.

    "certificate": {
       "thumbprint": "[parameters('secCertificateThumbprint')]",
       "x509StoreName": "[parameters('certificateStoreValue')]"
         },
    "certificateSecondary": {
        "thumbprint": "[parameters('certificateThumbprint')]",
        "x509StoreName": "[parameters('certificateStoreValue')]"
        }
      },
    

    属性现在应如下所示The properties should now look like this
    Json_Pub_Setting3

  5. 所有 Microsoft.Compute/virtualMachineScaleSets 资源定义进行更改 - 查找 Microsoft.Compute/virtualMachineScaleSets 资源定义。Make Changes to all the Microsoft.Compute/virtualMachineScaleSets resource definitions - Locate the Microsoft.Compute/virtualMachineScaleSets resource definition. 滚动到 "vaultCertificates":,位于 "OSProfile" 下。Scroll to the "vaultCertificates": , under "OSProfile". 应该会看到类似下面的屏幕。it should look something like this.

    Json_Pub_Setting4

    向其添加 secCertificateUrlValue。Add the secCertificateUrlValue to it. 使用以下代码片段:use the following snippet:

    {
        "certificateStore": "[parameters('certificateStoreValue')]",
        "certificateUrl": "[parameters('secCertificateUrlValue')]"
    }
    
    

    现在,生成的 Json 应如下所示。Now the resulting Json should look something like this. Json_Pub_Setting5Json_Pub_Setting5

备注

请确保已对模板中的 Nodetypes/Microsoft.Compute/virtualMachineScaleSets 资源定义重复执行了步骤 4 和 5。Make sure that you have repeated steps 4 and 5 for all the Nodetypes/Microsoft.Compute/virtualMachineScaleSets resource definitions in your template. 如果缺少其中一个,证书将无法安装在该虚拟机规模集上,并且在群集(包括停止运行的群集)上产生不可预知的结果(如果最终没有群集可用于安全性的有效证书)。If you miss one of them, the certificate will not get installed on that virtual machine scale set and you will have unpredictable results in your cluster, including the cluster going down (if you end up with no valid certificates that the cluster can use for security. 因此在继续之前,请仔细检查。So double check, before proceeding further.

编辑模板文件,反映前面添加的新参数Edit your template file to reflect the new parameters you added above

如果参考了 git-repo 中的示例,便可开始更改示例 5-VM-1-NodeTypes-Secure.parameters_Step2.JSONIf you are using the sample from the git-repo to follow along, you can start to make changes in The sample 5-VM-1-NodeTypes-Secure.parameters_Step2.JSON

编辑 Resource Manager 模板参数文件,添加 secCertificateThumbprint 和 secCertificateUrlValue 的两个新参数。Edit your Resource Manager Template parameter File, add the two new parameters for secCertificateThumbprint and secCertificateUrlValue.

"secCertificateThumbprint": {
  "value": "thumbprint value"
},
"secCertificateUrlValue": {
  "value": "Refers to the location URL in your key vault where the certificate was uploaded, it is should be in the format of https://<name of the vault>.vault.azure.cn:443/secrets/<exact location>"
},

将模板部署到 AzureDeploy the template to Azure

  • 现在,可以将模板部署到 Azure。You are now ready to deploy your template to Azure. 请打开 Azure PS 版本 1(或更高版本)的命令提示符。Open an Azure PS version 1+ command prompt.
  • 登录到 Azure 帐户,选择特定的 Azure 订阅。Sign in to your Azure Account and select the specific azure subscription. 对于有权访问多个 Azure 订阅的用户而言,这是一个重要步骤。This is an important step for folks who have access to more than one azure subscription.
Connect-AzAccount -Environment AzureChinaCloud
Select-AzSubscription -SubscriptionId <Subscription ID> 

部署模板之前先进行测试。Test the template prior to deploying it. 使用群集当前部署到的同一个资源组。Use the same Resource Group that your cluster is currently deployed to.

Test-AzResourceGroupDeployment -ResourceGroupName <Resource Group that your cluster is currently deployed to> -TemplateFile <PathToTemplate>

将模板部署到该资源组。Deploy the template to your resource group. 使用群集当前部署到的同一个资源组。Use the same Resource Group that your cluster is currently deployed to. 运行 New-AzResourceGroupDeployment 命令。Run the New-AzResourceGroupDeployment command. 无需指定模式,因为默认值为 incrementalYou do not need to specify the mode, since the default value is incremental.

备注

如果将 Mode 设置为 Complete,可能会无意中删除不在模板中的资源。If you set Mode to Complete, you can inadvertently delete resources that are not in your template. 因此请不要在此方案中使用该模式。So do not use it in this scenario.

New-AzResourceGroupDeployment -Name ExampleDeployment -ResourceGroupName <Resource Group that your cluster is currently deployed to> -TemplateFile <PathToTemplate>

下面是已填充数据的同一个 Powershell 命令示例。Here is a filled out example of the same powershell.

$ResourceGroup2 = "chackosecure5"
$TemplateFile = "C:\GitHub\Service-Fabric\ARM Templates\Cert Rollover Sample\5-VM-1-NodeTypes-Secure_Step2.json"
$TemplateParmFile = "C:\GitHub\Service-Fabric\ARM Templates\Cert Rollover Sample\5-VM-1-NodeTypes-Secure.parameters_Step2.json"

New-AzResourceGroupDeployment -ResourceGroupName $ResourceGroup2 -TemplateParameterFile $TemplateParmFile -TemplateUri $TemplateFile -clusterName $ResourceGroup2

部署完成后,使用新证书连接到群集,并执行一些查询。Once the deployment is complete, connect to your cluster using the new Certificate and perform some queries. 如果能够执行这些查询,If you are able to do. 则可以删除旧证书。Then you can delete the old certificate.

如果使用自签名证书,请务必将它们导入本地 TrustedPeople 证书存储。If you are using a self-signed certificate, do not forget to import them into your local TrustedPeople cert store.

######## Set up the certs on your local box
Import-PfxCertificate -Exportable -CertStoreLocation Cert:\CurrentUser\TrustedPeople -FilePath c:\Mycertificates\chackdanTestCertificate9.pfx -Password (ConvertTo-SecureString -String abcd123 -AsPlainText -Force)
Import-PfxCertificate -Exportable -CertStoreLocation Cert:\CurrentUser\My -FilePath c:\Mycertificates\chackdanTestCertificate9.pfx -Password (ConvertTo-SecureString -String abcd123 -AsPlainText -Force)

以下快速参考提供了用于连接到安全群集的命令For quick reference here is the command to connect to a secure cluster

$ClusterName= "chackosecure5.chinanorth.cloudapp.chinacloudapi.cn:19000"
$CertThumbprint= "70EF5E22ADB649799DA3C8B6A6BF7SD1D630F8F3" 

Connect-serviceFabricCluster -ConnectionEndpoint $ClusterName -KeepAliveIntervalInSec 10 `
    -X509Credential `
    -ServerCertThumbprint $CertThumbprint  `
    -FindType FindByThumbprint `
    -FindValue $CertThumbprint `
    -StoreLocation CurrentUser `
    -StoreName My

以下快速参考提供了用于获取群集运行状况的命令For quick reference here is the command to get cluster health

Get-ServiceFabricClusterHealth 

将客户端证书部署到群集。Deploying client certificates to the cluster.

可以使用与上述步骤 5 相同的步骤,将证书从 keyvault 部署到节点。You can use the same steps as outlined in the preceding Steps 5 to have the certificates deployed from a keyvault to the Nodes. 只需定义并使用不同的参数。You just need define and use different parameters.

添加或删除客户端证书Adding or removing Client certificates

除群集证书外,还可添加客户端证书来执行 Service Fabric 群集上的管理操作。In addition to the cluster certificates, you can add client certificates to perform management operations on a Service Fabric cluster.

可以添加两种类型的客户端证书 - 管理证书或只读证书。You can add two kinds of client certificates - Admin or Read-only. 然后,可以使用这些证书在群集上控制对管理操作和查询操作的访问。These then can be used to control access to the admin operations and Query operations on the cluster. 默认情况下,群集证书会添加到允许的管理证书列表。By default, the cluster certificates are added to the allowed Admin certificates list.

可以指定任意数量的客户端证书。you can specify any number of client certificates. 每次执行添加/删除操作都会导致对 Service Fabric 群集的配置进行更新Each addition/deletion results in a configuration update to the Service Fabric cluster

通过门户添加管理或只读客户端证书Adding client certificates - Admin or Read-Only via portal

  1. 导航到“安全性”部分,并选择“安全性”部分顶部的“+ 身份验证”按钮。Navigate to the Security section, and select the '+ Authentication' button on top of the security section.

  2. 在“添加身份验证”部分中,选择“身份验证类型”-“只读客户端”或“管理员客户端”On the 'Add Authentication' section, choose the 'Authentication Type' - 'Read-only client' or 'Admin client'

  3. 现在选择授权方法。Now choose the Authorization method. 向 Service Fabric 指出是要使用使用者名称还是指纹来查找此证书。This indicates to Service Fabric whether it should look up this certificate by using the subject name or the thumbprint. 一般来说,使用使用者名称授权方法并不是一种良好的安全做法。In general, it is not a good security practice to use the authorization method of subject name.

    添加客户端证书

使用门户删除管理或只读客户端证书Deletion of Client Certificates - Admin or Read-Only using the portal

若要删除辅助证书,以防将其用于群集安全,请导航到“安全性”部分,并从特定证书的上下文菜单中选择“删除”选项。To remove a secondary certificate from being used for cluster security, Navigate to the Security section and select the 'Delete' option from the context menu on the specific certificate.

将应用程序证书添加到虚拟机规模集Adding application certificates to a virtual machine scale set

若要将用于应用程序的证书部署到群集,请参阅此示例 Powershell 脚本To deploy a certificate you use for your applications to your cluster, see this sample Powershell script.

后续步骤Next steps

有关群集管理的详细信息,请阅读以下文章:Read these articles for more information on cluster management: