使用 PowerShell 在 Azure SQL 数据库中的多个数据库之间同步数据Use PowerShell to sync data between multiple databases in Azure SQL Database
Azure SQL 数据库
此 Azure PowerShell 示例将 SQL 数据同步配置为在 Azure SQL 数据库中的多个数据库之间同步数据。This Azure PowerShell example configures SQL Data Sync to sync data between multiple databases in Azure SQL Database.
如果没有 Azure 试用版订阅,请在开始前创建一个试用版订阅。If you don't have an Azure trail subscription, create a trial subscription before you begin.
备注
本文进行了更新,以便使用新的 Azure PowerShell Az 模块。This article has been updated to use the new Azure PowerShell Az module. 你仍然可以使用 AzureRM 模块,至少在 2020 年 12 月之前,它将继续接收 bug 修补程序。You can still use the AzureRM module, which will continue to receive bug fixes until at least December 2020. 若要详细了解新的 Az 模块和 AzureRM 兼容性,请参阅新 Azure Powershell Az 模块简介。To learn more about the new Az module and AzureRM compatibility, see Introducing the new Azure PowerShell Az module. 有关 Az 模块安装说明,请参阅安装 Azure PowerShell。For Az module installation instructions, see Install Azure PowerShell.
本教程需要 Az PowerShell 1.4.0 或更高版本。This tutorial requires Az PowerShell 1.4.0 or later. 如果需要进行升级,请参阅 Install Azure PowerShell module(安装 Azure PowerShell 模块)。If you need to upgrade, see Install Azure PowerShell module. 此外,还需要运行 Connect-AzAccount -EnvironmentName AzureChinaCloud
以创建与 Azure 的连接。You also need to run Connect-AzAccount -EnvironmentName AzureChinaCloud
to create a connection with Azure.
有关 SQL 数据同步的概述,请参阅使用 Azure 中的 SQL 数据同步跨多个云和本地数据库同步数据。For an overview of SQL Data Sync, see Sync data across multiple cloud and on-premises databases with SQL Data Sync in Azure.
重要
目前,SQL 数据同步不支持 Azure SQL 托管实例。SQL Data Sync does not support Azure SQL Managed Instance at this time.
先决条件Prerequisites
- 使用 AdventureWorksLT 示例数据库在 Azure SQL 数据库中创建数据库作为中心数据库。Create a database in Azure SQL Database from an AdventureWorksLT sample database as a hub database.
- 在同步数据库所在的相同区域中创建 Azure SQL 数据库中的数据库。Create a database in Azure SQL Database in the same region as the sync database.
- 在运行示例前更新参数占位符。Update the parameter placeholders before running the example.
示例Example
using namespace Microsoft.Azure.Commands.Sql.DataSync.Model
using namespace System.Collections.Generic
# hub database info
$subscriptionId = "<subscriptionId>"
$resourceGroupName = "<resourceGroupName>"
$serverName = "<serverName>"
$databaseName = "<databaseName>"
# sync database info
$syncDatabaseResourceGroupName = "<syncResourceGroupName>"
$syncDatabaseServerName = "<syncServerName>"
$syncDatabaseName = "<syncDatabaseName>"
# sync group info
$syncGroupName = "<syncGroupName>"
$conflictResolutionPolicy = "HubWin" # can be HubWin or MemberWin
$intervalInSeconds = 300 # sync interval in seconds (must be no less than 300)
# member database info
$syncMemberName = "<syncMemberName>"
$memberServerName = "<memberServerName>"
$memberDatabaseName = "<memberDatabaseName>"
$memberDatabaseType = "SqlServerDatabase" # can be AzureSqlDatabase or SqlServerDatabase
$syncDirection = "Bidirectional" # can be Bidirectional, Onewaymembertohub, Onewayhubtomember
# sync agent info
$syncAgentName = "<agentName>"
$syncAgentResourceGroupName = "<syncAgentResourceGroupName>"
$syncAgentServerName = "<syncAgentServerName>"
$syncMemberResourceId = "/subscriptions/$subscriptionId/resourceGroups/$resourceGroupName/providers/Microsoft.Sql/servers/$serverName/databases/$syncMemberDBName"
# temp file to save the sync schema
$tempFile = $env:TEMP+"\syncSchema.json"
# list of included columns and tables in quoted name
$includedColumnsAndTables = "[SalesLT].[Address].[AddressID]",
"[SalesLT].[Address].[AddressLine2]",
"[SalesLT].[Address].[rowguid]",
"[SalesLT].[Address].[PostalCode]",
"[SalesLT].[ProductDescription]"
$metadataList = [System.Collections.ArrayList]::new($includedColumnsAndTables)
Connect-AzAccount -EnvironmentName AzureChinaCloud
Select-AzSubscription -SubscriptionId $subscriptionId
# use if it's safe to show password in script, otherwise use PromptForCredential
# $user = "username"
# $password = ConvertTo-SecureString -String "password" -AsPlainText -Force
# $credential = New-Object -TypeName "System.Management.Automation.PSCredential" -ArgumentList $user, $password
$credential = $Host.ui.PromptForCredential("Need credential",
"Please enter your user name and password for server "+$serverName+".database.chinacloudapi.cn",
"",
"")
# create a new sync group (if you use private link, make sure to manually approve it)
Write-Host "Creating Sync Group "$syncGroupName"..."
New-AzSqlSyncGroup -ResourceGroupName $resourceGroupName -ServerName $serverName -DatabaseName $databaseName -Name $syncGroupName `
-SyncDatabaseName $syncDatabaseName -SyncDatabaseServerName $syncDatabaseServerName -SyncDatabaseResourceGroupName $syncDatabaseResourceGroupName `
-ConflictResolutionPolicy $conflictResolutionPolicy -DatabaseCredential $credential -UsePrivateLinkConnection | Format-list
# use if it's safe to show password in script, otherwise use PromptForCredential
# $user = "username"
# $password = ConvertTo-SecureString -String "password" -AsPlainText -Force
# $credential = New-Object -TypeName "System.Management.Automation.PSCredential" -ArgumentList $user, $password
$credential = $Host.ui.PromptForCredential("Need credential",
"Please enter your user name and password for server "+$serverName+".database.chinacloudapi.cn",
"",
"")
# add a new sync member (if you use private link, make sure to manually approve it)
Write-Host "Adding member"$syncMemberName" to the sync group..."
New-AzSqlSyncMember -ResourceGroupName $resourceGroupName -ServerName $serverName -DatabaseName $databaseName `
-SyncGroupName $syncGroupName -Name $syncMemberName -MemberDatabaseType $memberDatabaseType -SyncAgentResourceGroupName $syncAgentResourceGroupName `
-SyncAgentServerName $syncAgentServerName -SyncAgentName $syncAgentName -SyncDirection $syncDirection -SqlServerDatabaseID $syncAgentInfo.DatabaseId `
-SyncMemberAzureDatabaseResourceId $syncMemberResourceId -UsePrivateLinkConnection | Format-list
# update existing sync member to use private link connection
Update-AzSqlSyncMember `
-ResourceGroupName $resourceGroupName -ServerName $serverName -DatabaseName $databaseName -SyncGroupName $syncGroupName -Name $syncMemberName `
-MemberDatabaseCredential $memberDatabaseCredential -SyncMemberAzureDatabaseResourceId $syncMemberResourceId -UsePrivateLinkConnection $true
# update existing sync group and remove private link connection
Update-AzSqlSyncGroup `
-ResourceGroupName $resourceGroupName -ServerName $serverName -DatabaseName $databaseName -Name $syncGroupName -UsePrivateLinkConnection $false
# run the following Get-AzSqlSyncGroup/ Get-AzSqlSyncMember commands to confirm that a private link has been setup for Data Sync, if you decide to use private link.
# Get-AzSqlSyncMember returns information about one or more Azure SQL Database Sync Members. Specify the name of a sync member to see information for only that sync member.
Get-AzSqlSyncMember `
-ResourceGroupName $resourceGroupName -ServerName $serverName -DatabaseName $databaseName -SyncGroupName $syncGroupName -Name $syncMemberName ` | Format-List
# Get-AzSqlSyncGroup returns information about one or more Azure SQL Database Sync Groups. Specify the name of a sync group to see information for only that sync group.
Get-AzSqlSyncGroup `
-ResourceGroupName $resourceGroupName -ServerName $serverName -DatabaseName $databaseName ` | Format-List
# approve private endpoint connection, if you decide to use private link
Approve-AzPrivateEndpointConnection `
-Name myPrivateEndpointConnection -ResourceGroupName myResourceGroup -ServiceName myPrivateLinkService
# refresh database schema from hub database, specify the -SyncMemberName parameter if you want to refresh schema from the member database
Write-Host "Refreshing database schema from hub database..."
$startTime = Get-Date
Update-AzSqlSyncSchema -ResourceGroupName $resourceGroupName -ServerName $serverName -DatabaseName $databaseName -SyncGroupName $syncGroupName
# waiting for successful refresh
$startTime = $startTime.ToUniversalTime()
$timer=0
$timeout=90
# check the log and see if refresh has gone through
Write-Host "Check for successful refresh..."
$isSucceeded = $false
while ($isSucceeded -eq $false) {
Start-Sleep -s 10
$timer=$timer+10
$details = Get-AzSqlSyncSchema -SyncGroupName $syncGroupName -ServerName $serverName -DatabaseName $databaseName -ResourceGroupName $resourceGroupName
if ($details.LastUpdateTime -gt $startTime) {
Write-Host "Refresh was successful"
$isSucceeded = $true
}
if ($timer -eq $timeout) {
Write-Host "Refresh timed out"
break;
}
}
# get the database schema
Write-Host "Adding tables and columns to the sync schema..."
$databaseSchema = Get-AzSqlSyncSchema -ResourceGroupName $ResourceGroupName -ServerName $ServerName `
-DatabaseName $DatabaseName -SyncGroupName $SyncGroupName `
$databaseSchema | ConvertTo-Json -depth 5 -Compress | Out-File "C:\Users\OnPremiseServer\AppData\Local\Temp\syncSchema.json"
$newSchema = [AzureSqlSyncGroupSchemaModel]::new()
$newSchema.Tables = [List[AzureSqlSyncGroupSchemaTableModel]]::new();
# add columns and tables to the sync schema
foreach ($tableSchema in $databaseSchema.Tables) {
$newTableSchema = [AzureSqlSyncGroupSchemaTableModel]::new()
$newTableSchema.QuotedName = $tableSchema.QuotedName
$newTableSchema.Columns = [List[AzureSqlSyncGroupSchemaColumnModel]]::new();
$addAllColumns = $false
if ($MetadataList.Contains($tableSchema.QuotedName)) {
if ($tableSchema.HasError) {
$fullTableName = $tableSchema.QuotedName
Write-Host "Can't add table $fullTableName to the sync schema" -foregroundcolor "Red"
Write-Host $tableSchema.ErrorId -foregroundcolor "Red"
continue;
}
else {
$addAllColumns = $true
}
}
foreach($columnSchema in $tableSchema.Columns) {
$fullColumnName = $tableSchema.QuotedName + "." + $columnSchema.QuotedName
if ($addAllColumns -or $MetadataList.Contains($fullColumnName)) {
if ((-not $addAllColumns) -and $tableSchema.HasError) {
Write-Host "Can't add column $fullColumnName to the sync schema" -foregroundcolor "Red"
Write-Host $tableSchema.ErrorId -foregroundcolor "Red"
}
elseif ((-not $addAllColumns) -and $columnSchema.HasError) {
Write-Host "Can't add column $fullColumnName to the sync schema" -foregroundcolor "Red"
Write-Host $columnSchema.ErrorId -foregroundcolor "Red"
}
else {
Write-Host "Adding"$fullColumnName" to the sync schema"
$newColumnSchema = [AzureSqlSyncGroupSchemaColumnModel]::new()
$newColumnSchema.QuotedName = $columnSchema.QuotedName
$newColumnSchema.DataSize = $columnSchema.DataSize
$newColumnSchema.DataType = $columnSchema.DataType
$newTableSchema.Columns.Add($newColumnSchema)
}
}
}
if ($newTableSchema.Columns.Count -gt 0) {
$newSchema.Tables.Add($newTableSchema)
}
}
# convert sync schema to JSON format
$schemaString = $newSchema | ConvertTo-Json -depth 5 -Compress
# work around a PowerShell bug
$schemaString = $schemaString.Replace('"Tables"', '"tables"').Replace('"Columns"', '"columns"').Replace('"QuotedName"', '"quotedName"').Replace('"MasterSyncMemberName"','"masterSyncMemberName"')
# save the sync schema to a temp file
$schemaString | Out-File $tempFile
# update sync schema
Write-Host "Updating the sync schema..."
Update-AzSqlSyncGroup -ResourceGroupName $resourceGroupName -ServerName $serverName `
-DatabaseName $databaseName -Name $syncGroupName -Schema $tempFile
$syncLogStartTime = Get-Date
# trigger sync manually
Write-Host "Trigger sync manually..."
Start-AzSqlSyncGroupSync -ResourceGroupName $resourceGroupName -ServerName $serverName -DatabaseName $databaseName -SyncGroupName $syncGroupName
# check the sync log and wait until the first sync succeeded
Write-Host "Check the sync log..."
$isSucceeded = $false
for ($i = 0; ($i -lt 300) -and (-not $isSucceeded); $i = $i + 10) {
Start-Sleep -s 10
$syncLogEndTime = Get-Date
$syncLogList = Get-AzSqlSyncGroupLog -ResourceGroupName $resourceGroupName -ServerName $serverName -DatabaseName $databaseName `
-SyncGroupName $syncGroupName -StartTime $syncLogStartTime.ToUniversalTime() -EndTime $syncLogEndTime.ToUniversalTime()
if ($synclogList.Length -gt 0) {
foreach ($syncLog in $syncLogList) {
if ($syncLog.Details.Contains("Sync completed successfully")) {
Write-Host $syncLog.TimeStamp : $syncLog.Details
$isSucceeded = $true
}
}
}
}
if ($isSucceeded) {
# enable scheduled sync
Write-Host "Enable the scheduled sync with 300 seconds interval..."
Update-AzSqlSyncGroup -ResourceGroupName $resourceGroupName -ServerName $serverName -DatabaseName $databaseName `
-Name $syncGroupName -IntervalInSeconds $intervalInSeconds
}
else {
# output all log if sync doesn't succeed in 300 seconds
$syncLogEndTime = Get-Date
$syncLogList = Get-AzSqlSyncGroupLog -ResourceGroupName $resourceGroupName -ServerName $serverName -DatabaseName $databaseName `
-SyncGroupName $syncGroupName -StartTime $syncLogStartTime.ToUniversalTime() -EndTime $syncLogEndTime.ToUniversalTime()
if ($synclogList.Length -gt 0) {
foreach ($syncLog in $syncLogList) {
Write-Host $syncLog.TimeStamp : $syncLog.Details
}
}
}
清理部署Clean up deployment
运行示例脚本后,可以运行以下命令,删除资源组以及与其关联的所有资源。After you run the sample script, you can run the following command to remove the resource group and all resources associated with it.
Remove-AzResourceGroup -ResourceGroupName $ResourceGroupName
Remove-AzResourceGroup -ResourceGroupName $SyncDatabaseResourceGroupName
脚本说明Script explanation
此脚本使用以下命令。This script uses the following commands. 表中的每条命令链接到特定于命令的文档。Each command in the table links to command-specific documentation.
命令Command | 注释Notes |
---|---|
New-AzSqlSyncAgentNew-AzSqlSyncAgent | 创建新的同步代理。Creates a new Sync Agent. |
New-AzSqlSyncAgentKeyNew-AzSqlSyncAgentKey | 生成与同步代理相关联的代理密钥。Generates the agent key associated with the Sync Agent. |
Get-AzSqlSyncAgentLinkedDatabaseGet-AzSqlSyncAgentLinkedDatabase | 获取有关同步代理的所有信息。Get all the information for the Sync Agent. |
New-AzSqlSyncMemberNew-AzSqlSyncMember | 向同步组中添加新成员。Add a new member to the sync group. |
Update-AzSqlSyncSchemaUpdate-AzSqlSyncSchema | 刷新数据库架构信息。Refreshes the database schema information. |
Get-AzSqlSyncSchemaGet-AzSqlSyncSchema | 获取数据库架构信息。Get the database schema information. |
Update-AzSqlSyncGroupUpdate-AzSqlSyncGroup | 更新同步组。Updates the sync group. |
Start-AzSqlSyncGroupSyncStart-AzSqlSyncGroupSync | 触发同步。Triggers a sync. |
Get-AzSqlSyncGroupLogGet-AzSqlSyncGroupLog | 检查同步日志。Checks the Sync Log. |
后续步骤Next steps
有关 Azure PowerShell 的详细信息,请参阅 Azure PowerShell 文档。For more information about Azure PowerShell, see Azure PowerShell documentation.
可以在 Azure SQL 数据库 PowerShell 脚本中找到更多 SQL 数据库 PowerShell 脚本示例。Additional SQL Database PowerShell script samples can be found in Azure SQL Database PowerShell scripts.
有关 SQL 数据同步的详细信息,请参阅:For more information about SQL Data Sync, see:
- 概述 - 使用 Azure 中的 SQL 数据同步跨多个云和本地数据库同步数据Overview - Sync data across multiple cloud and on-premises databases with SQL Data Sync in Azure
- 设置数据同步Set up Data Sync
- 使用 Azure 门户 - 教程:设置 SQL 数据同步,以在 Azure SQL 数据库和 SQL Server 之间同步数据Use the Azure portal - Tutorial: Set up SQL Data Sync to sync data between Azure SQL Database and SQL Server
- 使用 PowerShell - 使用 PowerShell 在 Azure SQL 数据库中的数据库和 SQL Server 之间同步数据Use PowerShell - Use PowerShell to sync data between a database in Azure SQL Database and SQL Server
- Data Sync Agent - Azure 中 SQL 数据同步的 Data Sync AgentData Sync Agent - Data Sync Agent for SQL Data Sync in Azure
- 最佳做法 - Azure 中 SQL 数据同步的最佳做法Best practices - Best practices for SQL Data Sync in Azure
- 监视 - 使用 Azure Monitor 日志监视 SQL 数据同步Monitor - Monitor SQL Data Sync with Azure Monitor logs
- 故障排除 - 排查 Azure 中的 SQL 数据同步问题Troubleshoot - Troubleshoot issues with SQL Data Sync in Azure
- 更新同步架构Update the sync schema
- 使用 Transact-SQL - 在 Azure 中的 SQL 数据同步中自动复制架构更改Use Transact-SQL - Automate the replication of schema changes in SQL Data Sync in Azure
- 使用 PowerShell - 使用 PowerShell 更新现有同步组中的同步架构Use PowerShell - Use PowerShell to update the sync schema in an existing sync group
有关 SQL 数据库的详细信息,请参阅:For more information about SQL Database, see: