使用 Azure 数据工厂向/从 Azure Database for MySQL 复制数据Copy data to and from Azure Database for MySQL using Azure Data Factory

本文概述了如何使用 Azure 数据工厂中的复制活动从 Azure Database for MySQL 复制数据。This article outlines how to use the Copy Activity in Azure Data Factory to copy data from Azure Database for MySQL. 它是基于概述复制活动总体的复制活动概述一文。It builds on the copy activity overview article that presents a general overview of copy activity.

此连接器专用于 Azure Database for MySQL 服务This connector is specialized for Azure Database for MySQL service. 若要从位于本地或云中的通用 MySQL 数据库复制数据,请使用 MySQL 连接器To copy data from generic MySQL database located on-premises or in the cloud, use MySQL connector.

支持的功能Supported capabilities

以下活动支持此 Azure Database for MySQL 连接器:This Azure Database for MySQL connector is supported for the following activities:

可将数据从 Azure Database for MySQL 复制到任何支持的接收器数据存储。You can copy data from Azure Database for MySQL to any supported sink data store. 或者,可将数据从任何支持的源数据存储复制到 Azure Database for MySQL。Or, you can copy data from any supported source data store to Azure Database for MySQL. 有关复制活动支持作为源/接收器的数据存储列表,请参阅支持的数据存储表。For a list of data stores that are supported as sources/sinks by the copy activity, see the Supported data stores table.

Azure 数据工厂提供内置的驱动程序用于启用连接,因此无需使用此连接器手动安装任何驱动程序。Azure Data Factory provides a built-in driver to enable connectivity, therefore you don't need to manually install any driver using this connector.

入门Getting started

可以通过以下工具或 SDK 之一结合使用复制活动和管道。You can use one of the following tools or SDKs to use the copy activity with a pipeline. 选择链接,查看分步说明:Select a link for step-by-step instructions:

对于特定于 Azure Database for MySQL 连接器的数据工厂实体,以下部分提供有关用于定义这些实体的属性的详细信息。The following sections provide details about properties that are used to define Data Factory entities specific to Azure Database for MySQL connector.

链接服务属性Linked service properties

Azure Database for MySQL 链接服务支持以下属性:The following properties are supported for Azure Database for MySQL linked service:

属性Property 说明Description 必选Required
typetype type 属性必须设置为:AzureMySqlThe type property must be set to: AzureMySql Yes
connectionStringconnectionString 指定连接到 Azure Database for MySQL 实例所需的连接信息。Specify information needed to connect to the Azure Database for MySQL instance.
将此字段标记为 SecureString,以便安全地将其存储在数据工厂中。Mark this field as a SecureString to store it securely in Data Factory. 还可以将密码放在 Azure 密钥保管库中,并从连接字符串中拉取 password 配置。You can also put password in Azure Key Vault and pull the password configuration out of the connection string. 有关更多详细信息,请参阅以下示例和将凭据存储在 Azure 密钥保管库中一文。Refer to the following samples and Store credentials in Azure Key Vault article with more details.
Yes
connectViaconnectVia 用于连接到数据存储的集成运行时The Integration Runtime to be used to connect to the data store. 如果数据存储位于专用网络,则可以使用 Azure Integration Runtime 或自承载集成运行时。You can use Azure Integration Runtime or Self-hosted Integration Runtime (if your data store is located in private network). 如果未指定,则使用默认 Azure Integration Runtime。If not specified, it uses the default Azure Integration Runtime. No

典型的连接字符串为 Server=<server>.mysql.database.chinacloudapi.cn;Port=<port>;Database=<database>;UID=<username>;PWD=<password>A typical connection string is Server=<server>.mysql.database.chinacloudapi.cn;Port=<port>;Database=<database>;UID=<username>;PWD=<password>. 你可以根据自己的情况设置更多属性:More properties you can set per your case:

属性Property 说明Description 选项Options 必须Required
SSLModeSSLMode 此选项指定驱动程序在连接到 MySQL 时是否使用 SSL 加密和验证。This option specifies whether the driver uses SSL encryption and verification when connecting to MySQL. 例如E.g. SSLMode=<0/1/2/3/4> DISABLED (0) / PREFERRED (1) (Default) / REQUIRED (2) / VERIFY_CA (3) / VERIFY_IDENTITY (4)DISABLED (0) / PREFERRED (1) (Default) / REQUIRED (2) / VERIFY_CA (3) / VERIFY_IDENTITY (4) No
UseSystemTrustStoreUseSystemTrustStore 此选项指定是使用系统信任存储中的 CA 证书还是使用指定 PEM 文件中的 CA 证书。This option specifies whether to use a CA certificate from the system trust store, or from a specified PEM file. 例如E.g. UseSystemTrustStore=<0/1>; Enabled (1) / Disabled (0) (Default)Enabled (1) / Disabled (0) (Default) No

示例:Example:

{
    "name": "AzureDatabaseForMySQLLinkedService",
    "properties": {
        "type": "AzureMySql",
        "typeProperties": {
            "connectionString": {
                "type": "SecureString",
                "value": "Server=<server>.mysql.database.chinacloudapi.cn;Port=<port>;Database=<database>;UID=<username>;PWD=<password>"
            }
        },
        "connectVia": {
            "referenceName": "<name of Integration Runtime>",
            "type": "IntegrationRuntimeReference"
        }
    }
}

示例:在 Azure 密钥保管库中存储密码Example: store password in Azure Key Vault

{
    "name": "AzureDatabaseForMySQLLinkedService",
    "properties": {
        "type": "AzureMySql",
        "typeProperties": {
            "connectionString": {
                 "type": "SecureString",
                 "value": "Server=<server>.mysql.database.chinacloudapi.cn;Port=<port>;Database=<database>;UID=<username>;"
            },
            "password": { 
                "type": "AzureKeyVaultSecret", 
                "store": { 
                    "referenceName": "<Azure Key Vault linked service name>", 
                    "type": "LinkedServiceReference" 
                }, 
                "secretName": "<secretName>" 
            }
        },
        "connectVia": {
            "referenceName": "<name of Integration Runtime>",
            "type": "IntegrationRuntimeReference"
        }
    }
}

数据集属性Dataset properties

有关可用于定义数据集的各部分和属性的完整列表,请参阅数据集一文。For a full list of sections and properties available for defining datasets, see the datasets article. 本部分提供 Azure Database for MySQL 数据集支持的属性列表。This section provides a list of properties supported by Azure Database for MySQL dataset.

要从 Azure Database for MySQL 复制数据,请将数据集的 type 属性设置为 AzureMySqlTableTo copy data from Azure Database for MySQL, set the type property of the dataset to AzureMySqlTable. 支持以下属性:The following properties are supported:

属性Property 说明Description 必选Required
typetype 数据集的 type 属性必须设置为:AzureMySqlTableThe type property of the dataset must be set to: AzureMySqlTable Yes
tableNametableName MySQL 数据库中的表名。Name of the table in the MySQL database. 否(如果指定了活动源中的“query”)No (if "query" in activity source is specified)

示例Example

{
    "name": "AzureMySQLDataset",
    "properties": {
        "type": "AzureMySqlTable",
        "linkedServiceName": {
            "referenceName": "<Azure MySQL linked service name>",
            "type": "LinkedServiceReference"
        },
        "typeProperties": {
            "tableName": "<table name>"
        }
    }
}

复制活动属性Copy activity properties

有关可用于定义活动的各部分和属性的完整列表,请参阅管道一文。For a full list of sections and properties available for defining activities, see the Pipelines article. 本部分提供 Azure Database for MySQL 源和接收器支持的属性列表。This section provides a list of properties supported by Azure Database for MySQL source and sink.

Azure Database for MySQL 用作源Azure Database for MySQL as source

若要从 Azure Database for MySQL 复制数据,复制活动的 source 节需要支持以下属性:To copy data from Azure Database for MySQL, the following properties are supported in the copy activity source section:

属性Property 说明Description 必选Required
typetype 复制活动 source 的 type 属性必须设置为:AzureMySqlSourceThe type property of the copy activity source must be set to: AzureMySqlSource Yes
queryquery 使用自定义 SQL 查询读取数据。Use the custom SQL query to read data. 例如:"SELECT * FROM MyTable"For example: "SELECT * FROM MyTable". 否(如果指定了数据集中的“tableName”)No (if "tableName" in dataset is specified)
queryCommandTimeoutqueryCommandTimeout 查询请求超时前的等待时间。默认值为 120 分钟 (02:00:00)The wait time before the query request times out. Default is 120 minutes (02:00:00) No

示例:Example:

"activities":[
    {
        "name": "CopyFromAzureDatabaseForMySQL",
        "type": "Copy",
        "inputs": [
            {
                "referenceName": "<Azure MySQL input dataset name>",
                "type": "DatasetReference"
            }
        ],
        "outputs": [
            {
                "referenceName": "<output dataset name>",
                "type": "DatasetReference"
            }
        ],
        "typeProperties": {
            "source": {
                "type": "AzureMySqlSource",
                "query": "<custom query e.g. SELECT * FROM MyTable>"
            },
            "sink": {
                "type": "<sink type>"
            }
        }
    }
]

Azure Database for MySQL 作为接收器Azure Database for MySQL as sink

将数据复制到 Azure Database for MySQL 时,复制活动的 sink 节支持以下属性:To copy data to Azure Database for MySQL, the following properties are supported in the copy activity sink section:

属性Property 说明Description 必选Required
typetype 复制活动接收器的 type 属性必须设置为:AzureMySqlSinkThe type property of the copy activity sink must be set to: AzureMySqlSink Yes
preCopyScriptpreCopyScript 每次运行时将数据写入 Azure Database for MySQL 之前,为要执行的复制活动指定 SQL 查询。Specify a SQL query for the copy activity to execute before writing data into Azure Database for MySQL in each run. 可以使用此属性清除预加载的数据。You can use this property to clean up the preloaded data. No
writeBatchSizewriteBatchSize 当缓冲区大小达到 writeBatchSize 时,会将数据插入 Azure Database for MySQL 表。Inserts data into the Azure Database for MySQL table when the buffer size reaches writeBatchSize.
允许的值为 integer(表示行数)。Allowed value is integer representing number of rows.
否(默认值为 10,000)No (default is 10,000)
writeBatchTimeoutwriteBatchTimeout 超时之前等待批插入操作完成时的等待时间。Wait time for the batch insert operation to complete before it times out.
允许的值为 Timespan。Allowed values are Timespan. 示例为 00:30:00(30 分钟)。An example is 00:30:00 (30 minutes).
否(默认值为 00:00:30)No (default is 00:00:30)

示例:Example:

"activities":[
    {
        "name": "CopyToAzureDatabaseForMySQL",
        "type": "Copy",
        "inputs": [
            {
                "referenceName": "<input dataset name>",
                "type": "DatasetReference"
            }
        ],
        "outputs": [
            {
                "referenceName": "<Azure MySQL output dataset name>",
                "type": "DatasetReference"
            }
        ],
        "typeProperties": {
            "source": {
                "type": "<source type>"
            },
            "sink": {
                "type": "AzureMySqlSink",
                "preCopyScript": "<custom SQL script>",
                "writeBatchSize": 100000
            }
        }
    }
]

Lookup 活动属性Lookup activity properties

若要了解有关属性的详细信息,请查看 Lookup 活动To learn details about the properties, check Lookup activity.

Azure Database for MySQL 的数据类型映射Data type mapping for Azure Database for MySQL

从 Azure Database for MySQL 复制数据时,以下映射用于从 MySQL 数据类型映射到 Azure 数据工厂临时数据类型。When copying data from Azure Database for MySQL, the following mappings are used from MySQL data types to Azure Data Factory interim data types. 若要了解复制活动如何将源架构和数据类型映射到接收器,请参阅架构和数据类型映射See Schema and data type mappings to learn about how copy activity maps the source schema and data type to the sink.

Azure Database for MySQL 数据类型Azure Database for MySQL data type 数据工厂临时数据类型Data factory interim data type
bigint Int64
bigint unsigned Decimal
bit Boolean
bit(M), M>1 Byte[]
blob Byte[]
bool Int16
char String
date Datetime
datetime Datetime
decimal Decimal, String
double Double
double precision Double
enum String
float Single
int Int32
int unsigned Int64
integer Int32
integer unsigned Int64
long varbinary Byte[]
long varchar String
longblob Byte[]
longtext String
mediumblob Byte[]
mediumint Int32
mediumint unsigned Int64
mediumtext String
numeric Decimal
real Double
set String
smallint Int16
smallint unsigned Int32
text String
time TimeSpan
timestamp Datetime
tinyblob Byte[]
tinyint Int16
tinyint unsigned Int16
tinytext String
varchar String
year Int32

后续步骤Next steps

有关 Azure 数据工厂中复制活动支持作为源和接收器的数据存储的列表,请参阅支持的数据存储For a list of data stores supported as sources and sinks by the copy activity in Azure Data Factory, see supported data stores.