使用 Azure 数据工厂从 MySQL 复制数据Copy data from MySQL using Azure Data Factory

适用于:是 Azure 数据工厂是 Azure Synapse Analytics(预览版)APPLIES TO: yesAzure Data Factory yesAzure Synapse Analytics (Preview)

本文概述了如何使用 Azure 数据工厂中的复制活动从 MySQL 数据库复制数据。This article outlines how to use the Copy Activity in Azure Data Factory to copy data from a MySQL database. 它是基于概述复制活动总体的复制活动概述一文。It builds on the copy activity overview article that presents a general overview of copy activity.

备注

若要从或向 Azure Database for MySQL 服务复制数据,请使用专用的 Azure Database for MySQL 连接器To copy data from or to Azure Database for MySQL service, use the specialized Azure Database for MySQL connector.

支持的功能Supported capabilities

以下活动支持此 MySQL 连接器:This MySQL connector is supported for the following activities:

可以将数据从 MySQL 数据库复制到任何支持的接收器数据存储。You can copy data from MySQL database to any supported sink data store. 有关复制活动支持作为源/接收器的数据存储列表,请参阅支持的数据存储表。For a list of data stores that are supported as sources/sinks by the copy activity, see the Supported data stores table.

具体而言,此 MySQL 连接器支持 MySQL 版本 5.6、5.7 和 8.0。Specifically, this MySQL connector supports MySQL version 5.6, 5.7 and 8.0.

先决条件Prerequisites

如果数据存储是以下方式之一配置的,则需要设置自承载集成运行时才能连接到此数据存储:If your data store is configured in one of the following ways, you need to set up a self-hosted integration runtime to connect to the data store:

  • 数据存储位于本地网络内部、Azure 虚拟网络内部或 Amazon 虚拟私有云内。The data store is located inside an on-premises network, inside an Azure virtual network, or inside Amazon Virtual Private Cloud.
  • 数据存储是一种托管的云数据服务,只有在防火墙规则中列入允许列表的 IP 才能访问该服务。The data store is a managed cloud data service where the access is restricted to IPs that are whitelisted in the firewall rules.

集成运行时提供内置 MySQL 驱动程序(从版本 3.7 开始),因此无需手动安装任何驱动程序。The Integration Runtime provides a built-in MySQL driver starting from version 3.7, therefore you don't need to manually install any driver.

入门Getting started

若要使用管道执行复制活动,可以使用以下工具或 SDK 之一:To perform the Copy activity with a pipeline, you can use one of the following tools or SDKs:

对于特定于 MySQL 连接器的数据工厂实体,以下部分提供有关用于定义这些实体的属性的详细信息。The following sections provide details about properties that are used to define Data Factory entities specific to MySQL connector.

链接服务属性Linked service properties

MySQL 链接的服务支持以下属性:The following properties are supported for MySQL linked service:

属性Property 说明Description 必须Required
typetype type 属性必须设置为:MySqlThe type property must be set to: MySql Yes
connectionStringconnectionString 指定连接到 Azure Database for MySQL 实例所需的连接信息。Specify information needed to connect to the Azure Database for MySQL instance.
还可以将密码放在 Azure 密钥保管库中,并从连接字符串中拉取 password 配置。You can also put password in Azure Key Vault and pull the password configuration out of the connection string. 有关更多详细信息,请参阅以下示例和在 Azure 密钥保管库中存储凭据一文。Refer to the following samples and Store credentials in Azure Key Vault article with more details.
Yes
connectViaconnectVia 用于连接到数据存储的集成运行时The Integration Runtime to be used to connect to the data store. 先决条件部分了解更多信息。Learn more from Prerequisites section. 如果未指定,则使用默认 Azure Integration Runtime。If not specified, it uses the default Azure Integration Runtime. No

典型的连接字符串为 Server=<server>;Port=<port>;Database=<database>;UID=<username>;PWD=<password>A typical connection string is Server=<server>;Port=<port>;Database=<database>;UID=<username>;PWD=<password>. 你可以根据自己的情况设置更多属性:More properties you can set per your case:

属性Property 说明Description 选项Options 必须Required
SSLModeSSLMode 此选项指定驱动程序在连接到 MySQL 时是否使用 TLS 加密和验证。This option specifies whether the driver uses TLS encryption and verification when connecting to MySQL. 例如,SSLMode=<0/1/2/3/4>E.g., SSLMode=<0/1/2/3/4> DISABLED (0) / PREFERRED (1) (Default) / REQUIRED (2) / VERIFY_CA (3) / VERIFY_IDENTITY (4)DISABLED (0) / PREFERRED (1) (Default) / REQUIRED (2) / VERIFY_CA (3) / VERIFY_IDENTITY (4) No
UseSystemTrustStoreUseSystemTrustStore 此选项指定是使用系统信任存储中的 CA 证书还是使用指定 PEM 文件中的 CA 证书。This option specifies whether to use a CA certificate from the system trust store, or from a specified PEM file. 例如E.g. UseSystemTrustStore=<0/1>; Enabled (1) / Disabled (0) (Default)Enabled (1) / Disabled (0) (Default) No

示例:Example:

{
    "name": "MySQLLinkedService",
    "properties": {
        "type": "MySql",
        "typeProperties": {
            "connectionString": "Server=<server>;Port=<port>;Database=<database>;UID=<username>;PWD=<password>"
        },
        "connectVia": {
            "referenceName": "<name of Integration Runtime>",
            "type": "IntegrationRuntimeReference"
        }
    }
}

示例:在 Azure 密钥保管库中存储密码Example: store password in Azure Key Vault

{
    "name": "MySQLLinkedService",
    "properties": {
        "type": "MySql",
        "typeProperties": {
            "connectionString": "Server=<server>;Port=<port>;Database=<database>;UID=<username>;",
            "password": { 
                "type": "AzureKeyVaultSecret", 
                "store": { 
                    "referenceName": "<Azure Key Vault linked service name>", 
                    "type": "LinkedServiceReference" 
                }, 
                "secretName": "<secretName>" 
            }
        },
        "connectVia": {
            "referenceName": "<name of Integration Runtime>",
            "type": "IntegrationRuntimeReference"
        }
    }
}

如果使用的是具有以下有效负载的 MySQL 链接服务,它仍然按原样受支持,但建议使用新的版本。If you were using MySQL linked service with the following payload, it is still supported as-is, while you are suggested to use the new one going forward.

先前的有效负载:Previous payload:

{
    "name": "MySQLLinkedService",
    "properties": {
        "type": "MySql",
        "typeProperties": {
            "server": "<server>",
            "database": "<database>",
            "username": "<username>",
            "password": {
                "type": "SecureString",
                "value": "<password>"
            }
        },
        "connectVia": {
            "referenceName": "<name of Integration Runtime>",
            "type": "IntegrationRuntimeReference"
        }
    }
}

数据集属性Dataset properties

有关可用于定义数据集的各部分和属性的完整列表,请参阅数据集一文。For a full list of sections and properties available for defining datasets, see the datasets article. 本部分提供 MySQL 数据集支持的属性列表。This section provides a list of properties supported by MySQL dataset.

若要从 MySQL 复制数据,支持以下属性:To copy data from MySQL, the following properties are supported:

属性Property 说明Description 必须Required
typetype 数据集的 type 属性必须设置为:MySqlTableThe type property of the dataset must be set to: MySqlTable Yes
tableNametableName MySQL 数据库中的表名。Name of the table in the MySQL database. 否(如果指定了活动源中的“query”)No (if "query" in activity source is specified)

示例Example

{
    "name": "MySQLDataset",
    "properties":
    {
        "type": "MySqlTable",
        "typeProperties": {},
        "schema": [],
        "linkedServiceName": {
            "referenceName": "<MySQL linked service name>",
            "type": "LinkedServiceReference"
        }
    }
}

如果使用 RelationalTable 类型数据集,该数据集仍按原样受支持,但我们建议今后使用新数据集。If you were using RelationalTable typed dataset, it is still supported as-is, while you are suggested to use the new one going forward.

复制活动属性Copy activity properties

有关可用于定义活动的各部分和属性的完整列表,请参阅管道一文。For a full list of sections and properties available for defining activities, see the Pipelines article. 本部分提供 MySQL 源支持的属性列表。This section provides a list of properties supported by MySQL source.

以 MySQL 作为源MySQL as source

从 MySQL 复制数据时,复制活动的 source 节支持以下属性:To copy data from MySQL, the following properties are supported in the copy activity source section:

属性Property 说明Description 必须Required
typetype 复制活动 source 的 type 属性必须设置为:MySqlSourceThe type property of the copy activity source must be set to: MySqlSource Yes
查询query 使用自定义 SQL 查询读取数据。Use the custom SQL query to read data. 例如:"SELECT * FROM MyTable"For example: "SELECT * FROM MyTable". 否(如果指定了数据集中的“tableName”)No (if "tableName" in dataset is specified)

示例:Example:

"activities":[
    {
        "name": "CopyFromMySQL",
        "type": "Copy",
        "inputs": [
            {
                "referenceName": "<MySQL input dataset name>",
                "type": "DatasetReference"
            }
        ],
        "outputs": [
            {
                "referenceName": "<output dataset name>",
                "type": "DatasetReference"
            }
        ],
        "typeProperties": {
            "source": {
                "type": "MySqlSource",
                "query": "SELECT * FROM MyTable"
            },
            "sink": {
                "type": "<sink type>"
            }
        }
    }
]

如果使用 RelationalSource 类型源,该源仍按原样受支持,但我们建议今后使用新源。If you were using RelationalSource typed source, it is still supported as-is, while you are suggested to use the new one going forward.

MySQL 的数据类型映射Data type mapping for MySQL

从 MySQL 复制数据时,以下映射用于从 MySQL 数据类型映射到 Azure 数据工厂临时数据类型。When copying data from MySQL, the following mappings are used from MySQL data types to Azure Data Factory interim data types. 若要了解复制活动如何将源架构和数据类型映射到接收器,请参阅架构和数据类型映射See Schema and data type mappings to learn about how copy activity maps the source schema and data type to the sink.

MySQL 数据类型MySQL data type 数据工厂临时数据类型Data factory interim data type
bigint Int64
bigint unsigned Decimal
bit(1) Boolean
bit(M), M>1 Byte[]
blob Byte[]
bool Int16
char String
date Datetime
datetime Datetime
decimal Decimal, String
double Double
double precision Double
enum String
float Single
int Int32
int unsigned Int64
integer Int32
integer unsigned Int64
long varbinary Byte[]
long varchar String
longblob Byte[]
longtext String
mediumblob Byte[]
mediumint Int32
mediumint unsigned Int64
mediumtext String
numeric Decimal
real Double
set String
smallint Int16
smallint unsigned Int32
text String
time TimeSpan
timestamp Datetime
tinyblob Byte[]
tinyint Int16
tinyint unsigned Int16
tinytext String
varchar String
year Int

Lookup 活动属性Lookup activity properties

若要了解有关属性的详细信息,请查看 Lookup 活动To learn details about the properties, check Lookup activity.

后续步骤Next steps

有关 Azure 数据工厂中复制活动支持作为源和接收器的数据存储的列表,请参阅支持的数据存储For a list of data stores supported as sources and sinks by the copy activity in Azure Data Factory, see supported data stores.