Azure 数据工厂中带分隔符的文本格式Delimited text format in Azure Data Factory

• 02/01/2021
• • W

适用于： Azure 数据工厂 Azure Synapse Analytics

若要 分析带分隔符的文本文件或以带分隔符的文本格式写入数据，请遵循此文章中的说明。Follow this article when you want to parse the delimited text files or write the data into delimited text format.

以下连接器支持带分隔符的文本格式：Amazon S3、Azure Blob、Azure Data Lake Storage Gen2、Azure 文件存储、文件系统、FTP、Google 云存储、HDFS、HTTP 和 SFTP。Delimited text format is supported for the following connectors: Amazon S3, Azure Blob, Azure Data Lake Storage Gen2, Azure File Storage, File System, FTP, Google Cloud Storage, HDFS, HTTP, and SFTP.

数据集属性Dataset properties

有关可用于定义数据集的各部分和属性的完整列表，请参阅数据集一文。For a full list of sections and properties available for defining datasets, see the Datasets article. 本部分提供带分隔符的文本数据集支持的属性列表。This section provides a list of properties supported by the delimited text dataset.

下面是 Azure Blob 存储上的带分隔符的文本数据集的示例：Below is an example of delimited text dataset on Azure Blob Storage:

{
    "name": "DelimitedTextDataset",
    "properties": {
        "type": "DelimitedText",
        "linkedServiceName": {
            "referenceName": "<Azure Blob Storage linked service name>",
            "type": "LinkedServiceReference"
        },
        "schema": [ < physical schema, optional, retrievable during authoring > ],
        "typeProperties": {
            "location": {
                "type": "AzureBlobStorageLocation",
                "container": "containername",
                "folderPath": "folder/subfolder",
            },
            "columnDelimiter": ",",
            "quoteChar": "\"",
            "escapeChar": "\"",
            "firstRowAsHeader": true,
            "compressionCodec": "gzip"
        }
    }
}

复制活动属性Copy activity properties

有关可用于定义活动的各部分和属性的完整列表，请参阅管道一文。For a full list of sections and properties available for defining activities, see the Pipelines article. 本部分提供带分隔符的文本源和接收器支持的属性列表。This section provides a list of properties supported by the delimited text source and sink.

带分隔符的文本作为源Delimited text as source

复制活动的 __source* * 节支持以下属性。The following properties are supported in the copy activity __source** section.

formatSettings 下支持的 带分隔符的文本读取设置：Supported delimited text read settings under formatSettings:

"activities": [
    {
        "name": "CopyFromDelimitedText",
        "type": "Copy",
        "typeProperties": {
            "source": {
                "type": "DelimitedTextSource",
                "storeSettings": {
                    "type": "AzureBlobStorageReadSettings",
                    "recursive": true
                },
                "formatSettings": {
                    "type": "DelimitedTextReadSettings",
                    "skipLineCount": 3,
                    "compressionProperties": {
                        "type": "ZipDeflateReadSettings",
                        "preserveZipFileNameAsFolder": false
                    }
                }
            },
            ...
        }
        ...
    }
]

带分隔符的文本作为接收器Delimited text as sink

复制活动的 _sink* 节支持以下属性。The following properties are supported in the copy activity __sink** section.

formatSettings 下支持的 带分隔符的文本写入设置：Supported delimited text write settings under formatSettings:

映射数据流属性Mapping data flow properties

在映射数据流中，可以在以下数据存储中读取和写入带分隔符的文本格式：Azure Blob 存储和 Azure Data Lake Storage Gen2。In mapping data flows, you can read and write to delimited text format in the following data stores: Azure Blob Storage, and Azure Data Lake Storage Gen2.

源属性Source properties

下表列出了带分隔符的文本源支持的属性。The below table lists the properties supported by a delimited text source. 你可以在“源选项”选项卡中编辑这些属性。You can edit these properties in the Source options tab.

源示例Source example

下图是映射数据流中带分隔符的文本源配置的示例。The below image is an example of a delimited text source configuration in mapping data flows.

关联的数据流脚本为：The associated data flow script is:

source(
    allowSchemaDrift: true,
    validateSchema: false,
    multiLineRow: true,
    wildcardPaths:['*.csv']) ~> CSVSource

接收器属性Sink properties

下表列出了带分隔符的文本接收器支持的属性。The below table lists the properties supported by a delimited text sink. 可以在“设置”选项卡中编辑这些属性。You can edit these properties in the Settings tab.

接收器示例Sink example

下图是映射数据流中带分隔符的文本接收器配置的示例。The below image is an example of a delimited text sink configuration in mapping data flows.

关联的数据流脚本为：The associated data flow script is:

CSVSource sink(allowSchemaDrift: true,
    validateSchema: false,
    truncate: true,
    skipDuplicateMapInputs: true,
    skipDuplicateMapOutputs: true) ~> CSVSink

后续步骤Next steps

• 复制活动概述Copy activity overview
• 映射数据流Mapping data flow
• Lookup 活动Lookup activity
• GetMetadata 活动GetMetadata activity