Azure 数据工厂中的 JSON 格式JSON format in Azure Data Factory

• 02/01/2021
• • W

适用于： Azure 数据工厂 Azure Synapse Analytics

如果要 分析 JSON 文件或以 JSON 格式写入数据，请遵循本文中的说明。Follow this article when you want to parse the JSON files or write the data into JSON format.

以下连接器支持 JSON 格式：Amazon S3、Azure Blob、Azure Data Lake Storage Gen2、Azure 文件存储、文件系统、FTP、Google 云存储、HDFS、HTTP 和 SFTP。JSON format is supported for the following connectors: Amazon S3, Azure Blob, Azure Data Lake Storage Gen2, Azure File Storage, File System, FTP, Google Cloud Storage, HDFS, HTTP, and SFTP.

数据集属性Dataset properties

有关可用于定义数据集的各部分和属性的完整列表，请参阅数据集一文。For a full list of sections and properties available for defining datasets, see the Datasets article. 本部分提供 JSON 数据集支持的属性列表。This section provides a list of properties supported by the JSON dataset.

下面是 Azure Blob 存储上的 JSON 数据集的示例：Below is an example of JSON dataset on Azure Blob Storage:

{
    "name": "JSONDataset",
    "properties": {
        "type": "Json",
        "linkedServiceName": {
            "referenceName": "<Azure Blob Storage linked service name>",
            "type": "LinkedServiceReference"
        },
        "schema": [ < physical schema, optional, retrievable during authoring > ],
        "typeProperties": {
            "location": {
                "type": "AzureBlobStorageLocation",
                "container": "containername",
                "folderPath": "folder/subfolder",
            },
            "compression": {
                "type": "gzip"
            }
        }
    }
}

复制活动属性Copy activity properties

有关可用于定义活动的各部分和属性的完整列表，请参阅管道一文。For a full list of sections and properties available for defining activities, see the Pipelines article. 本部分提供 JSON 源和接收器支持的属性列表。This section provides a list of properties supported by the JSON source and sink.

若要了解如何从 JSON 文件中提取数据并将其映射到接收器数据存储/格式（反之亦然），请参阅架构映射。Learn about how to extract data from JSON files and map to sink data store/format or vice versa from schema mapping.

以 JSON 作为源JSON as source

复制活动的 __source* * 节支持以下属性。The following properties are supported in the copy activity __source** section.

formatSettings 下支持的“JSON 读取设置”：Supported JSON read settings under formatSettings:

JSON 作为接收器JSON as sink

复制活动的 _sink* 节支持以下属性。The following properties are supported in the copy activity __sink** section.

formatSettings 下支持的 JSON 写入设置：Supported JSON write settings under formatSettings:

JSON 文件模式JSON file patterns

从 JSON 文件复制数据时，复制活动可自动检测并分析以下 JSON 文件模式。When copying data from JSON files, copy activity can automatically detect and parse the following patterns of JSON files. 将数据写入 JSON 文件时，可以在复制活动接收器上配置文件模式。When writing data to JSON files, you can configure the file pattern on copy activity sink.

• 类型 I：setOfObjectsType I: setOfObjects

  每个文件都包含单一对象、JSON Lines 或串联的对象。Each file contains single object, JSON lines, or concatenated objects.

  • 单一对象 JSON 示例single object JSON example

    {
        "time": "2015-04-29T07:12:20.9100000Z",
        "callingimsi": "466920403025604",
        "callingnum1": "678948008",
        "callingnum2": "567834760",
        "switch1": "China",
        "switch2": "Germany"
    }
    

  • JSON Lines（接收器的默认值）JSON Lines (default for sink)

    {"time":"2015-04-29T07:12:20.9100000Z","callingimsi":"466920403025604","callingnum1":"678948008","callingnum2":"567834760","switch1":"China","switch2":"Germany"}
    {"time":"2015-04-29T07:13:21.0220000Z","callingimsi":"466922202613463","callingnum1":"123436380","callingnum2":"789037573","switch1":"US","switch2":"UK"}
    {"time":"2015-04-29T07:13:21.4370000Z","callingimsi":"466923101048691","callingnum1":"678901578","callingnum2":"345626404","switch1":"Germany","switch2":"UK"}
    

  • 串连的 JSON 示例concatenated JSON example

    {
        "time": "2015-04-29T07:12:20.9100000Z",
        "callingimsi": "466920403025604",
        "callingnum1": "678948008",
        "callingnum2": "567834760",
        "switch1": "China",
        "switch2": "Germany"
    }
    {
        "time": "2015-04-29T07:13:21.0220000Z",
        "callingimsi": "466922202613463",
        "callingnum1": "123436380",
        "callingnum2": "789037573",
        "switch1": "US",
        "switch2": "UK"
    }
    {
        "time": "2015-04-29T07:13:21.4370000Z",
        "callingimsi": "466923101048691",
        "callingnum1": "678901578",
        "callingnum2": "345626404",
        "switch1": "Germany",
        "switch2": "UK"
    }
    

• 类型 II：arrayOfObjectsType II: arrayOfObjects

  每个文件包含对象的数组。Each file contains an array of objects.

  [
      {
          "time": "2015-04-29T07:12:20.9100000Z",
          "callingimsi": "466920403025604",
          "callingnum1": "678948008",
          "callingnum2": "567834760",
          "switch1": "China",
          "switch2": "Germany"
      },
      {
          "time": "2015-04-29T07:13:21.0220000Z",
          "callingimsi": "466922202613463",
          "callingnum1": "123436380",
          "callingnum2": "789037573",
          "switch1": "US",
          "switch2": "UK"
      },
      {
          "time": "2015-04-29T07:13:21.4370000Z",
          "callingimsi": "466923101048691",
          "callingnum1": "678901578",
          "callingnum2": "345626404",
          "switch1": "Germany",
          "switch2": "UK"
      }
  ]
  

映射数据流属性Mapping data flow properties

在映射数据流中，你可以在下列数据存储中以 JSON 格式进行读取和写入：Azure Blob 存储和 Azure Data Lake Storage Gen2。In mapping data flows, you can read and write to JSON format in the following data stores: Azure Blob Storage, and Azure Data Lake Storage Gen2.

源属性Source properties

下表列出了 json 源支持的属性。The below table lists the properties supported by a json source. 你可以在“源选项”选项卡中编辑这些属性。You can edit these properties in the Source options tab.

源格式选项Source format options

在数据流中使用 JSON 数据集作为源时，你可以设置五个额外的设置。Using a JSON dataset as a source in your data flow allows you to set five additional settings. 可以在“源选项”选项卡的“JSON 设置”可折叠列表下找到这些设置。 。These settings can be found under the JSON settings accordion in the Source Options tab.

默认Default

默认情况下，将使用以下格式读取 JSON 数据。By default, JSON data is read in the following format.

{ "json": "record 1" }
{ "json": "record 2" }
{ "json": "record 3" }

单个文档Single document

如果选择了“单个文档”，则映射数据流会从每个文件中读取一个 JSON 文档。If Single document is selected, mapping data flows read one JSON document from each file.

File1.json
{
    "json": "record 1"
}
File2.json
{
    "json": "record 2"
}
File3.json
{
    "json": "record 3"
}

不带引号的列名称Unquoted column names

如果选择了“不带引号的列名称”，则映射数据流会读取不在引号中的 JSON 列。If Unquoted column names is selected, mapping data flows reads JSON columns that aren't surrounded by quotes.

{ json: "record 1" }
{ json: "record 2" }
{ json: "record 3" }

具有注释Has comments

如果 JSON 数据具有 C 或 C++ 样式的注释，请选择“具有注释”。Select Has comments if the JSON data has C or C++ style commenting.

{ "json": /** comment **/ "record 1" }
{ "json": "record 2" }
{ /** comment **/ "json": "record 3" }

带单引号Single quoted

如果 JSON 字段和值使用单引号而非双引号，请选择“带单引号”。Select Single quoted if the JSON fields and values use single quotes instead of double quotes.

{ 'json': 'record 1' }
{ 'json': 'record 2' }
{ 'json': 'record 3' }

使用反斜杠转义Backslash escaped

如果使用反斜杠对 JSON 数据中的字符进行转义，请选择“使用反斜杠转义”。Select Backslash escaped if backslashes are used to escape characters in the JSON data.

{ "json": "record 1" }
{ "json": "\} \" \' \\ \n \\n record 2" }
{ "json": "record 3" }

接收器属性Sink Properties

下表列出了 json 接收器支持的属性。The below table lists the properties supported by a json sink. 可以在“设置”选项卡中编辑这些属性。You can edit these properties in the Settings tab.

在派生列中创建 JSON 结构Creating JSON structures in a derived column

你可以通过派生列表达式生成器将复杂的列添加到数据流中。You can add a complex column to your data flow via the derived column expression builder. 在派生列转换中，添加一个新列并通过单击蓝色框打开表达式生成器。In the derived column transformation, add a new column and open the expression builder by clicking on the blue box. 若要使列成为复杂列，可以手动输入 JSON 结构，或者使用 UX 以交互方式添加子列。To make a column complex, you can enter the JSON structure manually or use the UX to add subcolumns interactively.

使用表达式生成器 UXUsing the expression builder UX

在输出架构侧窗格中，将鼠标指针悬停在某个列上，并单击加号图标。In the output schema side pane, hover over a column and click the plus icon. 选择“添加子列”，使列成为复杂类型。Select Add subcolumn to make the column a complex type.

你可以通过相同的方式添加更多的列和子列。You can add additional columns and subcolumns in the same way. 对于每个非复杂字段，可以在右侧的表达式编辑器中添加表达式。For each non-complex field, an expression can be added in the expression editor to the right.

手动输入 JSON 结构Entering the JSON structure manually

若要手动添加 JSON 结构，请添加一个新列，然后在编辑器中输入表达式。To manually add a JSON structure, add a new column and enter the expression in the editor. 表达式采用以下常规格式：The expression follows the following general format:

@(
    field1=0,
    field2=@(
        field1=0
    )
)

如果为名为“complexColumn”的列输入了此表达式，则会将其作为以下 JSON 写入到接收器：If this expression were entered for a column named "complexColumn", then it would be written to the sink as the following JSON:

{
    "complexColumn": {
        "field1": 0,
        "field2": {
            "field1": 0
        }
    }
}

完整分层定义的示例手动脚本Sample manual script for complete hierarchical definition

@(
    title=Title,
    firstName=FirstName,
    middleName=MiddleName,
    lastName=LastName,
    suffix=Suffix,
    contactDetails=@(
        email=EmailAddress,
        phone=Phone
    ),
    address=@(
        line1=AddressLine1,
        line2=AddressLine2,
        city=City,
        state=StateProvince,
        country=CountryRegion,
        postCode=PostalCode
    ),
    ids=[
        toString(CustomerID), toString(AddressID), rowguid
    ]
)

后续步骤Next steps

• 复制活动概述Copy activity overview
• 映射数据流Mapping data flow
• Lookup 活动Lookup activity
• GetMetadata 活动GetMetadata activity