使用 Node.js 以编程方式生成 LUIS 应用Build a LUIS app programmatically using Node.js

LUIS 提供与 LUIS 网站功能相同的编程 API。LUIS provides a programmatic API that does everything that the LUIS website does. 如果有预先存在的数据,这样可以节省时间,而且以编程方式创建 LUIS 应用比手动输入信息快。This can save time when you have pre-existing data and it would be faster to create a LUIS app programmatically than by entering information by hand.

注意

此文档尚未使用最新 LUIS 门户的文本和屏幕截图进行更新。This document has not been updated with text and screenshots for the latest LUIS portal.

必备条件Prerequisites

  • 登录 LUIS 网站,并在“帐户设置”中找到创作密钥Sign in to the LUIS website and find your authoring key in Account Settings. 使用此密钥调用 Authoring API。You use this key to call the Authoring APIs.
  • 如果没有 Azure 订阅,可在开始前创建一个试用帐户If you don't have an Azure subscription, create a Trial before you begin.
  • 本文从用户请求的一家虚拟公司的 CSV 格式日志文件开始。This article starts with a CSV for a hypothetical company's log files of user requests. 可从此处下载。Download it here.
  • 使用 NPM 安装最新的 Node.js。Install the latest Node.js with NPM. 此处下载它。Download it from here.
  • [建议] 用于 IntelliSense 和调试的 Visual Studio Code 可从此处免费下载。[Recommended] Visual Studio Code for IntelliSense and debugging, download it from here for free.

本文中的所有代码都可在 Azure-Samples 语言理解 GitHub 存储库中找到。All of the code in this article is available on the Azure-Samples Language Understanding GitHub repository.

将预先存在的数据映射到意向和实体Map preexisting data to intents and entities

即使系统在创建时未考虑使用 LUIS,如果它包含映射到用户不同操作的文本数据,也许能够从现有用户输入类别映射到 LUIS 中的意向。Even if you have a system that wasn't created with LUIS in mind, if it contains textual data that maps to different things users want to do, you might be able to come up with a mapping from the existing categories of user input to intents in LUIS. 如果可标识用户所说的重要单词或短语,这些单词可能会映射到实体。If you can identify important words or phrases in what the users said, these words might map to entities.

打开 IoT.csv 文件。Open the IoT.csv file. 它包含对虚构家庭自动化服务的用户查询日志,包括分类方式、用户所说的内容以及一些包含有用信息的列。It contains a log of user queries to a hypothetical home automation service, including how they were categorized, what the user said, and some columns with useful information pulled out of them.

预先存在数据的 CSV 文件

可以看到“RequestType”列可能是意向,“Request”列显示了一个示例陈述********。You see that the RequestType column could be intents, and the Request column shows an example utterance. 如果其他字段出现在陈述中,则可能是实体。The other fields could be entities if they occur in the utterance. 由于有意向、实体和示例陈述,因此需要一个简单的示例应用。Because there are intents, entities, and example utterances, you have the requirements for a simple, sample app.

从非 LUIS 数据生成 LUIS 应用的步骤Steps to generate a LUIS app from non-LUIS data

从 CSV 文件生成新的 LUIS 应用:To generate a new LUIS app from the CSV file:

  • 分析 CSV 文件中的数据:Parse the data from the CSV file:
    • 转换为可以使用创作 API 上传到 LUIS 的格式。Convert to a format that you can upload to LUIS using the Authoring API.
    • 从已分析的数据收集有关意向和实体的信息。From the parsed data, gather information about intents and entities.
  • 调用创作 API 以执行以下操作:Make authoring API calls to:
    • 创建应用。Create the app.
    • 添加从已分析数据中收集的意向和实体。Add intents and entities that were gathered from the parsed data.
    • 创建 LUIS 应用后,可以从已分析的数据中添加示例陈述。Once you have created the LUIS app, you can add the example utterances from the parsed data.

可以在 index.js 文件的最后一部分中看到此程序流。You can see this program flow in the last part of the index.js file. 复制或下载此代码并将其保存在 index.jsCopy or download this code and save it in index.js.

var path = require('path');

const parse = require('./_parse');
const createApp = require('./_create');
const addEntities = require('./_entities');
const addIntents = require('./_intents');
const upload = require('./_upload');

// Change these values
const LUIS_authoringKey = "YOUR_AUTHORING_KEY";
const LUIS_appName = "Sample App - build from IoT csv file";
const LUIS_appCulture = "en-us"; 
const LUIS_versionId = "0.1";

// NOTE: final output of add-utterances api named utterances.upload.json
const downloadFile = "./IoT.csv";
const uploadFile = "./utterances.json"

// The app ID is returned from LUIS when your app is created
var LUIS_appId = ""; // default app ID
var intents = [];
var entities = [];


/* add utterances parameters */
var configAddUtterances = {
    LUIS_subscriptionKey: LUIS_authoringKey,
    LUIS_appId: LUIS_appId,
    LUIS_versionId: LUIS_versionId,
    inFile: path.join(__dirname, uploadFile),
    batchSize: 100,
    uri: "https://api.cognitive.azure.cn/luis/api/v2.0/apps/{appId}/versions/{versionId}/examples"
};

/* create app parameters */
var configCreateApp = {
    LUIS_subscriptionKey: LUIS_authoringKey,
    LUIS_versionId: LUIS_versionId,
    appName: LUIS_appName,
    culture: LUIS_appCulture,
    uri: "https://api.cognitive.azure.cn/luis/api/v2.0/apps/"
};

/* add intents parameters */
var configAddIntents = {
    LUIS_subscriptionKey: LUIS_authoringKey,
    LUIS_appId: LUIS_appId,
    LUIS_versionId: LUIS_versionId,
    intentList: intents,
    uri: "https://api.cognitive.azure.cn/luis/api/v2.0/apps/{appId}/versions/{versionId}/intents"
};

/* add entities parameters */
var configAddEntities = {
    LUIS_subscriptionKey: LUIS_authoringKey,
    LUIS_appId: LUIS_appId,
    LUIS_versionId: LUIS_versionId,
    entityList: entities,
    uri: "https://api.cognitive.azure.cn/luis/api/v2.0/apps/{appId}/versions/{versionId}/entities"
};

/* input and output files for parsing CSV */
var configParse = {
    inFile: path.join(__dirname, downloadFile),
    outFile: path.join(__dirname, uploadFile)
};

// Parse CSV
parse(configParse)
    .then((model) => {
        // Save intent and entity names from parse
        intents = model.intents;
        entities = model.entities;
        // Create the LUIS app
        return createApp(configCreateApp);

    }).then((appId) => {
        // Add intents
        LUIS_appId = appId;
        configAddIntents.LUIS_appId = appId;
        configAddIntents.intentList = intents;
        return addIntents(configAddIntents);

    }).then(() => {
        // Add entities
        configAddEntities.LUIS_appId = LUIS_appId;
        configAddEntities.entityList = entities;
        return addEntities(configAddEntities);

    }).then(() => {
        // Add example utterances to the intents in the app
        configAddUtterances.LUIS_appId = LUIS_appId;
        return upload(configAddUtterances);

    }).catch(err => {
        console.log(err.message);
    });

分析 CSVParse the CSV

包含 CSV 中的陈述的列条目必须被分析为 LUIS 可以理解的 JSON 格式。The column entries that contain the utterances in the CSV have to be parsed into a JSON format that LUIS can understand. 此 JSON 格式必须包含一个 intentName 字段,该字段标识陈述的意向。This JSON format must contain an intentName field that identifies the intent of the utterance. 它还必须包含一个 entityLabels 字段,如果陈述中没有实体,该字段可以是空的。It must also contain an entityLabels field, which can be empty if there are no entities in the utterance.

例如,“打开灯”条目映射到此 JSON:For example, the entry for "Turn on the lights" maps to this JSON:

        {
            "text": "Turn on the lights",
            "intentName": "TurnOn",
            "entityLabels": [
                {
                    "entityName": "Operation",
                    "startCharIndex": 5,
                    "endCharIndex": 6
                },
                {
                    "entityName": "Device",
                    "startCharIndex": 12,
                    "endCharIndex": 17
                }
            ]
        }

在此示例中,intentName 来自 CSV 文件中“Request”列标题下的用户请求,entityName 来自具有密钥信息的其他列****。In this example, the intentName comes from the user request under the Request column heading in the CSV file, and the entityName comes from the other columns with key information. 例如,如果有“操作”或“设备”条目,并且该字符串也出现在实际请求中,则可将其标记为实体********。For example, if there's an entry for Operation or Device, and that string also occurs in the actual request, then it can be labeled as an entity. 下面的代码演示此分析过程。The following code demonstrates this parsing process. 可以复制或下载它并将其保存在 _parse.jsYou can copy or download it and save it to _parse.js.

// node 7.x
// built with streams for larger files

const fse = require('fs-extra');
const path = require('path');
const lineReader = require('line-reader');
const babyparse = require('babyparse');
const Promise = require('bluebird');

const intent_column = 0;
const utterance_column = 1;
var entityNames = [];

var eachLine = Promise.promisify(lineReader.eachLine);

function listOfIntents(intents) {
    return intents.reduce(function (a, d) {
        if (a.indexOf(d.intentName) === -1) {
            a.push(d.intentName);
        }
        return a;
    }, []);

}

function listOfEntities(utterances) {
    return utterances.reduce(function (a, d) {        
        d.entityLabels.forEach(function(entityLabel) {
            if (a.indexOf(entityLabel.entityName) === -1) {
                a.push(entityLabel.entityName);
            }     
        }, this);
        return a;
    }, []);
}

var utterance = function (rowAsString) {

    let json = {
        "text": "",
        "intentName": "",
        "entityLabels": [],
    };

    if (!rowAsString) return json;

    let dataRow = babyparse.parse(rowAsString);
    // Get intent name and utterance text 
    json.intentName = dataRow.data[0][intent_column];
    json.text = dataRow.data[0][utterance_column];
    // For each column heading that may be an entity, search for the element in this column in the utterance.
    entityNames.forEach(function (entityName) {
        entityToFind = dataRow.data[0][entityName.column];
        if (entityToFind != "") {
            strInd = json.text.indexOf(entityToFind);
            if (strInd > -1) {
                let entityLabel = {
                    "entityName": entityName.name,
                    "startCharIndex": strInd,
                    "endCharIndex": strInd + entityToFind.length - 1
                }
                json.entityLabels.push(entityLabel);
            }
        }
    }, this);
    return json;

};


const convert = async (config) => {

    try {

        var i = 0;

        // get inFile stream
        inFileStream = await fse.createReadStream(config.inFile, 'utf-8')

        // create out file
        var myOutFile = await fse.createWriteStream(config.outFile, 'utf-8');
        var utterances = [];

        // read 1 line at a time
        return eachLine(inFileStream, (line) => {

            // skip first line with headers
            if (i++ == 0) {

                // csv to baby parser object
                let dataRow = babyparse.parse(line);

                // populate entityType list
                var index = 0;
                dataRow.data[0].forEach(function (element) {
                    if ((index != intent_column) && (index != utterance_column)) {
                        entityNames.push({ name: element, column: index });
                    }
                    index++;
                }, this);

                return;
            }

            // transform utterance from csv to json
            utterances.push(utterance(line));

        }).then(() => {
            console.log("intents: " + JSON.stringify(listOfIntents(utterances)));
            console.log("entities: " + JSON.stringify(listOfEntities(utterances)));
            myOutFile.write(JSON.stringify({ "converted_date": new Date().toLocaleString(), "utterances": utterances }));
            myOutFile.end();
            console.log("parse done");
            console.log("JSON file should contain utterances. Next step is to create an app with the intents and entities it found.");

            var model = 
            {
                intents: listOfIntents(utterances),
                entities: listOfEntities(utterances)                
            }
            return model;

        });

    } catch (err) {
        throw err;
    }

}

module.exports = convert;

创建 LUIS 应用Create the LUIS app

将数据分析到 JSON 后,将其添加到 LUIS 应用。Once the data has been parsed into JSON, add it to a LUIS app. 下面的代码创建 LUIS 应用。The following code creates the LUIS app. 复制或下载它,并将其保存到 _create.jsCopy or download it, and save it into _create.js.

// node 7.x
// uses async/await - promises

var rp = require('request-promise');
var fse = require('fs-extra');
var path = require('path');



// main function to call
// Call Apps_Create
var createApp = async (config) => {
    
        try {
    
            // JSON for the request body
            // { "name": MyAppName, "culture": "en-us"}
            var jsonBody = { 
                "name": config.appName, 
                "culture": config.culture
            };
    
            // Create a LUIS app
            var createAppPromise = callCreateApp({
                uri: config.uri,
                method: 'POST',
                headers: {
                    'Ocp-Apim-Subscription-Key': config.LUIS_subscriptionKey
                },
                json: true,
                body: jsonBody
            });
    
            let results = await createAppPromise;

            // Create app returns an app ID
            let appId = results.response;  
            console.log(`Called createApp, created app with ID ${appId}`);
            return appId;

    
        } catch (err) {
            console.log(`Error creating app:  ${err.message} `);
            throw err;
        }
    
    }

// Send JSON as the body of the POST request to the API
var callCreateApp = async (options) => {
    try {

        var response; 
        if (options.method === 'POST') {
            response = await rp.post(options);
        } else if (options.method === 'GET') { // TODO: There's no GET for create app
            response = await rp.get(options);
        }
        // response from successful create should be the new app ID
        return { response };

    } catch (err) {
        throw err;
    }
} 

module.exports = createApp;

添加意向Add intents

创建应用后需要向其添加意向。Once you have an app, you need to intents to it. 下面的代码创建 LUIS 应用。The following code creates the LUIS app. 复制或下载它,并将其保存到 _intents.jsCopy or download it, and save it into _intents.js.


var rp = require('request-promise');
var fse = require('fs-extra');
var path = require('path');
var request = require('requestretry');

// time delay between requests
const delayMS = 1000;

// retry recount
const maxRetry = 5;

// retry request if error or 429 received
var retryStrategy = function (err, response, body) {
    let shouldRetry = err || (response.statusCode === 429);
    if (shouldRetry) console.log("retrying add intent...");
    return shouldRetry;
}

// Call add-intents
var addIntents = async (config) => {
    var intentPromises = [];
    config.uri = config.uri.replace("{appId}", config.LUIS_appId).replace("{versionId}", config.LUIS_versionId);

    config.intentList.forEach(function (intent) {
        config.intentName = intent;
        try {

            // JSON for the request body
            var jsonBody = {
                "name": config.intentName,
            };

            // Create an intent
            var addIntentPromise = callAddIntent({
                // uri: config.uri,
                url: config.uri,
                fullResponse: false,
                method: 'POST',
                headers: {
                    'Ocp-Apim-Subscription-Key': config.LUIS_subscriptionKey
                },
                json: true,
                body: jsonBody,
                maxAttempts: maxRetry,
                retryDelay: delayMS,
                retryStrategy: retryStrategy
            });
            intentPromises.push(addIntentPromise);

            console.log(`Called addIntents for intent named ${intent}.`);

        } catch (err) {
            console.log(`Error in addIntents:  ${err.message} `);

        }
    }, this);

    let results = await Promise.all(intentPromises);
    console.log(`Results of all promises = ${JSON.stringify(results)}`);
    let response = results;


}

// Send JSON as the body of the POST request to the API
var callAddIntent = async (options) => {
    try {

        var response;        
        response = await request(options);
        return { response: response };

    } catch (err) {
        console.log(`Error in callAddIntent:  ${err.message} `);
    }
}

module.exports = addIntents;

添加实体Add entities

以下代码向 LUIS 应用添加实体。The following code adds the entities to the LUIS app. 复制或下载它,并将其保存到 _entities.jsCopy or download it, and save it into _entities.js.

// node 7.x
// uses async/await - promises

const request = require("requestretry");
var rp = require('request-promise');
var fse = require('fs-extra');
var path = require('path');

// time delay between requests
const delayMS = 1000;

// retry recount
const maxRetry = 5;

// retry request if error or 429 received
var retryStrategy = function (err, response, body) {
    let shouldRetry = err || (response.statusCode === 429);
    if (shouldRetry) console.log("retrying add entity...");
    return shouldRetry;
}

// main function to call
// Call add-entities
var addEntities = async (config) => {
    var entityPromises = [];
    config.uri = config.uri.replace("{appId}", config.LUIS_appId).replace("{versionId}", config.LUIS_versionId);

    config.entityList.forEach(function (entity) {
        try {
            config.entityName = entity;
            // JSON for the request body
            // { "name": MyEntityName}
            var jsonBody = {
                "name": config.entityName,
            };

            // Create an app
            var addEntityPromise = callAddEntity({
                url: config.uri,
                fullResponse: false,
                method: 'POST',
                headers: {
                    'Ocp-Apim-Subscription-Key': config.LUIS_subscriptionKey
                },
                json: true,
                body: jsonBody,
                maxAttempts: maxRetry,
                retryDelay: delayMS,
                retryStrategy: retryStrategy
            });
            entityPromises.push(addEntityPromise);

            console.log(`called addEntity for entity named ${entity}.`);

        } catch (err) {
            console.log(`Error in addEntities:  ${err.message} `);
            //throw err;
        }
    }, this);
    let results = await Promise.all(entityPromises);
    console.log(`Results of all promises = ${JSON.stringify(results)}`);
    let response = results;// await fse.writeJson(createResults.json, results);


}

// Send JSON as the body of the POST request to the API
var callAddEntity = async (options) => {
    try {

        var response;        
        response = await request(options);
        return { response: response };

    } catch (err) {
        console.log(`error in callAddEntity: ${err.message}`);
    }
}

module.exports = addEntities;

添加表达Add utterances

LUIS 应用中定义了实体和意向后,可以添加陈述。Once the entities and intents have been defined in the LUIS app, you can add the utterances. 下面的代码使用 Utterances_AddBatch API,它允许每次添加最多 100 个陈述。The following code uses the Utterances_AddBatch API, which allows you to add up to 100 utterances at a time. 复制或下载它,并将其保存到 _upload.jsCopy or download it, and save it into _upload.js.

// node 7.x
// uses async/await - promises

var rp = require('request-promise');
var fse = require('fs-extra');
var path = require('path');
var request = require('requestretry');

// time delay between requests
const delayMS = 500;

// retry recount
const maxRetry = 5;

// retry request if error or 429 received
var retryStrategy = function (err, response, body) {
    let shouldRetry = err || (response.statusCode === 429);
    if (shouldRetry) console.log("retrying add examples...");
    return shouldRetry;
}

// main function to call
var upload = async (config) => {

    try{
      
        // read in utterances
        var entireBatch = await fse.readJson(config.inFile);

        // break items into pages to fit max batch size
        var pages = getPagesForBatch(entireBatch.utterances, config.batchSize);

        var uploadPromises = [];

        // load up promise array
        pages.forEach(page => {
            config.uri = "https://api.cognitive.azure.cn/luis/api/v2.0/apps/{appId}/versions/{versionId}/examples".replace("{appId}", config.LUIS_appId).replace("{versionId}", config.LUIS_versionId)
            var pagePromise = sendBatchToApi({
                url: config.uri,
                fullResponse: false,
                method: 'POST',
                headers: {
                    'Ocp-Apim-Subscription-Key': config.LUIS_subscriptionKey
                },
                json: true,
                body: page,
                maxAttempts: maxRetry,
                retryDelay: delayMS,
                retryStrategy: retryStrategy
            });

            uploadPromises.push(pagePromise);
        })

        //execute promise array
        
        let results =  await Promise.all(uploadPromises)
        console.log(`\n\nResults of all promises = ${JSON.stringify(results)}`);
        let response = await fse.writeJson(config.inFile.replace('.json','.upload.json'),results);

        console.log("upload done");

    } catch(err){
        throw err;        
    }

}
// turn whole batch into pages batch 
// because API can only deal with N items in batch
var getPagesForBatch = (batch, maxItems) => {

    try{
        var pages = []; 
        var currentPage = 0;

        var pageCount = (batch.length % maxItems == 0) ? Math.round(batch.length / maxItems) : Math.round((batch.length / maxItems) + 1);

        for (let i = 0;i<pageCount;i++){

            var currentStart = currentPage * maxItems;
            var currentEnd = currentStart + maxItems;
            var pagedBatch = batch.slice(currentStart,currentEnd);

            var j = 0;
            pagedBatch.forEach(item=>{
                item.ExampleId = j++;
            });

            pages.push(pagedBatch);

            currentPage++;
        }
        return pages;
    }catch(err){
        throw(err);
    }
}

// send json batch as post.body to API
var sendBatchToApi = async (options) => {
    try {

        var response = await request(options);
        //return {page: options.body, response:response};
        return {response:response};
    }catch(err){
        throw err;
    }   
}   

module.exports = upload;

运行代码Run the code

安装 Node.js 依赖项Install Node.js dependencies

在终端/命令行中,从 NPM 安装 Node.js 依赖项。Install the Node.js dependencies from NPM in the terminal/command line.

> npm install

更改配置设置Change Configuration Settings

若要使用此应用程序,需要将 index.js 文件中的值更改为自己的终结点密钥,并提供希望应用拥有的名称。In order to use this application, you need to change the values in the index.js file to your own endpoint key, and provide the name you want the app to have. 还可以设置应用的区域性或更改版本号。You can also set the app's culture or change the version number.

打开 index.js 文件,并在文件顶部更改这些值。Open the index.js file, and change these values at the top of the file.

// Change these values
const LUIS_programmaticKey = "YOUR_AUTHORING_KEY";
const LUIS_appName = "Sample App";
const LUIS_appCulture = "en-us";
const LUIS_versionId = "0.1";

运行脚本Run the script

使用 Node.js 从终端/命令行运行脚本。Run the script from a terminal/command line with Node.js.

> node index.js

or

> npm start

应用程序进度Application progress

应用程序运行时,命令行显示进度。While the application is running, the command line shows progress. 命令行输出包括 LUIS 的响应格式。The command line output includes the format of the responses from LUIS.

> node index.js
intents: ["TurnOn","TurnOff","Dim","Other"]
entities: ["Operation","Device","Room"]
parse done
JSON file should contain utterances. Next step is to create an app with the intents and entities it found.
Called createApp, created app with ID 314b306c-0033-4e09-92ab-94fe5ed158a2
Called addIntents for intent named TurnOn.
Called addIntents for intent named TurnOff.
Called addIntents for intent named Dim.
Called addIntents for intent named Other.
Results of all calls to addIntent = [{"response":"e7eaf224-8c61-44ed-a6b0-2ab4dc56f1d0"},{"response":"a8a17efd-f01c-488d-ad44-a31a818cf7d7"},{"response":"bc7c32fc-14a0-4b72-bad4-d345d807f965"},{"response":"727a8d73-cd3b-4096-bc8d-d7cfba12eb44"}]
called addEntity for entity named Operation.
called addEntity for entity named Device.
called addEntity for entity named Room.
Results of all calls to addEntity= [{"response":"6a7e914f-911d-4c6c-a5bc-377afdce4390"},{"response":"56c35237-593d-47f6-9d01-2912fa488760"},{"response":"f1dd440c-2ce3-4a20-a817-a57273f169f3"}]
retrying add examples...

Results of add utterances = [{"response":[{"value":{"UtteranceText":"turn on the lights","ExampleId":-67649},"hasError":false},{"value":{"UtteranceText":"turn the heat on","ExampleId":-69067},"hasError":false},{"value":{"UtteranceText":"switch on the kitchen fan","ExampleId":-3395901},"hasError":false},{"value":{"UtteranceText":"turn off bedroom lights","ExampleId":-85402},"hasError":false},{"value":{"UtteranceText":"turn off air conditioning","ExampleId":-8991572},"hasError":false},{"value":{"UtteranceText":"kill the lights","ExampleId":-70124},"hasError":false},{"value":{"UtteranceText":"dim the lights","ExampleId":-174358},"hasError":false},{"value":{"UtteranceText":"hi how are you","ExampleId":-143722},"hasError":false},{"value":{"UtteranceText":"answer the phone","ExampleId":-69939},"hasError":false},{"value":{"UtteranceText":"are you there","ExampleId":-149588},"hasError":false},{"value":{"UtteranceText":"help","ExampleId":-81949},"hasError":false},{"value":{"UtteranceText":"testing the circuit","ExampleId":-11548708},"hasError":false}]}]
upload done

打开 LUIS 应用Open the LUIS app

该脚本完成后,可以登录 LUIS,查看“我的应用”下创建的 LUIS 应用****。Once the script completes, you can sign in to LUIS and see the LUIS app you created under My Apps. 应该能够看到在 TurnOn、TurnOff 和 None 意向下添加的陈述************。You should be able to see the utterances you added under the TurnOn, TurnOff, and None intents.

TurnOn 意向

后续步骤Next steps

其他资源Additional resources

此示例应用程序使用以下 LUIS API:This sample application uses the following LUIS APIs: