对 LUIS 门户中的 1000 个话语执行批处理测试Batch testing with 1000 utterances in LUIS portal

批处理测试会验证活动训练版本,以判断其预测准确性。Batch testing validates your active trained version to measure its prediction accuracy. 批处理测试可帮助你查看活动版本中每个意向和实体的准确性,并使用图表显示结果。A batch test helps you view the accuracy of each intent and entity in your active version, displaying results with a chart. 查看批处理测试结果以采取适当的措施来提高准确性,例如,如果应用经常无法识别正确的意向或在话语中标记实体,则向意向添加更多示例话语。Review the batch test results to take appropriate action to improve accuracy, such as adding more example utterances to an intent if your app frequently fails to identify the correct intent or labeling entities within the utterance.

批处理测试的组数据Group data for batch test

对于 LUIS 来说,用于批处理测试的表达必须是全新,这一点很重要。It is important that utterances used for batch testing are new to LUIS. 如果有话语数据集,请将话语划分为三个集:添加到意向的示例话语、从已发布的终结点接收的话语,以及在训练 LUIS 后用于对其进行批处理测试的话语。If you have a data set of utterances, divide the utterances into three sets: example utterances added to an intent, utterances received from the published endpoint, and utterances used to batch test LUIS after it is trained.

话语数据集A data set of utterances

提交话语批处理文件(称为数据集 ),以用于批处理测试。Submit a batch file of utterances, known as a data set, for batch testing. 该数据集是一个 JSON 格式的文件,包含最多 1,000 个标记为非重复 的话语。The data set is a JSON-formatted file containing a maximum of 1,000 labeled non-duplicate utterances. 可以在一个应用中测试最多 10 个数据集。You can test up to 10 data sets in an app. 如果需要测试更多数据集,请删除数据集,然后添加新数据集。If you need to test more, delete a data set and then add a new one.

规则Rules
*无重复表达*No duplicate utterances
1000 个表达以内1000 utterances or less

*重复项被视为完全字符串匹配,不匹配的项将先进行标记。*Duplicates are considered exact string matches, not matches that are tokenized first.

允许在批处理测试中使用的实体Entities allowed in batch tests

即使批处理文件数据中没有对应的实体,模型中的所有自定义实体也会出现在批处理测试实体筛选器中。All custom entities in the model appear in the batch test entities filter even if there are no corresponding entities in the batch file data.

批处理文件格式Batch file format

批处理文件包含表达。The batch file consists of utterances. 每个表达都必须具有任何你预测可被检测到的机器学习实体随附的预期意向预测。Each utterance must have an expected intent prediction along with any machine-learned entities you expect to be detected.

使用实体的意向的批处理语法模板Batch syntax template for intents with entities

使用以下模板启动批处理文件:Use the following template to start your batch file:

[
  {
    "text": "example utterance goes here",
    "intent": "intent name goes here",
    "entities": 
    [
        {
            "entity": "entity name 1 goes here",
            "startPos": 14,
            "endPos": 23
        },
        {
            "entity": "entity name 2 goes here",
            "startPos": 14,
            "endPos": 23
        }
    ]
  }
]

批处理文件使用 startPos 和 endPos 属性来记录实体的开始和结束。The batch file uses the startPos and endPos properties to note the beginning and end of an entity. 值从零开始,不得以空格开始或结束。The values are zero-based and should not begin or end on a space. 这与使用 startIndex 和 endIndex 属性的查询日志不同。This is different from the query logs, which use startIndex and endIndex properties.

批处理测试中的角色Roles in batch testing

注意

批处理测试中不支持实体角色。Entity roles are not supported in batch testing.

不使用实体的意向的批处理语法模板Batch syntax template for intents without entities

使用以下模板启动没有实体的批处理文件:Use the following template to start your batch file without entities:

[
  {
    "text": "example utterance goes here",
    "intent": "intent name goes here",
    "entities": []
  }
]

如果不想测试实体,请包含 entities 属性并将值设置为空数组 []If you do not want to test entities, include the entities property and set the value as an empty array, [].

导入批处理文件的常见错误Common errors importing a batch

常见错误包括:Common errors include:

  • 超过 1,000 个表达More than 1,000 utterances
  • 不具有实体属性的话语 JSON 对象。An utterance JSON object that doesn't have an entities property. 此属性可以是空数组。The property can be an empty array.
  • 在多个实体中标记的字词Word(s) labeled in multiple entities
  • 实体标签以空格开头或结尾。Entity label starting or ending on a space.

批处理测试状态Batch test state

LUIS 跟踪每个数据集的最后一次测试的状态。LUIS tracks the state of each dataset's last test. 这包括大小(批处理中的表达数)、上次运行日期和最后结果(成功预测的表达数)。This includes the size (number of utterances in the batch), last run date, and last result (number of successfully predicted utterances).

批处理测试结果Batch test results

批处理测试结果是散点图,称为错误矩阵。The batch test result is a scatter graph, known as an error matrix. 此图表对批处理文件中的话语和当前模型的预测意向和实体进行四个方面的比较。This graph is a 4-way comparison of the utterances in the batch file and the current model's predicted intent and entities.

“假正”和“假负”部分上的数据点指示错误,应对其进行调查。Data points on the False Positive and False Negative sections indicate errors, which should be investigated. 如果所有数据点都在“真正”和“真负”部分上,则你的应用在此数据集上的准确性很好。If all data points are on the True Positive and True Negative sections, then your app's accuracy is perfect on this dataset.

图表的四个部分

此图表可基于 LUIS 的当前定型来帮助你查找 LUIS 预测不正确的表达。This chart helps you find utterances that LUIS predicts incorrectly based on its current training. 结果将按图表的每个区域来显示。The results are displayed per region of the chart. 选择图表上的单个点,以查看表达信息,或选择区域名称,以查看该区域的表达结果。Select individual points on the graph to review the utterance information or select region name to review utterance results in that region.

批处理测试

结果中的错误Errors in the results

批处理测试中的错误指示意向未按批处理文件中的指示进行预测。Errors in the batch test indicate intents that are not predicted as noted in the batch file. 在图表的两个红色部分中指示错误。Errors are indicated in the two red sections of the chart.

假正部分指示表达匹配了不应匹配的意向或实体。The false positive section indicates that an utterance matched an intent or entity when it shouldn't have. 假负指示表达未匹配应匹配的意向或实体。The false negative indicates an utterance did not match an intent or entity when it should have.

修复批处理错误Fixing batch errors

如果在批处理测试中出现错误,可以向意向添加更多表达,和/或在实体中标记更多表达,以帮助 LUIS 在意向间进行区分。If there are errors in the batch testing, you can either add more utterances to an intent, and/or label more utterances with the entity to help LUIS make the discrimination between intents. 如果你已添加了表达,且对其进行了标记,但在批处理测试中仍收到预测错误,请考虑添加[短语列表功能,其中包含特定于域的词汇,以帮助 LUIS 更快地理解。If you have added utterances, and labeled them, and still get prediction errors in batch testing, consider adding a phrase list feature with domain-specific vocabulary to help LUIS learn faster.

后续步骤Next steps