如何使用仪表板来改善应用How to use the Dashboard to improve your app

使用示例言语时查找并修复已训练应用的意向问题。Find and fix problems with your trained app's intents when you are using example utterances. 摘要仪表板显示整体性的应用信息,并突出显示应予以修复的意向。The summary dashboard displays overall app information, with highlights of intents that should be fixed.

查看仪表板分析是一个迭代过程,请在更改和改进模型时重复此过程。Review Dashboard analysis is an iterative process, repeat as you change and improve your model.

对于在意向中不包含任何示例言语的应用(称为仅限模式的应用),此页不会提供相关的分析。This page will not have relevant analysis for apps that do not have any example utterances in the intents, known as pattern-only apps.

在仪表板中可以修复哪些问题?What issues can be fixed from dashboard?

在仪表板中可以解决以下三种问题:The three problems addressed in the dashboard are:

问题Issue 图表颜色Chart color 说明Explanation
数据不平衡Data imbalance - 当示例言语的数量存在显著的差异时,将出现此问题。This occurs when the quantity of example utterances varies significantly. 所有意向需要有大致相同的示例言语数量 - None 意向除外。All intents need to have roughly the same number of example utterances - except the None intent. None 意向的数量应该只占应用中言语总数的 10%-15%。It should only have 10%-15% of the total quantity of utterances in the app.

如果数据不平衡但意向准确度超过特定的阈值,则不会将这种不平衡报告为一种问题。If the data is imbalanced but the intent accuracy is above certain threshold, this imbalance is not reported as an issue.

请从此问题着手 - 它可能是其他问题的根本原因。Start with this issue - it may be the root cause of the other issues.
不明确的预测Unclear predictions 橙色Orange 如果由于使用负采样或者将更多示例言语添加到意向,而导致最前面的意向与下一个意向的评分足够接近,以致下一次训练时评分将会掉转,则就会出现此问题。This occurs when the top intent and the next intent's scores are close enough that they may flip on the next training, due to negative sampling or more example utterances added to intent.
错误的预测Incorrect predictions 红色Red 未针对标记的意向(示例言语所在的意向)预测示例言语时,会出现此问题。This occurs when an example utterance is not predicted for the labeled intent (the intent it is in).

正确的预测以蓝色表示。Correct predictions are represented with the color blue.

仪表板将显示这些问题,告知哪些意向受到影响,并建议采取哪些措施来改进应用。The dashboard shows these issues and tells you which intents are affected and suggests what you should do to improve the app.

训练应用之前Before app is trained

训练应用之前,摘要仪表板不包含任何修复建议。Before you train the app, the summary dashboard does not contain any suggestions for fixes. 训练应用即可看到这些建议。Train your app to see these suggestions.

检查发布状态Check your publishing status

“发布状态”卡片包含有关活动版本的上次发布情况的信息。The Publishing status card contains information about the active version's last publish.

请检查活动的版本是否为要修复的版本。Check that the active version is the version you want to fix.

仪表板将显示应用的外部服务、发布区域,以及聚合的终结点访问次数。

其中还会显示任何外部服务、发布区域,以及聚合的终结点访问次数。This also shows any external services, published regions, and aggregated endpoint hits.

检查训练评估Review training evaluation

“训练评估”卡片按区域显示应用总体准确度的聚合摘要。The Training evaluation card contains the aggregated summary of your app's overall accuracy by area. 评分指示意向的质量。The score indicates intent quality.

“训练评估”卡片包含有关应用总体准确度的信息的第一个区域。

图表以不同的颜色指示正确预测的意向和问题区域。The chart indicates the correctly predicted intents and the problem areas with different colors. 根据建议改善应用时,此评分会提高。As you improve the app with the suggestions, this score increases.

建议的修复按问题类型进行区分,对于你的应用而言最为重要。The suggested fixes are separated out by problem type and are the most significant for your app. 若要按意向检查和修复问题,请使用页面底部的**有错误的意向** 卡片。If you would prefer to review and fix issues per intent, use the Intents with errors card at the bottom of the page.

每个问题区域包含需要修复的意向。Each problem area has intents that need to be fixed. 选择意向名称时,“意向”页将会打开,其中包含一个应用于言语的筛选器。When you select the intent name, the Intent page opens with a filter applied to the utterances. 使用此筛选器可以专注于处理导致问题的言语。This filter allows you to focus on the utterances that are causing the problem.

比较不同版本中的更改Compare changes across versions

对应用进行更改之前创建新版本。Create a new version before making changes to the app. 在新版本中,对意向的示例言语做出建议的更改,然后重新训练。In the new version, make the suggested changes to the intent's example utterances, then train again. 在仪表板页的“训练评估”卡片上,使用“显示已训练版本中的更改”来比较更改。On the Dashboard page's Training evaluation card, use the Show change from trained version to compare the changes.

比较不同版本中的更改

通过添加或编辑示例言语并重新训练来修复版本Fix version by adding or editing example utterances and retraining

修复应用的主要方法是添加或编辑示例言语,然后重新训练。The primary method of fixing your app will be to add or edit example utterances and retrain. 新的或更改的言语需要遵循不同言语的准则。The new or changed utterances need to follow guidelines for varied utterances.

添加示例言语的操作应由具备以下经验的人员来执行:Adding example utterances should be done by someone who:

  • 对于不同意向中的言语具有较高程度的了解。has a high degree of understanding of what utterances are in the different intents.
  • 知道在哪种情况下,一个意向中的言语可能会与另一个意向发生混淆。knows how utterances in one intent may be confused with another intent.
  • 能够决定是否应该将两个经常相互混淆的意向折叠为单个意向。is able to decide if two intents, which are frequently confused with each other, should be collapsed into a single intent. 如果是这种情况,则必须使用实体提取不同的数据。If this is the case, the different data must be pulled out with entities.

检查数据不平衡问题Review data imbalance

请从此问题着手 - 它可能是其他问题的根本原因。Start with this issue - it may be the root cause of the other issues.

数据不平衡意向列表显示需要添加更多言语才能纠正数据不平衡问题的意向。The data imbalance intent list shows intents that need more utterances in order to correct the data imbalance.

若要解决此问题To fix this issue:

  • 请将更多言语添加到意向,然后重新训练。Add more utterances to the intent then train again.

除非摘要仪表板上有建议,否则不要将言语添加到 None 意向。Do not add utterances to the None intent unless that is suggested on the summary dashboard.

提示

使用该页上的第三个部分 - 包含“言语(数目)”设置的“每个意向的言语”可以快速直观地了解哪些意向需要更多的言语。Use the third section on the page, Utterances per intent with the Utterances (number) setting, as a quick visual guide of which intents need more utterances.
使用“言语(数目)”查找存在数据不平衡的意向。Use 'Utterances (number)' to find intents with data imbalance.

检查错误的预测Review incorrect predictions

错误的预测意向列表显示包含言语的意向,这些言语用作特定意向的示例,但已针对不同的意向做了预测。The incorrect prediction intent list shows intents that have utterances, which are used as examples for a specific intent, but are predicted for different intents.

若要解决此问题To fix this issue:

  • 编辑言语,使之与该意向更为相关,然后重新训练。Edit utterances to be more specific to the intent and train again.
  • 如果言语过于接近,请合并意向,然后重新训练。Combine intents if utterances are too closely aligned and train again.

检查不明确的预测Review unclear predictions

不明确的预测意向列表显示包含言语的意向,这些言语的预测评分与它们最靠近的其他言语没有足够大的差异,以致于在使用负采样的情况下,下一次训练时这些言语的第一个意向可能会发生变化。The unclear prediction intent list shows intents with utterances with prediction scores that are not far enough way from their nearest rival, that the top intent for the utterance may change on the next training, due to negative sampling.

若要修复此问题To fix this issue;

  • 编辑言语,使之与该意向更为相关,然后重新训练。Edit utterances to be more specific to the intent and train again.
  • 如果言语过于接近,请合并意向,然后重新训练。Combine intents if utterances are too closely aligned and train again.

每个意向的言语Utterances per intent

此卡片显示不同意向的总体应用运行状况。This card shows the overall app health across the intents. 在修复意向并重新训练的过程中,请不断地概览一下此卡片中的问题。As you fix intents and retrain, continue to glance at this card for issues.

以下图表显示了一个适当平衡的应用,其中几乎不存在任何要修复的问题。The following chart shows a well-balanced app with almost no issues to fix.

以下图表显示了一个适当平衡的应用,其中几乎不存在任何要修复的问题。

以下图表显示了一个很不平衡的应用,其中包含许多要修复的问题。The following chart shows a poorly balanced app with many issues to fix.

以下图表显示了一个适当平衡的应用,其中几乎不存在任何要修复的问题。

将鼠标悬停在每个意向的条块上可以获取有关该意向的信息。Hover over each intent's bar to get information about the intent.

以下图表显示了一个适当平衡的应用,其中几乎不存在任何要修复的问题。

使用“排序依据”功能可按问题类型排列意向,以便可以专注于处理存在该问题的最严重意向。Use the Sort by feature to arrange the intents by issue type so you can focus on the most problematic intents with that issue.

有错误的意向Intents with errors

使用此卡片可以检查特定意向的问题。This card allows you to review issues for a specific intent. 此卡片的默认视图是问题最严重的意图,让你知道要将工作重心放在何处。The default view of this card is the most problematic intents so you know where to focus your efforts.

使用“有错误的意向”卡片可以检查特定意向的问题。

顶部的圆环图表以三种问题类型显示意向的问题。The top donut chart shows the issues with the intent across the three problem types. 如果出现了这三种类型的问题,每种问题类型会在下面绘制其自身的图表,以及另外两种问题类型的图表。If there are issues in the three problem types, each type has its own chart below, along with any rival intents.

按问题和百分比筛选意向Filter intents by issue and percentage

在该卡片的此部分,可以查找超出错误阈值的示例言语。This section of the card allows you to find example utterances that are falling outside your error threshold. 理想情况下,正确的预测应该占有很高的比例。Ideally you want correct predictions to be significant. 该百分比由业务和客户驱动。That percentage is business and customer driven.

请确定最符合业务需求的阈值百分比。Determine the threshold percentages that you are comfortable with for your business.

使用筛选器可以查找存在特定问题的意向:The filter allows you to find intents with specific issue:

筛选器Filter 建议的百分比Suggested percentage 目的Purpose
问题最严重的意向Most problematic intents - 从此处着手 - 修复此意向中的言语比其他修复方法更能改善应用。Start here - Fixing the utterances in this intent will improve the app more than other fixes.
正确的预测低于Correct predictions below 60%60% 这是选定意向中正确的、但置信度评分低于阈值的言语的百分比。This is the percentage of utterances in the selected intent that are correct but have a confidence score below the threshold.
不明确的预测高于Unclear predictions above 15%15% 这是选定意向中与最靠近的其他意向相混淆的言语的百分比。This is the percentage of utterances in the selected intent that are confused with the nearest rival intent.
错误的预测高于Incorrect predictions above 15%15% 这是选定意向中未正确预测的言语的百分比。This is the percentage of utterances in the selected intent that are incorrectly predicted.

正确的预测阈值Correct prediction threshold

你的有把握预测的置信度评分是什么?What is a confident prediction confidence score to you? 在应用开发的初期,目标可能是 60%。At the beginning of app development, 60% may be your target. 使用“正确的预测低于”和百分比 60% 查找选定意向中需要修复的所有言语。Use the Correct predictions below with the percentage of 60% to find any utterances in the selected intent that need to be fixed.

不明确或错误的预测阈值Unclear or incorrect prediction threshold

使用这两个筛选器可以查找选定意向中超出阈值的言语。These two filters allow you to find utterances in the selected intent beyond your threshold. 可将这两个百分比视为错误百分比。You can think of these two percentages as error percentages. 如果 10-15% 的预测错误率对你而较为合适,将筛选器阈值设置为 15% 即可找到高于此值的所有言语。If you are comfortable with a 10-15% error rate for predictions, set the filter threshold to 15% to find all utterances above this value.