什么是 Azure 内容审查器?What is Azure Content Moderator?

重要

现在,将对此服务的所有 HTTP 请求强制执行 TLS 1.2。TLS 1.2 is now enforced for all HTTP requests to this service.

Azure 内容审查器是一种 AI 服务,可用于处理可能的冒犯性、危险或不可取内容。Azure Content Moderator is an AI service that lets you handle content that is potentially offensive, risky, or otherwise undesirable. 它包括由 AI 提供支持的内容审核服务(用于扫描文本、图像和视频以及自动应用内容标记),以及审阅工具(面向一组人工审阅者的在线审查器环境)。It includes the AI-powered content moderation service which scans text, image, and videos and applies content flags automatically, as well as the Review tool, an online moderator environment for a team of human reviewers.

你可能想在应用中内置内容筛选软件,以符合法规或维护用户的预期环境。You may want to build content filtering software into your app to comply with regulations or maintain the intended environment for your users.

使用场合Where it's used

下面是软件开发人员或团队需要使用内容审核服务的一些场景:The following are a few scenarios in which a software developer or team would require a content moderation service:

  • 在联机市场中审查产品目录和其他用户生成的内容。Online marketplaces that moderate product catalogs and other user-generated content.
  • 在游戏公司中审查用户生成的游戏项目和聊天室。Gaming companies that moderate user-generated game artifacts and chat rooms.
  • 在社交消息平台中审查用户添加的图像、文本和视频。Social messaging platforms that moderate images, text, and videos added by their users.
  • 企业媒体公司对其内容进行集中式审查。Enterprise media companies that implement centralized moderation for their content.
  • K-12 教育解决方案提供商为学生和教师筛选掉不当的内容。K-12 education solution providers filtering out content that is inappropriate for students and educators.

重要

不能使用内容审查器检测非法儿童剥削图像。You cannot use Content Moderator to detect illegal child exploitation images. 不过,合格组织可以使用 PhotoDNA 云服务来筛查此类内容。However, qualified organizations can use the PhotoDNA Cloud Service to screen for this type of content.

组成部分What it includes

内容审查器服务包含多个可以通过 REST 调用和 .NET SDK 使用的 Web 服务 API。The Content Moderator service consists of several web service APIs available through both REST calls and a .NET SDK. 它还包括评审工具,允许人工审阅者帮助服务并改进或微调其审查功能。It also includes the Review tool, which allows human reviewers to aid the service and improve or fine-tune its moderation function.

审查 APIModeration APIs

内容审查器服务包括了审查 API,这些 API 检查内容中是否存在可能不适当或令人反感的材料。The Content Moderator service includes Moderation APIs, which check content for material that is potentially inappropriate or objectionable.

内容审查器审查 API 的框图

下表介绍了各种类型的审查 API。The following table describes the different types of moderation APIs.

API 组API group 说明Description
文本审查Text moderation 扫描文本中是否存在冒犯性内容、明确的或暗示性的色情内容、不雅内容和个人数据。Scans text for offensive content, sexually explicit or suggestive content, profanity, and personal data.
自定义术语列表Custom term lists 扫描文本中是否存在自定义术语列表以及内置的术语。Scans text against a custom list of terms along with the built-in terms. 根据你自己的内容策略,使用自定义列表阻止或允许内容。Use custom lists to block or allow content according to your own content policies.
图像审查Image moderation 扫描图像中是否存在成人内容或不雅内容,通过光学字符识别 (OCR) 功能检测图像中的文本,以及检测人脸。Scans images for adult or racy content, detects text in images with the Optical Character Recognition (OCR) capability, and detects faces.
自定义图像列表Custom image lists 针对自定义图像列表对图像进行扫描。Scans images against a custom list of images. 使用自定义图像列表筛选掉常常反复出现的内容的实例,这些实例不需再次分类。Use custom image lists to filter out instances of commonly recurring content that you don't want to classify again.
视频审查Video moderation 扫描视频中是否存在成人内容或挑逗性内容,并针对上述内容返回时间标记。Scans videos for adult or racy content and returns time markers for said content.

查看 APIReview APIs

审阅 API 允许你将审查管道与人工审阅者进行集成。The Review APIs let you integrate your moderation pipeline with human reviewers. 使用作业审阅工作流操作在审阅工具(下文)中创建和自动执行人工干预工作流。Use the Jobs, Reviews, and Workflow operations to create and automate human-in-the-loop workflows with the Review tool (below).

备注

工作流 API 在 .NET SDK 中尚不可用,但可以与 REST 终结点一起使用。The Workflow API is not yet available in the .NET SDK but can be used with the REST endpoint.

内容审查器审阅 API 的框图

审阅工具Review tool

内容审查器服务还包括基于 Web 的审阅工具,其中托管的内容审阅可供审阅人处理。The Content Moderator service also includes the web-based Review tool, which hosts the content reviews for human moderators to process. 人工输入不会训练服务,但是将服务与人工审阅团队组合起来以后,开发人员就可以在效率和准确性之间取得适当的平衡。The human input doesn't train the service, but the combined work of the service and human review teams allows developers to strike the right balance between efficiency and accuracy. 评审工具还为多种内容审查器资源提供用户友好型前端。The Review tool also provides a user-friendly front end for several Content Moderator resources.

内容审查器评审工具主页

数据隐私和安全性Data privacy and security

与所有认知服务一样,使用内容审查器服务的开发人员应该了解 Microsoft 针对客户数据的政策。As with all of the Cognitive Services, developers using the Content Moderator service should be aware of Microsoft's policies on customer data. 请参阅 Microsoft 信任中心上的“认知服务”页面来了解详细信息。See the Cognitive Services page on the Microsoft Trust Center to learn more.

后续步骤Next steps

若要开始在 Web 门户上使用内容审查器,请按照尝试在 Web 上使用内容审查器进行操作。To get started using Content Moderator on the web portal, follow Try Content Moderator on the web. 或者,完成客户端库快速入门以实现代码中的基本方案。Or, complete a client library quickstart to implement the basic scenarios in code.