如何使用真实示例为 Azure Cosmos DB 中的数据建模和分区How to model and partition data on Azure Cosmos DB using a real-world example

本文基于多个 Azure Cosmos DB 概念,例如数据建模分区预配吞吐量,演示如何完成一个真实数据设计练习。This article builds on several Azure Cosmos DB concepts like data modeling, partitioning, and provisioned throughput to demonstrate how to tackle a real-world data design exercise.

如果你平时主要使用关系数据库,可能在设计数据模型方面已经形成了自己的习惯和直觉。If you usually work with relational databases, you have probably built habits and intuitions on how to design a data model. 由于具体的约束,加上 Azure Cosmos DB 的独特优势,其中的大部分最佳做法不能产生很好的效果,甚至可能会生成欠佳的解决方案。Because of the specific constraints, but also the unique strengths of Azure Cosmos DB, most of these best practices don't translate well and may drag you into suboptimal solutions. 本文旨在引导你完成 Azure Cosmos DB 中的真实用例建模的整个过程,包括项的建模,以及实体共置和容器分区。The goal of this article is to guide you through the complete process of modeling a real-world use-case on Azure Cosmos DB, from item modeling to entity colocation and container partitioning.

方案The scenario

对于本练习,我们假设有一个博客平台域,用户可在其中创建帖子For this exercise, we are going to consider the domain of a blogging platform where users can create posts. 用户还可以点赞评论这些贴子。Users can also like and add comments to those posts.

提示

本文以斜体突出显示了某些词语,这些词语表示我们的模型需要处理的“事情”类型。We have highlighted some words in italic; these words identify the kind of "things" our model will have to manipulate.

将更多的要求添加到规范:Adding more requirements to our specification:

  • 首页显示最近创建的帖子的源。A front page displays a feed of recently created posts,
  • 我们可以提取某个用户的所有帖子、对某个帖子发表的所有评论,以及某个帖子的所有点赞数。We can fetch all posts for a user, all comments for a post and all likes for a post,
  • 帖子将连同其作者的用户名以及这些帖子获得的评论数和点赞数一起返回。Posts are returned with the username of their authors and a count of how many comments and likes they have,
  • 评论和点赞也会连同评论者和点赞者的用户名一起返回。Comments and likes are also returned with the username of the users who have created them,
  • 以列表形式显示时,帖子只需显示其内容的截断的摘要。When displayed as lists, posts only have to present a truncated summary of their content.

标识主要访问模式Identify the main access patterns

在开始之前,让我们通过标识解决方案的访问模式,来为初始规范提供某种结构。To start, we give some structure to our initial specification by identifying our solution's access patterns. 设计 Azure Cosmos DB 的数据模型时,必须了解模型需要为哪些请求提供服务,以确保模型能够有效地为这些请求提供服务。When designing a data model for Azure Cosmos DB, it's important to understand which requests our model will have to serve to make sure that the model will serve those requests efficiently.

为使整个过程更易于遵循,我们借用了 CQRS 中的某个词汇表,将这些不同的请求分类为命令或查询,其中,命令表示写入请求(即,更新系统的意图),查询表示只读的请求。To make the overall process easier to follow, we categorize those different requests as either commands or queries, borrowing some vocabulary from CQRS where commands are write requests (that is, intents to update the system) and queries are read-only requests.

下面是平台必须公开的请求列表:Here is the list of requests that our platform will have to expose:

  • [C1] 创建/编辑用户[C1] Create/edit a user
  • [Q1] 检索用户[Q1] Retrieve a user
  • [C2] 创建/编辑帖子[C2] Create/edit a post
  • [Q2] 检索帖子[Q2] Retrieve a post
  • [Q3] 以短格式列出用户的帖子[Q3] List a user's posts in short form
  • [C3] 创建评论[C3] Create a comment
  • [Q4] 列出帖子的评论[Q4] List a post's comments
  • [C4] 为帖子点赞[C4] Like a post
  • [Q5] 列出帖子的点赞数[Q5] List a post's likes
  • [Q6] 以短格式列出最近创建的 x 个帖子(源)[Q6] List the x most recent posts created in short form (feed)

在此阶段,我们尚未考虑每个实体(用户、帖子等)将要包含的详细信息。As this stage, we haven't thought about the details of what each entity (user, post etc.) will contain. 针对关系存储进行设计时,此步骤往往是要处理的最初几个步骤之一,因为我们需要确定这些实体在表、列、外键等方面如何进行转换。对于在写入时不会实施任何架构的文档数据库,基本上不必要予以考虑。This step is usually among the first ones to be tackled when designing against a relational store, because we have to figure out how those entities will translate in terms of tables, columns, foreign keys etc. It is much less of a concern with a document database that doesn't enforce any schema at write.

必须从一开始就标识访问模式的主要原因在于,请求列表将会成为我们的测试套件。The main reason why it is important to identify our access patterns from the beginning, is because this list of requests is going to be our test suite. 每当循环访问数据模型时,我们都会遍历每个请求,并检查其性能和可伸缩性。Every time we iterate over our data model, we will go through each of the requests and check its performance and scalability.

V1:第一个版本V1: A first version

从两个容器着手:userspostsWe start with two containers: users and posts.

用户容器Users container

此容器仅存储用户项:This container only stores user items:

{
  "id": "<user-id>",
  "username": "<username>"
}

我们按 id 将此容器分区,这意味着,该容器中的每个逻辑分区仅包含一个项。We partition this container by id, which means that each logical partition within that container will only contain one item.

帖子容器Posts container

此容器包含帖子、评论和点赞数:This container hosts posts, comments, and likes:

{
  "id": "<post-id>",
  "type": "post",
  "postId": "<post-id>",
  "userId": "<post-author-id>",
  "title": "<post-title>",
  "content": "<post-content>",
  "creationDate": "<post-creation-date>"
}

{
  "id": "<comment-id>",
  "type": "comment",
  "postId": "<post-id>",
  "userId": "<comment-author-id>",
  "content": "<comment-content>",
  "creationDate": "<comment-creation-date>"
}

{
  "id": "<like-id>",
  "type": "like",
  "postId": "<post-id>",
  "userId": "<liker-id>",
  "creationDate": "<like-creation-date>"
}

我们按 postId 将此容器分区,这意味着,该容器中的每个逻辑分区包含一个帖子、对该帖子的所有评论,以及该帖子的所有点赞数。We partition this container by postId, which means that each logical partition within that container will contain one post, all the comments for that post and all the likes for that post.

请注意,我们在此容器存储的项中引入了一个 type 属性,以区分此容器包含的三种实体类型。Note that we have introduced a type property in the items stored in this container to distinguish between the three types of entities that this container hosts.

另外,我们已选择引用相关的数据而不是嵌入这些数据(有关这些概念的详细信息,请查看此部分),因为:Also, we have chosen to reference related data instead of embedding it (check this section for details about these concepts) because:

  • 用户可以创建的帖子数没有上限;there's no upper limit to how many posts a user can create,
  • 帖子可以是任意长度;posts can be arbitrarily long,
  • 帖子产生的评论数和点赞数没有上限;there's no upper limit to how many comments and likes a post can have,
  • 我们希望能够在无需更新帖子本身的情况下,为帖子添加评论或点赞。we want to be able to add a comment or a like to a post without having to update the post itself.

模型的性能如何?How well does our model perform?

现在,让我们评估第一个版本的性能和可伸缩性。It's now time to assess the performance and scalability of our first version. 对于前面标识的每个请求,我们将测量其延迟,及其消耗的请求单位数。For each of the requests previously identified, we measure its latency and how many request units it consumes. 这种测量是针对某个虚构数据集进行的。该数据集包含 100,000 个用户,每个用户发布了 5 到 50 个帖子,每个帖子产生了 25 条评论和 100 次点赞。This measurement is done against a dummy data set containing 100,000 users with 5 to 50 posts per user, and up to 25 comments and 100 likes per post.

[C1] 创建/编辑用户[C1] Create/edit a user

可以直截了当地实现此请求,因为我们只需在 users 容器中创建或更新某个项。This request is straightforward to implement as we just create or update an item in the users container. 得益于 id 分区键,请求将合理分散在所有分区之间。The requests will nicely spread across all partitions thanks to the id partition key.

将单个项写入用户容器

延迟Latency RU 开销RU charge “性能”Performance
7 毫秒7 ms 5.71 RU5.71 RU

[Q1] 检索用户[Q1] Retrieve a user

通过读取 users 容器中的相应项来检索用户。Retrieving a user is done by reading the corresponding item from the users container.

从用户容器检索单个项

延迟Latency RU 开销RU charge “性能”Performance
2 毫秒2 ms 1 RU1 RU

[C2] 创建/编辑帖子[C2] Create/edit a post

类似于 [C1] ,我们只需写入到 posts 容器。Similarly to [C1], we just have to write to the posts container.

将单个项写入帖子容器

延迟Latency RU 开销RU charge “性能”Performance
9 毫秒9 ms 8.76 RU8.76 RU

[Q2] 检索帖子[Q2] Retrieve a post

首先检索 posts 容器中的相应文档。We start by retrieving the corresponding document from the posts container. 但这并不足够,根据规范,我们还需要聚合帖子作者的用户名以及此帖子产生的评论和点赞数,这需要发出 3 个附加的 SQL 查询。But that's not enough, as per our specification we also have to aggregate the username of the post's author and the counts of how many comments and how many likes this post has, which requires 3 additional SQL queries to be issued.

检索帖子并聚合附加数据

每个附加查询根据相应容器的分区键进行筛选,而我们恰好需要使用分区来最大化性能和可伸缩性。Each of the additional queries filters on the partition key of its respective container, which is exactly what we want to maximize performance and scalability. 但是,我们最终需要执行四个操作才能返回一个帖子,因此,我们将在下一次迭代中改进此方法。But we eventually have to perform four operations to return a single post, so we'll improve that in a next iteration.

延迟Latency RU 开销RU charge “性能”Performance
9 毫秒9 ms 19.54 RU19.54 RU

[Q3] 以短格式列出用户的帖子[Q3] List a user's posts in short form

首先,必须使用一个 SQL 查询来检索所需的帖子。该查询会提取对应于该特定用户的帖子。First, we have to retrieve the desired posts with a SQL query that fetches the posts corresponding to that particular user. 但是,我们还需要发出附加的查询来聚合作者的用户名以及评论数和点赞数。But we also have to issue additional queries to aggregate the author's username and the counts of comments and likes.

检索用户的所有帖子并聚合其附加数据

此实现存在许多缺点:This implementation presents many drawbacks:

  • 必须针对第一个查询返回的每个帖子,发出用于聚合评论数和点赞数的查询;the queries aggregating the counts of comments and likes have to be issued for each post returned by the first query,
  • 主查询不会根据 posts 容器的分区键进行筛选,导致扇出并在整个容器中进行分区扫描。the main query does not filter on the partition key of the posts container, leading to a fan-out and a partition scan across the container.
延迟Latency RU 开销RU charge “性能”Performance
130 毫秒130 ms 619.41 RU619.41 RU

[C3] 创建评论[C3] Create a comment

通过在 posts 容器中写入相应的项来创建评论。A comment is created by writing the corresponding item in the posts container.

将单个项写入帖子容器

延迟Latency RU 开销RU charge “性能”Performance
7 毫秒7 ms 8.57 RU8.57 RU

[Q4] 列出帖子的评论[Q4] List a post's comments

首先使用一个查询提取该帖子的所有评论,同样,我们也需要单独聚合每条评论的用户名。We start with a query that fetches all the comments for that post and once again, we also need to aggregate usernames separately for each comment.

检索帖子的所有评论并聚合其附加数据

尽管主查询会根据容器的分区键进行筛选,但单独聚合用户名会降低总体性能。Although the main query does filter on the container's partition key, aggregating the usernames separately penalizes the overall performance. 稍后我们将会改进。We'll improve that later on.

延迟Latency RU 开销RU charge “性能”Performance
23 毫秒23 ms 27.72 RU27.72 RU

[C4] 为帖子点赞[C4] Like a post

类似于 [C3] ,我们将在 posts 容器中创建相应的项。Just like [C3], we create the corresponding item in the posts container.

将单个项写入帖子容器

延迟Latency RU 开销RU charge “性能”Performance
6 毫秒6 ms 7.05 RU7.05 RU

[Q5] 列出帖子的点赞数[Q5] List a post's likes

类似于 [Q4] ,我们将查询该帖子的点赞数,然后聚合点赞者的用户名。Just like [Q4], we query the likes for that post, then aggregate their usernames.

检索帖子的所有点赞并聚合其附加数据

延迟Latency RU 开销RU charge “性能”Performance
59 毫秒59 ms 58.92 RU58.92 RU

[Q6] 以短格式列出最近创建的 x 个帖子(源)[Q6] List the x most recent posts created in short form (feed)

我们通过查询 posts 容器来提取最近的帖子(按创建日期的降序排序),然后聚合每个帖子的用户名以及评论数和点赞数。We fetch the most recent posts by querying the posts container sorted by descending creation date, then aggregate usernames and counts of comments and likes for each of the posts.

检索最近的帖子并聚合其附加数据

同样,我们的初始查询不会根据 posts 容器的分区键进行筛选,这会触发高开销的扇出。但这一次情况更糟,因为我们的目标是一个大得多的结果集,并要使用 ORDER BY 子句将结果排序,因此会消耗更多的请求单位。Once again, our initial query doesn't filter on the partition key of the posts container, which triggers a costly fan-out. This one is even worse as we target a much larger result set and sort the results with an ORDER BY clause, which makes it more expensive in terms of request units.

延迟Latency RU 开销RU charge “性能”Performance
306 毫秒306 ms 2063.54 RU2063.54 RU

反映 V1 的性能Reflecting on the performance of V1

分析在上一部分遇到的性能问题,我们可以发现存在两类主要问题:Looking at the performance issues we faced in the previous section, we can identify two main classes of problems:

  • 某些请求要求发出多个查询来收集需要返回的所有数据;some requests require multiple queries to be issued in order to gather all the data we need to return,
  • 某些查询不会根据它们所针对的容器的分区键进行筛选,导致发生扇出,使可伸缩性受到阻碍。some queries don't filter on the partition key of the containers they target, leading to a fan-out that impedes our scalability.

让我们解决上述每个问题,从第一个问题开始。Let's resolve each of those problems, starting with the first one.

V2:引入反规范化以优化读取查询V2: Introducing denormalization to optimize read queries

之所以需要在某些情况下发出附加请求,是因为初始请求的结果不包含需要返回的所有数据。The reason why we have to issue additional requests in some cases is because the results of the initial request doesn't contain all the data we need to return. 使用 Azure Cosmos DB 之类的非关系数据存储时,通常可以通过反规范化数据集中的数据来解决此类问题。When working with a non-relational data store like Azure Cosmos DB, this kind of issue is commonly solved by denormalizing data across our data set.

在本示例中,我们将修改帖子项,以添加帖子作者的用户名,以及评论数和点赞数:In our example, we modify post items to add the username of the post's author, the count of comments and the count of likes:

{
  "id": "<post-id>",
  "type": "post",
  "postId": "<post-id>",
  "userId": "<post-author-id>",
  "userUsername": "<post-author-username>",
  "title": "<post-title>",
  "content": "<post-content>",
  "commentCount": <count-of-comments>,
  "likeCount": <count-of-likes>,
  "creationDate": "<post-creation-date>"
}

此外,我们将修改评论和点赞项,以添加评论者和点赞者的用户名:We also modify comment and like items to add the username of the user who has created them:

{
  "id": "<comment-id>",
  "type": "comment",
  "postId": "<post-id>",
  "userId": "<comment-author-id>",
  "userUsername": "<comment-author-username>",
  "content": "<comment-content>",
  "creationDate": "<comment-creation-date>"
}

{
  "id": "<like-id>",
  "type": "like",
  "postId": "<post-id>",
  "userId": "<liker-id>",
  "userUsername": "<liker-username>",
  "creationDate": "<like-creation-date>"
}

反规范化评论数和点赞数Denormalizing comment and like counts

我们要实现的目的是,每次添加评论或点赞时,都会递增相应帖子中的 commentCountlikeCountWhat we want to achieve is that every time we add a comment or a like, we also increment the commentCount or the likeCount in the corresponding post. 由于 posts 容器已按 postId 分区,新项(评论或点赞)及其相应帖子将位于同一个逻辑分区中。As our posts container is partitioned by postId, the new item (comment or like) and its corresponding post sit in the same logical partition. 因此,我们可以使用某个存储过程来执行该操作。As a result, we can use a stored procedure to perform that operation.

现在,在创建评论 ( [C3] ) 时,我们不仅需要在 posts 容器中添加新项,而且还要针对该容器调用以下存储过程:Now when creating a comment ([C3]), instead of just adding a new item in the posts container we call the following stored procedure on that container:

function createComment(postId, comment) {
  var collection = getContext().getCollection();

  collection.readDocument(
    `${collection.getAltLink()}/docs/${postId}`,
    function (err, post) {
      if (err) throw err;

      post.commentCount++;
      collection.replaceDocument(
        post._self,
        post,
        function (err) {
          if (err) throw err;

          comment.postId = postId;
          collection.createDocument(
            collection.getSelfLink(),
            comment
          );
        }
      );
    })
}

此存储过程采用帖子 ID 和新评论的正文作为参数,然后:This stored procedure takes the ID of the post and the body of the new comment as parameters, then:

  • 检索帖子retrieves the post
  • 递增 commentCountincrements the commentCount
  • 替换帖子replaces the post
  • 添加新评论adds the new comment

由于存储过程是作为原子事务执行的,可以保证 commentCount 的值和实际评论数始终保持同步。As stored procedures are executed as atomic transactions, it is guaranteed that the value of commentCount and the actual number of comments will always stay in sync.

添加新的点赞来递增 likeCount 时,我们将显式调用类似的存储过程。We obviously call a similar stored procedure when adding new likes to increment the likeCount.

反规范化用户名Denormalizing usernames

对于用户名,需要采用不同的方法,因为用户不仅位于不同的分区中,而且还位于不同的容器中。Usernames require a different approach as users not only sit in different partitions, but in a different container. 如果必须反规范化不同分区和容器中的数据,可以使用源容器的更改源When we have to denormalize data across partitions and containers, we can use the source container's change feed.

在本示例中,每当用户更新其用户名时,我们都会使用 users 容器的更改源来做出反应。In our example, we use the change feed of the users container to react whenever users update their usernames. 如果发生这种情况,我们会针对 posts 容器调用另一个存储过程来传播更改:When that happens, we propagate the change by calling another stored procedure on the posts container:

将用户名反规范化为帖子容器

function updateUsernames(userId, username) {
  var collection = getContext().getCollection();

  collection.queryDocuments(
    collection.getSelfLink(),
    `SELECT * FROM p WHERE p.userId = '${userId}'`,
    function (err, results) {
      if (err) throw err;

      for (var i in results) {
        var doc = results[i];
        doc.userUsername = username;

        collection.upsertDocument(
          collection.getSelfLink(),
          doc);
      }
    });
}

此存储过程采用用户 ID 和用户的新用户名作为参数,然后:This stored procedure takes the ID of the user and the user's new username as parameters, then:

  • 提取与 userId 匹配的所有项(可能是帖子、评论或点赞)fetches all items matching the userId (which can be posts, comments, or likes)
  • 对于其中的每个项for each of those items
    • 替换 userUsernamereplaces the userUsername
    • 替换项replaces the item

重要

此操作的开销较大,因为需要针对 posts 容器的每个分区执行此存储过程。This operation is costly because it requires this stored procedure to be executed on every partition of the posts container. 假设大多数用户在注册期间选择了适当的用户名,并且以后永远不会更改此用户名,因此,极少运行这种更新。We assume that most users choose a suitable username during sign-up and won't ever change it, so this update will run very rarely.

V2 有多大的性能提升?What are the performance gains of V2?

[Q2] 检索帖子[Q2] Retrieve a post

完成反规范化后,只需提取单个项即可处理该请求。Now that our denormalization is in place, we only have to fetch a single item to handle that request.

从帖子容器检索单个项

延迟Latency RU 开销RU charge “性能”Performance
2 毫秒2 ms 1 RU1 RU

[Q4] 列出帖子的评论[Q4] List a post's comments

同样,我们无需发出额外的请求来提取用户名,最终只需运行一个可以根据分区键进行筛选的查询。Here again, we can spare the extra requests that fetched the usernames and end up with a single query that filters on the partition key.

检索帖子的所有评论

延迟Latency RU 开销RU charge “性能”Performance
4 毫秒4 ms 7.72 RU7.72 RU

[Q5] 列出帖子的点赞数[Q5] List a post's likes

列出点赞时,情况完全相同。Exact same situation when listing the likes.

检索帖子的所有点赞

延迟Latency RU 开销RU charge “性能”Performance
4 毫秒4 ms 8.92 RU8.92 RU

V3:确保所有请求都可缩放V3: Making sure all requests are scalable

分析我们的总体性能改进,可以发现仍有两个请求未得到完全优化: [Q3][Q6]Looking at our overall performance improvements, there are still two requests that we haven't fully optimized: [Q3] and [Q6]. 这些请求涉及到不根据其所针对的容器的分区键进行筛选的查询。They are the requests involving queries that don't filter on the partition key of the containers they target.

[Q3] 以短格式列出用户的帖子[Q3] List a user's posts in short form

此请求已受益于 V2 中引入的改进,可以免除附加的查询。This request already benefits from the improvements introduced in V2, which spares additional queries.

检索用户的所有帖子

但是,剩余的查询仍不根据 posts 容器的分区键进行筛选。But the remaining query is still not filtering on the partition key of the posts container.

查明此问题的原因其实非常简单:The way to think about this situation is actually simple:

  1. 此请求必须根据 userId 进行筛选,因为我们需要提取特定用户的所有帖子 This request has to filter on the userId because we want to fetch all posts for a particular user
  2. 它的性能之所以不佳,是因为它是针对 posts 容器执行的,而该容器的分区依据不是 userIdIt doesn't perform well because it is executed against the posts container, which is not partitioned by userId
  3. 明白地讲,我们需要针对某个容器执行此请求来解决性能问题,该容器的分区依据为 userIdStating the obvious, we would solve our performance problem by executing this request against a container that is partitioned by userId
  4. 正好我们已有这样一个容器:users 容器!It turns out that we already have such a container: the users container!

因此,我们通过将整个帖子复制到 users 容器,来引入第二级反规范化。So we introduce a second level of denormalization by duplicating entire posts to the users container. 这样,我们便可以有效地获取只按一个不同维度分区的帖子副本,从而可以更有效地按帖子的 userId 来检索帖子。By doing that, we effectively get a copy of our posts, only partitioned along a different dimensions, making them way more efficient to retrieve by their userId.

users 容器现在包含 2 种类型的项:The users container now contains 2 kinds of items:

{
  "id": "<user-id>",
  "type": "user",
  "userId": "<user-id>",
  "username": "<username>"
}

{
  "id": "<post-id>",
  "type": "post",
  "postId": "<post-id>",
  "userId": "<post-author-id>",
  "userUsername": "<post-author-username>",
  "title": "<post-title>",
  "content": "<post-content>",
  "commentCount": <count-of-comments>,
  "likeCount": <count-of-likes>,
  "creationDate": "<post-creation-date>"
}

请注意:Note that:

  • 我们在用户项中引入了 type 字段,以便将用户与帖子区分开来;we have introduced a type field in the user item to distinguish users from posts,
  • 我们还在用户项中添加了 userId 字段,该字段似乎是多余的,因为存在 id 字段。但是,该字段是必需的,因为 users 容器现在按 userId 分区(而不像以前一样按 id 分区)we have also added a userId field in the user item, which is redundant with the id field but is required as the users container is now partitioned by userId (and not id as previously)

若要实现这种反规范化,我们将再次使用更改源。To achieve that denormalization, we once again use the change feed. 这一次,我们将对 posts 容器的更改源做出反应,以将任何新的或更新的帖子调度到 users 容器。This time, we react on the change feed of the posts container to dispatch any new or updated post to the users container. 由于列出帖子不需要返回其完整内容,我们可以在列出过程中截断帖子。And because listing posts does not require to return their full content, we can truncate them in the process.

将帖子反规范化为用户容器

现在,可将查询路由到 users 容器,并根据该容器的分区键进行筛选。We can now route our query to the users container, filtering on the container's partition key.

检索用户的所有帖子

延迟Latency RU 开销RU charge “性能”Performance
4 毫秒4 ms 6.46 RU6.46 RU

[Q6] 以短格式列出最近创建的 x 个帖子(源)[Q6] List the x most recent posts created in short form (feed)

在此处必须处理类似的情况:尽管实现 V2 中引入的反规范化后无需运行附加的查询,但是,剩余的查询仍不会根据容器的分区键进行筛选:We have to deal with a similar situation here: even after sparing the additional queries left unnecessary by the denormalization introduced in V2, the remaining query does not filter on the container's partition key:

检索最近的帖子

遵循相同的方法最大化此请求的性能和可伸缩性要求只命中一个分区。Following the same approach, maximizing this request's performance and scalability requires that it only hits one partition. 这是一种可行的做法,因为我们只需返回有限数量的项;若要填充博客平台的主页,我们只需获取 100 个最近的帖子,而无需通过整个数据集分页。This is conceivable because we only have to return a limited number of items; in order to populate our blogging platform's home page, we just need to get the 100 most recent posts, without the need to paginate through the entire data set.

为了优化这最后一个请求,我们在设计中引入了第三个容器,该容器专门为此请求提供服务。So to optimize this last request, we introduce a third container to our design, entirely dedicated to serving this request. 将帖子反规范化为该新的 feed 容器:We denormalize our posts to that new feed container:

{
  "id": "<post-id>",
  "type": "post",
  "postId": "<post-id>",
  "userId": "<post-author-id>",
  "userUsername": "<post-author-username>",
  "title": "<post-title>",
  "content": "<post-content>",
  "commentCount": <count-of-comments>,
  "likeCount": <count-of-likes>,
  "creationDate": "<post-creation-date>"
}

此容器按 type(始终为项中的 post)分区。This container is partitioned by type, which will always be post in our items. 这可以确保此容器中的所有项位于同一个分区。Doing that ensures that all the items in this container will sit in the same partition.

若要实现反规范化,我们只需挂接前面引入的更改源管道,以将帖子调度到该新容器。To achieve the denormalization, we just have to hook on the change feed pipeline we have previously introduced to dispatch the posts to that new container. 要记住的一个要点是,需要确保只存储 100 个最近的帖子;否则,容器内容可能会增大到超过分区的最大大小。One important thing to bear in mind is that we need to make sure that we only store the 100 most recent posts; otherwise, the content of the container may grow beyond the maximum size of a partition. 为此,可以在每次将文档添加到容器中时,调用 post-triggerThis is done by calling a post-trigger every time a document is added in the container:

将帖子反规范化为源容器

下面是用于截断集合的 post-trigger 的正文:Here's the body of the post-trigger that truncates the collection:

function truncateFeed() {
  const maxDocs = 100;
  var context = getContext();
  var collection = context.getCollection();

  collection.queryDocuments(
    collection.getSelfLink(),
    "SELECT VALUE COUNT(1) FROM f",
    function (err, results) {
      if (err) throw err;

      processCountResults(results);
    });

  function processCountResults(results) {
    // + 1 because the query didn't count the newly inserted doc
    if ((results[0] + 1) > maxDocs) {
      var docsToRemove = results[0] + 1 - maxDocs;
      collection.queryDocuments(
        collection.getSelfLink(),
        `SELECT TOP ${docsToRemove} * FROM f ORDER BY f.creationDate`,
        function (err, results) {
          if (err) throw err;

          processDocsToRemove(results, 0);
        });
    }
  }

  function processDocsToRemove(results, index) {
    var doc = results[index];
    if (doc) {
      collection.deleteDocument(
        doc._self,
        function (err) {
          if (err) throw err;

          processDocsToRemove(results, index + 1);
        });
    }
  }
}

最后一步是将查询重新路由到新的 feed 容器:The final step is to reroute our query to our new feed container:

检索最近的帖子

延迟Latency RU 开销RU charge “性能”Performance
9 毫秒9 ms 16.97 RU16.97 RU

结论Conclusion

让我们分析一下在不同设计版本中引入的总体性能和可伸缩性改进。Let's have a look at the overall performance and scalability improvements we have introduced over the different versions of our design.

V1V1 V2V2 V3V3
[C1][C1] 7 毫秒 / 5.71 RU7 ms / 5.71 RU 7 毫秒 / 5.71 RU7 ms / 5.71 RU 7 毫秒 / 5.71 RU7 ms / 5.71 RU
[Q1][Q1] 2 毫秒 / 1 RU2 ms / 1 RU 2 毫秒 / 1 RU2 ms / 1 RU 2 毫秒 / 1 RU2 ms / 1 RU
[C2][C2] 9 毫秒 / 8.76 RU9 ms / 8.76 RU 9 毫秒 / 8.76 RU9 ms / 8.76 RU 9 毫秒 / 8.76 RU9 ms / 8.76 RU
[Q2][Q2] 9 毫秒 / 19.54 RU9 ms / 19.54 RU 2 毫秒 / 1 RU2 ms / 1 RU 2 毫秒 / 1 RU2 ms / 1 RU
[Q3][Q3] 130 毫秒 / 619.41 RU130 ms / 619.41 RU 28 毫秒 / 201.54 RU28 ms / 201.54 RU 4 毫秒 / 6.46 RU4 ms / 6.46 RU
[C3][C3] 7 毫秒 / 8.57 RU7 ms / 8.57 RU 7 毫秒 / 15.27 RU7 ms / 15.27 RU 7 毫秒 / 15.27 RU7 ms / 15.27 RU
[Q4][Q4] 23 毫秒 / 27.72 RU23 ms / 27.72 RU 4 毫秒 / 7.72 RU4 ms / 7.72 RU 4 毫秒 / 7.72 RU4 ms / 7.72 RU
[C4][C4] 6 毫秒 / 7.05 RU6 ms / 7.05 RU 7 毫秒 / 14.67 RU7 ms / 14.67 RU 7 毫秒 / 14.67 RU7 ms / 14.67 RU
[Q5][Q5] 59 毫秒 / 58.92 RU59 ms / 58.92 RU 4 毫秒 / 8.92 RU4 ms / 8.92 RU 4 毫秒 / 8.92 RU4 ms / 8.92 RU
[Q6][Q6] 306 毫秒 / 2063.54 RU306 ms / 2063.54 RU 83 毫秒 / 532.33 RU83 ms / 532.33 RU 9 毫秒 / 16.97 RU9 ms / 16.97 RU

我们已优化一个读取密集型方案We have optimized a read-heavy scenario

你可能已注意到,我们在牺牲写入请求(命令)性能的情况下,集中精力改善了读取请求(查询)的性能。You may have noticed that we have concentrated our efforts towards improving the performance of read requests (queries) at the expense of write requests (commands). 在许多情况下,写入操作现在会通过更改源触发后续的反规范化,因此,其计算开销更大,且具体化的时间更长。In many cases, write operations now trigger subsequent denormalization through change feeds, which makes them more computationally expensive and longer to materialize.

这种方案是合理的,因为博客平台(类似于大多数社交应用)是读取密集型的,这意味着,它需要服务的读取请求数量往往比写入请求数量要高几个数量级。This is justified by the fact that a blogging platform (like most social apps) is read-heavy, which means that the amount of read requests it has to serve is usually orders of magnitude higher than the amount of write requests. 因此,提高所要执行的写入请求的开销是有利的,这可以降低读取请求的开销,并提高其性能。So it makes sense to make write requests more expensive to execute in order to let read requests be cheaper and better performing.

执行极端优化后我们发现, [Q6] 的开销已从 2000 多个 RU 降到了 17 个 RU;这种改进是通过反规范化帖子实现的,每个项的开销大约为 10 RU。If we look at the most extreme optimization we have done, [Q6] went from 2000+ RUs to just 17 RUs; we have achieved that by denormalizing posts at a cost of around 10 RUs per item. 由于我们要服务的源请求比帖子创建或更新请求要多得多,在考虑到总体节省的情况下,这种反规范化带来的开销可以忽略不计。As we would serve a lot more feed requests than creation or updates of posts, the cost of this denormalization is negligible considering the overall savings.

可以增量方式应用反规范化Denormalization can be applied incrementally

本文中探讨的可伸缩性改进涉及到反规范化,以及复制整个数据集中的数据。The scalability improvements we've explored in this article involve denormalization and duplication of data across the data set. 请注意,不一定要在第 1 天就完成这些优化。It should be noted that these optimizations don't have to be put in place at day 1. 根据分区键筛选的查询在大规模执行时的性能更佳,但是,如果极少调用跨分区查询或者针对有限的数据集调用此类查询,则其性能也完全可接受。Queries that filter on partition keys perform better at scale, but cross-partition queries can be totally acceptable if they are called rarely or against a limited data set. 如果你只是要生成一个原型,或者要推出一款用户群较小并且受控的产品,则也许可以在今后再实施这些改进;重要的是监视模型的性能,以便可以确定是否以及何时将其投放生产。If you're just building a prototype, or launching a product with a small and controlled user base, you can probably spare those improvements for later; what's important then is to monitor your model's performance so you can decide if and when it's time to bring them in.

用于将更新分发到其他容器的更改源会持久存储所有这些更新。The change feed that we use to distribute updates to other containers store all those updates persistently. 这样,便可以请求所有更新,因为即使系统已包含大量的数据,创建容器和启动反规范化的视图也是一次性的同步操作。This makes it possible to request all updates since the creation of the container and bootstrap denormalized views as a one-time catch-up operation even if your system already has a lot of data.

后续步骤Next steps

完成这篇有关数据建模和分区实践的简介文章后,建议接下来阅读以下文章,以了解本文涉及的概念:After this introduction to practical data modeling and partitioning, you may want to check the following articles to review the concepts we have covered: