使用可靠集合Working with Reliable Collections

Service Fabric 通过可靠集合向 .NET 开发人员提供有状态的编程模型。Service Fabric offers a stateful programming model available to .NET developers via Reliable Collections. 具体而言,Service Fabric 提供可靠字典和可靠队列类。Specifically, Service Fabric provides reliable dictionary and reliable queue classes. 在使用这些类时,状态是分区的(实现伸缩性)、复制的(实现可用性),并在分区内进行事务处理(实现 ACID 语义)。When you use these classes, your state is partitioned (for scalability), replicated (for availability), and transacted within a partition (for ACID semantics). 让我们看一下可靠字典对象的典型用法,并看一看它究竟在做些什么。Let's look at a typical usage of a reliable dictionary object and see what it's actually doing.

try
{
   // Create a new Transaction object for this partition
   using (ITransaction tx = base.StateManager.CreateTransaction())
   {
      // AddAsync takes key's write lock; if >4 secs, TimeoutException
      // Key & value put in temp dictionary (read your own writes),
      // serialized, redo/undo record is logged & sent to secondary replicas
      await m_dic.AddAsync(tx, key, value, cancellationToken);

      // CommitAsync sends Commit record to log & secondary replicas
      // After quorum responds, all locks released
      await tx.CommitAsync();
   }
   // If CommitAsync isn't called, Dispose sends Abort
   // record to log & all locks released
}
catch (TimeoutException)
{
   // choose how to handle the situation where you couldn't get a lock on the file because it was 
   // already in use. You might delay and retry the operation
}

可靠字典对象上的所有操作(无法恢复的 ClearAsync 除外)都需要一个 ITransaction 对象。All operations on reliable dictionary objects (except for ClearAsync, which is not undoable), require an ITransaction object. 此对象与在单个分区中对任何可靠字典和/或可靠队列对象尝试进行的任何及所有更改具有关联性。This object has associated with it any and all changes you're attempting to make to any reliable dictionary and/or reliable queue objects within a single partition. 可通过调用分区的 StateManager 的 CreateTransaction 方法获取 ITransaction 对象。You acquire an ITransaction object by calling the partition's StateManager's CreateTransaction method.

在上面的代码中,ITransaction 对象传递到可靠字典的 AddAsync 方法。In the code above, the ITransaction object is passed to a reliable dictionary's AddAsync method. 在内部,接受键的字典方法采用与键关联的读取器/写入器锁。Internally, dictionary methods that accept a key take a reader/writer lock associated with the key. 如果此方法修改键的值,则在键上使用写入锁;如果此方法只读取键的值,则在键上使用读取锁。If the method modifies the key's value, the method takes a write lock on the key and if the method only reads from the key's value, then a read lock is taken on the key. 由于 AddAsync 将键值修改成新的传入值,因此使用键的写入锁。Since AddAsync modifies the key's value to the new, passed-in value, the key's write lock is taken. 因此,如果有 2(或更多个)线程尝试在同一时间添加相同的键值,则一个线程将获取写入锁,另一个线程会阻塞。So, if 2 (or more) threads attempt to add values with the same key at the same time, one thread will acquire the write lock, and the other threads will block. 默认情况下,方法最多阻塞 4 秒以获取锁,4 秒后方法会引发 TimeoutException。By default, methods block for up to 4 seconds to acquire the lock; after 4 seconds, the methods throw a TimeoutException. 方法重载存在可让你根据需要传递显式超时值。Method overloads exist allowing you to pass an explicit timeout value if you'd prefer.

通常,编写代码响应 TimeoutException 的方式是捕获它,然后重试整个操作(如以上代码中所示)。Usually, you write your code to react to a TimeoutException by catching it and retrying the entire operation (as shown in the code above). 在我的简单代码中,我只调用了每次传递 100 毫秒的 Task.Delay。In my simple code, I'm just calling Task.Delay passing 100 milliseconds each time. 但实际上,最好改用某种形式的指数退让延迟。But, in reality, you might be better off using some kind of exponential back-off delay instead.

获取锁后,AddAsync 会在与 ITransaction 对象关联的内部临时字典中添加键和值对象引用。Once the lock is acquired, AddAsync adds the key and value object references to an internal temporary dictionary associated with the ITransaction object. 这就完成了读取自己编写的语义。This is done to provide you with read-your-own-writes semantics. 也就是说,在调用 AddAsync 之后,稍后对 TryGetValueAsync 的调用(使用相同的 ITransaction 对象)将返回值,即使尚未提交事务。That is, after you call AddAsync, a later call to TryGetValueAsync (using the same ITransaction object) will return the value even if you have not yet committed the transaction. 接下来,AddAsync 将键和值对象序列化为字节数组,并将这些字节数组附加到本地节点的日志文件。Next, AddAsync serializes your key and value objects to byte arrays and appends these byte arrays to a log file on the local node. 最后,AddAsync 将字节数组发送给所有辅助副本,使其具有相同的键/值信息。Finally, AddAsync sends the byte arrays to all the secondary replicas so they have the same key/value information. 即使键/值信息已写入日志文件,在提交其关联的事务之前,这些信息不被视为字典的一部分。Even though the key/value information has been written to a log file, the information is not considered part of the dictionary until the transaction that they are associated with has been committed.

在上述代码中,调用 CommitAsync 会提交所有事务操作。In the code above, the call to CommitAsync commits all of the transaction's operations. 具体而言,它将提交信息附加到本地节点的日志文件,同时将提交记录发送给所有辅助副本。Specifically, it appends commit information to the log file on the local node and also sends the commit record to all the secondary replicas. 回复副本的仲裁(多数)后,所有数据更改将被视为永久性,并释放通过 ITransaction 对象操作的任何键关联锁,使其他线程/事务可以操作相同的键及其值。Once a quorum (majority) of the replicas has replied, all data changes are considered permanent and any locks associated with keys that were manipulated via the ITransaction object are released so other threads/transactions can manipulate the same keys and their values.

如果未调用 CommitAsync(通常是因为引发了异常),则会释放 ITransaction 对象。If CommitAsync is not called (usually due to an exception being thrown), then the ITransaction object gets disposed. 在释放未提交的 ITransaction 对象时,Service Fabric 会将中止信息追加到本地节点的日志文件,且不需要将任何信息发送到任何辅助副本。When disposing an uncommitted ITransaction object, Service Fabric appends abort information to the local node's log file and nothing needs to be sent to any of the secondary replicas. 然后将释放通过事务操作的任何与键关联的锁。And then, any locks associated with keys that were manipulated via the transaction are released.

常见陷阱及其规避方法Common pitfalls and how to avoid them

现在你已了解可靠集合在内部的工作原理,让我们了解一些常见的误用。Now that you understand how the reliable collections work internally, let's take a look at some common misuses of them. 参阅以下代码:See the code below:

using (ITransaction tx = StateManager.CreateTransaction())
{
   // AddAsync serializes the name/user, logs the bytes,
   // & sends the bytes to the secondary replicas.
   await m_dic.AddAsync(tx, name, user);

   // The line below updates the property's value in memory only; the
   // new value is NOT serialized, logged, & sent to secondary replicas.
   user.LastLogin = DateTime.UtcNow;  // Corruption!

   await tx.CommitAsync();
}

使用常规 .NET 字典时,可以在字典中添加键/值,然后更改属性的值(例如 LastLogin)。When working with a regular .NET dictionary, you can add a key/value to the dictionary and then change the value of a property (such as LastLogin). 不过,此代码无法对可靠字典正常运行。However, this code will not work correctly with a reliable dictionary. 我们前面讨论过:调用 AddAsync 将键/值对象序列化成字节数组,并将数组存储到本地文件,并将它们发送到辅助副本。Remember from the earlier discussion, the call to AddAsync serializes the key/value objects to byte arrays and then saves the arrays to a local file and also sends them to the secondary replicas. 稍后如果更改属性,只会更改内存中的属性值,而不会影响本地文件或发送到副本的数据。If you later change a property, this changes the property's value in memory only; it does not impact the local file or the data sent to the replicas. 如果进程崩溃,内存中的内容将全部丢失。If the process crashes, what's in memory is thrown away. 启动新的进程或另一个副本变成主副本时,旧属性值是可用的值。When a new process starts or if another replica becomes primary, then the old property value is what is available.

再次强调,上面这种错误是很容易发生的。I cannot stress enough how easy it is to make the kind of mistake shown above. 只有在进程崩溃时才能发现错误。And, you will only learn about the mistake if/when the process goes down. 编写代码的正确方式是只需反转两行:The correct way to write the code is simply to reverse the two lines:

using (ITransaction tx = StateManager.CreateTransaction())
{
   user.LastLogin = DateTime.UtcNow;  // Do this BEFORE calling AddAsync
   await m_dic.AddAsync(tx, name, user);
   await tx.CommitAsync();
}

这是另一个常见的错误:Here is another example showing a common mistake:

using (ITransaction tx = StateManager.CreateTransaction())
{
   // Use the user's name to look up their data
   ConditionalValue<User> user = await m_dic.TryGetValueAsync(tx, name);

   // The user exists in the dictionary, update one of their properties.
   if (user.HasValue)
   {
      // The line below updates the property's value in memory only; the
      // new value is NOT serialized, logged, & sent to secondary replicas.
      user.Value.LastLogin = DateTime.UtcNow; // Corruption!
      await tx.CommitAsync();
   }
}

同样地,使用常规 .NET 字典时,以上代码以常见的模式正常运行:开发人员使用键查询值。Again, with regular .NET dictionaries, the code above works fine and is a common pattern: the developer uses a key to look up a value. 如果值存在,开发人员会更改属性的值。If the value exists, the developer changes a property's value. 不过,使用可靠集合时,此代码会出现前面所述的相同问题: 将对象分配给可靠集合后,你不得修改该对象However, with reliable collections, this code exhibits the same problem as already discussed: you MUST not modify an object once you have given it to a reliable collection.

在可靠集合中更新值的正确方式是获取对现有值的引用,并将此引用所引用的对象视为不可变。The correct way to update a value in a reliable collection, is to get a reference to the existing value and consider the object referred to by this reference immutable. 然后创建新的对象,即原始对象的完全相同副本。Then, create a new object that is an exact copy of the original object. 现在,可以修改此新对象的状态,将新对象写入集合,以便将它序列化为字节数组、附加到本地文件并发送到副本。Now, you can modify the state of this new object and write the new object into the collection so that it gets serialized to byte arrays, appended to the local file and sent to the replicas. 提交更改之后,内存中的对象、本地文件和所有副本都处于完全一致的状态。After committing the change(s), the in-memory objects, the local file, and all the replicas have the same exact state. 大功告成!All is good!

以下代码演示在可靠集合中更新值的正确方式:The code below shows the correct way to update a value in a reliable collection:

using (ITransaction tx = StateManager.CreateTransaction())
{
   // Use the user's name to look up their data
   ConditionalValue<User> currentUser = await m_dic.TryGetValueAsync(tx, name);

   // The user exists in the dictionary, update one of their properties.
   if (currentUser.HasValue)
   {
      // Create new user object with the same state as the current user object.
      // NOTE: This must be a deep copy; not a shallow copy. Specifically, only
      // immutable state can be shared by currentUser & updatedUser object graphs.
      User updatedUser = new User(currentUser);

      // In the new object, modify any properties you desire
      updatedUser.LastLogin = DateTime.UtcNow;

      // Update the key's value to the updateUser info
      await m_dic.SetValue(tx, name, updatedUser);
      await tx.CommitAsync();
   }
}

定义不可变的数据类型以防止编程器错误Define immutable data types to prevent programmer error

理想情况下,我们希望编译器能够在意外生成改变对象状态的代码、而此对象又不该改变时报告错误。Ideally, we'd like the compiler to report errors when you accidentally produce code that mutates state of an object that you're supposed to consider immutable. 但是 C# 编译器做不到这一点。But, the C# compiler does not have the ability to do this. 因此,为了避免潜在的编程器错误,我们强烈建议将可靠集合使用的类型定义为不可变类型。So, to avoid potential programmer bugs, we highly recommend that you define the types you use with reliable collections to be immutable types. 具体而言,这意味着你要坚持使用核心值类型(例如数字 [Int32、UInt64 等]、DateTime、Guid、TimeSpan 等)。Specifically, this means that you stick to core value types (such as numbers [Int32, UInt64, etc.], DateTime, Guid, TimeSpan, and the like). 也可以使用 String。You can also use String. 最好是避免集合属性,因为将其序列化和反序列化经常会降低性能。It is best to avoid collection properties as serializing and deserializing them can frequently hurt performance. 但是,如果希望使用集合属性,强烈建议使用 .NET 的不可变集合库 (System.Collections.Immutable)。However, if you want to use collection properties, we highly recommend the use of .NET's immutable collections library (System.Collections.Immutable). 可以从 https://nuget.org 下载此库。此外,我们建议尽可能地密封类,并将字段设为只读。This library is available for download from https://nuget.org. We also recommend sealing your classes and making fields read-only whenever possible.

以下 UserInfo 类型演示如何利用上述建议定义不可变类型。The UserInfo type below demonstrates how to define an immutable type taking advantage of aforementioned recommendations.

[DataContract]
// If you don't seal, you must ensure that any derived classes are also immutable
public sealed class UserInfo
{
   private static readonly IEnumerable<ItemId> NoBids = ImmutableList<ItemId>.Empty;

   public UserInfo(String email, IEnumerable<ItemId> itemsBidding = null) 
   {
      Email = email;
      ItemsBidding = (itemsBidding == null) ? NoBids : itemsBidding.ToImmutableList();
   }

   [OnDeserialized]
   private void OnDeserialized(StreamingContext context)
   {
      // Convert the deserialized collection to an immutable collection
      ItemsBidding = ItemsBidding.ToImmutableList();
   }

   [DataMember]
   public readonly String Email;

   // Ideally, this would be a readonly field but it can't be because OnDeserialized
   // has to set it. So instead, the getter is public and the setter is private.
   [DataMember]
   public IEnumerable<ItemId> ItemsBidding { get; private set; }

   // Since each UserInfo object is immutable, we add a new ItemId to the ItemsBidding
   // collection by creating a new immutable UserInfo object with the added ItemId.
   public UserInfo AddItemBidding(ItemId itemId)
   {
      return new UserInfo(Email, ((ImmutableList<ItemId>)ItemsBidding).Add(itemId));
   }
}

ItemId 类型也是不可变类型,如下所示:The ItemId type is also an immutable type as shown here:

[DataContract]
public struct ItemId
{
   [DataMember] public readonly String Seller;
   [DataMember] public readonly String ItemName;
   public ItemId(String seller, String itemName)
   {
      Seller = seller;
      ItemName = itemName;
   }
}

架构版本控制(升级)Schema versioning (upgrades)

就内部而言,可靠集合使用 .NET 的 DataContractSerializer 串行化对象。Internally, Reliable Collections serialize your objects using .NET's DataContractSerializer. 串行化对象保存在主副本的本地磁盘中,并传输到辅助副本。The serialized objects are persisted to the primary replica's local disk and are also transmitted to the secondary replicas. 随着服务日趋成熟,你可能想要更改服务所需的数据种类(架构)。As your service matures, it's likely you'll want to change the kind of data (schema) your service requires. 必须十分谨慎地对待数据的版本控制方法。Approach versioning of your data with great care. 首先但同样重要的是,始终必须能够反序列化旧数据。First and foremost, you must always be able to deserialize old data. 具体而言,这意味着反序列化代码必须无限向后兼容:服务代码的版本 333 必须能够对 5 年前服务代码第 1 版放在可靠集合中的数据进行操作。Specifically, this means your deserialization code must be infinitely backward compatible: Version 333 of your service code must be able to operate on data placed in a reliable collection by version 1 of your service code 5 years ago.

此外,服务代码一次只升级一个域。Furthermore, service code is upgraded one upgrade domain at a time. 因此,在升级期间,同时执行两个不同版本的服务代码。So, during an upgrade, you have two different versions of your service code running simultaneously. 必须避免新版本的服务代码使用新的架构,因为旧版的服务代码可能无法处理新的架构。You must avoid having the new version of your service code use the new schema as old versions of your service code might not be able to handle the new schema. 应该尽可能将每个版本的服务都设计成向前兼容 1 个版本。When possible, you should design each version of your service to be forward compatible by one version. 具体而言,这意味着 V1 的服务代码应当能够忽略它不显式处理的任何架构元素。Specifically, this means that V1 of your service code should be able to ignore any schema elements it does not explicitly handle. 但是,它必须能够保存它不显式了解的任何数据,并且在更新字典键或值时将它写回。However, it must be able to save any data it doesn't explicitly know about and write it back out when updating a dictionary key or value.

警告

尽管可以修改键的架构,但必须确保键的哈希代码和相等算法是稳定的。While you can modify the schema of a key, you must ensure that your key's hash code and equals algorithms are stable. 如果更改其中任一算法的工作方式,再也无法在可靠字典中查询键。If you change how either of these algorithms operate, you will not be able to look up the key within the reliable dictionary ever again. .NET 字符串可以用作键,但请使用字符串本身作为键,不要使用 String.GetHashCode 的结果作为键。.NET Strings can be used as a key but use the string itself as the key--do not use the result of String.GetHashCode as the key.

另外,也可以执行通称为两阶段升级的功能。Alternatively, you can perform what is typically referred to as a two upgrade. 通过两阶段升级,可以将服务从 V1 升级到 V2:V2 包含知道如何处理新架构更改的代码,但此代码不会执行。With a two-phase upgrade, you upgrade your service from V1 to V2: V2 contains the code that knows how to deal with the new schema change but this code doesn't execute. 当 V2 代码读取 V1 数据时,它在其上操作并写入 V1 数据。When the V2 code reads V1 data, it operates on it and writes V1 data. 然后,在跨所有升级域的升级都完成之后,就可以通知运行中的 V2 实例,升级已完成。Then, after the upgrade is complete across all upgrade domains, you can somehow signal to the running V2 instances that the upgrade is complete. (通知方式之一是推出配置升级;这就是两阶段升级。)现在,V2 实例可以读取 V1 数据,将它转换成 V2 数据、操作它,并写出为 V2 数据。(One way to signal this is to roll out a configuration upgrade; this is what makes this a two-phase upgrade.) Now, the V2 instances can read V1 data, convert it to V2 data, operate on it, and write it out as V2 data. 当其他实例读取 V2 数据时,不需要转换它,只要操作并写出 V2 数据即可。When other instances read V2 data, they do not need to convert it, they just operate on it, and write out V2 data.

后续步骤Next Steps

若要了解如何创建向前兼容的数据约定,请参阅向前兼容的数据协定To learn about creating forward compatible data contracts, see Forward-Compatible Data Contracts

若要了解版本控制数据协定的最佳做法,请参阅数据协定版本控制To learn best practices on versioning data contracts, see Data Contract Versioning

若要了解如何实现版本容错的数据协定,请参阅版本容错的序列化回调To learn how to implement version tolerant data contracts, see Version-Tolerant Serialization Callbacks

若要了解如何提供可跨多个版本互操作的数据结构,请参阅 IExtensibleDataObjectTo learn how to provide a data structure that can interoperate across multiple versions, see IExtensibleDataObject