Building scalable cloud databases

Applies to: Azure SQL Database

Scaling out databases can be easily accomplished using scalable tools and features for Azure SQL Database. In particular, you can use the Elastic Database client library to create and manage scaled-out databases. This feature lets you easily develop sharded applications using hundreds—or even thousands—of databases in Azure SQL Database.

To download:

• The Java version of the library, see Maven Central Repository.
• The .NET version of the library, see NuGet.

Documentation

1. Get started with Elastic Database tools
2. Elastic Database features
3. Shard map management
4. Migrate existing databases to scale out
5. Data dependent routing
6. Multi-shard queries
7. Adding a shard using Elastic Database tools
8. Upgrade client library apps
9. Elastic queries overview
10. Elastic Database tools glossary
11. Elastic Database client library with Entity Framework
12. Elastic Database client library with Dapper
13. Split-merge tool
14. Performance counters for shard map manager
15. FAQ for Elastic Database tools

Client capabilities

Scaling out applications using sharding presents challenges for both the developer as well as the administrator. The client library simplifies the management tasks by providing tools that let both developers and administrators manage scaled-out databases. In a typical example, there are many databases, known as "shards," to manage. Customers are co-located in the same database, and there is one database per customer (a single-tenant scheme). The client library includes these features:

• Shard map management: A special database called the "shard map manager" is created. Shard map management is the ability for an application to manage metadata about its shards. Developers can use this functionality to register databases as shards, describe mappings of individual sharding keys or key ranges to those databases, and maintain this metadata as the number and composition of databases evolves to reflect capacity changes. Without the Elastic Database client library, you would need to spend a lot of time writing the management code when implementing sharding. For details, see Shard map management.

• Data dependent routing: Imagine a request coming into the application. Based on the sharding key value of the request, the application needs to determine the correct database based on the key value. It then opens a connection to the database to process the request. Data dependent routing provides the ability to open connections with a single easy call into the shard map of the application. Data dependent routing was another area of infrastructure code that is now covered by functionality in the Elastic Database client library. For details, see Data dependent routing.

• Multi-shard queries (MSQ): Multi-shard querying works when a request involves several (or all) shards. A multi-shard query executes the same T-SQL code on all shards or a set of shards. The results from the participating shards are merged into an overall result set using UNION ALL semantics. The functionality as exposed through the client library handles many tasks, including: connection management, thread management, fault handling, and intermediate results processing. MSQ can query up to hundreds of shards. For details, see Multi-shard querying.

In general, customers using Elastic Database tools can expect to get full T-SQL functionality when submitting shard-local operations as opposed to cross-shard operations that have their own semantics.

Next steps

• Elastic Database client library (Java, .NET) - to download the library.

• Get started with Elastic Database tools - to try the sample app that demonstrates client functions.

• GitHub (Java, .NET) - to make contributions to the code.

• Azure SQL Database elastic query overview - to use elastic queries.

• Moving data between scaled-out cloud databases - for instructions on using the split-merge tool.

Related content

Not using elastic database tools yet? Check out our Getting Started Guide.