Skip to content
master
Switch branches/tags
Code

Latest commit

65379: kvserver: actuate load-based replica rebalancing under heterogeneous localities r=aayushshah15 a=aayushshah15

This commit teaches the `StoreRebalancer` to make load-based rebalancing
decisions that are meaningful within the context of the replication constraints
placed on the ranges being relocated and the set of stores that can legally
receive replicas for such ranges.

Previously, the `StoreRebalancer` would compute the QPS underfull and overfull
thresholds based on the overall average QPS being served by all stores in the
cluster. Notably, this included stores that were in replication zones that
would not satisfy required constraints for the range being considered for
rebalancing. This meant that the store rebalancer would effectively never be
able to rebalance ranges within the stores inside heavily loaded replication
zones (since all the _valid_ stores would be above the overfull thresholds).

This patch is a move away from the bespoke relocation logic in the
`StoreRebalancer`. Instead, we have the `StoreRebalancer` rely on the
rebalancing logic used by the `replicateQueue` that already has the machinery
to compute load based signals for candidates _relative to other comparable
stores_. The main difference here is that the `StoreRebalancer` uses this
machinery to promote convergence of QPS across stores, whereas the
`replicateQueue` uses it to promote convergence of range counts. A series of
preceeding commits in this patchset generalize the existing replica rebalancing
logic, and this commit teaches the `StoreRebalancer` to use it.

This generalization also addresses another key limitation (see #62992) of the
`StoreRebalancer` regarding its inability to make partial improvements to a
range. Previously, if the `StoreRebalancer` couldn't move a range _entirely_
off of overfull stores, it would give up and not even move the subset of
replicas it could. This is no longer the case.

Resolves #61883
Resolves #62992
Resolves #31135

/cc @cockroachdb/kv

Release justification: fixes a set of major limitations behind numerous support escalations

Release note (performance improvement): QPS-based rebalancing is now
aware of different constraints placed on different replication zones. This
means that heterogeneously loaded replication zones (for instance, regions)
will achieve a more even distribution of QPS within the stores inside each
such zone.


Co-authored-by: Aayush Shah <aayush.shah15@gmail.com>
fa9acb4

Files

Permalink
Failed to load latest commit information.
Type
Name
Latest commit message
Commit time
pkg
Sep 8, 2021
Sep 7, 2021


CockroachDB is a cloud-native distributed SQL database designed to build, scale, and manage modern, data-intensive applications.

What is CockroachDB?

CockroachDB is a distributed SQL database built on a transactional and strongly-consistent key-value store. It scales horizontally; survives disk, machine, rack, and even datacenter failures with minimal latency disruption and no manual intervention; supports strongly-consistent ACID transactions; and provides a familiar SQL API for structuring, manipulating, and querying data.

For more details, see our FAQ or architecture document.

Docs

For guidance on installation, development, deployment, and administration, see our User Documentation.

Starting with CockroachCloud

We can run CockroachDB for you, so you don't have to run your own cluster.

See our online documentation: Quickstart with CockroachCloud

Starting with CockroachDB

  1. Install CockroachDB: using a pre-built executable or build it from source.
  2. Start a local cluster and connect to it via the built-in SQL client.
  3. Learn more about CockroachDB SQL.
  4. Use a PostgreSQL-compatible driver or ORM to build an app with CockroachDB.
  5. Explore core features, such as data replication, automatic rebalancing, and fault tolerance and recovery.

Client Drivers

CockroachDB supports the PostgreSQL wire protocol, so you can use any available PostgreSQL client drivers to connect from various languages.

Deployment

  • CockroachCloud - Steps to create a free CockroachCloud cluster on your preferred Cloud platform.
  • Manual - Steps to deploy a CockroachDB cluster manually on multiple machines.
  • Cloud - Guides for deploying CockroachDB on various cloud platforms.
  • Orchestration - Guides for running CockroachDB with popular open-source orchestration systems.

Need Help?

Building from source

See our wiki for more details.

Contributing

We welcome your contributions! If you're looking for issues to work on, try looking at the good first issue list. We do our best to tag issues suitable for new external contributors with that label, so it's a great way to find something you can help with!

See our wiki for more details.

Engineering discussions take place on our public mailing list, cockroach-db@googlegroups.com. Also please join our Community Slack (there's a dedicated #contributors channel!) to ask questions, discuss your ideas, and connect with other contributors.

Design

For an in-depth discussion of the CockroachDB architecture, see our Architecture Guide. For the original design motivation, see our design doc.

Licensing

Current CockroachDB code is released under a combination of two licenses, the Business Source License (BSL) and the Cockroach Community License (CCL).

When contributing to a CockroachDB feature, you can find the relevant license in the comments at the top of each file.

For more information, see the Licensing FAQs.

Comparison with Other Databases

To see how key features of CockroachDB stack up against other databases, check out CockroachDB in Comparison.

See Also