(9) How does Uber's ringpop work in a layman's terms? - Quora

(9) How does Uber's ringpop work in a layman's terms? - Quora

I hope this is layman's enough. Please let me know if it isn't...

Think of how distributed applications (services, in the Service-Oriented Architecture sense) are traditionally provisioned and run. That is, as a set of independent worker processes (let's call them instances from now on). Each of these instances are, for the most part, treated equally and get their fair share of traffic. These instances also don't know about one another. They act as if they are the only ones serving traffic for your application! How dare they! :P Their use is orchestrated by something else, like a load-balancer, in front of them.

This model is all well and good, but not the Ringpop model[1]. The very foundations of Ringpop are based on a membership protocol derived from SWIM. A membership protocol gives the ability for these independent instances to discover one another, establish connections for communication between one another and provides a very powerful building block to implement more advanced features like an actor model, (weak) leader election, work delegation, distributed querying and command execution, request coalescing, application-layer caching and more.

Ringpop has applied consistent hashing on top of its membership protocol as a part of its core offering. Consistent hashing is all about partitioning a keyspace and assigning ownership to each of the partitions. The keyspace is mapped onto a range of your choosing, but are typically a hash of the IDs of the entities/objects in your application. The owners are the individual instances of your application that were discovered through Ringpop's membership protocol. In short, certain objects in your application are owned by certain instances. Ownership is distributed evenly, but not in the same way a round-robin scheduler might distribute evenly.

What happens when membership changes, for instance, when you want to add capacity to your application or some hosts running your application fail? Ringpop automatically detects those changes and integrates them into its membership list and consistent hash ring. This integration may result in partitions being reassigned or rebalanced (aka new owners for the objects in your application).

Why is any of this good? When you want certain guarantees or deterministic behavior out of your application at tremendous scale often not everything can be homogenous. Not everything can serve every request or store all of the data and even if they could, sometimes it's extremely hard to do efficiently.

The other nice part about Ringpop is that, for any application embedding it, its clients remain completely ignorant of the underlying sharding scheme. Clients neither know nor have to care about who the right instance is to serve a request. They may send a request anywhere and Ringpop forwards it to the correct owner.

In summary, Ringpop provides scalable, fault-tolerant application-layer sharding. More fundamentally, it provides the instances of a distributed application with the ability to cooperate and coordinate in a variety of ways. Choose your own adventure!

Providing more clarity around Ringpop and its intended use is not only a fun problem to work through, but it's also my job. I hope this helps!

[1] Ringpop does not preclude your application from having a load-balancer in front of it. In fact, it still ought to!

Read full article from (9) How does Uber's ringpop work in a layman's terms? - Quora