Redis and its C# implementation | miafish

Redis and its C# implementation | miafish

When we use Redis?

Redis is a fantastic choice if you want a highly scalable data store shared by multiple processes, multiple applications, or multiple servers. As just an inter-process communication mechanism it is tough to beat(such as dictionary). The fact that you can communicate cross-platform, cross-server, or cross-application just as easily makes it a pretty great choice for many use cases. Its speed also makes it great as a caching layer.

How does Redis work?

Redis is key value cache and store. why it is highly scalable and can be shared by multiple processes, multiple applications or multiple servers? Let us see the design of Redis.

Redis Cluster data sharding

hash slot: there are 16384 hash slots in Redis Cluster.

Eg: Every node in a Redis Cluster is responsible of a subset of the hash slots, so for example you may have a cluster with 3 nodes, where:

• Node A contains hash slots from 0 to 5500.
• Node B contains hash slots from 5501 to 11000.
• Node C contains hash slots from 11001 to 16384.

Redis Cluster mater-slave model

when the cluster is created (or at a latter time) we add a slave node to every master, so that the final cluster is composed of A, B, C that are masters, and A1, B1, C1 that are slaves, the system is able to continue if node B fails(B1 will become master and has copy data of B).

Internal work

A,B,C nodes. B with foo inside

1. Client => A : Cluster Hints
2. A => Client: a map of hash slots with nodes
3. Client => B: get foo
4. B => Client: bar

Fault tolerance

A guesses B is failing, as the latest PING request timed out. A will not take any action without any other hint.

C sends a PONG to A. with the gossip section containing  about B; C also think B is failing.

At this point  A marks B as failed, and notifies the information to all the other nodes in the cluster, that will mark the node as failing.

If B will ever return back, the first time he will PING any nodes of the cluster. It will be notified to shut down ASAP. As intermitting clients are not good for the clients.

Note: only way to rejoin a Redis cluster after massive crash is : Redis-trib by hand

Redis Partitioning

• Redis Cluster (internal system)
• Twemproxy (mid layer)
• Clients supporting consistent hashing (client side)

How Redis expires keys

Redis keys are expired in two ways: a passive way, and an active way.

A key is actively expired simply when some client tries to access it, and the key is found to be timed out.

Of course this is not enough as there are expired keys that will never be accessed again. This keys should be expired anyway, so periodically Redis test a few keys at random among keys with an expire set. All the keys that are already expired are deleted from the keyspace.

Specifically this is what Redis does 10 times per second:

1. Test 100 random keys from the set of keys with an associated expire.
2. Delete all the keys found expired.
3. If more than 25 keys were expired, start again from step 1.

This is a trivial probabilistic algorithm, basically the assumption is that our sample is representative of the whole key space, and we continue to expire until the percentage of keys that are likely to be expired is under 25%

This means that at any given moment the maximum amount of keys already expired that are using memory is at max equal to max amount of write operations per second divided by 4.

Read full article from Redis and its C# implementation | miafish