When eventual consistency is REALLY eventual - AWS S3

When eventual consistency is REALLY eventual - AWS S3

When Eventual Consistency is Really Eventual Amazon's S3 storage services promises extreme durability (11 9s) but only eventual consistency.  Applications must be designed to accommodate this relaxed consistency model.  Because consistency is often achieved within just seconds, it is easy to be lulled into habits that fail when the latency to consistency is much higher.  Can your system handle very high latency operations? On March 13, we observed an extreme example of eventual consistency in S3.  The "eventual" nature of S3 during that time is illustrated in the graph below.  (I would like to emphasize that the situation described below had no impact on our customer experience because it affected only background, batch processing.) Total Number of S3 Objects to Process as a Function of Time The graph shows the number of items in a collection of several hundred queues.  We buffer messages in the queues for an hour and then process the previous hour's messages.

Read full article from When eventual consistency is REALLY eventual - AWS S3