The Log: What every software engineer should know about real-time data's unifying abstraction | LinkedIn Engineering

The Log: What every software engineer should know about real-time data's unifying abstraction | LinkedIn Engineering

Zookeeper handles much of the system co-ordination (perhaps with a bit of help from higher-level abstractions like Helix or Curator).

Mesos and YARN do process virtualization and resource management

Embedded libraries like Lucene and LevelDB do indexing

Netty, Jetty and higher-level wrappers like Finagle and rest.li handle remote communication

Avro, Protocol Buffers, Thrift, and umpteen zillion other libraries handle serialization

Kafka and Bookeeper provide a backing log.

Read full article from The Log: What every software engineer should know about real-time data's unifying abstraction | LinkedIn Engineering