多服务器通讯层应该如何设计—一次code review小记 – 后端技术 by Tim Yang



多服务器通讯层应该如何设计—一次code review小记 – 后端技术 by Tim Yang

对于服务器内部的通讯功能,已经有TCP提供可靠传输,已经有类似Apache MINA这样的框架来提供上层的封装,业务逻辑代码完全可以直接基于socket或MINA这样的框架来直接传送数据,我们为什么还需要包装一个独立的通讯层?

目前考虑到的原因有

  • Connection pool作用。为了节点间更高的throughput, 通常两节点之间会使用n个固定连接来传输。n可设为CPU内核数大小。
  • 多节点互相连接的处理。一个节点需要连接n个节点发送信息。如果这部分逻辑放在业务调用层来实现,业务层会变得臃肿。
  • 处理自动重连逻辑,单个节点可以停用并重新启用。将这部分逻辑封装到通讯层,上层调用会更优雅。
  • 服务器之间通讯很短时间内不稳定(比如10s左右的中断)的处理。如果IDC内部或IDC之间通讯不稳定会造成这种现象。我们希望这种短暂的中断,不要引起业务逻辑重新选择节点,造成大量数据迁移及cache重构带来的内部振荡。
  • 海量数据传输最优化,大部分时间单个节点单向传输会>10k个数据包/s,独立的通讯层容易测试观察该环节是否存在瓶颈。


Read full article from 多服务器通讯层应该如何设计—一次code review小记 – 后端技术 by Tim Yang


No comments:

Post a Comment

Labels

Algorithm (219) Lucene (130) LeetCode (97) Database (36) Data Structure (33) text mining (28) Solr (27) java (27) Mathematical Algorithm (26) Difficult Algorithm (25) Logic Thinking (23) Puzzles (23) Bit Algorithms (22) Math (21) List (20) Dynamic Programming (19) Linux (19) Tree (18) Machine Learning (15) EPI (11) Queue (11) Smart Algorithm (11) Operating System (9) Java Basic (8) Recursive Algorithm (8) Stack (8) Eclipse (7) Scala (7) Tika (7) J2EE (6) Monitoring (6) Trie (6) Concurrency (5) Geometry Algorithm (5) Greedy Algorithm (5) Mahout (5) MySQL (5) xpost (5) C (4) Interview (4) Vi (4) regular expression (4) to-do (4) C++ (3) Chrome (3) Divide and Conquer (3) Graph Algorithm (3) Permutation (3) Powershell (3) Random (3) Segment Tree (3) UIMA (3) Union-Find (3) Video (3) Virtualization (3) Windows (3) XML (3) Advanced Data Structure (2) Android (2) Bash (2) Classic Algorithm (2) Debugging (2) Design Pattern (2) Google (2) Hadoop (2) Java Collections (2) Markov Chains (2) Probabilities (2) Shell (2) Site (2) Web Development (2) Workplace (2) angularjs (2) .Net (1) Amazon Interview (1) Android Studio (1) Array (1) Boilerpipe (1) Book Notes (1) ChromeOS (1) Chromebook (1) Codility (1) Desgin (1) Design (1) Divide and Conqure (1) GAE (1) Google Interview (1) Great Stuff (1) Hash (1) High Tech Companies (1) Improving (1) LifeTips (1) Maven (1) Network (1) Performance (1) Programming (1) Resources (1) Sampling (1) Sed (1) Smart Thinking (1) Sort (1) Spark (1) Stanford NLP (1) System Design (1) Trove (1) VIP (1) tools (1)

Popular Posts