去分库分表的亿级数据NewSQL实践之旅



去分库分表的亿级数据NewSQL实践之旅

所以说,上述的特点非常契合目前我们在数据库应用设计上碰到的问题。经过评估,我们开始着手安排部署和测试,在备份、监控、故障恢复和扩展几个方面有以下的一些心得:


  • 官方提供了一套基于 mydumper 和 myloader 的工具套件,是基于逻辑备份的方式,对于迁移来说是很便捷的;

  • 监控用的是应用内置上报 Prometheus 的方案,可以写脚本与自有的监控系统对接;

  • 有状态的KV层采用的是 Raft 协议来保证高可用,Leader 的选举机制满足了故障自恢复的需求;

  • 无状态的 TiDB-Server 查询层可以在前段搭建LVS 或HAProxy来保证故障的切换;

  • KV 层的 Region 分裂保证了集群无感知的扩展。


测试之后发现数据库运维中比较重要的几项都已经闭环的解决了,但有得有失,实测结论是:


  • TiDB 是分布式结构,有网络以及多副本开销,导致 Sysbench OLTP 测试中 单 server 的读性能不如 MySQL,但在大数据量下性能相较于MySQL不会产生明显下滑,且弹性扩展能力评估后可以满足业务的峰值需求;


Read full article from 去分库分表的亿级数据NewSQL实践之旅


No comments:

Post a Comment

Labels

Algorithm (219) Lucene (130) LeetCode (97) Database (36) Data Structure (33) text mining (28) Solr (27) java (27) Mathematical Algorithm (26) Difficult Algorithm (25) Logic Thinking (23) Puzzles (23) Bit Algorithms (22) Math (21) List (20) Dynamic Programming (19) Linux (19) Tree (18) Machine Learning (15) EPI (11) Queue (11) Smart Algorithm (11) Operating System (9) Java Basic (8) Recursive Algorithm (8) Stack (8) Eclipse (7) Scala (7) Tika (7) J2EE (6) Monitoring (6) Trie (6) Concurrency (5) Geometry Algorithm (5) Greedy Algorithm (5) Mahout (5) MySQL (5) xpost (5) C (4) Interview (4) Vi (4) regular expression (4) to-do (4) C++ (3) Chrome (3) Divide and Conquer (3) Graph Algorithm (3) Permutation (3) Powershell (3) Random (3) Segment Tree (3) UIMA (3) Union-Find (3) Video (3) Virtualization (3) Windows (3) XML (3) Advanced Data Structure (2) Android (2) Bash (2) Classic Algorithm (2) Debugging (2) Design Pattern (2) Google (2) Hadoop (2) Java Collections (2) Markov Chains (2) Probabilities (2) Shell (2) Site (2) Web Development (2) Workplace (2) angularjs (2) .Net (1) Amazon Interview (1) Android Studio (1) Array (1) Boilerpipe (1) Book Notes (1) ChromeOS (1) Chromebook (1) Codility (1) Desgin (1) Design (1) Divide and Conqure (1) GAE (1) Google Interview (1) Great Stuff (1) Hash (1) High Tech Companies (1) Improving (1) LifeTips (1) Maven (1) Network (1) Performance (1) Programming (1) Resources (1) Sampling (1) Sed (1) Smart Thinking (1) Sort (1) Spark (1) Stanford NLP (1) System Design (1) Trove (1) VIP (1) tools (1)

Popular Posts