System Scalability notes | miafish

System Scalability notes | miafish

1. server scale: every server contains exactly the same codebase and does not store any user-related data, like sessions or profile pictures, on local disc or memory.

way to do: this user-related data need to be stored in a centralized data store which is accessible to all your application servers. It can be an external database or an external persistent cache, like Redis

2. Database scale: if there is no much join in query, try NOSql.

way to do: more database servers, sharding, change to NOSql.

3. cache: do not touch database if you can

A cache is a simple key-value store and it should reside as a buffering layer between your application and your data storage. Whenever your application has to read data it should at first try to retrieve the data from your cache. Only if it's not in the cache should it then try to get the data from the main data source

4. cache object instead of database query

when your class has finished the "assembling" of the data array, directly store the data array, or better yet the complete instance of the class, in the cache! This allows you to easily get rid of the object whenever something did change and makes the overall operation of your code faster and more logical.

5. async: implement it as much as you can

Hadoop would be a good desgin

• Can't use just one database. Use many databases, partitioned horizontally and vertically.
• Because of partitioning, forget about referential integrity or cross-domain JOINs.
• Forget about 100% data integrity.
• At large scale, cost is a problem: hardware, databases, licenses, storage, power.
• Once you're large, spammers and data-scrapers come a-knocking.
• Cache!
• Use asynchronous flows.
• Reporting and analytics are challenging; consider them up-front when designing the system.
• Expect the system to fail.
• Don't underestimate your growth trajectory.

Read full article from System Scalability notes | miafish