AWS S3 vs Google Cloud vs Azure:
Cloud Storage Performance
Cloud Storage Performance Dec 29, 2015 I'm building a new cloud product that quickly processes large amounts of scientific data. Our largest customer dataset so far is about 3,000 tables, each 40 to 80 MB and totaling 150 GB, which we aim to process in 10 seconds or less. Each table can be processed independently, so we parallelize heavily to reach this goal — our deployment uses 1000 vCPUs or more as needed. The tricky part is rapidly reading that much data into memory. Right now about 80% of the computation time is spent reading data, and that leads to the focus of this three-part post: cloud storage performance. As part of solving this problem, I evaluated a few different approaches: object storage, database-backed storage and attached storage, each of which I'll be detailing in separate posts. Midway through writing this, Google happened to released their multi-cloud PerfKitBenchmarker. As far as I can find,
Read full article from AWS S3 vs Google Cloud vs Azure:
Cloud Storage Performance
No comments:
Post a Comment