Key concepts for scaling systems

There are many different tricks to build systems that scale, but they all follow the same underlying principles:

1. Caching: What expensive computations can we store and then serve again without recomputing them? This works both for computations but also for static website content.
2. Decoupling producers and consumers: Add a queue between a producer and consumer to decouple them. This provides a buffer and safety for if one of them inevitably falls over.
3. Separate reads from writes: Often the read and write workload is vastly different. Can we exploit this and create a main write database and multiple follower read replicas?
4. Horizontal scaling & load balancing: Can we scale our systems in such a way that we can simply add more machines to handle the load? This works for both servers (autoscaling) but also for databases (sharding). This does mean that we have to balance and manage this load.

Image is from Alex Xu’s book.

Subscribe

I blog about how to grow as a machine learning engineer! Liked this article and want to hear more? Join 40+ others and subscribe!