Article originally posted on InfoQ. Visit InfoQ

As the flagship track at QCon, Architectures You’ve Always Wondered About showcases real-world examples of innovator companies pushing the limits with modern software systems. At recent conferences in San Francisco and London, the speakers clearly showed what “scalable” can really mean, from a trillion messages to exabytes of data. This eMag brings together several of these stories and hopefully provides advice and inspiration for your future projects.

At Dropbox, it is crucial to provide high performance and availability for the data storage solutions they provide. Facundo Agriel goes deep into the details about Dropbox’s exabyte-scale blob storage system for Magic Pocket. In addition to all the software that handles millions of queries per second, the physical hardware design must also be considered. Because each storage device contains over 100 drives, and hardware devices can fail, the system has to be self-healing. The architecture has to consider this and many other factors to achieve Dropbox’s performance targets while remaining cost-effective.

Monzo serves 7 million banking customers daily, on an architecture of thousands of microservices that they deploy hundreds of times a day. Suhail Patel covers some of the specific technologies they use, including Cassandra and Kubernetes, and how a focus on platform engineering and developer experience has allowed them to scale up, even with a relatively lean engineering team. Because nothing ever goes entirely as planned, there were some mistakes and incidents along the way, but those led to some valuable lessons and a better system in the end.

Cloudflare provides DDoS protection for websites around the globe, and their services have relied on Kafka to send over 1 trillion messages. However, decoupling those microservices, and the teams that build them, required dedicated effort. Changing the message format from JSON to Protobuf, and the development of custom tools, led to more structured messages and better fault tolerance, which in turn reduced the cognitive load of teams and increased adoption. Andrea Medda and Matt Boyle explain how balancing technical and socio-technical needs is necessary to create a successful solution.

The eMag wraps up by looking at a new architecture pattern, which Bilgin Ibryam calls cloud-bound applications. This is the evolution of cloud-native architecture, and moves the abstraction of compute-centric concerns to an abstraction of application integration concerns. The CNCF project Dapr provides APIs that speak to these integration needs and allow architects to design a clean separation around core business logic.

Free download