Architecture Weekly #102

Architecture Weekly Issue #102. Articles, books, and playlists on architecture and related topics. Split by sections, highlighted with complexity: 🤟 means hardcore, 👷‍♂️ is technically applicable right away,  🍼 - is an introduction to the topic or an overview. Now in telegram and Substack as well.

Highlights

Scalable OLTP in the Cloud 🤟

Murat discusses a paper, where it is shown that OLTP databases are not scalable(yeah, Oracle, PostgreSQL and MS SQL) due to the nature of MVCC implementation. The writing and reading transactions require synchronization, and ever read transactions do that as well. The paper includes a DB design which would solve the problems, but relaxing the guarantees is required. Get a cup of strong tea, and be ready for some hard thinking while reading :)

Scalable OLTP in the Cloud: What’s the BIG DEAL?
This paper is from Pat Helland, the apostate philosopher of database systems, overall a superb person, and a good friend of mine. The paper...

#paper #db #cloud

Overview of Distributed PostgreSQL Architectures 🤟

We talk a lot about distributed databases, but barely talk about why we need them in the first place. In this post you will find a nice explanation why availability and reliability requirements inevitably lead us to such architectures and what options do we have with 5 different variants. 16 minute long-read inside.

An Overview of Distributed PostgreSQL Architectures | Crunchy Data Blog
Marco just joined Crunchy Data and he reflects on his career in distributed systems in this post. He provides an overview of several options for approaching distributed Postgres workloads and the pros and cons of each approach.

#db #distrubutedsystems

The Scary Thing About Automating Deploys 👷‍♂️

I learned from this article that Slack is using a Monolith and deploying it 30 times a day automatically with the help of their ReleaseBot. But it wasn't always the case, and developers had to guide the deployment step by step. The main problem was understanding if everything goes well. Eventually, they figured it out with the anomaly detection, which led to fully automated deployment. Great stuff!  

The Scary Thing About Automating Deploys - Slack Engineering
Most of Slack runs on a monolithic service simply called “The Webapp”. It’s big – hundreds of developers create hundreds of changes every week. Deploying at this scale is a unique challenge. When people talk about continuous deployment, they’re often thinking about deploying to systems as soon as ch…

#sre

Follow-Up

Fallacies of Distributed Computing 🍼

Distributed Systems are hard for various reasons, including the misconceptions or fallacies. Find a series of article which explain the fallacies and their root causes in a lightweight manner and provide advice on how to handle them.

Fallacy #1: The network is reliable
Anyone with a cable or DSL modem knows how temperamental network connections can be. The Internet just stops working, and the only way to get it going again is to unplug it for 15 seconds. (Or, put…

#distributedsystems

5 Ways to Share Code Between Lambda Functions 👷‍♂️

Serverless adoption is on the rise, and one of the problems while using lambdas is sharing the code between like modules, or common ways to connect to s3, dynamodb and other services. Find an article with 5 different ways of sharing hte code between lambda functions, including absolutely ridiculous ones like storing code on s3 and download it in runtime :)

5 Ways to Share Code Between Lambda Functions
How to Share Code Between Lambda Functions?

#aws #serverless

Refactoring Legacy Code with the Strangler Fig Pattern 👷‍♂️

We all faced the situations with God-objects: when a class is handling several responsibilites at once. The refactoring of such object are usually tedious and error-prone. Shopify is telling their approach with the Strangler Fig Pattern. Steps and Diagrams included!

Refactoring Legacy Code with the Strangler Fig Pattern
There are strategies for reducing the size and responsibilities of large objects. Here’s one that worked for us at Shopify, an all-in-one commerce platform supporting over 1M merchants across the globe.

#refactoring

What is Scalability Anyway? 🍼

You can try to give your own definition of scalability, and you may find one by Marc Brooker: "A system is scalable in the range where the cost of adding incremental work is approximately constant". That means you can build a graph of cost for different solutions, which Marc is doing in his post to come to the conclusion that serverless systems ARE actually scalable according to this definition.

What is Scalability Anyway? - Marc’s Blog

#scalability #serverless

End-to-End Tracing at Canva 🍼

Canva shared the story of implementing e2e tracing in their high-loaded product. Switching to OpenTelemetry, picking up observability provider and solving technical issues with OTEL in Frontend - everything in this blog post!

End-to-end Tracing - Canva Engineering Blog
How Canva implemented end-to-end tracing and are using it to drive operational insights

#observability #casestudy

How eBay Export Billion-Scale Graphs on Transactional Graph Databases 👷‍♂️

There is not much case studies shared on using Graph Databases in the internet, and here is a good piece on how eBay found a nice solution to making queries on the large database. Instead of backup/restore functionality they found that disaster recovery setup works much faster, which allows to have a copy of a transaction graph database for subsequent queries. Tons of details inside.  

How We Export Billion-Scale Graphs on Transactional Graph Databases
By leveraging Apache Spark, eBay Graph Database can export a graph with billions of vertices and edges.

#db #graph #casestudy

WARNING 🇺🇦

The brutal and unjustified war against Ukraine continues. If you want to help Ukraine directly visit this fund.

Big thanks to Nikita, Anatoly, Oleksandr, Dima, Pavel B, Pavel, Robert, Roman, Iyri, Andrey, Lidia, Vladimir, August, Roman, Egor, Roman, Evgeniy, Nadia, Daria, Dzmitry, Mikhail, Nikita and Dmytro for supporting the newsletter. They receive early access to the articles, influence the content and participate in the closed group where we discuss the architecture problems. They also see my daily updates on all the things I am working on. Join them at Patreon or Boosty!