Architecture Weekly #35

Architecture Weekly Issue #35. Articles, books, and playlists on architecture and related topics. Split by sections, highlighted with complexity: 🤟 means hardcore, 👷‍♂️ is technically applicable right away,  🍼 - introduction to the topic or an overview. Now in telegram as well.

WARNING 🇺🇦

It's already been 207 days since Russia's crazy, brutal, an unjustified war against Ukraine. We condemn this war and want it to stop ASAP. We continue this newsletter so you can advance your skill and help the millions of Ukrainian people in any way possible.

Databases

SQLite Internals: Pages & B-trees 🤟

SQLite is a small database famous for it's reliability, small size, usage in the mobile applications and the fact that the LOCs of tests for it is greater that the lines of actual DB code. In this article you will find how SQLite encodes the rows, why we need pages and how b-trees allow to find data in log(N) time.

SQLite Internals: Pages & B-trees
Let’s open a hex editor and see what this thing is made of

Building a scalable neobanking platform from scratch on CockroachDB  👷‍♂️

Small though interesting case study of a neobanking startup looking for a database solution for the financial ledger. Typically, banks go with Oracle's banking engine or build something on top of Postgres. There are still some alternatives, but Fi decided to go with Cockroach DB. Follow the article to know why.

Building a scalable neobank platform from scratch on CockroachDB
How Fi is using CockroachDB as a financial ledger for India’s first neobanking platform for millennials

PostgreSQL HA & Kubernetes 👷‍♂️

Speaking of PostgreSQL: the question "Can I run my database in K8s" is really popular. Ralph Soika shared the ways you can combine running a PostgreSQL together with Kubernetes(either outside of it or within), including the network block storage. More in the article.

PostgreSQL HA & Kubernetes – Ralph’s Open Source Blog

Cloud

Streaming data to BigQuery from Kafka on GCP using Dataflow 👷‍♂️

This week I stumbled upon a question on how to use data from Kafka to make fast search(preferably using SQL) and be able to update or delete the data. One of the several options is using GCP's BigQuery offering, but you need to get the data there first. Thomas de Lazzari described the receipt how to do that.

Kafka to BigQuery using Dataflow
In this article, we evaluate two different methods to connect Kafka to BigQuery using Dataflow

Videos

Understanding Microservices with Distributed Tracing 👷‍♂️

When you have a monolith application, it's really easy to understand what happens: you grab a log in Kibana for the app and see what happens. But when you break it down to 200+ services, you just can't find the answer. One of the solutions is instrumentation based distributed tracing. Watch a video by Lita Cho, where she describes how it works at Lyft.

Distributed Tracing and Complex World

Having a previous one, can't forget sharing another video on distributed tracing which more about the theory and open standards, which drive the distributed tracing further. Get a video from GOTO; conference.

General

Data Warehouse Architecture 👷‍♂️

Following the same task I realized that having some ingest from Kafka to perform searching, post-processing or analytical tasks sounds like a part of the bigger solution - Data Warehouse. Find an article on how it can be designed.

Data Warehouse Architecture, Components & Diagram Concepts
This data warehouse architecture tutorial covers all the basic to advance stuff like definitions, characteristics, architectures, components, data marts, and more.

DDD Beyond the Basics: Mastering Multi-Bounded Context  🍼

We all love DDD. One of the questions that we usually have is how to connect the bounded context. We know, that Anti-corruption layer pattern is one of the ways. Mario Bittencourt shows, how it can be implemented, and which smaller patterns can help to do that.

DDD Beyond the Basics: Mastering Multi-Bounded Context Integration
Domain-Driven Design (DDD) has been around since 2003 when it was introduced by Eric Evans. Because of the potential benefits it can bring…

Improving Post Search at LinkedIn 🍼

We all receive the letters from LinkedIn saying "Similar jobs" or "You might be also interested in this company". Sometimes they are even relevant! It appears the task to create those recommendations is pretty complex, and their engineering blog has an article on how LI solves it.

Improving Post Search at LinkedIn
Co-authors: Amol Chandla, Jagadeesan Sundaresan, Siddharth Pratap Singh, Saipriya Kumar, and Anand Kishore

Architecting Apache Kafka for GDPR Compliance 🍼

GDPR is a regulation which among many things requires companies to delete the data of a particular customer at his or her will. But if you have the data in Kafka, how do you do that? Grab a receipt down below.

Architecting Apache Kafka for GDPR Compliance
A complete-ish, long-form guide to addressing Apache Kafka for GDPR with event-driven architecture.

Liked the newsletter? Consider helping to run it at Patreon or Boosty. Patrons and Boosty subscribers of certain level also get an access to private Architecture Community. Big thanks to Nikita, Anatoly, Oleksandr, Dima and Pavel for already supporting the newsletter.