Architecture Weekly #105

Architecture Weekly Issue #105. Articles, books, and playlists on architecture and related topics. Split by sections, highlighted with complexity: 🀟 means hardcore, πŸ‘·β€β™‚οΈ is technically applicable right away,  🍼 - is an introduction to the topic or an overview. Now in telegram and Substack as well.

If you're interested in the technologies, development approaches and overall business of our little startup in the compliance field subscribe on Patreon and Boosty, as I shared an article recently on how we added a second product to our architecture last week.

Highlights

Data Structures for Data-Intensive Applications 🀟

This paper provides an insightful guide on choosing and designing data structures for managing large amounts of data efficiently. It explains the trade-offs between speed, storage, and updates, helping developers and architects make informed decisions for their systems. Through clear concepts like the RUM framework, it highlights how understanding data usage and hardware can lead to better performance in applications ranging from databases to operating systems.

#db #performance

Fundamentals of Availability for System Design Interview 🍼

One of the requirements you need to fullfil during an interview is availability. There are several items you need to understand about what does it mean for business, how we measure availability and how to design for it. Checkout here the fundamentals, and remember, if you need help preparing for such an interview, you can always request a mock system design here.

Master the Fundamentals of Availability for Systems Design Interview
Master availability to choose the best technologies for your applications and pass on a Systems Design interview. Click here for more!

#systemdesign

connect() - why are you so slow? 🀟

While this post by Cloudflare explains the mechanics of choosing a source port while establishing a tcp connection, it mostly amazes me how the deep network mechanics can affect the overall system performance much more than the design decisions. Β 

connect() - why are you so slow?
This is our story of what we learned about the connect() implementation for TCP in Linux. Both its strong and weak points. How connect() latency changes under pressure, and how to open connection so that the syscall latency is deterministic and time-bound.

#performance

Follow-Up

An intuition for distributed consensus in OLTP systems πŸ‘·β€β™‚οΈ

Building a mental model for distributed systems it not trivial. Phil Eaton writes a long read which explains it in relatively simple terms touching on Raft, linearizability and implications for availability. Great read!

An intuition for distributed consensus in OLTP systems | notes.eatonphil.com
An intuition for distributed consensus in OLTP systems

#distributedsystems

S3-FIFO: Scalable and Efficient Cache Eviction πŸ‘·β€β™‚οΈ

The most prominient critic of FIFO cache is low hit rate. However, improving it by altering from one to 3 queues efficiently fixes the problem. Find a paper, visualization and statictis on the s3fifo website.

#algorithm #performance

RADIO Framework for System Design interview 🍼

Many folks are struggling with the structure to follow on the system design interview. I mean they jump from requirements to drawing, then back to requirements, then they are getting lost. But there are couple of approaches to be an interviewee, and one of them is RADIO: Requirements, Architecture, Data, Interfaces and Deep Dive.

RADIO Framework for System Design | Apply to Any Front End System Design Interview
Apply the RADIO Framework to Front End System Design Interviews to guide yourself in a structured manner. A good structure is half the battle won.

#interview

Transforming Postgres into a Fast OLAP Database πŸ‘·β€β™‚οΈ

PostgreSQL is frequently a default choice for new systems, but only for OLTP. In order to get analytics, we would at other solutions like ClickHouse. ParadeDB claims that with the new pg_analytics extension you can turn PostgreSQL into an OLAP storage by deltalake tables. Find the benchmarks and details inside!

pg_analytics: Transforming Postgres into a Fast OLAP Database - ParadeDB

#db

The state of Kubernetes jobs in 2023 Q4 🍼

This newsletter is supposed to help you advance your skills and grow in your career or land a better job. Let's take a look at the state of Kubernetes jobs market worldwide. Numbers, distribution, required skilled and the titles - in the research below.

The state of Kubernetes jobs in 2023 Q4
A deep dive into what skills and knowledge is required to get hired in a Kubernetes position.

#kubernetes

WARNING πŸ‡ΊπŸ‡¦

The brutal and unjustified war against Ukraine continues. If you want to help Ukraine directly visit this fund.

Big thanks to Nikita, Anatoly, Oleksandr, Dima, Pavel B, Pavel, Robert, Roman, Iyri, Andrey, Lidia, Vladimir, August, Roman, Egor, Roman, Evgeniy, Nadia, Daria, Dzmitry, Mikhail, Nikita and Dmytro for supporting the newsletter. They receive early access to the articles, influence the content and participate in the closed group where we discuss the architecture problems. They also see my daily updates on all the things I am working on. Join them at Patreon or Boosty!