Architecture Weekly #47

Architecture Weekly Issue #47. Articles, books, and playlists on architecture and related topics. Split by sections, highlighted with complexity: 🤟 means hardcore, 👷‍♂️ is technically applicable right away,  🍼 - is an introduction to the topic or an overview. Now in telegram as well.

WARNING 🇺🇦

It's already been 298 days since Russia's crazy, brutal and unjustified war against Ukraine. We condemn this war and want it to stop ASAP. We continue this newsletter so you can advance your skill and help the millions of Ukrainian people in any way possible. If you want to help directly, visit this fund.

Video

Hey, folks! I got a bit sick, so the video is coming earlier next week. Sorry for that.

Highlights

This week highlights are all about databases. First of all, Bolt and PingCAP held a technical meetup covering the TitaniumDB architecture and the necessity of migration from MySQL to TitaniumDB for Bolt. Next I include an article about DB sharding following it up with a practical example of balancing the shards and moving data by Shopify. Find it all below!

TitaniumDB and Bolt 👷‍♂️

On 6th of December there was a first technical meetup of Bolt and PingCAP combined. There we discussed how Bolt used to have MySQL as a primary database, what were the issues, it's journey to Vitesse and finally to TitaniumDB. A fascinating journey I would say!

#database #titaniumdb

Database Sharding Explained 🍼

Database sharding is a technique used to horizontally partition a database into smaller, faster, more easily managed parts called shards. There are several approaches to sharding, including range-based, hash-based, and directory-based sharding. Sharding is a complex process that requires careful planning and is not suitable for all databases or workloads. However, for large, high-performance databases that need to scale, sharding can be an effective way to improve performance, scalability, and availability. Find a guide on the sharding strategies below.

Database Sharding: Concepts, Examples, and Strategies
Database sharding divides data into smaller chunks and distributes it across different database nodes. Learn more about sharding practices and strategies.

#database #sharding

Shard Balancing with confidence 👷‍♂️

Continuing the topic of sharding, I want to share an astonishing article by Shopify. First of all, I like the approach of picking up a sharding strategy: they create a hypothesis and then test it with real data in the simulated environment, thus being able to pick a best strategy. Secondly, the article contains the real example of moving a high traffic shop from one shard to another keeping the integrity in place and avoiding downtime. I am impressed!  

Shard Balancing: Moving Shops Confidently with Zero-Downtime at Terabyte-scale
This post explains how we’re able to balance our MySQL shards by migrating shops across shards—entirely online and with virtually zero consumer-facing downtime.

#sharding #database

Follow-Up

Diagrams as Code 🍼

Creating the diagrams with code is almost mainstream now with Mermaid, PlantUML and their integration into GitHub, Confluence and other tools. Today we share a new project which allows doing the same - with Python Code now.

Diagrams · Diagram as Code
Diagram as Code

#diagrams #documentation

A curated list for Awesome K8s Security 👷‍♂️

Kubernetes provides many capabilities, but with great power comes great responsibility. In the K8s case - it's the cluster security. Grab a very nice list of blogs, videos, books and CVEs on Kubertentes Security.

GitHub - magnologan/awesome-k8s-security: A curated list for Awesome Kubernetes Security resources
A curated list for Awesome Kubernetes Security resources - GitHub - magnologan/awesome-k8s-security: A curated list for Awesome Kubernetes Security resources

#security #k8s #kubernetes

Abstraction is Expensive 🤟

We build complex systems via abstractions of different levels: be it the network, files, virtual machines etc. If the abstraction is misaligned you end up with a cost: let's say you pick a wrong schema for a DB and instead of a single look up spend time doing scans. In the blog post Nima Badizadegan from Google shares his thoughts on why abstractions are expensive and gives some great examples even with TCP or memory allocators.

Abstraction is Expensive - Speculative Branches

#abstraction #vision

Essential features of an Event Store 🍼

Whatever you use for event store: a custom built piece, a library or a complete solution, you should have some expectations from it like being append-only, supporting event streams, event stream versioning, and others. Find the full list in the article down below.

Essential features of an Event Store for Event Sourcing
An Event Store, whether it is an out-of-the-box solution of custom-built, needs to offer certain essential features and might offer some…

#eventsourcing #eventstore

Designing an Engineering Strategy. Part 1 🍼

Part of the Enterprise Architect, Director of Engineering and other senior roles is designing and executing an engineering strategy, which is way to solve a big business problem from the engineering perspective. Sounds interesting? Start with a first post from a series of designing an engineering strategy.

Designing an Engineering Strategy. Part I
What the heck is Strategy?

#leadership #strategy #ea

What is a service mesh? 🍼

Mesh term is used so frequently: data mesh, network mesh, service mesh. Folks from Koyeb blog decided to explain what is a service mesh, what it typically consist of, and how it can help to solve the problems for the developers of microservices.

What is a service mesh?
Wondering what is a service mesh? Get all of your service mesh questions answered with this short and sweet primer. We take a look at how the two crucial parts of a service mesh, the control plane and the data plane, work together to handle interservice communication.

#servicemesh #microservice

Like the newsletter? Consider helping to run it at Patreon or Boosty. The funds go to pay for the hosting and some software like a Camo Studio license. Patrons and Boosty subscribers of a certain level also get access to a private Architecture Community. Big thanks to Nikita, Anatoly, Oleksandr, Dima, Pavel, Robert, Roman and Andrei for already supporting the newsletter.