Last time, we talked about 8 fallacies of distributed computing and why our systems end up being more complex than we ever imagined. In our last conclusion, we mentioned that two nodes already have a lot of aspects to consider, let alone a microservice architecture. Microservice architectures have many additional…

Know the three main items of a good design review — This article will take backend development as an example, but the same concepts can be applied to various fields. Before explaining how to prepare a good design review, let’s first introduce what a design review should have. As an architect, I’m going to tell you what I want to see. …

Explained with a code sample — This article is the last of this series. We have already described the problem encountered in data-oriented design. In this article, we will introduce a better way to tackle a feature requirement. We continue the previous example, a sign-in mission, and try a different…

Software design is an evolving process. Every large system starts from a tiny system. When a problem is encountered in the existing architecture but cannot be solved, the system will begin to evolve. Every evolution is accompanied by some technical selections. What problems should be solved? What price will it…

Top-down and Bottom-up, which one is better — Last time we talked about how to do technical selection, the key is to understand the requirements completely, so that we can pick the most fit among the options. Remember, it’s the right fit, not the best, because there is no perfect solution. Nevertheless, sometimes we don’t just make one…

There are so many tradeoffs, how to carry on? — Last week, an engineer asked me how to make a technical selection. And he provided a real-world example. Their organization wanted to implement an event-driven architecture, and there were three options available right now — MQTT, AMQP, and Kafka — but he didn’t know how to make a decision. My…

New challenges lead to new tools, then new challenges? — In the data pipeline, we often talk about ETL, aka Extract-Transform-Load, which is actually a very simple process as follows.

The difference between the perspective of a backend engineer and a data engineer — Although I have been working with data in the past, this year, I have been working closely with data engineers and finally fully realized the concept of data-centric. We all know that applications are made up of data, so as a backend engineer or architect, we must be involved in…

Demonstrating some common practices of SQL Window Function — When doing ETL, we always write data into the database one batch at a time, which contains the snapshot of each batch. these snapshots are basically historical data and may contain a lot of duplicates, so how do we find the data we need in these batches? This article will…