Last time, we talked about 8 fallacies of distributed computing and why our systems end up being more complex than we ever imagined. In our last conclusion, we mentioned that two nodes already have a lot of aspects to consider, let alone a microservice architecture. Microservice architectures have many additional…

Know the three main items of a good design review — This article will take backend development as an example, but the same concepts can be applied to various fields. Before explaining how to prepare a good design review, let’s first introduce what a design review should have. As an architect, I’m going to tell you what I want to see. …

Explained with a code sample — This article is the last of this series. We have already described the problem encountered in data-oriented design. In this article, we will introduce a better way to tackle a feature requirement. We continue the previous example, a sign-in mission, and try a different design flow. …

Software design is an evolving process. Every large system starts from a tiny system. When a problem is encountered in the existing architecture but cannot be solved, the system will begin to evolve. Every evolution is accompanied by some technical selections. What problems should be solved? What price will it…

Start from a small step — Last time we talked about the evolution of the data infrastructure, the whole evolution process is as follows. Stage 0: None Stage 1: Batching Stage 2: Streaming Stage 3: Integrated Stage 4: Automation Stage 5: Decentralization But remember? The evolution from stage 1 to stage 2 was a bit weird…

From Monolith to Self-service Platform — All systems start as a small monolith. In the beginning, when resources and manpower are not sufficient, the monolith is the choice we have to make, even the data infrastructure is no exception. But as requirements increase, there are more and more scenarios that cannot be achieved by the current…

Understanding Hot-Cold Separation — In this article, we are going to introduce how to archive the audit logs stored in MongoDB to AWS S3, but before we start, let’s understand why we need it and the meaning of the need. Audit logs are records of user actions on the system, including signing in and…

Three approaches to improve database elasticity — Last week we talked about the difference between scalability and elasticity, and the concept of solutions was mentioned at a high level. In this article, let’s take a look at some practical solutions. First, if you haven’t read the last article, I suggest you read it first. This time we…

Understanding the concept of streaming architecture — Last time, we introduced streaming processing. In order to be able to handle batch and real-time data with a more pure infrastructure, we introduced Kafka and the streaming framework. In this article, we will introduce a common design pattern for streaming, enrichment, and examine what benefits the streaming framework can…