Why So Few Qualified Architects?
A Proven Theory of the Architect’s Difficulties
As an experienced architect, I have interviewed many candidates. But I have always had one question in my mind: why is it so hard to find candidates who meet the expectations?
Recently, after reading a study, I was able to answer this question in a more systematic way. The topic of this study is: Talent vs Luck: the role of randomness in success and failure.
It is also the 2022 Ig Informal Lecture for the Economics Prize.
Before we explain the connection between the architect’s difficulty in finding and this study, let’s quickly summarise the study.
To make a long story short, a person’s success has nothing to do with talent, but only with luck. To put it further, “Success is 1% genius and 99% luck”.
However, Thomas Edison told us that genius is 1% inspiration and 99% perspiration. In other words, to be successful, effort is only 0.99%.
Why are so few candidates qualified to be architects?
Because being an architect is not something you can do just by working hard, it’s very much a matter of chance.
Wait, that’s a bit arbitrary, isn’t it?
In fact, every single one of those interviewed was a good senior engineer — yes, every single one.
They all excel in their positions and have a good grasp of what they are responsible for. Even a team leader or manager who hadn’t actually written code in a long time still had a deep understanding of the service they were working on.
However, these candidates were eliminated.
The reason was simple: not enough experience. Yes, I agree that they can all fully handle one or a few microservices. But the volume of these microservices was just not there, either because they lacked the volume of traffic and high-frequency reads and writes, or because they lacked the complexity and interaction with other systems.
This kind of experience is no match for an architect of a high-volume e-commerce platform.
Did you notice a key point? They all do their jobs well or even brilliantly, but they are still inexperienced.
Specialist vs. Generalist
This is actually the difference between I-shaped talents and T-shaped talents, i.e. Specialist and Generalist.
If you want to become a qualified architect, it is not enough just to do a good job. But to be honest, even if you work very hard, you may not be qualified. As I mentioned at the beginning, whether you can become an architect or not is mostly a matter of luck.
- If you are lucky enough to have worked for a large enterprise, then there is a little more possibility.
- If, luckily, the company is willing to invest in various training, then there are more possibilities.
- If you are lucky enough to have friendships with architects, then you have a little more possibilities.
Can’t we learn the skills we should have by our own efforts?
Yes, but it’s really hard.
If you don’t see how wide the world is, you can’t imagine what you need to learn.
There is a Chinese idiom, “Frog in the bottom of a well”. A frog who lives in a well all his life cannot know the outside world.
Back to the architect’s perspective, let’s look at a few real-world examples.
How do you know you need to learn caching if a system works well with one database and one API service?
How do you know you need to learn sharding if the amount of data can always be carried by a single database?
How do you know you need to learn message queuing if the logic of the system is simple and can be responded in a short time?
More specifically, if a system does not have any search requirement, how do you know you need to learn Elasticsearch?
All these are the vision that an architect needs to have, so do you know why most of the time you need to rely on “luck”?
Even so
There are still possibilities for us to try our best.
- Read more books.
- Attend more seminars.
- Visit more blogs.
Even if we don’t know what’s missing, we can still become what we want to be by learning through various channels.
Architects are on a never-ending journey of learning.
Conclusion
We’ve talked a lot about what it takes to be an architect, and explained why it’s so hard to find a qualified candidate, but I have to say that being an architect is just one of the “choices” in a career.
It doesn’t mean an architect is superior, it doesn’t mean an architect is invincible, and it doesn’t mean an architect is better than a senior engineer.
If learning is your hobby, then architect is a good choice.
But a senior software developer is also a good choice.