What does 1 trillion yuan of GMV mean for Chinese e-commerce companies?
Back a decade ago, the answer was: it meant they had to pay $20 billion to ioE (IBM, Oracle, EMC) three U.S. companies, and they had to pay every year. That's even more than they could make that year.
If this happens, many of China's later giant-like Internet companies will become the hit workers of American companies.
But ten years later, this is not the case, and it all starts with a scientist named Yang Zhenkun.
Many years ago, he followed his teacher Academician Wang Xuan to create a laser illumination row.
Now he has created the world's first truly distributed database, OceanBase.
First, the internal entrepreneurship of three twists and turns
In fact, OceanBase has been facing a crisis of premature death since the day it was established in Alibaba.
Yang Zhenkun told Lei Fengwang about this, just shyly laughing, speaking in the old folk dialect with a slight Hubei accent, and his speech was not hurried or slow.
But he's well aware of the importance of the matter: "If we hadn't done OceanBase at the time, there would have been only one way for other companies to do database, and that would have been to 'wear a vest'—take the open source stuff, change it to your own 'vest,' and do it." ”
In Ali's previous strategic meeting, the strategic direction of "going to IOE" was insisted upon by Dr. Wang Jian, Ali's chief technology officer, and the previous statement of 20 billion yuan was calculated by him.
Chairman of Alibaba Group Technology Committee Wang Jian (Name: Ph.D.)
Wei Zhe, former CEO of Alibaba, recalled the scene of the day for Leifeng Network.
In order to increase his persuasiveness, Wang Jian even pulled up the retired CTO Wu Jiong, let him talk about the situation of Amazon and Google from a technical point of view, as an overall preparation; and then from the actuarial calculation of finance, he converted 1 trillion yuan of GMV into data volume, and then converted it into the number of IBM minicomputers, Oracle databases, and EMC storage quantities, and finally converted to the specific price.
After Wang Jian finished calculating, everyone on Ali's executive committee was convinced, and it was considered to solve this problem from the executive level. (The on-site discussion on the same day was very exciting, please see the revised version of "Boiling Fifteen Years" for details.) ）
However, it is not enough to convince the people above, the performers below are Oracle, EMC engineers, "go to IOE" is actually to smash their jobs, how can they support it.
When Yang Zhenkun started the OceanBase project, he heard more negative and negative voices than supporting voices.
As an innovator, going against the wind is a price that must be borne, and if you can't reach the other shore, you will fall to the ground and hear the overwhelming laughter.
Two years, this is the deadline ali gave Yang Zhenkun, and it is also the time when Ali executives can withstand the pressure for him. Within two years, the company fully supports the achievements; if you can't make achievements, you can only roll up and leave.
With such pressure, Yang Zhenkun set off with the OceanBase team.
The system that Yang Zhenkun wants to do is a native distributed database. If successful, OceanBase will have the opportunity to help enterprises build a new generation of core systems.
Before OceanBase, Oracle-dominated giants used "centralized databases." If the processing power and capacity of the database are analogous to "horse-drawn carts", the solution of the centralized database is to make the "horses" bigger, make the stand-alone machine more powerful, and drive the storage and processing of data.
But centralized databases also have problems. On the one hand, centralized databases, born decades ago, are too old to cope with exponentially increasing capacity demands. When the growth of the "horse" is difficult to keep up with the speed of the "car", then the collapse of the system is only a matter of time. Companies can still increase the capacity of databases by investing more money, but the cost will also be greatly improved.
On the other hand, the centralized database has poor risk prevention and control capabilities. If one machine in a centralized database fails, the entire system could be at risk of collapsing.
Since it is difficult to support alone, why not "collect firewood for everyone"? Already quite researched on distribution, Yang Zhenkun thought that many "ponies" could be used to pull this "big car" together.
That's where OceanBase, the native distributed database, came up with the idea, and in 2010, no one in the world came up with that idea. Yang Zhenkun's arm can be said to be "unprecedented".
Distributed databases are more scalable than traditional centralized databases. In the face of rapidly growing demand, OceanBase does not rely on stand-alone machines, and can achieve simple and inexpensive capacity expansion. At the same time, the original minicomputer is also tens of millions, in the OceanBase distributed concept, cheap stand-alone machines can also be used as servers, from hundreds of thousands to tens of thousands, OceanBase is cheaper than Oracle more than a little and a half.
Distributed databases are also more disaster-tolerant. Since OceanBase "doesn't put eggs in one basket," the database can still run stably when a stand-alone machine in the system has a problem. "Three places and five centers, three computer rooms in the same city", the system of ants is so designed, as long as most of the equipment is not down at the same time, the operation of the system will not be affected.
HTAP (Mixed Transaction Analysis Processing), which is another great tool of OceanBase.
The functionality of databases can be roughly divided into two categories – OLTP and OLAP.
OLTP means "online transaction and transaction processing", that is, "bookkeeping". Every expenditure of the buyer, every single sale of the merchant, and every piece of data generated on the platform must be clearly recorded in different categories.
OLAP means "online analytical processing", that is, "data analysis". The ability to analyze user preferences and analyze operational data is among them.
Users of databases have both "bookkeeping" and "analysis" needs, but these two needs usually need to be met by two different database systems, and there are few reliable, one-stop solutions on the market. Operating both systems naturally increases the cost and difficulty.
Like the left and right brains, "bookkeeping" and "analysis" are two different ways of processing data. "Bookkeeping" requires real-time interaction with the system, data is stored "row by row", while analysis requires vertical comparison, and data is stored "column by column".
What OceanBase does is to merge the "left brain" and "right brain" to form a one-stop solution. Convenient, but also cheap. It is not easy to put "row memory" and "column memory" together, and the distributed architecture design provides technical feasibility for the HTAP model.
Although the system design is good, but to be honest, 2 years is really not enough.
OceanBase's results in the first two years are difficult to say satisfactory, and Yang Zhenkun naturally knows this. At the end of 2012, many of the features were still not implemented, and the team was not complete enough. At the first level of the product, OceanBase lost.
The lack of business is another challenge that OceanBase encounters. Of course, the business groups within Ali are not "one-minded", the business bosses have to carry their own business indicators, the headquarters should also take into account their interests in making decisions, the database is the underlying system, and they cannot be forced to "change what they say and change". Under the departmental wall, OceanBase has not yet established a sense of trust with various business units when it first arrives, and can only "sell" its own products from the bottom up, which is naturally half the work.
With the 2-year period approaching, OceanBase faces a situation where the company could be shut down at any time.
Under the internal and external difficulties, Yang Zhenkun still got a second chance.
According to legend, Yang Zhenkun flew to the Ali headquarters in Hangzhou and found his old colleague Wang Jian at Microsoft. Wang Jian was the CTO of Alibaba at the time, and it was also the most difficult time for him in Alibaba Cloud. He was well aware of the difficulty of Yang Zhenkun's scientific research in Alibaba, and he could also see the potential of OceanBase in the future database competition, so he issued a paper transfer order after consulting with relevant leaders.
Soon after, OceanBase was adjusted to the Alipay system.
But when asked about this time, the first thing that Yang Zhenkun thought of was Cheng Li, the current CTO of Ali, whose name was Lu Su. After Yang Zhenkun and the team came to Alipay, as a "god-like figure" in the hearts of Ali technical people, Lu Su was also optimistic about the prospect of a native distributed database, so he helped Yang Zhenkun gain a firm foothold in Alipay, which also laid the groundwork for OceanBase's future rise.
Alibaba CTO Cheng Li (name: Lu Su)
Second, in 2014, the horn of the counterattack sounded
In 2014, at Alipay, Yang Zhenkun and OceanBase finally waited for a once-in-a-lifetime opportunity.
That year, the trading volume of Singles' Day is expected to hit a record again. In the face of another big test, Alipay's internal database engineers are facing a great enemy, and they have begun the stress test of database running volume in full swing.
At the moment of the enemy, Yang Zhenkun and his team are still sitting on the cold bench. After many times of active request, Alipay only let OceanBase bear 1% of the business turnover. Another year of disappointment, and the turnaround point that belongs to OceanBase seems out of reach.
As mentioned earlier, OceanBase's functions and structure are much stronger than Oracle's, why hasn't Alipay fully promoted OceanBase applications?
The IOE system is the PUA of the database industry. You know it's expensive, you know it's not good, you know there's a cheaper, more convenient solution, but you're willing to believe it. In the ToB business, trust is the most precious.
"How do you make sure OceanBase doesn't lose a penny of Alipay users?" Lu Su's words had also asked Yang Zhenkun.
Although IOE is not good, IOE wins in a steady way. Many technical leaders are reluctant to use new technologies and products, and problems with old products can be blamed on the product, and problems with new products can only be blamed on themselves.
At the end of the day, there is no use without trust, and there is no trust without use. OceanBase struggles in such an endless cycle.
Just when OceanBase's future was still unclear, a "bad" news came: Oracle collapsed!
It turned out that in the running volume test, Oracle, which had been dominant in the ant, had collapsed repeatedly, and the pressure capacity was only 90% of the expected.
I can't care if OceanBase is reliable or not, and I can't do it at this time. The technical team had to make a difficult decision: to urgently enable OceanBase.
The traffic of Singles' Day is increasing year by year, and the centralized Oracle will always fall to the altar one day. Yang Zhenkun has long been waiting for this day, and the opportunity for OceanBase to counterattack has come!
So OceanBase was ordered to take over 10% of Alipay's 2014 Double Eleven traffic.
This is an opportunity, but why not be heavier than The pressure of Tarzan? Yang Zhenkun knows that if he and his team make a difference, then he and OceanBase will lose trust in the company forever, and no one will dare to use it again. Even if it can bring benefits to the business later, it will not help.
In this battle, OceanBase can only succeed, not fail.
On the eve of Singles' Day, OceanBase was fully staffed. Ant Group CEO Peng Lei (flower name Lucy) previously served as the chief human resources officer within Alibaba, knowing people and making good use of them. She came to OceanBase's office overnight, both as a mobilization and as a supervisor.
The change was so huge, it was difficult to say that Peng Lei's heart was not worried. She asked Yang Zhenkun if he had confidence, and Yang happened to be standing by the window. He pushed open the window, Hangzhou in November has begun to turn cold, the office building under the pedestrians are hurried. Yang Zhenkun knew that his and OceanBase's future all depended on this, he said:
"Something went wrong and we're ready to jump out of here."
Seven years later, OceanBase has long since become the "head horse" in the domestic database. Talking about this, Yang Zhenkun only said that it was a joke. And the pressure, anxiety and determination of "becoming a benevolent person if you don't succeed" are like clouds of smoke and have become jokes many years later.
Open Source – OceanBase's Dreams, Frustrations, and Redemptions
Open source is the romance of many procedural apes.
Although it seems very puzzling to the outside world - to expose their hard-earned code to the public for free, OceanBase has resolutely chosen open source.
Putting aside the idealism of technical people, OceanBase chose open source for the following 3 reasons:
On the one hand, the database needs to run volume. In the process of sufficient testing and feedback, the database can find operational problems and make organic adjustments.
OceanBase, as a new concept database product, naturally needs practical testing and verification. Open source is to give OceanBase the opportunity to constantly "fight monsters and upgrades".
On the other hand, as just mentioned, trust is an important barrier to database products and a cornerstone of cooperation. Users want to entrust data to the database, almost to entrust their important "family property" to pay.
OceanBase makes all of its code and secrets public, and it also tells users that they are trustworthy. Based on this trust, OceanBase also expects to form its own open source community and expand its user base.
If someone uses it, there will be trust; with trust, more people will use it, and it will naturally produce more benefits.
Yang Zhenkun has enough confidence that even if OceanBase is open source for public use, it is difficult for others to understand the mystery and mechanism. As mentioned earlier, databases are an extremely complex project. Like chips, even if the finished product can be obtained, it is difficult to reverse engineer it. Even if it is open source, the moat is there.
The famous database MySQL has been open source for more than 20 years, and the core technology is still in the hands of the development team. This technical barrier is mapped to open source and is a core capability of a database.
So in 2011, OceanBase, which was still part of the Taobao system at that time, open sourced for the first time, and this open source experience is difficult to call satisfactory.
For OceanBase, this open source is quite "catching ducks on the shelves". OceanBase was still dominated by a single-point write model until 2014, and it was only two years later that it officially completed version 1.0.
In 2011, in order to cater to the trend of full open source of the Taobao system, but also the enthusiasm of technical people, OceanBase hastily announced open source, and the actual public is only a "semi-finished product". This version was still dominated by single-point writing at the time, and it was not at all what the team had ideally looked like.
Through this open source, another question is also mapped out: what is the positioning of OceanBase? Is it a support department for the ant business, or a generalized tool?
To support, OceanBase needs more internal-company requirements. Some of the "tailor-made" requirements may even go beyond the capabilities of a regular database. If you are led by this demand, it is likely to cause wear and tear on the construction period, and it is easy to narrow your own path.
What Yang Zhenkun and the team want to do has always been a generalized tool. However, the cognitive gap makes OceanBase, which is open source for the first time, unable to get enough support from the business team, and the development team has to devote its main energy to the development of the 1.0 version of the product, and it is difficult to allocate manpower to maintain the continuous update of the open source version.
It's hard to say who's right and who's wrong, it's just that the positions are different.
No one is available, the first open source OceanBase is finally no longer updated, and this attempt is finally gone.
After that, the OceanBase team, which was deeply involved in development, finally saw the payoff.
Inside Alipay, OceanBase gradually took over Oracle's previous "business." In order to make the "go to Oracle" process more "silky", OceanBase strives to make the process "uncoded".
On March 15, 2015, Yang Zhenkun proudly announced on his Weibo: "Since last Friday, 100% of Taobao/Tmall/Juhuasuan transactions on Alipay have been on OceanBase. You may not feel anything. ”
A year later, OceanBase finally took over the accounting database in its entirety. In an interview, Yang Zhenkun said: "This... It is a sign that OceanBase has truly gained a foothold in the financial system. ”
Such remarkable achievements have been achieved, and ali has also won awards internally, but Yang Zhenkun and the team's ambitions are not willing to stop here. Another year later, with the glamorous endorsement of Alipay's successful case, OceanBase was commercially available for the first time, and the first customer was Bank of Nanjing.
Alibaba's top brass have finally seen the advantages of OceanBase's native distributed database, and seen the possibility of subverting the industry with the HTAP model that can do both transactions and analysis. They formally agreed to oceanBase as a separate company to operate independently, which was also an important watershed in OceanBase's development.
On June 1, 2020, Beijing Aoxing Bes Technology Co., Ltd. was officially established. Yang Bing, former technical director of financial technology products of Ant Group, is the CEO, Yang Zhenkun is the chief scientist, and Yang Chuanhui (flower name: Rizhao), who has been in OceanBase since his inception, has been appointed as CTO.
Soon, OceanBase, which had just been "out of the house" for just 12 months, once again chose open source. After months of working documentation and code combing through, in June 2021, OceanBase was once again open sourced to the whole network.
June 1, 2021 OceanBase officially open source
Unlike the veiled "PR open source", OceanBase made a cruel move this time: they disclosed their entire kernel at once. They sink their hearts and just think about how to meet customer needs in an open source way. Following the instructions, a beginner can assemble OceanBase's database in 30 minutes.
What OceanBase does is user value, which is an important concept of their open source.
From the earliest few customers, to today's more than 400 customers in the whole industry; large to Alipay, online business banks, small to 66% of the total proportion of small enterprises, from the ant independent of the Ostar Bess is still helping all walks of life in the whole society.
"Let there be no difficult business in the world" is still flowing in the genes of each of them.
epilogue:
Although he was nearly a year old, Yang Zhenkun still had no intention of stagnating. On the road to technology-driven innovation, OceanBase's goal is to be "infinitely scalable and always-on", intended to create a database system that is not limited by expansion and failure, and this achievement may forever disrupt the database and even the Internet industry.
People who do technology are mostly simple. Yang Zhenkun and OceanBase have come for more than ten years, ups and downs, and finally so far. Sometimes he always thinks of his teacher, Wang Xuan, an academician of the Chinese Academy of Sciences and the Chinese Academy of Sciences, the "Bi Sheng" of contemporary China, who subverted the Chinese printing industry.
33 years ago, Yang Zhenkun became a disciple of Wang Xuan.
That year, Yang Zhenkun was 22 years old, young and freehand; Wang Xuan was 52 years old, which was also the year of Dangdang. Today, Wang Xuan has long passed away, and Yang Zhenkun has also dedicated more than thirty years of his youth to scientific research.
I longed for you back then; today, I still use you as a beacon.
Yang Zhenkun knew that, like his mentor, his life would be dedicated to China's scientific research, but he never had half remorse or half retreat.
In January 2022, after the DC 2021 Distributed Database Developer Conference hosted by CSDN and hosted by OceanBase, Leifeng Interviewed Yang Zhenkun and had a deep conversation with him. Here's a transcript of the interview:
The evolution of database technology
Leifeng Network: We know that there are different stages in the development of technology in history. Databases should also be no exception, and can be divided into several stages according to technological breakthroughs, can you review it for us?
Yang Zhenkun: In the beginning, it was actually related to the account. The earliest database is called a hierarchical database, and the function is relatively simple. And it is also because its function is relatively single, everyone has contacted about a few years later, invented a network of databases. The biggest representative is actually this system of IBM.
By the late '60s or '70s, the relational model was born. Once the two Turing Prize winners were at the ACM (International Computer Society) conference, the two factions of people argued: one faction said that the future must be a relational model, because the relational model has a stronger ability to express itself. Another school of thought says that the relational model is not practical. Structural complexity can lead to poor performance.
Since then, there has been no conclusion on this issue. So, from the publication of the relational model in 1970 to the entire 70s, there was no product.
It wasn't until the late 1970s and early 1980s that relational databases began to be productized. This model has been used ever since, until almost before the Internet. Centralized, essentially a stand-alone system. One of the key reasons is because it's too complex.
Leifeng Network: In order to solve this problem, what efforts have scientists made?
Yang Zhenkun: For databases, it has a special feature: it is difficult for people to operate databases in a natural language. So later, someone developed a database-specific language called SQL (Structured Query Language).
This SQL is not telling the database what to do, this one is too difficult to describe; it tells the database what I want and how you do it is your business.
Oracle is one of the earliest databases, namely Oracle, DB2, SQL Server, and PGSQL and MySQL. For more than 20 years, no database of this size was available. Databases are a very big challenge to this day, and it has led to the pattern of many years being roughly unchanged, and the development of a new database is particularly difficult.
In the Internet era, there are many new and completely different needs. Then we at OceanBase have this opportunity today.
Database sharding tables are not distributed databases
Leifeng Network: OceanBase defines itself as a "native" distributed database, how to understand this "native"?
Yang Zhenkun: I will make a very clear point: database shards are not distributed databases. We think of a native distributed database, which is a single database that looks like a centralized database using a single machine, capable of doing transactions and analysis in a set of databases. If you divide the database and table, it actually adds a middleware to multiple stand-alone databases.
However, this scheme does not actually have any way to ensure consistency between these multiple databases. For example, the first database is me and the second database is you. I transfer money to you, and it is possible that I transferred you out, but you did not receive it. This is possible for database-sharding databases, but not for our native distributed databases. OceanBase guarantees strong consistency and is a single database that can handle both transactions and analytics.
Leifeng Network: OceanBase has gone through many cold and hot summers under the Alipay system, before and after the application of OceanBase, what changes has Alipay produced?
Yang Zhenkun: In fact, since 2014, the company has also seen the value of our business, mainly because Alipay's database volume is too large. Even if your hardware is as stable as possible, it will eventually go wrong. It doesn't matter if you have a few units; you have hundreds, thousands of them, and it's bound to go wrong. At the same time, the cost of centralized database maintenance and repair is very large, and the staff has to go to the earliest gateway to fish out the information and compare it.
After using OceanBase, we deployed three replicas. Even if one machine is broken, the remaining machines are guaranteed to have the correct data and can be automatically recovered. In the past, as long as the main library was broken, they had to do manual reconciliation. No matter how bad the point, the staff must get up within a few minutes, otherwise in the system, the data that the user looks at will be completely wrong. But OceanBase can do the system automatically, the machine is broken, there are other machines can work, our system can continue to run. So this one is of great help to the business.
Such a system has improved the comfort of the staff a lot, and in the past, as long as there was a problem, no matter what time it was, it was necessary to respond to the emergency. Now on Double Eleven, the staff already has time to sit down and drink a cup of tea. That's the biggest change I've seen.
Go out of China and be the world's database
LeiFeng Network: At present, the call for the sinification of databases is getting higher and higher, what kind of position is China in terms of the entire international technical level?
Yang Zhenkun: If we talk about distributed databases, we should be in the leading position in the entire world. This is also related to our own "poor and white". We didn't have much of a centralized database foundation, and instead of competing with others on the original track and having new needs, we might as well go to the new track.
Instead, like the United States now it is more focused on concentration. We also regard the United States as a kind of "curve overtaking", or "cross-generational competition". The main reason is that we often talk about a concept called application-driven technological innovation. With China, there are more people than the size of the market, can americans compare? Right? That's how it goes out.
Leifeng Network: After that, it is certainly possible to export the database OceanBase to foreign countries.
Yang Zhenkun: I think it is necessary, why do we only circle ourselves in this one market? Whether in terms of cost or capability, there are some things that our system can do that the current database system can't do.
However, there is also a process in the development of our own products. There is such a large market in China, and we are still mainly facing the domestic market, eating through the domestic market, and improving OceanBase at the same time.
Although when to go to sea, we do not yet have a specific plan to announce. But I believe that the future is still very bright.
How do you see the metaverse and the future of Web 3.0?
Leifeng Network: What do you think about emerging concepts such as metaversity and WEB3.0?
Yang Zhenkun: Most of my time is still focused on our own products and technologies. To be honest, I really don't understand metaverses and Web 3.0.
But if I had to say it, the database itself is an infrastructure, and you can use it in the metaverse, in real space, in production, in entertainment, in games, wherever you want.
But the megatrends of the metaverse may be a positive for us. Because we are more capable of this, it is like saying that we have come all the way from Singles' Day, right? We should be the first database that can provide such a large amount of processing power to the system.
In fact, whether you do not care about the metaverse or some other trends, data is always going in the direction of "big", and the concurrent is also going in the direction of "big". The metacosm may be a scenario where one of the drivers is slightly larger, and we think there is no doubt about his tendency to go to distribution.
When it comes to Web 3.0, the whole database starts with the Internet, and the database is the foundation of the whole Internet, because you need to have storage and processing functions in the background of anything you do on the Internet. So whether it's web 1.0, 2.0 or 3.0, one of its foundations is actually that the foundation is dataization.
In fact, it confirms a sentence that distributed databases conform to the trend of the times.