Guest | Wu Junlong Wish China QA Director
Edit | Strict
In order to improve business capacity, many Internet companies are pinned on the "996" work system of overtime and hard work. However, such an approach has been proved in heated discussion and practice not only has little effect, but also reduces the enthusiasm of employees, and even plays a reverse effect, so that the entire R & D efficiency does not rise but falls. Therefore, in recent years, many large and medium-sized enterprises have begun to lay out research and development efficiency.
When companies and teams first get into contact with R&D effectiveness, they more or less have these questions: What exactly does R&D effectiveness refer to? What problems can R&D effectiveness practices help companies solve? What about companies that once vigorously promoted R&D effectiveness? Is the research effect problem solved? Will R&D effectiveness fall into the trap of "bringing down the team"?
Based on the above problems, we invited Mr. Wu Junlong, an expert in R&D effectiveness, to share his exploration in the field of R&D effectiveness. At the same time, Mr. Wu Junlong is also a lecturer on the special topic of QCon+ Case Study Club [Guide to Avoiding Pits for R&D Efficiency Improvement], and will bring the sharing of "Best Practices for People-oriented R&D Effectiveness Improvement", hoping to bring inspiration to everyone.
The following is an interview with Teacher Wu Junlong:
InfoQ: What have you been doing lately?
I am currently responsible for the formation, operation and efficiency improvement of the domestic test team at a cross-border e-commerce company. This is a small and medium-sized Internet company, the domestic mainly responsible for cross-border logistics, the business has a certain degree of complexity, but the overall size of the technical team is not large, so efficiency improvement is a more important work. My personal focus is mainly on quality improvement, that is, how to ensure the quality of product delivery quickly and well, which is also a long-term pursuit of my career.
The difficulties of companies of different sizes in promoting quality efficiency are different. For large factories, the difficulty is often how to break down organizational barriers, which needs to balance the interests of all parties; in small and medium-sized teams, the difficulty lies more in the imperfection of infrastructure, as well as the problem of team resources, which needs to balance the relationship between quality and efficiency. For the latter, my team is currently using a data-driven approach to visualize quality data, uncover the root causes of inefficiencies, and drive improvements to the most "painful" problems. Through more than half a year's efforts, the delivery throughput has increased by nearly 2 times, and the online missed test rate has dropped from 10% to 3%, basically meeting expectations.
InfoQ: How did you become interested in R&D effectiveness and embark on this career path?
I came from a test and development background, first in eBay China, when R&D efficiency had not yet risen in China, but it was not a new thing abroad. My team was originally called QE Infrastructure Team, which is mainly an automated tool, and the audience of these tool platforms is basically testers. Later we started to do test execution platform, which can be combined with the continuous integration pipeline to facilitate the rapid quality feedback after submitting the code, and we can also set up various quality access controls in the pipeline to facilitate developers to manage quality, so that it is linked to the developer's work. After that, our tool integrates the publishing platform, which can do artifact delivery and version management, which is linked to the work of the operations staff.
The coverage of these tools is basically the entry point that is now recognized by the industry as an entry point for R&D performance improvement. You can take a look at the following figure, which is a screenshot from Alibaba Cloud. Later, our team was renamed Productivity Team. Similar to Productivity, the words Efficiency are actually what we now mean by efficiency. In this regard, it is true that foreign countries are leading a lot.
Entry point for improving R&D efficiency
With a clear organizational positioning and a sound tool system, we began to force process improvements to balance the relationship between quality and performance, such as: code scanning without Blocker Issue to merge code; Smoke Test to pass to test; Full Regression Test to be released, unless the Leader approved; Canary environment Smoke Test through to cut official traffic, and so on. At this point in time, I, including our entire team, are no longer working in the role of the original test development, but as a performance driver to empower the daily work of each team.
After a few jobs, I stayed at a big factory like Alibaba Local Life (Hungry) and also worked in a startup. In addition to quality infrastructure construction, I am also responsible for capacity assurance and have also led a business test team, but the idea of improving efficiency is my focus that I have always been involved in my daily work. Watching everyone's work efficiency improve is a thing that has both a sense of happiness and achievement.
InfoQ: Have you encountered any unforgettable difficulties in your R&D performance improvement process? What are the summaries and inspirations?
The difficulties are very numerous, and I believe that every practitioner of R&D effectiveness has a bitter stomach. The conflicts of interest mentioned above, the imperfect infrastructure, the lack of team awareness, and even the lack of management awareness are not things that can be solved in a short time and a half.
In my personal experience, the biggest difficulty is not in technology and management, but in how to build an organization's consensus on R&D effectiveness goals. I ask a simple question, what kind of results do you think should be obtained from the improvement of R&D efficiency? Is your job easier, or are you doing more work? For management, is the team not required to work overtime? Or is the product delivery faster? If there are very different answers between different groups of the same company, then it is difficult for people to form a synergy, some people want to relax, some people want to toss, this matter is naturally difficult.
So before you're ready to build tools and optimize processes, think deeply about what our purpose is to drive R&D effectiveness in this company, and maintain communication with your management, reach consensus, and then influence your team and customers, which is the foundation of all R&D efficiency improvement work. Otherwise, you will encounter all kinds of incomprehension and all kinds of grievances in your work.
InfoQ: There are many definitions and statements about R&D effectiveness, from your personal point of view, how do you understand R&D effectiveness? What are its key points?
Indeed, there are many definitions of R&D effectiveness, and some are very academic. For example, "R&D effectiveness refers to the correctness of the use of behavioral purposes and means and the advantages of effects, which are reflected in the software research and development process." Such an obscure definition is estimated that most people cannot understand it.
I would like to share a definition of R&D efficacy that I personally prefer, called "the ability to continuously deliver effective value smoothly and with high quality", just 10 words, but a lot of information. Let me share my understanding:
Smooth: The flow process of expressing value is not hindered, which corresponds to the efficiency of flow, that is, the product cannot be blocked in one link for too long. This is a very important point, blocking often means that there is a bottleneck in one link, and if you do not intervene in time, it will affect other links downstream.
High quality: Improving efficiency is not at the expense of quality, the two are both necessary and wanted.
Continuous delivery: Deliver value at high frequency and in small batches so that all stakeholders in the project can keep abreast of the project's progress, identify risks, and respond to changes. In layman's terms, it is "small steps and fast running".
Effective Value: Focus on products and features that are truly useful to the target user.
I believe you have also seen that these points mentioned above are finally formed into a kind of ability, so the research and development efficiency is more of a long-term capacity building, rather than a short-term result. It's not that I can do well now, but I want to develop sustainably.
InfoQ: What are the points of attention for companies or teams in measuring R&D effectiveness?
Let's first answer a question: can the effectiveness of software development be measured?
（https://martinfowler.com/bliki/CannotMeasureProductivity.html）
I personally think that R&D effectiveness can be measured, but the measurement is relative, that is, we should not focus on the number of measurement results, let alone mechanically set the measurement indicators as KPIs, but should use the measurement results as a phenomenon to give us an entrance to drill down the root cause. This is an economical approach that I particularly recommend, especially when the team has limited resources.
As an example, I use Test Lead Time (from the time the tester receives the test task to the time when the tester actually starts testing) as a measure of flow efficiency in my current team. We expect most testing tasks to be completed within a week, so depending on the length of the test, the roll-out lead time should be less than 2 days. This indicator itself does not explain the problem, its significance is that if the lead time is much more than 2 days, we need to look for the root cause (note that at this time, we can not get the conclusion that the test efficiency is problematic, do not preconceived to give any absolute conclusions).
In the process of finding the root cause, we may also need to set up more indicators to support some of our guesses. For example, guessing that the quality of the R&D personnel is too poor to start the test in time, it is necessary to include the indicator of the return rate of the test work order; the guess tester is not high and cannot support so many test needs, at this time it is necessary to include the test throughput indicator, and so on. It can also be combined with some interviews and cases to further approach the root cause.
So you see, we gradually help us find the root cause of local inefficiency through the drilling down of the metric, so we can take measures to improve, and the results of the improvement will eventually be reflected in the metric. I think it's a more scientific approach to think of metrics as a guide rather than a referee.
Measurements are relative, and there is an important aspect of avoiding horizontal comparisons. This is also a measurement misunderstanding that occurs in many enterprises, like to engage in rankings, to see which department is lagging behind and needs to be improved, which is to absolutize the metrics that should be relative between different teams. Taking the test task lead time mentioned above as an example, in the Chinese team I am responsible for, we need to do 2 days, but in the US team, the demand iteration speed is slower, and 5 days does not affect the delivery of the final project on schedule. If we mechanically use 2 days as a standard to compare the effectiveness of the Chinese and American teams horizontally, it is not appropriate. Choosing metrics that are appropriate for the current situation of the team and focusing on the team's own effectiveness improvement, that is, "comparing yourself with yourself", is a better way to practice.
InfoQ: Many companies or teams see that other people's R&D efficiency improvement scheme works very well, but after applying it to the company, not only does it not improve their R&D efficiency very well, but even reduces the R&D efficiency of the team. What do you suggest?
I think this is precisely the charm of R & D effectiveness, the essence of this problem is that the improvement of R & D efficiency is a global work that requires a high degree of "dealing with people", and people are the biggest variable in R & D work, difficult to standardize, difficult to measure, difficult to control, and naturally it is difficult to have a very specific and universal means of efficiency improvement.
We all know that software development is an intellectual activity of human beings, but this almost indisputable assertion is a common blind spot in the thinking of many people. We always unconsciously think that we can rely on computers to solve all problems. There are many such examples, managers like to sell automated testing, thinking that automated testing can replace the work of testers, and then reduce the work intensity of personnel and improve efficiency. But if we just blindly introduce automated testing technology, without considering the skill level and acceptance of the current team personnel, then the result is likely to be that the labor cost invested in automated testing is higher than manual testing, not only does not improve efficiency, but also reduces efficiency.
I don't deny the value of automated testing, but consider the actual situation of the team when landing. The same is true for R&D performance improvement, many times, we are talking about the process to be standardized, the discussion of research tools to be universal, but these processes and tools are not suitable for our organization, are they user-friendly? There is still too little attention and thinking in this regard.
Since software research and development work is a human activity, then all the performance improvement work on top of this must take into account people's basic feelings and reasonable needs. Therefore, for the construction of R&D efficiency, my suggestion is four words: people-oriented. When we switch to specific work, we can actively learn from the excellent practices of our peers, but at the same time, we should understand the context behind these practices (the size and characteristics of the target team, the technical level and foundation of the target personnel, the company culture and values, the style of the manager, etc.), and then combine the current situation of our own team to analyze whether it is applicable, or what changes to make.
InfoQ: What do you think are the development trends in the industry in the future to further improve the efficiency of R&D?
I think that the development trend of R&D effectiveness mainly has two directions, one is the technical level, as far as possible for the "people" to do more things, so that the "people" do less things, such as low code, AI technology, etc.; the other is the management level, the collaboration between people is continuously optimized, as far as possible no blocking, no duplication of work. You see, no matter how it develops, the essence of people-oriented has not changed.
I am relatively optimistic about the development of the technical level. Today's young people rarely see programming with note punching, which was applied to the first computer ENAC in 1946. Less than 10 years later, the first high-level programming language, Fortran, was born (whether it was the first is controversial or not). Looking at today's programming languages, primary school students can easily get started, which is completely unimaginable in the past.
Back in the field of R&D effectiveness, the idea of low-code or even no-code, a term that has emerged in recent years (although this concept has long existed), now has a mature commercial-grade product. By dragging and dropping, a person who can't write code can also build simple applications. Whether it can be expanded to more complex application construction in the future is a point of view, and we remain concerned.
In terms of management, in the past, we have seen more general agile methods and lean management, etc., and now many enterprises have also begun to try more subdivided practices such as Kanban methods and requirements instantiation, which is a good thing. For the concept of R&D effectiveness, which is in the development stage, the fear is not to toss, but no one toss.
InfoQ: Finally, what to say to the partners who are interested in R&D effectiveness and the managers who want to try to improve R&D effectiveness!
Frankly speaking, in the field of R&D efficiency, domestic enterprises are generally lagging behind. We have experienced a long period of time by burning money, people tactics in exchange for higher market share, so as to achieve the winner-take-all years, research and development efficiency can be filled with people, with money. Now it is found that the policy is different, people are not so good at tricks, and it is more passive to start throttling. My personal feeling is that many companies are still in the stage of being forced to promote R&D efficiency, not really to improve efficiency, but to save money.
Satya Nadella, microsoft's current CEO, said: "There cannot be a more important thing for an engineer, for a product team, than to work on the systems that drive our productivity. So I would, any day of the week, trade off features for our own productivity.” Translated as "There is nothing more important for engineers and product teams than building a system that improves R&D effectiveness." In order to improve the efficiency of R&D, I am willing to abandon the delivery of some functions at any time. ”
The times are different, "big fish eat small fish" has gradually become history, "fast fish eat slow fish" is the main theme of today's era. Our managers should regard the improvement of R&D effectiveness as a strategic goal related to the fate of the company, which is the key to maintaining the competitiveness of the enterprise.
I also give three pieces of advice (keywords) for individuals working on R&D effectiveness: patience, studiousness, and openness. It's not easy to improve R&D performance, so don't try to solve all problems one-size-fits-all. If you meet a company that attaches great importance to R&D efficiency, work steadily for a few more years, and patiently precipitate your own methodology, this is your value, but also the value of the industry. In addition, R&D effectiveness as a hot field, the amount of knowledge is very large (of course, there will be a lot of bias), I hope you can keep learning, especially from your peers, think more. Finally, I would like to quote einstein's famous quote: "To solve the important problems we face, we must not stop at the level of thinking that made them in the first place." "We should have an open mind, try to break through ourselves, and use innovative thinking to promote the improvement of R&D efficiency."
The crowd gathered firewood and hoped that we could work together to contribute to the development and innovation of R&D efficiency.
Guest Profiles
Wu Junlong
Wish China QA Director
Former Senior Test Manager of Alibaba Local Life, graduated from the University of Science and Technology of China with a master's degree. In the software quality system, service capacity guarantee, service stability construction, software research and development efficiency and other fields of deep ploughing for many years, good at solving quality and efficiency problems through innovative means, has a number of domestic and foreign patents. He has been invited to give speeches at various technical conferences in the industry and disseminate advanced concepts and methodologies. Columnist of Geek Time's "Capacity Assurance Core Technology and Practice", author of the best-selling book "The Beauty of Software R&D Efficiency Improvement".
Event recommendations
How to upgrade the big data system from top to bottom for new workloads? On May 12-14, the QCon Global Software Development Conference will land at the Beijing International Convention Center, focusing on next-generation big data system architecture, distributed databases, and cloud-native data lakes. In addition, there are many hot topics such as business architecture, Rust, WASM, observable, business security and compliance, product design, new infrastructure of the front end, and organizational management, so as to discuss new technology trends with you. Buy tickets now to enjoy 20% discount, buy tickets immediately off 1760 yuan, scan the code to buy tickets immediately.