After fixing 13 million lines of code, I saved Apple $200 million, but I didn't get the promised tens of millions of shares

Author | Terry Lambert

Translated by | Sambodhi, Yan Shan

Planning | Tina

"The Mac OS X kernel has 13 million lines of code, and SVN shows that I wrote 7 percent of it."

Recently, on the Quora (similar to Zhihu) website, a Po owner asked a question: "What do you need to do to make the operating system UNIX certified?" Terry Lambert gave a brilliant answer, and given his identity as a former Apple engineer who worked on much of the Mac OS X kernel, the answer quickly gained widespread attention and tens of thousands of likes.

It's a fascinating story.

In the early years, when Apple introduced the Mac OS X operating system, it often used the "UNIX kernel compatible with the operating system" as a means of publicity. On its website, for example, there is a dedicated page of UNIX articles about Mac OS X.

The Open Group filed a lawsuit against it, claiming that it used the Unix name without authorization. Once Apple loses, it needs to pay $200 million in damages.

However, Apple insisted that it was free to use the Unix name, believing that it was not making false claims. But in order to cope with the lawsuit, Lambert recalls, Apple had two main solutions to the matter, either to buy the International Open Standards Organization for about $1 billion, or to quickly obtain Unix compatibility certification to resolve the lawsuit.

Jobs Steve entrusted this "secret" task of certification to former Apple technical chief Lambert, who needed to run compliance tests against the existing Mac OS source code, modifying it as it was tested, and the completion time was only given one year. This also meant that Lambert needed to know the 13 million lines of code in the Mac OS X kernel in a short period of time, which was very technically difficult.

Lambert was promised $20 million in shares and $10 million for individual Lambert after the mission was completed. In fact, this kind of task would require a team of twenty or thirty people to work for five years, but Lambert completed it in a year with a team of five, during which time he contributed about 2 million lines of code to hundreds of open source projects. However, Lambert did not receive the $10 million in compensation, and his stock was swallowed up by his leaders.

Some people say that there are no 10x programmers, but I don't think they have met a 100x programmer like you.

While we all know that Mac OS X eventually passed UNIX certification, that doesn't stop us from following Lambert's memories to learn the details of this interesting past.

Make Mac OS X a true UNIX

According to Lambert, there's a lot of work to be done to get Mac OS X unIX certified.

Resolving the lawsuit by making Mac OS X a true UNIX was actually good for the international open standards organization, which was losing the market as Linux became more popular.

The acquisition of the International Open Standards Organization for about $1 billion would give Apple the freedom to use the trademark; however, it would not exempt them from existing contractual obligations with Sun Microsystems, IBM, and others, who had already licensed the UNIX trademark.

Asked if Lambert could lead a team to work on the first option, he said he agreed on the premise that the project could be based on the project, instructing the rest of the organization to make the appropriate changes on their own code base, and that the commit rules could be handled fairly leniently.

Subsequently, Lambert was approved and certified. They first run the compliance test suite against the existing Mac OS source code, and because of the header file, the test suite immediately shows an error.

He and his colleague Ed Moy made two lines of changes, moving the type definition from where it should be. There is a change in one line, and the change in the other line is in the file where the type should actually exist.

Then run the test again and find that one of the header file errors in the first test is gone. So Lambert and others did a "world build" that rebuilt everything in Mac OS X, including iTunes. As a result of this change, nearly 150 items failed to rebuild, and iTunes was among them.

So Lambert and Ed took a closer look and fixed each of these projects so they could all build. They then went on to do another "world building" and everything was built.

It was also at that time that Lambert had access to all of Apple's source code. They then submitted high-priority bug fixes to the project, but some of them were immediately demoted in priority, while others only needed to do simple fixes because they helped provide the patches. Then Bertrand Serlet, vice president of engineering, re-prioritized the projects that had been demoted. Then, Lambert they commit the header file changes.

The project even "alarmed" Jobs

At this point, Lambert and they had to go back and conduct a feasibility assessment of the entire project. Given the prerequisites he set for the project, both he and Ed felt that this could continue within the time frame.

The project was also upgraded to Jobs Steve. Lambert continues to get executive approvals, after all, saving Apple a lot of money and improving marketing collateral for all Mac OS X servers.

They were promised $20 million in shares when they were completed. Lambert will get $10 million, and Ed and Karen Crippes will each get $5 million.

Lambert said he wore a lot of "hats at the time, not just a technical lead, but a de facto project manager."

This process is also visible to the naked eye and is long. Lambert estimated that it would take about a year, and it was a project team of 5 people: three mousekateers (not spelling mistakes), two contractors— one Len Lattanzi, in charge of user space code; and the other Jaime Delgadillo, who was responsible for full-time test automation and error archiving, and also provided as many patches as possible.

The project team also had two temporary contractors, one responsible for tool compliance and the other responsible for the manual pages. In addition, they can also bring in some people from other departments of Apple to do short-term work according to the situation.

A long year

The first big deal is that all the header files pass the tests so that the other tests in the test suite can start running. By that time, Lambert had actually committed all the changes to the header file to the rest of Mac OS X. When the Tiger (mac OS X v10.4) version was released, the header file was already standard.d.. However, this undermines the proper functioning of CodeWarrior, the Mac's IDE. Lambert says he's always wanted to solve the problem, but he never found an opportunity, and CodeWarrior is more or less damaged.

For the rest of Apple, Lambert just solved the "fix header" bug, which included many other bugs for individual header files, which took about three months.

Lambert has promised a year, so how can he fulfill his year's expectations?

Lambert recalls, "I knew that making mandatory changes to the header file, and the project changes associated with it, would be the largest single part of the project. Once we are able to do other tests, there will be a lot of 'easily achievable goals' to address in other ways. The process took more than two months, and we were always hesitant to submit the rules, but it was done quickly. Ed did most of the work of libSystem (libc + other system libraries) with my assistance and merged them together to remove some content from the namespace; that's why in /usr/include/sys, header files start with "_". ”

"While waiting for a commit, you can do other work in parallel, and that's what we do. There is still a lot of work to be done after achieving these easily achievable goals, such as rewriting the signal system in the core, but it is not so easy to achieve. ”

But Lambert and others have convinced Umesh (I won't tell you his last name), "because he doesn't want us to touch his pthreads code, and he wants to make some changes there anyway, and he's very happy to have this project as a means of going through those changes over and over again." Later, they also got "grudging" support from Mike Smith (Michael Smith, a senior Apple engineer) to rewrite the file-locking code.

The project team eventually "bought" Joe Sokol by asking questions about trap paths and around the signaling system stack frame saving. But Lambert stresses that among these people, Umesh was the most helpful for their deadlines.

Pull the trigger

Everything is in place, Lambert is ready to "pull the trigger", and the accident follows.

"They introduced Intel's code changes and let's wait another two weeks, but everything is messed up. So I spent three days reintegrate all the patches on the consistency branch into Intel's kernel code. By then, I knew all about the 13 million lines of code in the Mac OS X kernel. ”

Then everyone went back to the state of passing the test. Lambert was later told that he couldn't do integration for Tiger. But this will miss the deadline that has been set. Tiger delayed another six months before its release, again and again. This is a problem with Intel, not a kernel problem.

Lambert says, "We could have easily posted on Tiger, in line with the deadline we set ourselves. "But if he were to do the same job for Linux, it would probably take five years, and there would have to be twenty or thirty people." After all, there are too many Linux distributions, resulting in a large number of camps.

Lambert concluded by mentioning that the team has made a lot of contributions to the open source community, "We have received a lot of thanks from the open source community, especially the fixes that allowed bash to pass the test. You would never know how much Apple has contributed to the open source community as part of this project, at least for non-Apple people, and it's a secret project, so we're not promoting that fact. But I estimate that we contributed about 2 million lines of code to hundreds of open source projects that year. While there is a lot of gratitude, it is not attributable to the collective, so Apple has been accused of "using open source code but never giving back.". However, we fixed bugs for at least 15 GCC. And you don't know it at all. ”

So overall, it's a big project, especially for compliance. "Before everything Karen did, there were existing exceptions about self-certification, contracts, osf/1 Mach-based exceptions to get test exceptions, and so on." Lambert laments that this is indeed a long process.

Still, Lambert revealed in the comments section that he didn't end up getting the expected $10 million stake for some reasons.

Reference Links:

https://www.quora.com/What-goes-into-making-an-OS-to-be-Unix-compliant-certified