Reports from the Heart of the Machine
Edit: Egg sauce
At the 2021 summits, the number of papers using PyTorch is already at least 3 times that of TensorFlow, and this gap continues to widen.
From the early academic frameworks Caffe and Theano, to the later PyTorch and TensorFlow, since deep learning became the focus again in 2012, many machine learning frameworks have become the new favorites of researchers and industry workers.
At the end of 2018, Google launched a new JAX framework, and its popularity has been steadily increasing. Many researchers have high hopes that it will replace many deep learning frameworks such as TensorFlow.
However, PyTorch and TensorFlow are still the two major players in the ML framework field, and the power of other nascent frameworks is not yet comparable. The relationship between PyTorch and TensorFlow is one or the other, and the balance of power is quietly changing.
In October 2019, Horace He, an undergraduate at Cornell University and an intern with the PyTorch team, conducted statistics on the use of PyTorch and TensorFlow in academia. The results showed that researchers had flocked to PyTorch, but at the time, it seemed that the industry's first choice was still TensorFlow.
As shown in the chart below, since the middle of 2019, in the major summits of statistics, PyTorch has completed the anti-overtake of TensorFlow from the usage indicator.
Data collection time: October 2019.
At that time, the developer community was hotly debated: in the future, who can usher in the "highlight moment" in the ML framework dispute? Two years later, Horace He again gave the updated statistics.
Up to now, the proportion of PyTorch in the three top meetings of EMNLP, ACL and ICLR has exceeded 80%, and this proportion has remained above 70% in other conferences. In just two years, TensorFlow's living space has shrunk dramatically.
PyTorch's "overtaking" in academia
Specific to each summit, the author also shows detailed data in the chart:
Taking CVPR as an example, before CVPR 2018, TensorFlow was still more used than PyTorch, and in the following year, the situation immediately reversed.
The PyTorch usage rate in CVPR 2019 was 22.72% (294 articles), tensorFlow usage became 11.44% (148 articles); by CVPR 2020, the two figures became 28.49% (418 articles) and 7.7% (113 articles), respectively.
In ICML, ICLR, NeurIPS, these conferences, the same competitive situation is still the same:
PyTorch rode out, and TensorFlow continued to fall. In ICLR 2022, PyTorch usage was 32.20% (1091 articles), and TensorFlow fell to 6.14% (208 articles), opening up the gap fivefold.
Does TensorFlow have a future in academia?
So, how did TensorFlow, which was on the retreating side, get to where it is today?
In the Hackrnews community, this topic has sparked a hot discussion among developers:
"In academic publishing, it's critical to be able to compare your work to SOTA. If everyone else in your area uses a framework, then you should do the same. Over the past few years, Pytorch has been the framework I've followed the most."
"But one of the bright spots of Tensorflow is static charts. As the model becomes more dense and requires different parts to execute in parallel, we see some challenges in PyTorch's running model."
In the developer's opinion, if you want to do a lot of things in parallel, Tensorflow still has some features that other products can't match. It all depends on what you're doing.
Others say Tensorflow's decline is due to strategic lapses.
"I think Tensorflow made a bad move in academia because it was very difficult to use in earlier versions. Sure, it always performs better than PyTorch, but when you're a heavy PhD student, you care less about whether your code is efficient and more about whether your code works. Some people say that PyTorch debugging is relatively easy, so those early models were published in PyTorch, and many people came to PyTorch later."
What do you think?