laitimes

"Unintentional Algorithm Cruelty": How Delivery Riders Trapped in the System "Maintain the System"

"Artificial retardation"

Riders' perception of the system mostly comes from the use of the app and their own delivery work. In their daily work, riders refer to the background system in a variety of ways, including "delivery app", "technology", "background" and "mobile phone", and some riders directly use "platform" to call the dispatch system. For most riders, the technical systems in the background are "very complex" and "not very clear". In daily communication, the rider's perception of the system is usually very specific and full of situationality. They used their labor to understand and figure out algorithms, and to explain their understanding of technical systems. During the interview, the rest of the research team and I listened to many examples of the system, both large and small. Although these examples do not allow us to see the whole picture of the interaction between the whole system and labor, they are like a single piece of the puzzle, allowing us to "see the big from the small" through individual cases, and get a glimpse of the dynamic interaction between the system and labor.

In the actual delivery process, riders' complaints about dispatch routes and map guidance are so common that almost everyone has to say two words. Among them, the intelligent guidance of the map has the most complaints. In my fieldwork in 2017 and 2018, I found that at that time, many food delivery platforms were setting up the order delivery distance to calculate the straight-line distance instead of the actual distance distance. Since the algorithm needs to estimate the delivery time based on the delivery distance of the order, the difference between the actual distance and the straight-line distance increases the delivery pressure of the delivery staff. In this regard, Master Gao Zhe complained to me many times:

The straight-line distance is short, and the actual distance is much longer than that! Once, a list to Haojing Oriental showed that it was 3 kilometers, and guess how much I ran? 5.2 km! …… The system is not right on this map. What does it mean to show straight-line distance? We're not a helicopter, we just take off and land.

For calculating distances, the platform's map system was later adjusted to calculate time with actual distances instead of straight-line distances. However, some riders are still dissatisfied, saying that the distance displayed by the system has been processed and shortened, intentionally or unintentionally, and the actual delivery distance is often more than the distance displayed by the system. Most of the riders are not computational and technological, but they see the problems with these high-tech, complex algorithmic systems in the background as sometimes "naïve" and ridiculous. When they were chatting, they jokingly referred to the back-end system as "artificial retardation".

Yan Huitao (referred to as "Yan" in the following dialogue) is an "Eleme" rider near Panjiayuan, he is not tall, a little fat, and smiles when he speaks. Since 2017, he has been stationed at a restaurant chain, specializing in order delivery for the restaurant. After the interview, we added WeChat, sometimes through WeChat. One day, he approached me and asked me if I was still doing research on delivery riders, and said he was willing to provide me with some "funny things" about the system. In order to give me a better understanding, he first sent me a lot of screenshots of his phone (see Figure 10), and then began to talk about how the system was "vexatious", so that he was helpless.

Here's our WeChat conversation:

Iwa: Today's single-handed distribution is too unreasonable.

Me: Where is it unreasonable?

Iw: Is it reasonable to reverse the order? Time is running out right now.

Me: Is it retrograde?

Iw: Well, the route of the system is the retrograde route. You see, from 3 to 4, I had to cross a river and a high speed. Following the map, I had to make a big circle. It stands to reason that with the high speed as the boundary, the orders on both sides are almost impossible to send. I've sent it several times today!

Me: How did you get there?

Iwa: Look for the overpass and go backwards.

"Unintentional Algorithm Cruelty": How Delivery Riders Trapped in the System "Maintain the System"

Since the back-end system constantly adjusts the delivery route based on the rider's flow data, in many cases, the data with a lot of traffic has legitimacy. As a result, in many cases, the retrograde route that riders have to choose in order to save distance and shorten the delivery time will also be gradually adopted by the background map and developed into a recommended route. Every time he talked about this, Yan Huitao would laugh, and then threw me a sentence: "Lu Xun said it well, there is no road in this world, and when there are more people, it becomes a road!" ”

There are also riders who complain that the backstage system is "not good" and "not intelligent". A colleague in the Lao Gao site once complained that the app's Bluetooth recognition was inaccurate:

If you don't order to the store, you can't eat. (There is a Bluetooth in the background of the App) Bluetooth recognition to the store (function), some (time, the background is not accurate) the positioning of the merchant, you are obviously there, but you can't click to the store. You can't get to the store, technology has the final say. But at this time, if the customer cancels, it is (our) logistics responsibility, and the rider will be fined 500, and the day will be in vain.

Problems such as inaccurate positioning, inability to click on the next step, and confusion in dispatching are often mentioned by riders. Behind every technical problem, it is closely related to the rider's delivery "performance" and labor income. In order to ensure a smooth delivery process, riders have developed a lot of agency and flexibility, and they will keep track of the remaining time of the order, the time it takes to pick up the food at the store, the distance traveled, as well as the floor, whether there is an elevator, whether it needs to be accessed, etc. In the face of "artificial retardation", the rider will open up a specific set of logic that talks to the system in the process, and on this basis, realize the co-evolution of man and machine. Through the labor of delivering orders, the rider is constantly "trial and error" for the logic designed by the platform's algorithm. The labor undertaken by the rider is more like that of the "test end" engineer, who delivers orders again and again, verifying the accuracy of the system again and again. The only difference is that the test engineer is confronted with a computer screen, while the rider is tested on a three-dimensional, complex street. The process of "rider complaining – platform improvement" vividly illustrates the growth of intelligent algorithms.

"Nourishment System"

On an afternoon in the summer of 2021, we chatted with Da Qiang, Wang Xiaoli, Li Feilong, Donkey and other riders at a restaurant in Fangshan Food Street. During the small talk, I mentioned the delivery system, which opened up the conversation, and the riders expressed their opinions. In the heated discussions between you and me, I found that although everyone is full of various dissatisfaction with the platform's dispatch system, they have reached a consensus on one point, that is, as a rider, you need to maintain your own dispatch system. Brother Daqiang, who was sitting next to me, took a bite of rice and said angrily:

The system is grandpa, and it needs to be kept all the time! If you are good to it, it will be good to you!

Brother Da Qiang is a crowdsourcing rider on Catalpa Street in Fangshan Liangxiang. He was about thirty years old, burly, with a beer belly, riding a big motorcycle, straightforward, and loud talking. According to my understanding, he can be regarded as an "opinion leader" in Fangshan Food Street. He has been working for a long time, has a lot of experience, knows many rider brothers, and likes to communicate with everyone. When there is something that is not sure, the riders will consult with him. In the summer of 2021, my classmates and I went to Catalpa Street many times, and Brother Da Qiang enthusiastically introduced us to other riders, and also designated a rider for each of us to follow to run the single. I once ran with Brother Daqiang for an afternoon, and he rode fast and walked so fast that I had to trot all the way behind.

For me, "feeding systems" is a very interesting field topic. According to Brother Daqiang's feedback, the system is hierarchical, and only when the data is done well, the system will send itself a better list. As for how to "get your own data", Brother Da Qiang's answer is: "Run orders well." Here's how he explained:

First of all, you can't keep refusing orders. Reject once or twice, many times a day, and the backend will know - it has its own settings. When you reach a certain number, not only will you not receive orders, but you may also be banned. That's right, because you keep refusing orders. …… Also, keep running. Do you see those who are ranked high and have good performance, which one is not on duty all day? [They] rarely take time off. It is not possible to fish for three days and dry nets for two days. Data is all accumulated, not often run, no good data.

In the process of running orders, the riders keenly discovered that the back-end system relies on data accumulation to rate and dispatch delivery riders. This is especially true for crowdsourced riders who are grabbing orders. During the afternoon and evening rush hours, crowdsourcing riders appear on the streets, sitting in groups of three or five, or sitting in front of the snack bar, or leaning next to the electric car, and begin to bury their heads in "grabbing orders". I've heard riders complain more than once that an order comes out and everyone grabs it together, but in the end, it always ends up in the hands of a few people they know. Some people say it's because of the different speeds of different brands of mobile phones, some people say it's a matter of distance, and some people say it's because the person who grabbed it used cheating software.

But Brother Da Qiang doesn't think so. According to him, those fixed people can always grab orders because their "data is good, and they never pick orders", so the system tends to send them good orders. Brother Da Qiang has always insisted that "good data is earned through hard work". He is not optimistic about the riders who squat on the street all day long, "picking orders" and "rejecting orders". On the contrary, if you want to have good data, you need to put your head down.

"Unintentional Algorithm Cruelty": How Delivery Riders Trapped in the System "Maintain the System"

Stills from "Another Day of Hope".

Here, the hard work of the "maintenance system" coincides with the game mentioned above. Under the logic of continuous upgrading and evolution, the system needs riders to produce a steady stream of data to "feed" themselves. Under the logic of gamification, rider labor and the back-end system form a subtle interactive loop: riders with more orders and faster delivery time will get more orders, while riders with small orders and "picky orders" will not get positive feedback from the system and have a tendency to be marginalized. The algorithm system is gradually changing to an organized and hierarchical structure, in which continuity and compliance have become the main logic of the system's dispatch.

The classification of "dispatch" and "order grabbing" of the "flash delivery" platform is a vivid embodiment of the "maintenance system". The platform will divide the flash delivery staff into two categories: "dispatch" and "grab orders" according to the number of orders delivered by the riders and the working hours. The former refers to the platform taking the initiative to send orders to riders, and the level of "dispatching" is high, the order volume is large, and the order price is high; The latter requires riders to independently grab orders in the orders distributed by the platform, with a low level of "order grabbing", a small order volume, and a low order price. In order for a rider to maintain the level of "dispatch", he needs to work continuously and uninterruptedly. According to the regulations, a rider needs to complete a certain amount of order delivery every week and every day, and if the specified order volume is not completed for some reason, it will be downgraded to "grab orders".

"Algorithmic imagination" has always been an important field in algorithm research, which aims to explore how people know and understand the algorithm system itself. People with professional skills may extend their imagination of the application of algorithms through the internal mechanism of algorithms, while more people who have no way of knowing the "black box" of algorithms pay attention to the system settings of algorithms through the external performance of the algorithm system. In this theoretical exploration, "algorithmic imagination" is not just a kind of "imagination". Rather, it is a practical process, a generative process. An individual's imagination of algorithms has a direct impact on their personal lives and their interactions with algorithms. Long-term practical experience has developed a unique imagination of delivery riders for background algorithms. When Brother Da Qiang, Wang Xiaoli, Li Feilong, and Brother Donkey are together, they often discuss the algorithm "black box" in the background. Although they can't understand and explain why algorithms are like this in some situations and different in others, their speculations, questions, and ideas undoubtedly form "algorithmic gossip", and the expression of these words constitutes the imagination of delivery riders about algorithms, and also profoundly affects their order running work.

For example, the continuity required to "feed the system" can sometimes be difficult for riders when it comes to knowing that the system needs to be constantly "fed" through individual labor. Although the platform recruits workers with flexible and autonomous job positioning, there are continuous working hours requirements for riders in the actual labor process. It is not uncommon for riders to return to their hometowns for medical visits, autumn harvests, or other emergencies, and once these accidents interrupt the continuity of their work, their background data will become "unsightly" and the rider level will be downgraded. A rider on the "Flash" platform once described his situation sullenly on "Douyin":

You have to run every day. You see, I've been back to my hometown for a few days, and there is something going on, and this performance has come down. I can't get a big order, so I can only send some small orders around.

In 2014, web design consultant and author · Meyer coined the term "unintentional algorithmic cruelty" to describe a flaw in computer design — a lack of empathy. Indeed, in the algorithm design of food delivery platforms, the practice of "maintaining the system" through continuous labor is also full of controversy. As a mobile group in the social network, riders face many uncertainties in life and work. To "price" a rider's labor through a single algorithmic monitoring without allowing them to "get into the wrong" is to treat the rider as a machine that can consistently produce value, rather than as a person with an individualized life situation.

What's even more amazing is that in the accumulation of labor, the riders become emotionally dependent on the algorithmic system. They personify the app and make an effort to have a conversation with it. In this process, the "maintenance system" becomes a very interesting human-computer interaction practice. Words used to describe people, such as "very smart", "chicken thief", and "very calculating", are widely used to describe the dispatch system in the background of the platform. In addition to waiting for orders, riders will also "delve into the system". They derive their own understanding from the many rules and regulations on the app's website, share and discuss them with other riders, and even practice and verify them in the process of delivering orders. The labor practice of the rider's "inverse algorithm" will be expanded in detail in Chapter 6. What I want to emphasize here is the interactive process between the rider and the system that is both dependent and confrontational, cooperative and negotiating. In the process, there is even an interesting emotional dependence. In the discussion of the sociology of emotions, scholars believe that the arousal and connection of emotions are important characteristics of human beings. At the same time, emotions are influenced by many structural factors of society and culture. The interaction between the delivery rider and the technological system is an important factor in the emotional arousal of the delivery rider, and it is also very likely that we humans will face the future scenario of human-machine dialogue.

Carolyn · Marvin explains her understanding of technological evolution in her book When Old Technology Comes New. She believes that the evolution of technology is not only functional and instrumental, but also humanistic and social. This view is still applicable today in the evolution of algorithms and artificial intelligence technology. In today's profound transformation of the form of technology, the evolution of algorithms is definitely not a single logic told by simple coding, programs, and software. On the contrary, its evolution is deeply embedded in the process of platform labor. The evolution of algorithms needs to be based on the "behavioral data" of digital workers, and on top of this, the increasingly refined and rigorous management of workers will be shaped. This is what I said at the beginning of this chapter about "generative management."

The consequences of this kind of "backlash" deserve our deep consideration. As explained in "Delivery Riders, Trapped in the System", when the algorithm system becomes more and more stringent and the speed of riders becomes faster and faster, the entire food delivery industry falls into a "negative cycle" that is difficult to escape. It is precisely for this reason that the food delivery industry will be crowned with the meaning of "transitional labor", and a steady stream of people will join and a steady stream of people will leave. Today's food delivery system is more like a "technological singularity" that takes everyone involved into an unknown future.

Algorithmic ethics on the agenda

"Unintentional Algorithm Cruelty": How Delivery Riders Trapped in the System "Maintain the System"

Stills from "Another Day of Hope".

In today's world where smart technology is so "unleashed", it is indeed necessary for us to call for control and reflection on it. Especially when technology and capital are further bound, its powerful kinetic energy and the accompanying powerful destructive power are becoming increasingly apparent, and the Leviathan of technology seems to be becoming a reality. With the increasing popularity of the platform economy, there is an obvious conflict between the capital and public attributes of technology. The rider who is "trapped in the system" shows this contradiction. Riders participate in the production of algorithms, but cannot participate in the development of algorithm rules. The discussion of the ethics of algorithms shows the contradiction between the public logic and the private logic of technology.

Let's go back to the initial expectations set by technology. What is the original intention and essence of technology? Since the concept of technological development, human history has set the direction of technological development to make society and life better. This "beauty" is reflected in the development of technology, which allows more and more people to enjoy a convenient or abundant life. Technology may be abused, but the sharing and public welfare of technology are always its most important social attributes. As some scholars have pointed out, "public and public welfare, as opposed to narrow private and collective, are the most important value cores and evaluation criteria for the practice of technological socialization".

Taking a step back, whether from the perspective of theory or practice, the technical products produced by human intellectual labor must have an innate public character. This public characteristic is manifested in the fact that the emergence of technology will benefit the survival, life and development of human beings, help human beings further solve the overall survival dilemma, and continuously increase the public value of human life. For example, the initial emergence of Internet technology was based on the logic of interconnection of all mankind and free flow of information. Both the history of technology and the history of society show that the generation of a technology is definitely not the product of individualization and singularity, on the contrary, it is collective, interactive, and a product that combines multiple actors and has multiple characteristics. Coming back to algorithms, their production and generation also follow the logic of collective generation. Therefore, when such a collective technology evolves into a single decision-making and arbitrariness due to the intervention of many organized, structured and bureaucratic systems, the problem of technology is contrary to publicity.

The discussion of "delivery riders, trapped in the system" quickly aroused the attention of the whole society about the ethics of algorithm technology. This has attracted the attention of the government, and has also brought greater public opinion pressure to platform companies. After that, all parties finally began to actively reflect on how algorithms should be constructed and managed as a technological system with social power relations, and began to take action.

In September 2021, Meituan disclosed the rules for calculating riders' delivery time. For the estimation of the rider's delivery time, the back-end system has set up four algorithms: "historical data model estimation time", "estimation time under the characteristics of urban traffic status", "cumulative estimation time for each delivery scenario such as meal delivery to store pick-up" and "delivery distance estimation time", and promises to use the longest time among the four algorithms as the rider's "estimated delivery time". In addition, in order to reduce the situation of "people waiting for meals" in the meal pick-up process, riders can report through the App to get a certain amount of time to replenish. In response to the question about the goal of algorithm optimization, the platform has established three principles: first, adhere to fairness, reasonableness and people-oriented; The second is to adhere to fairness and coordination, and fully consider the interests of consumers, merchants, riders and other related parties on the platform; The third is to adhere to openness and transparency, and promptly disclose to the public the algorithm rules that affect the rights and interests of all parties. In November 2021, Meituan announced the "order delivery" algorithm, and through the information it provided, Meituan pointed out that the logic of order allocation integrates the three ends of riders, merchants and consumers, and adjusts the rider's time margin and the degree of routing to allocate orders.

Government policy is not far behind. On September 17, 2021, nine ministries and commissions, including the Cyberspace Administration of China, the Publicity Department of the CPC Central Committee, the Ministry of Education, the Ministry of Science and Technology, the Ministry of Industry and Information Technology, the Ministry of Public Security, the Ministry of Culture and Tourism, the State Administration for Market Regulation, and the State Administration of Radio and Television, issued the "Guiding Opinions on Strengthening the Comprehensive Governance of Internet Information Service Algorithms", clarifying the principles of algorithm ecological regulation: "The application of algorithms is fair and just, open and transparent, the development of algorithms is safe and controllable, and independent innovation, effectively preventing risks and hidden dangers caused by the abuse of algorithms." On January 4, 2022, the Cyberspace Administration of China, the Ministry of Industry and Information Technology, the Ministry of Public Security, and the State Administration for Market Regulation jointly issued the "Provisions on the Administration of Algorithmic Recommendations for Internet Information Services", pointing out that when providing algorithm-related services, "laws and regulations shall be observed, social morality and ethics shall be respected, business ethics and professional ethics shall be observed, and the principles of fairness and justice, openness and transparency, scientific reasonableness, and honesty and credibility shall be followed."

This is a positive feedback. The ethical issue of algorithms is on the agenda. Through the article "Food Delivery Riders, Trapped in the System", all parties in society have begun to pay attention to the production of data, the formulation of algorithmic rules, and the arising rights and interests of platform workers. In the platform economy under the logic of "generative management", the core issues of algorithm ethics reflection should be the subordination of data and the public participation of algorithm rules. Today, when intelligence is so closely linked to individual life and labor, the problem of data and algorithms "has expanded from the single dimension of simple 'individual information' and privacy protection to the three dimensions of 'individual rights and interests, enterprise competition and production relations'". Therefore, it is more necessary to return to the perspective of political economy to reflect on who produces the technological system, who should own it, and what kind of public value it can produce.

Technophilist Andrew · Feinberg put it this way in Alternative Modernity: "First, we are entering a new era characterized by generalized technologies that affect us in very unexpected ways; Second, it's about how we treat technology, because, perhaps for the first time in history, public participation is beginning to have a significant impact on the form of technological change. "In the post-epidemic era, the conflict of interests of nation-states and market-oriented consumption based on global industry and information technology have strengthened the aura, advantages and dominant psychology of the development of intelligent technology. Whoever has more technology and more data will have more initiative to determine the direction of future digitalization. For this reason, the production of digital technology, the determination of rules, and the confirmation of data rights have always existed. In particular, we need to be wary of the tendency of capital-based digital ideologies to erode the public nature of technology, which is not the future we want to do.

"Unintentional Algorithm Cruelty": How Delivery Riders Trapped in the System "Maintain the System"

(This article is excerpted from the book "Transitional Labor: Food Delivery Riders in the Platform Economy", published by The Paper with the permission of the publishing house)

Read on