Reports from the Heart of the Machine
Machine Heart Editorial Department
By looking at your financial resources on your mobile phone, the accuracy of poverty alleviation can be increased by up to 21%.
The COVID-19 pandemic has devastated many low- and middle-income countries, leading to widespread food insecurity and a sharp decline in living standards. In response to the crisis, governments and humanitarian organizations around the world have distributed social assistance to more than 1.5 billion people. However, they are facing a key challenge: with the available data, it remains a daunting task to quickly identify the target populations most in need of assistance.
In a paper recently published in Nature, "Machine Learning and Phone Data can Improve Targeting of Humanitarian Aid," researchers from the University of California, Berkeley, the University of Mannheim in Germany, and Northwestern University in the United States showed that the use of mobile phone network data can improve the relevance of humanitarian relief.
They used traditional survey data to train machine learning algorithms to identify poverty in users' mobile phone data. Trained algorithms can then prioritize assistance to the poorest mobile phone users.
The researchers evaluated the methodology by studying a flagship emergency cash transfer project (Novissi) in the West African country of Togo, which uses algorithms to allocate millions of dollars worth of COVID-19 relief assistance. In their analysis, they compared results under different goal-setting mechanisms, including exclusion errors (where the truly poor are wrongly considered unqualified), overall social welfare, and equity measures.
Compared to the Togolese government's geolocation targeting approach, the researchers used machine learning methods to reduce the exclusion error by 4–21%. Compared to methods that require comprehensive social registration (a hypothesis that such registration does not exist in Togo), the machine learning approach increased the exclusion error by 9–35%. These results underscore the ability of new data sources to complement traditional approaches in determining humanitarian assistance, especially in crisis environments where traditional data are missing or outdated.
Research background
Let's start with Togo's flagship emergency cash transfer project, Novissi. In April 2020, shortly after the first covid-19 cases appeared, the Togolese government launched the project. The economic restriction order led to the shutdown of many Togolenians and raised widespread food security concerns. The Novissi project aims to provide survival cash assistance to those most affected.
Project Address: https://publicadministration.un.org/zh/Themes/Digital-Government/Good-Practices-for-Digital-Government/Compendium/CompendiumID/472
However, when the Government of Togo first launched the Novissi project, there was no traditional social registration system available to assess eligibility for assistance, nor was it possible to allocate time or resources to build such a registration system during the COVID-19 pandemic. The most recent census, completed in 2011, did not contain information on the wealth or poverty of households. The recent National Living Standards Survey covered only a subset of households.
In this case, eligibility for assistance for the Novissi project is based on data contained in the national voter registration system updated in late 2019. Unfortunately, this approach does not include the poorest families in Togo in the novissi project.
The study aims to help the Togolese government expand the scope of the Novissi project's assistance from informal workers in the capital Lomé to the poorer people in rural areas, and in the process meet two of the Togolese government's stated policy objectives: to direct aid to the poorest geographical areas of the country; and to prioritize the distribution of aid to the poorest mobile phone users in those geographical areas.
Based on this, the researchers used machine learning algorithms to analyze non-traditional data from satellites to mobile phone networks, and ultimately improved the targeting of the poorest mobile phone users.
Conduct surveys of mobile phone users to determine the level of user wealth and consumption
The first step is to use machine learning algorithms for high-resolution satellite imagery to obtain a micro-estimate of Togo's wealth per 2.4 km × 2.4 km region. These estimates provide the relative wealth of all households in each small grid unit, which are then population-weighted averaged to arrive at the wealth estimate of the smallest administrative unit in Togo.
The second step is to process the mobile phone metadata provided by the two mobile phone operators in Togo through a machine learning algorithm to estimate the average daily consumption of each mobile phone user.
Specifically, the study obtained mobile phone metadata (Call Detail Recording (CDR)) for specific time periods for 2018-2021 from two mobile network operators in Togo. The study focused on three segments of mobile network data: October to December 2018, April to June 2019, and March to September 2020. The CDR data contains the following information. Calls: caller phone number, receiver phone number, call date and time, call duration, base station ID of the call; SMS message: sender phone number, receiver phone number, date and time of message, antenna ID that sent the message; mobile data usage: phone number, transaction date and time, data consumption (upload and download combined); mobile money transaction: sender phone number, receiver phone number (if peer-to-peer), transaction date and time, The amount of the transaction and the broad categories of transaction types (cash, cash, peer-to-peer, or bill payment).
The study surveyed a representative number of mobile phone users and used these surveys to measure each user's wealth or consumption, and then matched survey-based estimates with detailed metadata about each user's mobile phone usage history, using supervised machine learning algorithms to train the sample data to predict user wealth and consumption levels through mobile phone use. This second step is similar to the traditional proxy means test (PMT), but with two main differences: the study uses high-dimensional vectors of mobile phone features rather than low-dimensional vectors of assets to estimate wealth; the study uses machine learning algorithms designed to maximize out-of-sample prediction capabilities rather than traditional linear regression that maximizes goodness-of-fit within a sample.
In order to protect the confidentiality of the obtained data, the study pseudonymized the CDR by hashing each phone number into a unique ID before analysis. This data is stored on the university server, with access rights set. Before matching CDR records to survey responses, the study obtained informed consent from all study subjects in a telephone survey.
Accurate assessment
The study calls this combination of machine learning and cell phone data a phone-based approach. By comparing this approach with the localization error under the counterfactual approach: a geolocation method piloted by the government in the summer of 2020 (Togo admin-2 pole, i.e. poverty map of Togo's counties, 40 counties), poor states (Togo admin-3 level, 397 states); occupation-based positioning (including Novissi's initial targeting method for informal workers, and the best approach for the country's poorest occupational categories).
The study, which aims to help the poorest people in the 100 poorest states, found that the phone-based targeting approach greatly reduces the error of exclusion and errors of inclusion (non-poor people are mistakenly considered eligible) relative to other possible targeting methods of the Togolese government, as shown in Figures 1a and 1.
Using PMT as a measure of true poverty, telephone-based positioning (area under the curve (AUC) = 0.70) is superior to other possible methods of assistance for rural Novissi (e.g., AUC = 0.59-0.64 for geographic-scale localization).
Figure 1: Novissi target compared to alternative target
Table 1.
For more details, please read the original paper.