The copyright of the original article belongs to the WeChat public account
"Bring science home" all
撰文 | Skin
审校｜Ziv
Nowadays, AI technology is being widely used in all aspects of our lives, but some scientists remind everyone that it is better not to entrust your health to some apps for now, because they are likely to make mistakes and even kill people.
Let's take a look at some of the risks posed by AI. ｜giphy
In 2015, a family in Oregon, USA, was rushed to the hospital due to mushroom poisoning. And the reason why they eat poisonous mushrooms is because the app on their mobile phones tells them that these mushrooms are edible.
In 2023, something like this will happen again. William D.'s family ate mushrooms a lot, and identifying them was almost a matter of generations, but William D. felt that the advantage of his generation was the use of smartphones. So he took a photo of the mushrooms in his backyard and uploaded them to the "mushroom recognizer" on his phone, which told him it was an edible variety. But after eating it, he was poisoned.
Local medical staff said the mushroom was so toxin that one mushroom was enough to kill four people, and William D. ate four, thankfully he was admitted to the hospital early.
The mushroom that William D. eats is actually a type of Amanita phalloides, which is very poisonous. Reference 2
Rick Claypool, a public affairs researcher who also uses AI to identify poisonous mushrooms, has not experienced a serious poisoning incident like in the previous example, but when he realized how error-prone AI recognition is today, he decided to write a report to warn people not to trust the species of mushrooms identified by AI.
如今有许多用AI识别蘑菇的软件｜picturemushroom
In the report, he mentioned that today's social media platforms can actually make more people aware of the problem of mushrooms and poisoning, and experts and experienced people will also provide a variety of high-quality information and resources on the platform.
AI technology can be of great help in this process, but blindly trusting them is also a risky thing.
Rick Van de Poll, an expert in mycology, once said that even experts can hardly tell whether mushrooms are edible or not by sight alone, and that distinguishing them requires not only color and shape, but also sometimes the sense of smell and even the use of a microscope.
Some of the mushroom recognition apps mentioned in the report. Reference 4
In 2022, a group of researchers in Australia analyzed three apps that claimed to be able to use AI to identify wild mushrooms, that is, let AI judge the toxicity of mushrooms simply by a photo. As a result, they found that the accurate recognition rate of poisonous mushrooms by these apps was only about 50%. The top-performing correct recognition rate was 67%.
Among them, there is a probability that the APP will not recognize the highly virulent Amanita phalloides, and even misidentified it once or twice. A doctor in the United States mentioned that in recent years, although some people who have eaten Amanita have survived, they have had to undergo liver transplants and even have permanent neurological problems. Therefore, "the cost of severe poisoning and possible death is very real." ”
While this image doesn't seem to be a problem, it's actually generated by DALL-E 3, and the words on it are all meaningless. Reference 4
As mentioned in the report, while AI technology is very useful, if you want to market the features of an app, exaggerate the advantages and downplay the risks, it can involve serious harm. Therefore, the report suggests that companies that provide apps should first let users know where AI technology is being used, and secondly, remind users that AI may make mistakes and bear responsibility for related mistakes.
The reason why AI may make mistakes is not only because of inaccurate recognition with digital photos alone, but also because there is too much false information on the Internet.
A report by 404media found that there appear to be AI-generated books on Amazon that teach people "how to cook poisonous mushrooms" when they are actually nonsense.
What's even more dangerous is that these "poisonous mushroom guides" don't even suggest that the book was written by AI, and the signature appears to be a fictional author, and the "fake expert" is said to have published 19 books......
The books may have been generated by AI, but there are no hints on the cover. Reference 5
Other AIs will draw an AI to show a mushroom, or they may show the wrong details. Getting users to get false information.
The AI (left) says that porcini mushrooms look like this, but its depiction is not accurate. On the right is a real picture of porcini mushrooms. Reference 4
In 2023, Gizmodo's reporters also found that Google's AI gave ridiculous answers when using search engines. When searching for "how to cook Amanita phalloides", the search engine gives the answer: you can filter out the toxins with water......
And this answer is actually wrong. In response, Google at the time mentioned that it would adopt higher precautions to treat these sensitive topics related to health and limit the use of some AI.
The search engine content provided by the AI is fake, and experts suggest that eating Amanita in this way can be fatal. Reference 7
All in all, making good use of AI technology may require a better understanding and regulation of it, and not ignoring the risks it may bring.
Resources:
[1]https://www.oregonlive.com/pacific-northwest-news/2015/05/4_people_fall_ill_after_trusti.html
[2]https://edition.cnn.com/2023/12/29/health/mushrooms-poisoning-foraging/index.html
[3]https://edition.cnn.com/2023/12/29/health/mushrooms-poisoning-foraging/index.html
[4]https://www.citizen.org/article/mushroom-risk-ai-app-misinformation/?utm_source=VentureBeat&utm_campaign=cef30891a2-EMAIL_CAMPAIGN_2024_03_19_01_11_COPY_01&utm_medium=email&utm_term=0_-dff594d55a-%5BLIST_EMAIL_ID%5D&mc_cid=cef30891a2&mc_eid=6385387dc8
[5]https://www.404media.co/ai-generated-mushroom-foraging-books-amazon/
[6]https://www.tumblr.com/rebeccathenaturalist/726294696941518848/hey-there-fellow-naturalist-albeit-less?ref=404media.co
[7]https://gizmodo.com/google-search-ai-answers-slavery-benefits-1850758631
[8]https://www.tandfonline.com/doi/full/10.1080/15563650.2022.2162917