Zhi DongXi (public number: zhidxcom)
<b>Compile</b><b> the | Cheng Qian</b>
<b>Edited by | Li Shuiqing</b>
On Tuesday, the international authoritative journal "Nature" published a new ai (artificial intelligence) achievement, which is an AI image detection software for paper review, which can help editors review fake images in biology and other papers, misplace pictures, etc.
In scientific papers, the accuracy of experimental results often needs experimental data to prove. Some researchers may use computers to generate fake pictures, fabricate data, or use old data to duplicate the map in order to quickly complete papers or verify the results of their experiments. Finding modified and copied images in papers faster and more accurately has always been a top priority for academic journal editors.
Previously, in ten journals published by the American Association for Cancer Research (AACR), papers were subjected to an unusually additional examination before publication — reviewing images that appeared in articles.
Since January 2021, AACR has used AI software to re-examine other peer-reviewed manuscripts, and in addition to duplicate images, the software can also find fake images that have been rotated, stretched, or generated or modified using computers.
An example of a digital color transmission electron microscopy (TEM) operation on a picture of a virus in Proofig, with blue lines representing hundreds of identical features that ai-powered people use to compare
AI review software will be the future, and AACR is an early adopter of the technology. Whether intentionally or unintentionally, in order to avoid publishing tampered images in papers, many journals hire people to manually scan manuscripts for problems and then use censorship software to help check what they find.
But Nature understands that over the past year, multiple publishers have begun automating the review process, relying on artificial intelligence software to discover duplicated and modified images before the manuscript is published.
In the flow cytogram in the paper, there is a completely consistent final landing point of cells. (Image source is China Youth Network)
Daniel Evanko, director of journal operations at the Philadelphia Association of Pennsylvania, said AACR experimented with a variety of software products before finally opting for the service of Proofig, Israel's Rehovot's artificial intelligence detection software. "We are very happy with this." He added that he hopes AI will help researchers review to reduce the problems that arise after publication.
It's worth noting that when AI software labels images, professional editors are still needed to decide what to do. For example, if the same dataset appears twice in the text, but represents different meanings, duplicate images need to be preserved. Secondly, image duplication problems also arise due to simple copy and paste errors during manuscript assembly, but this happens unconsciously and is not deliberately deceiving the reader. Until then, these problems could only be resolved through discussions between editors and authors.
Plus, now that AI is effective enough and inexpensive, experts say that automated image inspection assistants could take the scientific publishing industry by storm in the coming years, just as using software to check for plagiarism became the norm in a decade. Publishing industry groups also say they are exploring ways to compare images from manuscripts in different journals.
Other image integrity experts agree with this trend, although they believe that there is currently no public comparison of various software products, and that automated inspections may produce excessive false positives or miss some pictures.
In the long run, reliance on software screening may also prompt fraudsters to use AI to trick software, just as some people avoid plagiarism filtering by tweaking text. Bernd Pulverer, editor-in-chief of EMBO Reports in Heidelberg, Germany, said: "I am concerned that we are engaged in an arms race with AI-based technologies, which could lead to deeply falsified images. ”
Researchers have been developing AI software for image inspection for years because they fear that academic misconduct in papers could wreak havoc on the scientific literature, which would far outweigh the damage caused by retracting an article or correcting a statement after something went wrong in an article.
Previously, Dutch microbiologist Elisabeth Bik, who independently or collaboratively published about 20,000 biomedical papers, was artificially analyzed by researchers in 2016 and the results showed that more than 4 percent of papers may contain problematic images. However, usually only about 1% of papers are corrected each year, and even fewer are retracted.
Dutch microbiologist Elisabeth Bik
"I know of about 20 companies around the world that are developing image inspection software." Mike Rossner, president of Image Data Integrity, a California-based biomedical research image processing consulting firm, said.
Last year, a group of scientific publishers around the world formed a working group to develop standards for software that screens for image problems; this year the group issued guidelines on how editors should deal with tampering with images, but have yet to develop guidelines for software.
Some academic groups and companies told Nature, journals and government agencies were experimenting with their AI software, but Proofig was the first to disclose customer information. In addition to AACR, the American Society of Clinical Investigation began using Proofig's software to review manuscripts in the journal Clinical Investigation (JCI) in July, said Sarah Jackson, executive editor of the Journal of The American Society of Clinical Investigation. Helen King, head of transformation at SAGE, an independent academic publishing company in London, said SAGE Publishing adopted the software in five of its life sciences journals in October.
Proofig's software extracts images from papers and compares them in pairs to find common features. In general, the paper is examined in about a minute or two, and Dror Kolodkin-Gal, founder of Proofig, says the software can also correct tricky problems, such as compression artifacts that can occur when compressing high-resolution raw data into smaller files. "Computers have an advantage over human vision." "Not only does the computer not get tired, it runs faster, but it's also not affected by size, location, orientation, overlap, partial replication, and a combination of these factors," he says. ”
Gar declined to discuss pricing in detail about the cost of image checking, but said contracts with publishers often charge based on the number of images in the paper, but also on the number of manuscripts. He said that amounted to a charge per sheet of paper "close to tens of dollars instead of a few hundred dollars."
Jackson said that in the journal Clinical Investigation, the software found more problems than previous staff manually reviewed. While staff still need to check the output of Proofig, it's important that the journal already has AI systems in place to handle a variety of image problems. "We really feel that rigorous data is an absolute hallmark of our journals. We think it's worth the time and money. Jackson said. Meanwhile, in the Journal of the American Association for Cancer Research, Ivanko said many authors were happy to have spotted duplicate errors in their pictures before they were published.
Meanwhile, Lausanne, Switzerland-based publisher Frontiers has developed its own image inspection software, AIRA (Artificial Intelligence Review Assistant), as part of an automated inspection system. A spokesperson said a team studying image integrity has been using AIRA internally since August 2020 to perform image checks on all submitted manuscripts. But most of the papers tagged by the software don't actually have problems, and only about 10 percent need to be followed up by the editorial team. Frontiers declined to say how many papers AIRA had tagged.
Image integrity experts, including BAK and Rosner, say they haven't tried AIRA or Proofig themselves, and it's difficult to evaluate software products that haven't yet used standardized testing for public comparison. Rosner added that in addition to repetition, it is also important to detect whether part of the image is deleted or cropped by PS. "AI software may be a useful addition to visual screening, but it may not be a substitute for the current form."
"However, I believe that this will eventually become the criteria for manuscript screening." Birk added.
Publishers who have not yet adopted AI software image screening have cited cost and reliability issues. A spokesman for PLOS at the American Public Library of Science said he was "eagerly" monitoring the progress of the study, tools that "reliably identify common image integrity issues and can be applied at scale." Dutch publishing group Elsevier said it was "still testing" the software, but some of its journals screened all papers before publication and examined images "using a combination of software tools and manual analysis."
In April 2020, academic publisher Wiley launched an image screening service for ad hoc manuscripts, which are currently used by more than 120 journals, but currently this is manual screening aided by software, a spokesperson said. Springer Nature, publisher of the journal Nature, said it was evaluating some external tools while collating data to train its own software, which would "combine complementary ARTIFICIAL intelligence and humans to identify problematic images."
"EMBO Press magazine still primarily uses manual screening, and I don't quite believe in the cost-benefit ratio of commercial products," Pulwell said. And Pulwell is a member of a cross-publisher working group that defines software standards, "and I have no doubt that we will soon have advanced tools." ”
Pulwell worries that fraudsters may understand how the software works and use AI software to generate fake images that neither people nor the software can detect. While no one has yet indicated that such images have appeared in research papers, a preprint published last year on BioRxiv, a website that provides distribution services, suggests that there may be falsified versions of biological images, such as Western blots, that cannot be distinguished from real data.
Western blot image in the paper in the journal "Cell Biochemistry" (Picture source is China Youth Network)
But researchers are tackling the problem, with Edward Delp, a computer scientist at Purdue University in West Lafayette, Indiana, leading a team in a project funded by the Defense Advanced Research Projects Agency that is working on software that detects fake images by artificial intelligence and focusing on fake biological images, such as microscope images and X-rays, and a paper describing the system is under review.
Light microscopy captures microscopic photographs of cells and tissues
Currently, AI image checking is usually done in manuscripts rather than papers, which will make it more and more computationally calculated. But commercial and academic software developers say this is technically feasible. Daniel Acuña, a computer scientist at Syracuse University in New York, successfully ran AI detection software on thousands of COVID-19-related preprints last year to look for duplicates.
Crossref is a U.S. nonprofit cooperative organization of more than 15,000 organizations responsible for organizing cross-thesis plagiarism checks, among other things. Bryan Vickery, Crossref's director of products in London, said they were currently conducting an investigation asking its members about concerns about tampering with images, the software they used and whether a "cross-publisher service" that could share images was feasible.
In December, STM Solutions, a subsidiary of STM, an industry group for academic publishers in Oxford, announced that it was developing a cloud platform to help publishers collaborate "to check for research integrity issues in submitted articles" while maintaining privacy and confidentiality. Matt McKay, an STM spokesman, said detecting image tampering, duplication and plagiarism across journals "occupies a significant position in our roadmap."
Academic misconduct in papers will have a greater impact on journals, authors, etc. Previously, academic journals relied on manual screening by editors to find image errors in papers, but due to the limitations of manual review, there will always be "fish that slip through the net". Therefore, with the continuous development and improvement of artificial intelligence technology, artificial intelligence detection technology will become the main means of image detection of papers.
However, because artificial intelligence technology cannot judge the image according to the situation, it still needs to be reviewed by manual editors in the end, but this still greatly reduces the workload of editing. Jackson said that in the journal of clinical investigation, the AI software found more problems than previous staff manually reviewed.
In addition, it is also crucial to establish a cloud data platform for artificial intelligence software review in the field of image detection, so that cross-publisher detection can effectively avoid academic misconduct due to different regions and disciplines.
Source: Nature