On March 23, 2022, in Nantong, Jiangsu Province, workers are stepping up the production of test kits in the production workshop of the new crown antigen detection kit. (People's Vision/Photo)
A nasal swab is taken, the reagent is inserted to dilute, and the liquid is dripped onto the test card. After waiting for a few moments, horizontal bars began to appear on the white film. If the result is "positive" by two bars, the tester should immediately report to the health department and wait for further confirmation of the nucleic acid test.
According to the incomplete statistics of southern weekend reporters, in the past less than a month, Jilin, Changchun, Shanghai, Hangzhou and other cities have carried out more than 20 million times of self-testing of new crown antigens. Since the opening of self-testing in China on March 11, 2022, antigen detection has become a powerful supplement to nucleic acid testing.
While promoting antigen testing, health departments go out of their way to remind the public of the limitations of this approach. "Antigen detection is less sensitive than nucleic acid testing. We emphasize the need to standardize sampling and detection. At the press conference of the joint prevention and control mechanism of the State Council on April 1, Wang Guiqiang, director of the Department of Infectious Diseases of the First Hospital of Peking University, said so.
How well does the antigen detection reagent perform? When does a missed test occur? What are the advantages and disadvantages between different brands of reagents? Who will check their quality?
At present, a number of third-party research institutions have studied the performance of adverogenic detection reagents. Most recently, on March 13, the Hong Kong Biotechnology Association, a nonprofit organization, released a report comparing 24 antigen tests purchased in the Hong Kong market, 17 of which were produced by mainland manufacturers, with mixed results.
"We were also surprised that the quality of the products was so far apart." Yu Changhai, chairman of the Hong Kong Biotechnology Association, told Southern Weekend.
As of April 6, the number of approved antigen detection reagents in China has reached 24. On March 14, the Medical Device Technology Evaluation Center of the State Food and Drug Administration issued the "Key Points for technical review of the registration of antigen detection reagents for the new coronavirus (trial)" (hereinafter referred to as the 2022 version of the new regulations), and the threshold for the registration of antigen detection reagents has been greatly increased.
In the eyes of industry insiders, the new regulations are like a sieve, blocking many products that do not perform well for medium and weak positive patients.
On April 3, 2022, in Guangfulin Street, Songjiang, Shanghai, volunteers delivered antigen detection kits without contact. (People's Vision/Photo)
The new regulations first mention the Ct value, and some enterprises withdraw their applications for registration
"The main thing is to tell us what materials to prepare, and don't report it." The R&D director of a pharmaceutical company in Anhui revealed that after the release of the 2022 version of the new regulations, the drug regulatory department has held online lectures in various provinces to publicize the new regulations to enterprises.
Mei Chuan, head of an antigen reagent manufacturer in Shanghai, said that Jiangsu Province also held similar training, with experts from the China Institute of Food and Drug Control (China Academy of Inspection) attending. "What the experts mean is that if you feel that the sensitivity of the product does not meet the requirements, do not send it over, and sending it over is only a workload to increase the workload of the Central Inspection Institute."
Meichuan's previous clinical trials did not meet the new regulations, and it currently intends to make up another clinical trial abroad.
March 14 was a watershed day for companies that submitted antigen testing reagent registrations. On this day, the new regulations replaced the "2019 Novel Coronavirus Antigen/Antibody Detection Reagent Registration Technical Review Points" (hereinafter referred to as the 2020 version of the regulations) issued in February 2020, and put forward more detailed review requirements.
Southern Weekend reporters compared the approval conditions of the US Food and Drug Administration (FDA), the European Union and other approvals and found that in the requirements of several key indicators, the 2022 version of the new regulations is at the forefront of the world.
Taking the sensitivity and specificity of the core indicators to evaluate the accuracy of antigen detection reagents as an example, in the 2020 version of the regulations, the minimum requirement for sensitivity is 90%, and the new regulations in the 2022 version have been increased to 95%. Sensitivity refers to the proportion of patients who detect a positive result, and specificity refers to the proportion of patients who detect a negative result in a disease-free subject. The higher these two numbers, the closer the accuracy of the proof product is to the results of nucleic acid testing.
A more important change is that the 2022 version of the new regulation specifies the Ct value of the detection reagent. The Ct value (Cycle Threshold) is an indicator of nucleic acid detection that refers to the number of cycles that the fluorescence signal in each reaction tube undergoes when it reaches a set threshold. In layman's terms, the Ct value can measure the virus concentration of the sample, and the higher the Ct value, the lower the virus concentration.
The new regulations require that different viral load samples in positive samples of clinical trials should have sufficient sample sizes: taking the positive judgment value of nucleic acid detection reagent Ct≤38 as an example, it is recommended that the number of positive samples with a ct value of ≤30 should not be less than 170, and the number of positive samples with a Ct value of >30 should not be less than 30.
The 2020 version of the regulations does not propose Ct value indicators, only requiring the enrollment population of clinical trials to "include some convalescent cases", nor does it require the number and proportion. The 2022 version of the new regulations is equivalent to requiring products to include weakly positive subjects while ensuring that product performance does not change much.
This poses a huge challenge for manufacturers of paragen detection reagents. Li Desheng, technical manager of BGI Gene R&D, told the Southern Weekend reporter, "Generally speaking, nucleic acid detection can still be detected well in Ct values of 30 to 35, but antigen reagents are already in a critical state at 30, and the detection compliance rate will be greatly reduced." ”
Southern Weekend reporters reviewed the clinical data of antigen detection products that have been listed overseas before March 14 and found that most of the manufacturers' indicators include specificity, sensitivity, consistency, product detection limits and other indicators, but do not involve Ct values. From independent research conducted after the introduction of the Ct value category by third-party institutions, many manufacturers cannot meet the requirements of the 2022 version of the new regulations.
Mei Chuan revealed that at present, there are more than 100 domestic enterprises applying for the registration of new crown antigen detection reagents. After the new rules were issued, dozens of companies have withdrawn their applications.
The performance of mainstream antigen reagents is uneven
How does ct value, or viral load, of COVID-19 patients relate to infectivity?
Virologist Chang Rongshan previously said in an interview with a Southern Weekend reporter that the sample with a general Ct value of about 35 is weak positive, but the concentration below 30 indicates that the concentration is not low. Jin Dongyan, a professor at the School of Biomedical Medicine of the University of Hong Kong's Li Ka Shing School of Medicine, told Southern Weekend that infected people with a general Ct value below 15 have a super transmission risk, and more than 21 are still transmissible, and there is basically no transmission until it reaches more than 30.
In the ninth edition of the new crown pneumonia diagnosis and treatment plan released by the National Health and Health Commission, mild cases can be discharged from the hospital after two consecutive nucleic acid detection N gene and ORF gene Ct values of the new coronavirus ≥35. In the previous version of the protocol, the discharge condition was two consecutive negative nucleic acid tests, and the ct value limit value of the nucleic acid test results was 40.
At a press conference on the prevention and control of the new crown pneumonia epidemic in Shanghai held on March 17, Zhang Wenhong, director of the Department of Infectious Diseases at Huashan Hospital affiliated to Fudan University, said that through the study, it was found that the basic infectivity was already very low when the Ct value threshold was adjusted to 30 times amplification. Patients with a Ct value of 35 are basically not infectious, and can be isolated at home.
Taken together, a Ct value of 30 is a safe baseline. The mainstream antigen detection reagents on the market, how does the detection performance under this standard? The answers are mixed.
The Paul Henrich Institute (PEI) participates in the evaluation of medical devices, vaccines and biologics in Germany. The institute has long carried out independent quality assessment of antigen detection reagents listed in Germany, and the institute tested sensitivity according to three intervals of Ct values less than or equal to 25, 25 to 30, and greater than or equal to 30. If the sensitivity of the product is less than 75% when the Ct value is below 25, it is judged to be unqualified and needs to be removed from the German certification list. At present, PEI has counted at least 245 products, of which 46 are judged to be unqualified.
The results of the evaluation also showed that among the 199 products that passed the test, when the Ct value reached the range of 25 to 30, the test results showed a huge difference: some manufacturers, such as Hefei Biohan, still had 95% product sensitivity. However, the sensitivity of South Korea's NanoEntek products is only 8.7%, the sensitivity of one Zhejiang manufacturer's products is only 4.3%, and the sensitivity of another Beijing manufacturer's products is even 0%.
It is important to note that PEI studies use laboratory samples, not clinical trials. In real-world studies, sensitivity decreases relatively flatly as Ct values increase.
In another study published in the international journal Viruses in February 2022, a team of Belgian scientists collected 116 positive samples infected with Delta or Omicron strains and tested the sensitivity of six antigen tests. When the Ct value is less than 25, the detection rate of all products is above 94%. In the range greater than 25, including Chinese products, the detection rate of 5 models fell to between 20% and 40%.
In a clinical trial of 36 commercially available mainstream antigen detection reagents around the world, the Geneva-based nonprofit Foundation for Innovative New Diagnostics distributed between 68.4% and 97.5% in 8 products from China when the Ct value was less than 33, and 4 were higher than 90%. It should be noted that each product was studied independently and its subjects were subjected to different situations.
According to a recent research report released by the Hong Kong Biotechnology Association, some of the products produced in Singapore, South Korea, the United States, China and other countries and regions can still detect positive samples after dilution several times, and some products even react positively to orange juice and tap water. Ultimately, the report recommended the use of four Chinese-made test reagents.
On March 17, 2022, the new crown antigen test kit was sold on the shelves of Tianjin pharmacies, and the public can register and purchase with an ID card. (People's Vision/Photo)
Ct value: the Eu "does not have a card", the United States has high requirements
One possible reason for the uneven quality of antigen testing in overseas markets is that the approval threshold is lower.
Southern Weekend reporters found that the requirements for Ct values in the review of many countries and regions are lower, and the requirements for overall sensitivity and specificity are lower than those of China's 2022 version of the new regulations.
Nowadays, many companies like to advertise that their own antigen detection reagents have obtained the EU CE certificate. However, an industry insider told Southern Weekend that the evaluation thresholds of different EU member states' certification authorities are different, "such as Germany and France will be stricter, and some small countries will be looser."
After obtaining the CE certificate, only if at least one member state obtains a performance assessment (e.g. German PEI) can it be included in the EU general whitelist and then used in all member states. The list was established in January 2021 and is administered by the EU Health and Safety Commission. The list requires that the product performance evaluation be indicated in the report the subject's Ct value. The overall sensitivity of the product is higher than 80%, or the detection rate of strongly positive patients with a Ct value of less than 25 is more than 90%, that is, it meets the requirements.
According to the incomplete statistics of the Southern Weekend reporter, there are currently 615 antigen rapid inspection products that have obtained the EU CE certificate, of which at least 317 are from China. There are a total of 203 products on the general white list, of which at least 109 are from China.
The U.S. Food and Drug Administration 's (FDA) Emergency Use Authorization (EUA) requirements are more stringent. Li Desheng said that when BGI applied, the FDA required companies to provide Ct value data for cases, and 10% to 20% of the samples in the subjects needed to have a Ct value of more than 30.
Li Desheng pointed out that the ABOVE REQUIREMENTS OF THE FDA show its emphasis on the differences in testing of people with different degrees of infection. Therefore, BGI combines the relevant requirements of the FDA to further refine the stratification of Ct values and increase the monitoring of product performance in several sections with Ct values below 25, between 25 and 30, and above 30. He said that the minimum sensitivity set by the FDA is 80%, but BGI gene products still have a detection rate of more than 90% at about 30 ct values.
On April 6, southern weekend reporters reviewed the FDA review requirements and found that the FDA's requirements for Ct values have been updated: the review requirements for the antigen detection reagent self-test kit are that 10% to 20% of the samples in the subjects need to be within 3 Ct values of the nucleic acid detection as a comparison.
At present, the FDA has authorized a total of 17 antigen self-test products EUA, of which 3 are from China. Southern Weekend reporters inquired about the fda's disclosure of the three Chinese manufacturers that have obtained EUA, Jiuan Medical, Aikang Biology, and Oriental Biologics, and found that although the published clinical data all counted the time when the subjects appeared symptoms, they did not show ct value information.
Mei Chuan does not think that the EU drug review agency is deliberately lowering the threshold, "I think that when the review regulations were first designed, the issue of Ct value may not have been considered." Everyone thinks that antigen testing is used for qualitative purposes, and if the detection rate is a little worse, it may not be too big a problem. ”
On March 24, 2022, in Jilin Province, citizens used the new crown antigen test kit to self-test. (People's Vision/Photo)
Clinical data of domestic products are difficult to find
"At the beginning of the new crown pneumonia epidemic, many antigen reagent manufacturers showed me clinical reports, and I sighed how it was so powerful, it was 100% (sensitivity)? Looking at it again, it was all done with critically ill patients. Yu Changhai said, "The data you see may come from different sample selection methods." But there is no provision in this regard (before). ”
Selecting patients with different disease courses for clinical trial may result in "inflated" performance data obtained. "It doesn't make sense to talk about sensitivity aside from ct values. Some manufacturers have found a bunch of very serious infected people, randomly test a 100% detection rate, you go to test the newly infected, the body virus content is relatively small, then test the sensitivity of the reagent. Umekawa said.
At the 2022 International Symposium on Antigen Detection of the New Coronavirus held on April 2, Zhang Wenhong said, "Many reagent companies will choose to test during the symptom time window when studying, but specific to the actual clinical application, due to different application scenarios, sensitivity and specificity will still be lower than laboratory data." ”
At present, on the EU white list, except for a few manufacturers such as Zhuhai Lituo and Hangzhou Longji mentioned the Ct value range, most domestic and foreign manufacturers only publish two numbers of sensitivity and specificity.
Of the 17 PRODUCTS APPROVED BY THE FDA, 4 PRODUCTS HAVE DATA ON THE INSTRUCTIONS PAGE FOR THE SAME ITEM TESTED WITH 11 DIFFERENT CT VALUES OF THE OMICRON STRAIN. The trial, conducted by the U.S. National Institutes of Health, showed that the best performing product was able to detect a sample with an upper limit of 25.8 ct values.
Jiuan Medical, which obtained EUA, once released data from its own products in the same trial in the announcement, and said that it was "100% detected". On January 12, the Shenzhen Stock Exchange's inquiry letter pointed out that In an announcement issued on January 7, Jiuan Medical said that its kit "detected 100% of the Samples of Omicron active virus with a maximum Ct value of 21.59 (n=5)" in the experiment, but did not disclose the other 7 negative samples.
Jiuan Medical replied to the inquiry letter of the Shenzhen Stock Exchange that its kit could not detect 6 samples with a Ct value greater than or equal to 23.87. Based on this, the Tianjin Securities Regulatory Bureau believes that Jiuan Medical has not disclosed all the experimental results before, and the disclosed information is incomplete. On February 22, it was decided to take supervision and management measures against two senior executives of Jiuan Medical.
Have the approved enterprises conducted clinical trials to calculate ct values in accordance with the 2022 version of the new regulations? Umekawa is negative. He said that under normal circumstances, the antigen detection reagent from contacting the hospital, to finding clinical patients of different ages, genders, and races, to carrying out testing, the whole process lasts at least several months.
This can also be confirmed by the other side. At present, most of the approved products are emergency approvals. Southern Weekend reporter found that in the enterprise registration information published on the official website of the State Food and Drug Administration, except for Nanjing Novizan, the registration certificates of other enterprises are only valid for about 1 year. The registration information prompts that "this product is only for emergency reserves", and the enterprise is required to issue a clinical report when renewing the registration.
On March 22, Southern Weekend reporters called a number of antigen detection reagent manufacturers that have been listed in China to inquire about their clinical data. Except for one vendor that provided sensitivity and specific figures, the rest of the vendors either said it was unclear whether the clinical trial counted Ct values or said it was inconvenient to disclose.
Li Jinming, deputy director of the Clinical Laboratory Center of the National Health Commission, said at a press conference held on March 15 that the sensitivity of antigen detection reagents approved by the mainland is between 75% and 98%, and the specificity is between 95% and 99%. Zhang Wenhong suggested at the April 2 symposium that in order to avoid false negatives, antigen detection reagents need to be measured at least every three days to reach the level of nucleic acid detection.
The Ct threshold is controversial
It should be clarified that although the Ct value is linked to the viral load, and the viral load is directly proportional to the infectivity, there is still controversy in the academic community as to whether the Ct value should be included in the testing standards of the new crown testing products.
A study published in the Lancet's preprint platform SSRN showed that the Harvard Medical School team compared the detection capabilities of three common antigen detection reagents in the United States to detect The Omicron strain, and found that when the sample Ct value reached 27.5 or more, all reagents lost their testing ability. However, the study emphasizes that "the spread of the new crown virus is related to the viral load, but also to the length of exposure, whether masks are worn, ventilation conditions, immune capacity, symptomatic and strain characteristics." Infectivity is a multidimensional phenomenon, so it is still difficult to set the threshold based on ct values alone. ”
"What reagents are used, what instruments are used, and different people in different laboratories can cause differences in Ct values." So it will take some time to turn it into a complete gold standard. Yu Changhai said.
Meichuan said that responsible manufacturers will continue to iterate products and improve quality. In the past two years, the research and development technology of new crown antigen reagents has made great progress. Therefore, past test data does not necessarily represent the performance of the current product.
For self-test users, the biggest impact is whether the operation method is standardized. "Empty the nasal cavity before the self-test, hum a few breaths heavily, and start a 'fierce' when taking the nose swab, the accuracy rate will be higher." A quality inspector of a Jiangsu antigen reagent manufacturer told southern weekend reporters.
Li Desheng pointed out that in the past, the general public lacked understanding of the Ct value, and after the promulgation of the new regulations, it is conducive to the public to understand the relationship between the Ct value and the viral load. This can also strengthen public awareness of timely antigen testing within 5 days of the onset of symptoms and when the viral load is relatively high.
Putting aside the influence of external factors, with the increase of orders, the quality of manufacturers' products will be more challenging. "If a large number of antigen tests are used, there will be a problem, that is, whether manufacturers can do a good job in quality control to ensure testing capabilities and testing efficiency." At the seminar on April 2, Zhang Wenhong said so.
Previously, there have been cases of antigen detection reagent companies "self-cutting financial road". On Jan. 10, British company Avacta said it would suspend sales of antigen detection product AftiDX because the company's testing found that AffiDX was less sensitive to the Omicron variant virus than other variants at lower viral loads. After the news, Avacta's stock price fell by as much as 27% in one day.
Alastair Smith, CEO of Avacta, said in a statement, "We believe that all antigen kits should be studied for Omicron sensitivity and the results should be shared to ensure that the public can have confidence in the results of these reagents." ”
(At the request of the interviewee, Umekawa is a pseudonym)
Southern Weekend reporter Haiyang Southern Weekend intern Huang Jiayu Ru Wenxin
![The NBA introduces new rules! There are two positive players who can sign one, and the league is fighting too hard to avoid a shutdown[fig]](data:image/gif;base64,R0lGODlhAQABAIAAAP///wAAACwAAAAAAQABAAACAkQBADs=)