Analytical Sensitivity and Effectiveness of Different SARS-CoV-2 Testing Options
Article Information
Nico Lelie1*, Marco Koppelman2, Harry van Drimmelen1, Sylvia Bruisten3
1Biologicals Quality Control (BioQControl), Heiloo, The Netherlands
2Sanquin, The National Screening Laboratory of Sanquin (NSS), Amsterdam, The Netherlands
3Public Health Laboratory, Department of Infectious Diseases, Public Health Service (GGD), Amsterdam, The Netherlands
*Corresponding author: Nico Lelie, Biologicals Quality Control (BioQControl), Heiloo, The Netherlands
Received: 20 June 2022; Accepted: 28 June 2022; Published: 06 August 2022
Citation: Nico Lelie, Marco Koppelman, Harry van Drimmelen, Sylvia Bruisten. Analytical Sensitivity and Effectiveness of Different SARS-CoV-2 Testing Options. Archives of Internal Medicine Research 5 (2022): 346-356.
View / Download Pdf Share at FacebookAbstract
We prepared Severe Acute Respiratory Syndrome-Coronavirus-2 (SARS-CoV-2) working standards from a pool of swab fluid samples for comparing the analytical sensitivity of different molecular detection methods and rapid antigen assays. The following 50% Limits of Detection (LOD) (and 95% Confidence Interval (CI)) were estimated in RNA copies/mL: Roche cobas PCR 1.8 (1.0-3.3), Hologic Aptima TMA 6.6 (4.4-9.9), DRW SAMBA 15 (7-30), Molgen LAMP 23 (13-42), Fluorecare antigen 50,000, Abbott Panbio antigen 75,000 and Roche antigen 100,000 copies/mL. One 50% Tissue Culture Infectious Dose (TCID50) of culture fluid was estimated to be equivalent to approximately 1000 RNA copies (2700- 4300 International Units) in our standard. When assuming this level as a proxy for the start of contagiousness in a log-linear ramp up phase model with 10-fold and 10,000-fold rise of viral load per day for the original B.1 (Wuhan) type and B.1.617.2 Delta variant respectively we estimated relative time points of first detectability of early infection from the 50% LODs. The four molecular assays would be able to detect the B1 (Wuhan) type 40-66 hours earlier than the 1000 copies/mL infectivity threshold, whereas the three antigen tests would become positive 41-48 hours later. Our modeling of analytical sensitivity data confirms that molecular assays are more reliable than antigen assays for identifying early infected asymptomatic individuals who are potentially infectious. However, if our assumption of approximately 4-fold more rapid viral replication after exposure to the Delta variant and the currently circulating less pathogenic Omicron variant is correct daily antigen self-tests may be a more practical alternative.
Keywords
SARS-CoV-2 RNA Assays, Antigen Assays, Working Standard, Early Detection
SARS-CoV-2 RNA Assays articles; Antigen Assays articles; Working Standard articles; Early Detection articles
Article Details
Abbreviations:
CI: Confidence Interval; IU: International Unit; LAMP: Loop-Mediated Isothermal Amplification; LOD: Limit of Detection; NAT: Nucleic Acid Amplification Technology; PCR: Polymerase Chain Reaction; SAMBA: Simple Amplification Based Assay; TCID50: 50% Tissue Culture Infectious Dose; TMA: Transcription Mediated Amplification; SARS-CoV-2: Severe Acute Respiratory Syndrome-Coronavirus-2; WHO: World Health Organization
1. Introduction
Several studies compared the clinical sensitivity of molecular assays and rapid antigen tests on nasopharyngeal or nasal swab samples for detection of severe acute respiratory syndrome-coronavirus 2 (SARS-CoV-2) infection [1-6]. The limitation of these comparison studies is that separate swabs must be taken for detection of SARS-CoV-2 RNA and antigen respectively, whereas these individuals were usually tested because of (mild) symptoms and already had higher viral loads. Only limited data is available from studies that directly compared the analytical sensitivity of different Nucleic Acid Amplification Technology (NAT) systems and lateral flow devices for rapid detection of SARS-CoV-2 antigen using serial dilutions of swab fluid with a known viral load [6-8]. In this report SARS-CoV-2 (B.1 Wuhan type) reference preparations from a pool of swab samples in Viral Transport Medium before and after inactivation by beta-propiolactone [9, 10] were used for comparison of the analytical sensitivity of different SARS-CoV-2 assays. These working standards were first quantified in NAT detectable SARS-CoV-2 RNA copies/mL using limiting dilution analysis and later - at the time the World Health Organization (WHO) International Standard became available - also in International Units (IU/mL).
Additionally, by comparison of NAT detection limits on USA-WA1-2020 culture fluid (reported in package insert of the Roche cobas SARS-CoV-2 assay) and on our working standard the amount of NAT detectable RNA copies per 50% Tissue Culture Infectious Dose (TCID50 ) was roughly estimated. Since the rapid antigen tests are known to miss a considerable proportion of infected individuals [1, 2] we also evaluated two NAT methods that were developed for faster detection of SARS-CoV-2 infection i.e. a loop-mediated isothermal amplification (LAMP) assay and a so called Simple amplification based assay (SAMBA) [11]. We compared their sensitivity with two widely used NAT systems as reference methods i.e. a real time Polymerase Chain Reaction (PCR) assay and a transcription mediated amplification (TMA) assay. By testing 1.5- and 3-fold fold dilution series of our native and inactivated working standards the limits of detection (LOD) of three rapid antigen tests were compared with those of four NAT methods. We then used the analytical sensitivity data to model the effectiveness of different SARS-CoV-2 detection options in identifying infected individuals during the early asymptomatic phase of infection. For this early ramp up phase model a log-linear growth curve of the viral load in the cells of the upper respiratory tract was assumed using best estimates for the viral doubling time and start of potential contagiousness of the B.1 (Wuhan) type and the B.1.617.2 Delta variant [12-14].
2. Material and Methods
2.1 Preparation of native and inactivated SARS-CoV-2 standards and reference panels
Working standards were prepared from a pool of remnant fluid of swab samples, kindly provided by the Public Health Laboratory (GGD) of Amsterdam. Samples were fully anonymized before use. The study procedure was evaluated by the Medical Ethics Committee of the Academic Medical Center in Amsterdam (W21_507 # 21.559) and deemed not to require a full review of the board. Swab samples in Viral Transport (GLY) Medium were collected in August to November 2020 at the time only the B.1 (Wuhan) type of SARS-CoV-2 virus circulated in the Netherlands. Samples were known to be tested positive using the Hologic TMA test with RLU values >1200. Approximately 90 mL of this pooled medium constituted the native SARS-CoV-2 standard. Seventy-five mL of this native pool was inactivated with 0.14% beta-propiolactone for 5 hours at 23°C, followed by 18 hours incubation at 2-8°C [9, 10] to produce the inactivated standard. This inactivated standard was used for preparing a 10 member (3 and 10-fold) dilution panel in a 2% human plasma solution in phosphate buffered saline (PBS), which was used for replicate testing by different NAT methods to determine the 95% and 50% LODs by probit analysis. In addition a 20 member panel composed of 1.5-fold dilutions of both the native and inactivated standard was prepared for comparing the analytical sensitivity of different rapid antigen assays. The reference panels were prepared by BioQControl (Heiloo, The Netherlands) via gravimetrically recorded dilution steps and were snap frozen in liquid nitrogen before storage at -80°C.
2.2 Quantification of SARS-CoV-2 RNA and antigen in working standards before and after inactivation
Ten-fold standard dilutions of both the native and inactivated SARS-CoV-2 standards were made in 2% plasma-PBS and tested in duplicate in the cobas PCR assay (Roche Molecular Systems) to compare the SARS CoV-2 RNA Ct values for both the ORF1ab and E gene targets. The yield after inactivation was determined by parallel line analysis on Ct values for the two PCR targets separately. A panel composed of 3 and 10-fold dilutions of the inactivated standard was tested in multiple replicate tests in different NAT methods to allow for estimation of the 95% and 50% LOD by probit analysis. The concentration in NAT detectable RNA copies/mL that was assigned to the inactivated standard was based on the 63% LOD in the most sensitive NAT method. We assumed that this assay reached 100% NAT efficiency or a 63% LOD of 1 detectable RNA copy per amplification reaction as follows from Poisson distribution in limiting dilution analysis. The concentration in RNA copies/mL in the inactivated working standard was then calculated by correcting for the input volume in the NAT method. In addition, 3 and 10-fold dilutions of the inactivated standard were tested in duplicate against similar concentrations of the WHO International Standard 20/146 in the cobas PCR assay. The conversion factor between NAT detectable RNA copies and International Units (IUs) was calculated by parallel line analysis on Ct values for both ORF1ab and E gene targets in the cobas assay. The concentration in the native standard was derived from the value assigned to the inactivated standard and the measured yield after inactivation, whereby the average was taken for the two PCR targets in the cobas assay. The yield of SARS-CoV-2 antigen after inactivation was determined by comparison of geometric mean detection endpoint titers on 1.5-fold dilutions of both the native and inactivated standard in three rapid antigen tests.
2.3 Estimation of SARS-CoV-2 viral load in culture fluid of known infectivity
We compared the 95% LOD expressed in TCID50/mL on USA-WA1-2020 culture fluid reported in the package insert of the Roche cobas assay with the 95% LOD expressed in NAT detectable RNA copies/mL assigned to our inactivated working standard to roughly estimate the dose of SARS-CoV-2 RNA copies per TCID50.
2.4 SARS-CoV-2 assays evaluated for analytical sensitivity
To determine the 95% and 50% LOD by probit analysis the inactivated SARS-CoV-2 standard dilution panel (with estimated viral RNA concentrations varying between 33,784 and 1.1 copies/mL) was tested in multiple replicates by the following NAT assays: Roche cobas PCR, Hologic Aptima TMA, LAMP [prototype assay, Molgen, Veenendaal, the Netherlands] and Diagnostics of the Real World (DRW) SAMBA II assay [11]. The 1.5-fold SARS-CoV-2 standard dilution panel with concentrations varying between 800,000 and 60,000 copies/mL of both the native and inactivated standards was tested in duplicate by three rapid antigen lateral flow devices, i.e. Panbio COVID-19 Ag Rapid Test (Abbott Rapid Diagnostics Jena GmbH), SARS-CoV-2 Rapid Antigen Test (Roche Diagnostics GmbH) and Fluorecare COVID-19 Spike Protein Test Kit (Shenzhen Microprofit Biotech Co, Ltd, kindly provided by Sander Brus, PreVViral, Amsterdam, The Netherlands). The latter Chinese assay exclusively targeted the spike protein, while the other kits targeted the nucleocapsid protein. For each of the antigen tests the panel members were mixed 1:1 with the sample buffer from the kits and the required volume was added to the reaction hole of the lateral flow devices. The panel members that gave a clearly visible line were scored as + whereas the next panel member(s) showing a faint line was recorded as ± or indeterminate. The viral concentration in the 1:1 mixed sample buffer with the last + and ± reactivity were recorded as endpoint titers. If the duplicate reactions were discrepant (+ and ± or ± and -) the series with the lowest endpoint titers were taken as final positive (+) and indeterminate (±) result.
2.5 Statistics
The 95%, 63% and 50% LODs on the SARS-CoV-2 analytical sensitivity panel were calculated by probit analysis and relative sensitivities of assays by parallel line probit analysis using SPSS software. The potency of the inactivated SARS-CoV-2 standards against the native standard and the WHO International Standard 20/146 was calculated from the Ct values in the cobas assay using parallel line analysis.
2.6 Assumptions for modeling assay conversion times during early infection
In our log-linear ramp up phase model we assumed
that viral load of the B.1 (Wuhan) type increased on average approximately 10-fold per day (mean viral doubling time 7.2 hours) during the initial phase of infection and on average 10,000-fold per day (mean viral doubling time 1.8 hours) with the B.1.617.2 Delta variant. These assumptions were based on observations during two outbreaks in Guangdong province in China [12]. Although these Chinese investigators observed a considerable variation in viral growth curves between individual cases during daily quantitative PCR testing they estimated a 1000-fold higher mean viral load at the day of PCR conversion with the Delta variant than with 19A/19B strains during an earlier outbreak in the beginning of 2020. The 50% LOD NAT conversion point of the most sensitive assay was arbitrarily set at time point zero and the 50% LOD NAT conversion points found with (the most sensitive targets of) the other assays were calculated using the assumptions for the viral growth curves mentioned above. The rapid antigen detection limits with a faint reaction line (±) on the 1.5 fold native standard dilutions were used to calculate the time intervals between the 50% NAT conversion points and the rapid antigen test conversion points.
3. Results
3.1 Quantification of SARS-CoV-2 RNA and antigen in working standard before and after inactivation
Three 10-fold dilutions of SARS-CoV-2 standards before and after inactivation were tested in duplicate in the cobas SARS-CoV-2 assay and Ct values for the two PCR gene targets are shown in Supplemental Table 1. With parallel line assay the distance between Ct values (95% confidence interval (CI)) before and after inactivation was 1.96 (1.79-2.12) for ORF1ab and 2.67 (2.57-2.75) for E gene targets respectively. The potency (CI) of the native standard was thus estimated to be 21.96 = 3.88 (3.46-4.35) fold higher than the inactivated standard based on ORF1ab gene PCR and 22.67 =6.34 (5.97-6.79) fold higher based on E gene PCR (on average a 4.96-fold difference). Hence, the recovery after treatment of the pool of swab fluid with beta-propiolactone was 25.8 (23.0-28.9)% for the ORF1ab gene and 15.7 (14.8-16.8)% for the E gene (yield on average 20.2%). From the proportions of reactive tests in three different NAT assays on inactivated standard dilutions the 63% LOD (and 95% confidence interval (CI)) was calculated by parallel line probit analysis and the results are shown in Supplemental Table 2.
We assumed that the NAT method with the highest sensitivity (the cobas assay for ORF1ab target) reached 100% NAT efficiency or a 63% LOD of 1 NAT detectable RNA copy per amplification reaction as follows from Poisson distribution statistics. Since for each replicate cobas PCR test a volume of 400 uL was used as input in the amplification reaction the concentration at the 63% LOD was thus set at 2.5 copies/mL for the cobas PCR ORF1ab assay. From the dilution factors we calculated that the viral load in the undiluted inactivated standard was then 3.38 x 106 NAT detectable RNA copies/mL and 4.96-fold higher (see above) in the native standard (1.68 x 107 copies/mL). With these assigned RNA copies/mL to the SARS-CoV-2 standards the calculated NAT efficiency was 100 (53-187)% and 54 (24-94)% in the cobas ORF1ab and E PCR tests respectively, whereas it would be 67 (42-102)% in the Aptima TMA assay. (Supplemental Table 2).When comparing the dilutional titers in three rapid antigen tests on the 1.5-fold dilution panel we estimated that the SARS-CoV-2 antigen concentration was 1.8 (range 1.3-2.5) fold lower in the beta-propiolactone-inactivated standard than the untreated standard, whereas the RNA concentration was 5.0 (3.9-6.3)-fold lower after inactivation. Hence, the antigenicity relative to the RNA concentration was found to be 2.7 (range 2.5-3.0) fold higher in the inactivated standard than in the native standard.
3.2 Calibration of inactivated SARS-CoV-2 working standard against the WHO International Standard
For calibration of the inactivated SARS-CoV-2
standard against the WHO 20/146 standard we tested dilution series in the cobas SARS-CoV-2 assay in duplicate (Supplemental Table 3a) and analyzed the Ct values using parallel line analysis with exception of the lowest concentration to improve parallelism. The potency difference (CI) or the amount of IUs per NAT detectable RNA copy was 4.29 (4.44-5.36) for the ORF1ab target and 2.68 (2.19-3.29) for the E gene target (Supplemental Table 3b).
Table 1: Proportion reactive results and average assay response values on an inactivated SARS-CoV-2 standard dilution panel tested in multiple replicates by four NAT methods.
Inactivated standard dilutions |
cobas ORF1ab gene |
cobas E gene |
Aptima |
||||
panel member |
RNA copies/mL |
r/n (%) |
mean Ct |
r/n (%) |
mean Ct |
r/n (%) |
mean RLU |
1 |
33784 |
4/4 (100%) |
26.92 |
4/4 (100%) |
27.71 |
2/2 (100%) |
1082 |
2 |
11250 |
4/4 (100% |
28.64 |
4/4 (100%) |
29.46 |
2/2 (100%) |
1090 |
3 |
3378 |
4/4 (100%) |
30.4 |
4/4 (100%) |
31.32 |
4/4 (100%) |
1095 |
4 |
1125 |
8/8 (100%) |
31.85 |
8/8 (100%) |
32.87 |
4/4 (100%) |
1093 |
5 |
337.8 |
8/8 (100%) |
33.15 |
8/8 (100%) |
34.21 |
14/14 (100%) |
1104 |
6 |
112.5 |
8/8 (100% |
34.69 |
8/8 (100%) |
35.87 |
14/14 (100%) |
1083 |
7 |
33.8 |
8/8 (100%) |
35.26 |
8/8 (100%) |
36.45 |
13/14 (93%) |
883 |
8 |
11.3 |
8/8 (100%) |
36.87 |
7/8 (88%) |
38.06 |
9/14 (64%) |
684 |
9 |
3.38 |
7/8 (88%) |
37.55 |
4/8 (50%) |
37.89 |
4/14 (29%) |
429 |
10 |
1.12 |
1/8 (13%) |
1/8 (13%) |
1/14 (7%) |
323 |
Viral state |
RNA copies/mL 1:1 mixture buffer |
Roche |
Abbott A |
Fluorecare |
Native |
A B |
A B |
A B |
|
400,000 |
+ + |
+ + |
+ + |
|
300,000 |
+ + |
+ + |
+ + |
|
225,000 |
+ + |
+ + |
+ + |
|
175,000 |
+ + |
+ + |
+ + |
|
125,000 |
± ± |
+ + |
+ + |
|
100,000 |
± ± |
± ± |
+ + |
|
75,000 |
- - |
± ± |
+ + |
|
50,000 |
- - |
- - |
± ± |
|
37,500 |
- - |
- - |
- - |
|
30,000 |
- - |
- - |
- - |
|
Inactivated |
400,000 |
+ + |
+ + |
+ + |
300,000 |
+ + |
+ + |
+ + |
|
225,000 |
+ + |
+ + |
+ + |
|
175,000 |
+ + |
+ + |
+ + |
|
125,000 |
+ + |
+ + |
+ + |
|
100,000 |
+ + |
+ + |
+ + |
|
75,000 |
+ ± |
+ + |
+ + |
|
50,000 |
± - |
+ + |
+ + |
|
37,500 |
- - |
± ± |
+ + |
|
30,000 |
- - |
- - |
± ± |
Table 2: Comparison of analytical sensitivity of three rapid antigen tests on 1.5 fold dilutions of SARS-CoV-2 working standards before and after inactivation by betapropiolactone.
Viral state |
RNA copies/mL 1:1 mixture buffer |
Roche |
Abbott A |
Fluorecare |
Native |
A B |
A B |
A B |
|
400,000 |
+ + |
+ + |
+ + |
|
300,000 |
+ + |
+ + |
+ + |
|
225,000 |
+ + |
+ + |
+ + |
|
175,000 |
+ + |
+ + |
+ + |
|
125,000 |
± ± |
+ + |
+ + |
|
100,000 |
± ± |
± ± |
+ + |
|
75,000 |
- - |
± ± |
+ + |
|
50,000 |
- - |
- - |
± ± |
|
37,500 |
- - |
- - |
- - |
|
30,000 |
- - |
- - |
- - |
|
Inactivated |
400,000 |
+ + |
+ + |
+ + |
300,000 |
+ + |
+ + |
+ + |
|
225,000 |
+ + |
+ + |
+ + |
|
175,000 |
+ + |
+ + |
+ + |
|
125,000 |
+ + |
+ + |
+ + |
|
100,000 |
+ + |
+ + |
+ + |
|
75,000 |
+ ± |
+ + |
+ + |
|
50,000 |
± - |
+ + |
+ + |
|
37,500 |
- - |
± ± |
+ + |
|
30,000 |
- - |
- - |
± ± |
Table 3: Comparison of LODs of different SARS-CoV-2 NAT and antigen assays and relative sensitivity factors on working standards before and after inactivation.
Test method |
Viral state of standard |
Replicate tests per dilution |
LOD reactivity |
RNA copies/mL (95%CI) |
Relative insensitvityǂ factorǂ |
Roche cobas PCR ORF1ab |
inactivated |
8 |
50% |
1.8 (1.0-3.3) |
1 |
95% |
8.3 (4.5-18.6) |
1 |
|||
Roche cobas PCR E gene |
inactivated |
8 |
50% |
3.5 (2.0-6.0) |
1.9 |
95% |
15.5 (8.7-34.6) |
1.9 |
|||
Roche MP6# PCR ORF1ab |
inactivated |
8 |
50% |
11.1 (5.9-20.1) |
6 |
95% |
49.8 (27-112) |
6 |
|||
Hologic Aptima TMA |
inactivated |
14 |
50% |
6.6 (4.4-9.9) |
3.6 |
95% |
29.7 (18.4-60.1) |
3.6 |
|||
Molgen LAMP |
inactivated |
6 |
50% |
22.8 (12.7-42.1) |
12.4 |
95% |
102.5 (54.1-251) |
12.4 |
|||
DRW SAMBA ORF |
inactivated |
8 |
50% |
15.0 (7.1-30.1) |
8.1 |
95% |
133 (44.3-447) |
16 |
|||
DRW SAMBA N |
inactivated |
8 |
50% |
150 (79.2-271) |
81.1 |
95% |
597 (315-4168) |
71.9 |
|||
Roche Antigen |
inactivated |
2 |
± |
75,000 |
41,667 |
+ |
100,000 |
12,048 |
|||
Abbott Antigen |
inactivated |
2 |
± |
37,500 |
20,833 |
+ |
50,000 |
6,024 |
|||
Fluorecare Antigen |
inactivated |
2 |
± |
30,000 |
16,667 |
+ |
37,500 |
4,518 |
|||
Roche Antigen |
native |
2 |
± |
100,000 |
55,556 |
+ |
175,000 |
21,084 |
|||
Abbott Antigen |
native |
2 |
± |
75,000 |
41,667 |
+ |
125,000 |
15,060 |
|||
Fluorecare Antigen |
native |
2 |
± |
50,000 |
27,778 |
+ |
75,000 |
9,036 |
|||
ǂ according to comparison of 50% and 95% LODs respectively for NAT methods as calculated by probit analysis. For antigen assays detection endpoint titers with indeterminate (±) reactivity were compared with the 50% LOD in cobas PCR, whereas endpoint titers with positive (+) reactivity were compared with the 95% LOD in cobas PCR #MP6 : minipools of six samples |
3.3 Analytical sensitivity of different NAT and rapid antigen assays
Table 1 gives the proportion of reactive results and mean measurement values on the inactivated standard dilutions for the four NAT methods. Table 2 shows the reactivity of three rapid antigen tests on 1.5 fold serial dilutions of the native and inactivated working standards. In Table 3 we combined the analytical sensitivity data of the four NAT systems and three rapid antigen assays. The cobas assay for ORF1ab was the most sensitive with 50% and 95% LOD of 1.8 (1.0-3.3) and 8.3 (4.5-18.6) copies/mL if used on single samples and 11.1 (5.9-20.1) and 49.8 (27-112) if used on minipools of 6 swab samples. In parallel line probit analysis the Aptima TMA assay was 3.6 (2.4-5.3) fold less sensitive than cobas PCR with 50% and 95% LODs of 6.6 (4.4-9.9) and 29.7 (18.4-60.1) copies/mL. The LAMP assay was 12.4 (6.7-25) fold less sensitive than the Roche assay with estimated 50% and 95% LODs of 23 (13-42) and 102 (54-251) copies/mL. The SAMBA II assay was evaluated later using the same reference panel and in a separate probit analysis the 50% LOD and 95% LOD were 15 (7-30) and 133 (44-447) copies/mL respectively. The analytical sensitivity of the rapid antigen assays on 1.5-fold dilutions of the native standard dilutions were 28,000 to 56,000 fold lower than the cobas PCR assay when comparing the lowest concentration giving indeterminate (±) antigen reactivity with the 50% LOD in PCR, whereas the LODs with rapid antigen positive (+) reactivity were 9,000 to 21,000-fold higher than the 95% LOD in the cobas assay. On our SARS-CoV-2 standard of the B.1 (Wuhan) type the Chinese Fluorecare assay targeting the spike protein was slightly more sensitive than the Abbott and Roche antigen assays targeting the nucleocapsid protein.
3.4 Relation between viral load and infectivity in culture fluid
The 95% LOD (CI) in the package insert of the cobas assay was estimated in TCID50/mL using culture fluid of the SARS-CoV-2 Strain USA-WA1-2020. For the ORF1ab gene the Roche package insert reported a 95% LOD (CI) of 0.007 (0.005-0.036) TCID50/mL, whereas on our working standard we estimated the 95% LOD at 8.3 (4.5-18.6) copies/mL for this PCR gene target. According to this comparison 1 TCID50 would be equivalent to 1186 (847-6098) PCR detectable RNA copies in our working standard.
3.5 Modeling of SARS-CoV-2 assay conversion time points during early infection
When assuming a log-linear ramp up phase model with a daily 10-fold increase of viral load for the B.1 (Wuhan) type and 10,000 fold per day for the B.1.617.2 Delta variant (see methods) the time intervals between assay conversion during early infection were estimated from the 50% LODs for the NAT options and the LODs with indeterminate (±) reactivity for the antigen assays (Figure and Supplemental Table 4). Assuming an infectivity threshold at 1 TCID50/mL or around 1000 copies/mL in swab fluid (see above) the 50% NAT conversion points were estimated to be 40-66 hours earlier for the B.1 (Wuhan) strain. By contrast, according to our model the rapid antigen assays would become positive 41-48 hours later than this infectivity threshold and therefore have a higher probability to miss contagious individuals in an early phase of infection than NAT assays. If our assumption of approximately 4-fold faster replication of the Delta variant is correct [12] the time intervals between assay conversion points become four-fold shorter (Figure, Supplemental Table 4).
Figure 1: Modelling time point of conversion of different NAT and rapid antigen detection options during early ramp up phase of SARS-CoV-2 infection in relation to potential infectivity level of swab fluid in tissue culture. The mean viral doubling time (with wide confidence limits) was estimated at 7.2 hour based on viral growth curves of the original B.1 (Wuhan) type virus and estimated to be four-fold shorter (1.8 hour) after infection with the Delta variant.
4. Discussion
In the present study we directly compared the analytical sensitivity of four different NAT assays and three lateral flow devices for SARS CoV-2 detection on dilution series of a pool of swab fluid samples before and after inactivation by beta-propiolactone. The viral load in the native and inactivated material was quantified in PCR detectable RNA copies/mL by limiting dilution analysis and in IU/mL against the WHO International Standard. We estimated the viral load in culture fluid of known infectivity and found that 1 TCID50 was equivalent to approximately 1000 NAT detectable RNA copies (2700-4300 IU) in our working standard. We were surprised that the antigen concentration relative to the RNA concentration was 2.7 (range 2.5-3.0) fold higher in the inactivated standard than in the native standard. We speculate that the beta-propiolactone treatment has destroyed or modified subgenomic RNA fragments from human cells that were present in the pool of swab GLY samples, but had less impact on full length RNA genomes packaged in virions and on the antigenicity of the nucleocapsid or spike protein. For discussion on further details of the standardization in RNA copies or IUs in swab fluid before and after inactivation and the relation to TCID50 we refer to the Appendix.
On 1.5 fold dilutions of the native working standard the rapid antigen tests were found to be 28,000 to 56,000 fold less sensitive than the 50% LOD (CI) of 1.8 (1.0-3.3) copies/mL achieved by the most sensitive NAT assay in our comparison study, which was the cobas PCR assay. The Aptima TMA assay was 3.6 fold less sensitive than this PCR assay, whereas the SAMBA and LAMP assays were 8-12 fold less sensitive. Assuming a ramp up phase model for the viral load with one Log10 increase of viral load per day [12] we estimated that the Fluorecare, Abbott and Roche antigen assays were able to detect early viral replication 4-5 days later than the Roche cobas and Hologic Aptima NAT assays, whereas the cobas PCR in MP6 format, SAMBA and LAMP would be approximately 0.5-1 day slower in detecting early viral replication than the most sensitive NAT options. More importantly, we estimated that the NAT assays were able to detect early viral replication 40-66 hours before the estimated tissue culture infectivity conversion point at approximately 1000 copies/mL in the ramp up phase, whereas the antigen assays became positive 41-48 hours later. Since in early infection the effect of neutralizing antibodies does not yet play a role we assumed that a threshold of 1 TCID50/mL in swab samples could serve as a proxy for potential infectiousness by droplets and aerosols from infected individuals.
Rapid antigen tests have been promoted as an effective tool in preventing SARS-CoV-2 transmission since they detected the majority of PCR positive symptomatic individuals and it was assumed that the probability of contagiousness was highest in antigen positive individuals [1, 2]. It is possible that the airborne transmissibility of SARS-CoV-2 between humans in close contact starts at a viral load that is lower than the estimated tissue culture infectivity limit of 1000 copies/mL in swab fluid. Taking considerable uncertainty ranges in the exact conversion time points of assays and the start of contagiousness into account we believe that our modeling of analytical sensitivity data helps to understand what the effectiveness is of using different SARS-CoV-2 test options in timely identifying potentially contagious subjects.
Our modeling of the time points of assay conversion during early virus replication in the upper respiratory tract matches with evaluation data of laminar flow antigen detection methods in large clinical studies in the Netherlands [1, 2]. One of these studies [2] found a clinical sensitivity of 59% with two widely used rapid antigen tests when asymptomatic individuals were tested 5 days after potential exposure, whereas a higher proportion of 73-84% of the exposed subjects with symptoms were antigen reactive. These proportions are compatible with our model of early dynamics of viral load and estimated antigen conversion points as shown in the Figure. The calculated time points of SARS-CoV-2 RNA and antigen conversion in the different assays were based on best estimates of the viral growth curve of the B1 (Wuhan) type and the Delta variant observed in Guangdong province in China [12]. To our knowledge this is the only study in which the early dynamics of the viral load of infected individuals after exposure to the original Wuhan type and Delta variant was followed by daily PCR testing.
The Chinese investigators observed a 1000-fold higher rise in viral load per day for the Delta variant. We therefore assumed approximately 4-fold more rapid replication of the Delta variant in our ramp up phase model. If this assumption is correct the assay conversion points and infectivity levels in the ramp up phase would be reached 4 times earlier by the Delta variant than by the original wildtype virus. Since viral loads of the Delta variant in vaccine breakthrough infections were found to be comparable to those in unvaccinated individuals [15, 16] a model of 10,000-fold increase of viral load per day in the early ramp up phase may be generalizable to all early infected individuals by the Delta variant regardless of the vaccination status. We acknowledge that there are several limitations with our ramp up phase model but think that in an early phase of infection viral load increases exponentially (log-linear) and can serve as an indicator of potential infectiousness. One can imagine that the effectiveness of testing and isolation of infected individuals during outbreaks of the Delta variant were even more dependent on the sensitivity of the assay and the rapid turnaround time of the test results than in the past with the original virus. We recommend the use of NAT methods rather than antigen assays in settings where reliable identification of contagious individuals is essential. If the infection rate in the target population is low one can also make use of more cost-effective NAT screening using pooled swab samples [17]. For diagnostic testing of the more rapidly replicating Delta variant a faster NAT assay such as the SAMBA and LAMP assay seems to be more suitable. However, if one receives the test results not immediately at the testing site but several hours later, one could just as well opt for a laboratory that uses a more sensitive NAT method and receive the test result within 24 hours. Alternatively, one could also perform daily antigen self-tests after exposure to the more rapidly replicating Delta and Omicron variants. Currently the less pathogenic Omicron variant is the dominant strain [18] for which testing and isolation of infected individuals becomes less critical.
In conclusion, we estimated up to 56,000 fold differences in detection limits between several SARS-CoV-2 detection methods on a pool of swab fluid samples before inactivation. Translating these analytical sensitivity data to assay conversion time points using a ramp up phase model helps us to better understand what the effectiveness is of the different testing options in detecting early (asymptomatic) infection in potentially contagious individuals.
Appendix
Standardization details
In this standardization study we compared LODs of
different NAT methods with the detection endpoint titers of rapid antigen tests on 1.5 fold dilutions of a pool of swab fluid samples before and after inactivation by beta-propiolactone. These working standards were quantified in NAT detectable RNA copies/mL by limiting dilution analysis and in IU/mL by comparison against the WHO 20/146 standard, whereby one NAT detectable RNA copy by the cobas PCR assay in the inactivated working standard was found to be equivalent to 4.29 (4.44-5.36) IUs for the ORF1ab target and 2.68 (2.19-3.29) IUs for the E gene. Hence the IU/copy conversion factors in the cobas PCR assay were found to be 1.60 (1.57-1.63) higher for the ORF1ab gene than for the E gene. Similarly we found that the amount of E gene targets was reduced 1.64 (1.55-1.72) fold more than the amount of ORF1ab gene targets by treatment of the working standard with beta-propiolactone. We speculate that this slight but significant difference in recovery is caused by presence of unequal amounts of subgenomic RNA of the ORF1ab and E genes derived from human cells that were present in the swab fluid pool before inactivation. Another explanation-although less likely-is that beta-propiolactone renders more ORF1ab than E gene targets undetectable in the cobas PCR assay. It may be that the inactivation of the cell culture-derived WHO International Standard by acid and heat treatment acts differently than the chemical inactivation of our working standard. However, the relative amount of detectable genomic and subgenomic RNA of the ORF1ab and E genes in the WHO standard preparation before inactivation is unknown. It must be noted that the difference in Ct value between the ORF1ab and E gene targets on the USA-WA1-2020 culture fluid used for determining analytical sensitivity in the Roche cobas package insert was on average 2.42 as compared to a difference of 0.92 on our inactivated standard. In terms of a potency (or relative detectability) this is a factor 5.37 versus 1.88 for the two preparations. This suggests that there is still 2.8 fold more of E gene RNA in our inactivated working standard than in the USA-WA1-2020 culture fluid. Hence there are significant differences in the potency of ORF1ab and E genes in different standards according to quantification in the cobas SARS-CoV-2 assay. Our calibration in RNA copies/mL was based on assuming 100 (53-187) % NAT efficiency of the ORF1ab gene in the Roche cobas assay and 67 (42-102) % in the Hologic Aptima test. If the amount of IUs would be equal to the true amount of RNA molecules the NAT efficiency of the two assays would be 23 (12-44) % and 16 (10-24) % respectively.
When comparing the relationship between viral load and infectivity in WA1-2020 culture fluid we estimated that 1 TCID50 would be equivalent to 1186 (847-6098) RNA copies of the ORF1ab gene in our working standard. According to the certificate of analysis NR-52281 (BEI resources) 1 TCID50 of the USA WA1-2020 strain would be equivalent to 7393 RNA copies according to quantification in Droplet Digital PCR. Our estimate was not compatible with the 95% positive tissue culture infectivity limit of 2.17 x 105 copies/mL in clinical studies [1, 2] according to an in house calibration curve of the Erasmus University Medical Center (EUMC) [1]. This limit corresponded with a Roche cobas E gene Ct value ≤30. A Ct value of 25 corresponded with 4.87 x 106 E gene copies/mL according to EUMC [1], which in our calibration in PCR detectable RNA copies/mL was 26-fold lower and corresponded with a concentration of 186,000 RNA copies/mL. Hence, the EUMC 95% positive culture limit of 2.17 x 105 EUMC copies/mL may be equivalent to approximately 8300 copies/mL according to our calibration, which level seemed compatible with our estimate of a 50% tissue culture infectivity limit of approximately 1000 PCR detectable RNA copies/mL. We were surprised that the antigen concentration relative to the RNA concentration was found to be 2.7 (2.5-3.0) fold higher in the inactivated standard than in the native standard. We speculate that the beta-propiolactone treatment has destroyed or modified subgenomic RNA fragments from human cells that were present in the pool of swab GLY samples, but had less impact on full length RNA genomes packaged in virions and on the antigenicity of the nucleocapsid or spike protein. Additionally, antigen epitopes hidden in immune complexes may have been released by beta-propiolactone, although this seems less likely since the virus in the GLY-pool must have been dominated by swab samples with very high viral load from antibody negative (or low reactive) individuals. In conclusion, there seem to be significant differences between standards in the amount of SARS-CoV-2 RNA copies and IUs for different NAT gene targets, which balance is also affected by inactivation. Therefore assay detection limits as well as the infectious dose in tissue culture are dependent on the reference preparation used for calibration in RNA copies.
References
- Iglòi Z, Velzing J, van Beek J, et al. Clinical evaluation of the Roche SD Biosensor rapid antigen test for SARS-CoV-2 in a municipal health service testing site. The Netherlands. Emerging Infectious Diseases 27 (2021): 1323-1329.
- Schuit E, Veldhuizen IK, Venekamp RP, et al. Diagnostic accuracy of rapid antigen tests in asymptomatic and presymptomatic close contacts of individuals with confirmed SARS-CoV-2 infection: cross sectional study. BMJ 347 (2021): 1676.
- Lindner AK, Nikolai O, Kausch F, et al. Head to head comparison of SARS-CoV-2 antigen-detecting rapid test with self-collected nasal swab versus professional-collected nasopharyngeal swab. Eur Respir J 57 (2021): 2003961
- Berger A, Ngo Nsoga MT, Perez-Rodriguez FJ, et al. Diagnostic accuracy of two commercial SARS-CoV-2 antigen-detecting rapid tests at the point of care in community-based testing centers. PloS ONE 16 (2021): e0248921.
- Nordgren J, Sharma S, Olsson H, et al. SARS-CoV-2 rapid antigen test: High sensitivity to detect infectious virus. J Clin Virol 140 (2021): 104846.
- Mak GCN, Lau SSY, Wong KKY, et al.: Analytical sensitivity and clinical sensitivity of the three rapid antigen detection kits for detection of SARS-CoV-2 virus. J Clin Virol 133 (2020): 104684.
- Perchetti GA, Huang ML, Mills MG, et al. Analytical Sensitivity of the Abbott Binax NOW COVID-19 Ag CARD. J Clin Microbiol 59 (2021): e02880-20.
- Schleibhauer H, Filomena A, Nitsche A, et al. Comparative sensitivity evaluation for 122 CE-marked rapid diagnostic tests for SARS-CoV-2 antigen, Germany, September to April 2021. Euro Surveill 26 (2020): 2100441.
- Stephan W, Dichtelmuller H, Prince AM, et al. Inactivation of the Hutchinson strain of hepatitis non-A, non-B virus in intravenous immunoglobulin by β-propiolactone. J Med Virol 26 (1988): 227-232
- Scheidler A, Rokos K, Reuter T, Ebermann R, Pauli G. Inactivation of viruses by beta-propiolactone in human cryo poor plasma and IgG concentrates. Biologicals 26 (1998): 135-144.
- Assennato SM, Ritchie AV, Nadala C, et al. Performance evaluation of the SAMBA II SARS-CoV-2 test for point-of-care detection of SARS-CoV-2. J. Clin Microbiol 59 (2020): e01262-20.
- Li B, Deng A, Li K, et al. Viral infection and transmission in large, well-traced outbreak caused by SARS-CoV-2 Delta variant. Nature Communications 14 (2022): 460 .
- Mlchochova P, Kemp S, Dhar MS, et al. SARS-CoV-2 B.1.617.2 Delta variant replication and immune evasion. Nature 599 (2021): 114-119.
- Pilcher DP, Westreich D, Hudgens MG. Group testing for Severe Acute Respiratory Syndrome-Coronavirus 2 to enable rapid scale-up of testing and real-time surveillance of incidence. JID (2020): 222
- Chia PY, Ong SWX, Chiew CJ, et al. Virological and serological kinetics of SARS-CoV-2 Delta variant vaccine-breakthrough infections: a multi-center cohort study. Clin Microbiol Infect 28 (2022): 612.
- Brown CM, Vostok J, Johnson H, et al. Outbreak of SARS-CoV-2 infections, including COVID-19 vaccine breakthrough infections, associated with large public gatherings – Barnstable County, Massachusetts, July 2021. MMWR 70 (2021): 31.
- Heaney K, Ritchie AV, Henry R, et al. Evaluation of sample pooling using the SAMBA II SARS-CoV-2 Test, Journal of Virological Methods (2021).
- Puhach O, Adea K, Hulo N, et al. Infectious viral load in unvaccinated and vaccinated individuals infected with ancestral Delta or Omicron SARS-CoV-2. Nature Medicine Epub (2022).
Supplemental Tables
Supplemental Table 1: Ct values in Roche cobas SARS-CoV-2 RNA test on standard dilutions before and after inactivation by beta-propiolactone.
Standard |
A-series |
B-series |
|||
ORF1ab |
E gene |
ORF1ab |
E gene |
||
Viral state |
Dilution |
Ct |
Ct |
Ct |
Ct |
Native |
10 |
21.95 |
22.1 |
21.78 |
21.96 |
Native |
100 |
25.05 |
25.13 |
24.91 |
25.18 |
Native |
1000 |
27.87 |
28.2 |
28.09 |
28.32 |
Inactivated |
10 |
23.54 |
24.55 |
23.77 |
24.67 |
Inactivated |
100 |
26.93 |
27.87 |
26.95 |
27.86 |
Inactivated |
1000 |
30.15 |
30.99 |
30.04 |
30.95 |
Supplemental Table 2: Estimation of SARS-CoV-2 RNA concentration in inactivated standard based on Poisson distribution assuming 100% NAT efficiency of most sensitive assay.
Assay |
Volume amplified/ input volume in assay µL |
n^ |
Recalibrated 63% LOD copies/mL (95% CI) |
relative sensitivity factor (95% CI) |
NAT efficiency (95% CI)# |
cobas PCR target 1 [ORF1ab] |
400/400 µL |
8 |
2.5 (1.33-4.73) |
1.00 (reference) |
100% (53-187)% |
cobas PCR target 2 [E] |
400/400 µL |
8 |
4.67 (2.66-8.52) |
0.53 (0.31-0.93) |
54% (29-94)% |
Aptima TMA |
167/500 µL |
14 |
8.95 (5.88-14.2) |
0.28 (0.19-0.42) |
67% (42-102)% |
^ number of replicates tested per member of P0356 SARS CoV-2 standard dilution panel # adjusted for amplification volume and assuming 100% efficiency of the Roche cobas assay for target 1 (ORF1ab) |
Supplemental Table 3a: Ct values on WHO and BioQ SARS-CoV-2 working standard dilution series in Roche cobas PCR assay.
WHO 20/146 |
Series A |
Series B |
||
ORF1ab |
E |
ORF1ab |
E |
|
IU/mL |
Ct1 |
Ct2 |
Ct1 |
Ct2 |
100000 |
27.63 |
27.78 |
27.31 |
27.38 |
30000 |
29.16 |
29.26 |
29.43 |
29.61 |
10000 |
30.71 |
30.82 |
30.39 |
30.61 |
3000 |
32.2 |
32.61 |
33.07 |
33.05 |
1000 |
32.94 |
33.37 |
33.27 |
33.77 |
BioQ inactivated |
ORF1ab |
E |
ORF1ab |
E |
RNA copies/mL |
Ct1 |
Ct2 |
Ct1 |
Ct2 |
33784 |
26.94 |
27.75 |
26.54 |
27.3 |
11249 |
28.74 |
29.6 |
28.78 |
29.68 |
3378 |
30.34 |
31.25 |
30.09 |
31.02 |
1125 |
31.75 |
32.75 |
31.92 |
32.56 |
338 |
33.17 |
34.03 |
33.26 |
34.21 |
Supplemental Table 3b: Potency of inactivated working standard against WHO 20/146 standard in Roche cobas SARS-CoV-2 PCR assay.
Target |
Parameter |
Value (95% CI) |
ORF1ab |
delta Ct |
2.10 (1.78-2.42) |
IU/copy |
4.29 (3.44-5.36) |
|
E |
delta Ct |
1.42 (1.13-1.72) |
IU/copy |
2.68 (2.19-3.29) |
Supplemental Table 4: Estimated assay conversion times relative to the most sensitive assay, i.e. cobas PCR (Table 4a) and relative to an assumed infectivity threshold of 1000 NAT detectable RNA copies/mL equivalent to 1 TCID50 (Table 4b) during ramp up phase of viremia of B.1 (Wuhan) type SARS-CoV-2 virus and the B.1.617.2 (delta) variant assuming a 10-fold and 10,000 fold daily rise of viral load respectively.
Supplemental Table 4a.
Assay |
50% LOD copies/mL^ |
B.1 (Wuhan) type |
B1.617.2 (Delta) variant |
|
days |
hours |
hours |
||
cobas PCR |
1.8 |
0 |
0 |
0 |
Aptima TMA |
6.6 |
0.56 |
13.4 |
3.3 |
cobas MP6 PCR |
11.1 |
0.78 |
18.7 |
4.7 |
SAMBA |
15 |
0.91 |
21.8 |
5.4 |
LAMP |
22.8 |
1.09 |
26.2 |
6.6 |
1 TCID50/mL |
1000 |
2.74 |
65.9 |
16.5 |
Fluorecare antigen |
50000 |
4.44 |
106.5 |
26.6 |
Panbio antigen |
75000 |
4.61 |
110.7 |
27.7 |
Roche antigen |
100000 |
4.74 |
113.7 |
28.4 |
^50% NAT LODs or ± antigen LODs on native standard before inactivation |
Supplemental Table 4b.
Assay |
50% LOD copies/mL^ |
B.1 (Wuhan) type |
B.1.617.2 (Delta) variant |
|
days |
hours |
hours |
||
cobas PCR |
1.8 |
-2.74 |
-65.9 |
-16.5 |
Aptima TMA |
6.6 |
-2.19 |
-52.5 |
-13.1 |
cobas MP6 PCR |
11.1 |
-1.97 |
-47.2 |
-11.8 |
SAMBA |
15 |
-1.84 |
-44.1 |
-11 |
LAMP |
22.8 |
-1.65 |
-39.6 |
-9.9 |
1 TCID50/mL |
1000 |
0 |
0 |
0 |
Fluorecare antigen |
50000 |
1.69 |
40.6 |
10.2 |
Panbio antigen |
75000 |
1.87 |
44.8 |
11.2 |
Roche antigen |
100000 |
1.99 |
47.8 |
12 |
^50% NAT LODs or ± antigen LODs on native standard before inactivation |