Reply to thread

Message

<blockquote data-quote="tomBitonti" data-source="post: 8103171" data-attributes="member: 13107">Background: I was reading the CNN Corona virus live stream today (06-Oct-2020), and read the following:&quot;Coronavirus testing in the US is still lagging, but Fauci says we're better off than we were a few months ago&quot;, from CNN’s Health Shelby Lin Erdman[URL unfurl=&quot;true&quot;]https://www.cnn.com/world/live-news/coronavirus-pandemic-10-06-20-intl/h_4502fde88e4fcce635ed840f91be1d14[/URL]The article continues:I was interested in how those accuracy numbers translate into false negative and false positive test results.First, there is a question of the meaning the &quot;accuracy&quot; percentages.&nbsp; I found this, which indicates that the numbers are likely &quot;positive percentage agreement&quot; (PPA) and &quot;negative percentage agreement&quot; (NPA) values, since the numbers very closely match this published chart of test results:&quot;Abbott rides to the rescue with $5 Covid-19 test&quot;[URL unfurl=&quot;true&quot;]https://www.evaluate.com/vantage/articles/news/policy-and-regulation/abbott-rides-rescue-5-covid-19-test[/URL]&quot;Abbott reports positive and negative percent agreement&quot;&quot;Accuracy of FDA-authorised antigen tests&quot;<table style='width: 100%'><tr><th>Date of EUA</th><th>Company</th><th>Test</th><th>PPA</th><th>NPA</th><th>Size*</th></tr><tr><td>Aug 26</td><td>Abbott</td><td>BinaxNow</td><td>97.1%</td><td>98.5%</td><td>102</td></tr><tr><td>Aug 18</td><td>LumiraDx</td><td>LumiraDx</td><td>97.6%</td><td>96.6%</td><td>257</td></tr><tr><td>Jul 2</td><td>Beckton Dickenson</td><td>BD Veritor</td><td>84%</td><td>100%</td><td>226</td></tr><tr><td>May 8</td><td>Quidel</td><td>Sofia Sars</td><td>96.7%</td><td>100%</td><td>209</td></tr></table>* Suspected positive sample sizeTo understand what &quot;positive percentage agreement&quot; and &quot;negative percentage agreement&quot; mean, I found the following:&quot;Statistical Guidance on Reporting Results from Studies Evaluating Diagnostic Tests&quot;[URL unfurl=&quot;true&quot;]https://www.fda.gov/media/71147/download[/URL]And the answer is ... complicated.&nbsp; PPA and NPA are measure of how well a test results match a non-standard reference test.&nbsp; Without knowing the accuracy of the reference test, understanding the Abbott test results is difficult.&nbsp; If the reference test was 100% accurate, then the PPA and NPA are percentage measurements of sensitivity and specificity.&nbsp; But, if the reference test is less than 100% accurrate, the PPA and NPA may indicate a lesser or a greater accuracy.&nbsp; For example, the test which is being compared to the reference test might agree with the reference test where the reference test is accurate, and dis-agree where the references test is inaccurate.The FDA has this to say:In any case, labeling PPA and NPA as &quot;accuracy&quot; seems quite incorrect.As an aside, two other values which are described by the FDA text are &quot;sensitivity&quot; and &quot;specificity&quot;:<ul>
<li data-xf-list-type="ul">Sensitivity refers to how often the test is positive when the condition of interest is present</li>
<li data-xf-list-type="ul">Specificity refers to how often the test is negative when the condition of interest is absent</li>
</ul>Although of only small value -- given the uncertainty of the meaning of the Abbot PPA and NPA values, I took the values as measuring &quot;accuracy&quot; and determined the proportion of false results for different virus distributions.&nbsp; That is, for populations with 90%, 50%, 10%, 1%, 0.1%, and 0.01% infected individuals:Populations descriptions:<table style='width: 100%'><tr><th></th><th>Percentage</th><th>Proportion</th><th>Number of Persons</th></tr><tr><td>Uninfected</td><td>10%</td><td> 1/10</td><td>100,000</td></tr><tr><td>Infected</td><td>90%</td><td> 9/10</td><td>900,000</td></tr><tr><td>Total</td><td>100%</td><td>10/10</td><td>1,000,000</td></tr><tr><td></td><td></td><td></td><td></td></tr><tr><td>Uninfected</td><td>50%</td><td> 5/10</td><td>500,000</td></tr><tr><td>Infected</td><td>50%</td><td> 5/10</td><td>500,000</td></tr><tr><td>Total</td><td>100%</td><td>10/10</td><td>1,000,000</td></tr><tr><td></td><td></td><td></td><td></td></tr><tr><td>Uninfected</td><td>90%</td><td> 9/10</td><td>900,000</td></tr><tr><td>Infected</td><td>10%</td><td> 1/10</td><td>100,000</td></tr><tr><td>Total</td><td>100%</td><td>10/10</td><td>1,000,000</td></tr><tr><td></td><td></td><td></td><td></td></tr><tr><td>Uninfected</td><td>99%</td><td> 99/100</td><td>990,000</td></tr><tr><td>Infected</td><td>1%</td><td>&nbsp; 1/100</td><td>10,000</td></tr><tr><td>Total</td><td>100%</td><td>100/100</td><td>1,000,000</td></tr><tr><td></td><td></td><td></td><td></td></tr><tr><td>Uninfected</td><td>99.9%</td><td> 999/1000</td><td>999,000</td></tr><tr><td>Infected</td><td>0.1%</td><td>&nbsp;&nbsp; 1/1000</td><td>1,000</td></tr><tr><td>Total</td><td>100.0%</td><td>1000/1000</td><td>1,000,000</td></tr><tr><td></td><td></td><td></td><td></td></tr><tr><td>Uninfected</td><td>99.99%</td><td> 9999/10000</td><td>999,900</td></tr><tr><td>Infected</td><td>0.01%</td><td>&nbsp; &nbsp; 1/10000</td><td>100</td></tr><tr><td>Total</td><td>100.00%</td><td>10000/10000</td><td>1,000,000</td></tr></table>Working through the numbers, I obtained the following test accuracy results.&nbsp; Here &quot;accuracy&quot; means &quot;Upon obtaining a test result, what is the chance that that result is correct?&quot;Unsurprisingly, as the proportion of the population which is infected drops, the number of false positives dominates the &quot;infected&quot; results, leading to increasingly worse accuracy.<table style='width: 100%'><tr><th>Population Percent Infected</th><th>Percent True Uninfected Results</th><th>Percent True Infected Results</th></tr><tr><td>90%</td><td>87.8%</td><td>99.7%</td></tr><tr><td>50%</td><td>98.5%</td><td>97.0%</td></tr><tr><td>10%</td><td>99.8%</td><td>78.5%</td></tr><tr><td>1%</td><td>99.98%</td><td>24.9%</td></tr><tr><td>0.1%</td><td>99.998%</td><td>3.1%</td></tr><tr><td>0.01%</td><td>99.9998%</td><td>0.327%</td></tr></table>Calculations:[CODE]Uninfected:&nbsp; &nbsp;&nbsp; 10%&nbsp; &nbsp; &nbsp;&nbsp; ( 1/10)&nbsp;&nbsp; 100,000&nbsp; U&nbsp; 97,000 I&nbsp;&nbsp; 3,000Infected:&nbsp; &nbsp; &nbsp;&nbsp; 90%&nbsp; &nbsp; &nbsp;&nbsp; ( 9/10)&nbsp;&nbsp; 900,000&nbsp; U&nbsp; 13,500 I 886,500Total:&nbsp; &nbsp; &nbsp; &nbsp;&nbsp; 100%&nbsp; &nbsp; &nbsp;&nbsp; (10/10) 1,000,000&nbsp; U 110,500 I 889,500 (U 87.8% I 99.7%)Uninfected:&nbsp; &nbsp;&nbsp; 50%&nbsp; &nbsp; &nbsp;&nbsp; ( 5/10)&nbsp;&nbsp; 500,000&nbsp; U 485,000 I&nbsp; 15,000Infected:&nbsp; &nbsp; &nbsp;&nbsp; 50%&nbsp; &nbsp; &nbsp;&nbsp; ( 5/10)&nbsp;&nbsp; 500,000&nbsp; U&nbsp;&nbsp; 7k500 I 492,500Total:&nbsp; &nbsp; &nbsp; &nbsp;&nbsp; 100%&nbsp; &nbsp; &nbsp;&nbsp; (10/10) 1,000,000&nbsp; U 492k500 I 507,500 (U 98.5% I 97.0%)Uninfected:&nbsp; &nbsp;&nbsp; 90%&nbsp; &nbsp; &nbsp;&nbsp; ( 9/10)&nbsp;&nbsp; 900,000&nbsp; U 873,000 I&nbsp; 27,000Infected:&nbsp; &nbsp; &nbsp;&nbsp; 10%&nbsp; &nbsp; &nbsp;&nbsp; ( 1/10)&nbsp;&nbsp; 100,000&nbsp; U&nbsp;&nbsp; 1,500 I&nbsp; 98,500Total:&nbsp; &nbsp; &nbsp; &nbsp;&nbsp; 100%&nbsp; &nbsp; &nbsp;&nbsp; (10/10) 1,000,000&nbsp; U 874,500 I 125,500 (U 99.8% I 78.5%)Uninfected:&nbsp; &nbsp;&nbsp; 99%&nbsp; &nbsp;&nbsp; ( 99/100)&nbsp;&nbsp; 990,000&nbsp; U 960,300 I&nbsp; 29,700Infected:&nbsp; &nbsp; &nbsp; &nbsp; 1%&nbsp; &nbsp;&nbsp; (&nbsp; 1/100)&nbsp; &nbsp; 10,000&nbsp; U&nbsp; &nbsp;&nbsp; 150 I&nbsp;&nbsp; 9,850Total:&nbsp; &nbsp; &nbsp; &nbsp;&nbsp; 100%&nbsp; &nbsp;&nbsp; (100/100) 1,000,000&nbsp; U 960,450 I&nbsp; 39,550 (U 99.98% I 24.9%)Uninfected:&nbsp;&nbsp; 99.9%&nbsp;&nbsp; ( 999/1000)&nbsp;&nbsp; 999,000&nbsp; U 969,030 I&nbsp; 29,970Infected:&nbsp; &nbsp; &nbsp; 0.1%&nbsp;&nbsp; (&nbsp;&nbsp; 1/1000)&nbsp; &nbsp;&nbsp; 1,000&nbsp; U&nbsp; &nbsp; &nbsp; 15 I&nbsp; &nbsp;&nbsp; 985Total:&nbsp; &nbsp; &nbsp;&nbsp; 100.0%&nbsp;&nbsp; (1000/1000) 1,000,000&nbsp; U 969,045 I&nbsp; 30,955 (U 99.998% I 3.1%)Uninfected:&nbsp; 99.99% ( 9999/10000)&nbsp;&nbsp; 999,900&nbsp; U 969,903.0 I 29,997.0Infected:&nbsp; &nbsp;&nbsp; 0.01% (&nbsp; &nbsp; 1/10000)&nbsp; &nbsp; &nbsp;&nbsp; 100&nbsp; U&nbsp; &nbsp; &nbsp;&nbsp; 1.5 I&nbsp; &nbsp;&nbsp; 98.5Total:&nbsp; &nbsp; &nbsp; 100.00% (10000/10000) 1,000,000&nbsp; U 969,904.5 I 30,095.5 (U 99.9998% I 0.327%)[/CODE]Any errors in the above are my own.Tom Bitonti</blockquote>

[QUOTE="tomBitonti, post: 8103171, member: 13107"] Background: I was reading the CNN Corona virus live stream today (06-Oct-2020), and read the following: "Coronavirus testing in the US is still lagging, but Fauci says we're better off than we were a few months ago", from CNN’s Health Shelby Lin Erdman [URL unfurl="true"]https://www.cnn.com/world/live-news/coronavirus-pandemic-10-06-20-intl/h_4502fde88e4fcce635ed840f91be1d14[/URL] The article continues: I was interested in how those accuracy numbers translate into false negative and false positive test results. First, there is a question of the meaning the "accuracy" percentages. I found this, which indicates that the numbers are likely "positive percentage agreement" (PPA) and "negative percentage agreement" (NPA) values, since the numbers very closely match this published chart of test results: "Abbott rides to the rescue with $5 Covid-19 test" [URL unfurl="true"]https://www.evaluate.com/vantage/articles/news/policy-and-regulation/abbott-rides-rescue-5-covid-19-test[/URL] "Abbott reports positive and negative percent agreement" "Accuracy of FDA-authorised antigen tests" [TABLE] [TR][TH]Date of EUA[/TH][TH]Company[/TH][TH]Test[/TH][TH]PPA[/TH][TH]NPA[/TH][TH]Size*[/TH][/TR] [TR][TD]Aug 26[/TD][TD]Abbott[/TD][TD]BinaxNow[/TD][TD][B]97.1%[/B][/TD][TD][B]98.5%[/B][/TD][TD]102[/TD][/TR] [TR][TD]Aug 18[/TD][TD]LumiraDx[/TD][TD]LumiraDx[/TD][TD]97.6%[/TD][TD]96.6%[/TD][TD]257[/TD][/TR] [TR][TD]Jul 2[/TD][TD]Beckton Dickenson[/TD][TD]BD Veritor[/TD][TD]84%[/TD][TD]100%[/TD][TD]226[/TD][/TR] [TR][TD]May 8[/TD][TD]Quidel[/TD][TD]Sofia Sars[/TD][TD]96.7%[/TD][TD]100%[/TD][TD]209[/TD][/TR] [/TABLE] * Suspected positive sample size To understand what "positive percentage agreement" and "negative percentage agreement" mean, I found the following: "Statistical Guidance on Reporting Results from Studies Evaluating Diagnostic Tests" [URL unfurl="true"]https://www.fda.gov/media/71147/download[/URL] And the answer is ... complicated. PPA and NPA are measure of how well a test results match a non-standard reference test. Without knowing the accuracy of the reference test, understanding the Abbott test results is difficult. If the reference test was 100% accurate, then the PPA and NPA are percentage measurements of sensitivity and specificity. But, if the reference test is less than 100% accurrate, the PPA and NPA may indicate a lesser or a greater accuracy. For example, the test which is being compared to the reference test might agree with the reference test where the reference test is accurate, and dis-agree where the references test is inaccurate. The FDA has this to say: In any case, labeling PPA and NPA as "accuracy" seems quite incorrect. As an aside, two other values which are described by the FDA text are "sensitivity" and "specificity": [LIST] [*]Sensitivity refers to how often the test is positive when the condition of interest is present [*]Specificity refers to how often the test is negative when the condition of interest is absent [/LIST] Although of only small value -- given the uncertainty of the meaning of the Abbot PPA and NPA values, I took the values as measuring "accuracy" and determined the proportion of false results for different virus distributions. That is, for populations with 90%, 50%, 10%, 1%, 0.1%, and 0.01% infected individuals: Populations descriptions: [TABLE] [TR][TH][/TH][TH]Percentage[/TH][TH]Proportion[/TH][TH]Number of Persons[/TH][/TR] [TR][TD]Uninfected[/TD] [TD]10%[/TD] [TD] 1/10[/TD] [TD]100,000[/TD][/TR] [TR][TD]Infected[/TD] [TD]90%[/TD] [TD] 9/10[/TD] [TD]900,000[/TD][/TR] [TR][TD]Total[/TD] [TD]100%[/TD] [TD]10/10[/TD] [TD]1,000,000[/TD][/TR] [TR][/TR] [TR][TD]Uninfected[/TD] [TD]50%[/TD] [TD] 5/10[/TD] [TD]500,000[/TD][/TR] [TR][TD]Infected[/TD] [TD]50%[/TD] [TD] 5/10[/TD] [TD]500,000[/TD][/TR] [TR][TD]Total[/TD] [TD]100%[/TD] [TD]10/10[/TD] [TD]1,000,000[/TD][/TR] [TR][/TR] [TR][TD]Uninfected[/TD] [TD]90%[/TD] [TD] 9/10[/TD] [TD]900,000[/TD][/TR] [TR][TD]Infected[/TD] [TD]10%[/TD] [TD] 1/10[/TD] [TD]100,000[/TD][/TR] [TR][TD]Total[/TD] [TD]100%[/TD] [TD]10/10[/TD] [TD]1,000,000[/TD][/TR] [TR][/TR] [TR][TD]Uninfected[/TD] [TD]99%[/TD] [TD] 99/100[/TD] [TD]990,000[/TD][/TR] [TR][TD]Infected[/TD] [TD]1%[/TD] [TD] 1/100[/TD] [TD]10,000[/TD][/TR] [TR][TD]Total[/TD] [TD]100%[/TD] [TD]100/100[/TD] [TD]1,000,000[/TD][/TR] [TR][/TR] [TR][TD]Uninfected[/TD] [TD]99.9%[/TD] [TD] 999/1000[/TD] [TD]999,000[/TD][/TR] [TR][TD]Infected[/TD] [TD]0.1%[/TD] [TD] 1/1000[/TD] [TD]1,000[/TD][/TR] [TR][TD]Total[/TD] [TD]100.0%[/TD] [TD]1000/1000[/TD] [TD]1,000,000[/TD][/TR] [TR][/TR] [TR][TD]Uninfected[/TD] [TD]99.99%[/TD] [TD] 9999/10000[/TD] [TD]999,900[/TD][/TR] [TR][TD]Infected[/TD] [TD]0.01%[/TD] [TD] 1/10000[/TD] [TD]100[/TD][/TR] [TR][TD]Total[/TD] [TD]100.00%[/TD] [TD]10000/10000[/TD] [TD]1,000,000[/TD][/TR] [/TABLE] Working through the numbers, I obtained the following test accuracy results. Here "accuracy" means "Upon obtaining a test result, what is the chance that that result is correct?" Unsurprisingly, as the proportion of the population which is infected drops, the number of false positives dominates the "infected" results, leading to increasingly worse accuracy. [TABLE] [TR] [TH]Population Percent Infected[/TH] [TH]Percent True Uninfected Results[/TH] [TH]Percent True Infected Results[/TH] [/TR] [TR][TD]90%[/TD][TD]87.8%[/TD][TD]99.7%[/TD][/TR] [TR][TD]50%[/TD][TD]98.5%[/TD][TD]97.0%[/TD][/TR] [TR][TD]10%[/TD][TD]99.8%[/TD][TD]78.5%[/TD][/TR] [TR][TD]1%[/TD][TD]99.98%[/TD][TD]24.9%[/TD][/TR] [TR][TD]0.1%[/TD][TD]99.998%[/TD][TD]3.1%[/TD][/TR] [TR][TD]0.01%[/TD][TD]99.9998%[/TD][TD]0.327%[/TD][/TR] [/TABLE] Calculations: [CODE] Uninfected: 10% ( 1/10) 100,000 U 97,000 I 3,000 Infected: 90% ( 9/10) 900,000 U 13,500 I 886,500 Total: 100% (10/10) 1,000,000 U 110,500 I 889,500 (U 87.8% I 99.7%) Uninfected: 50% ( 5/10) 500,000 U 485,000 I 15,000 Infected: 50% ( 5/10) 500,000 U 7k500 I 492,500 Total: 100% (10/10) 1,000,000 U 492k500 I 507,500 (U 98.5% I 97.0%) Uninfected: 90% ( 9/10) 900,000 U 873,000 I 27,000 Infected: 10% ( 1/10) 100,000 U 1,500 I 98,500 Total: 100% (10/10) 1,000,000 U 874,500 I 125,500 (U 99.8% I 78.5%) Uninfected: 99% ( 99/100) 990,000 U 960,300 I 29,700 Infected: 1% ( 1/100) 10,000 U 150 I 9,850 Total: 100% (100/100) 1,000,000 U 960,450 I 39,550 (U 99.98% I 24.9%) Uninfected: 99.9% ( 999/1000) 999,000 U 969,030 I 29,970 Infected: 0.1% ( 1/1000) 1,000 U 15 I 985 Total: 100.0% (1000/1000) 1,000,000 U 969,045 I 30,955 (U 99.998% I 3.1%) Uninfected: 99.99% ( 9999/10000) 999,900 U 969,903.0 I 29,997.0 Infected: 0.01% ( 1/10000) 100 U 1.5 I 98.5 Total: 100.00% (10000/10000) 1,000,000 U 969,904.5 I 30,095.5 (U 99.9998% I 0.327%) [/CODE] Any errors in the above are my own. Tom Bitonti [/QUOTE]

Verification