A claim of 99% sensitivity sounds good, and is often achievable. Any real system will also have false positives, so let’s say that we have a specificity (test true negative rate) of 99% as well. This is probably unrealistically good, most systems will false positive more. This sounds great.
However, Bayes’ theorem paints a very different picture.
If the prevalence of wanted criminals in the population is say 1/10000 (this is hard to guess), what are the odds that a person that is flagged is a wanted criminal?
The unintuitive answer is less than 1% of the time (~0.98%) will the suspect actually be a criminal.
By far the most important term is the prevalence of the thing you are testing for in the population, in this case criminality. Any dragnet facial recognition is invariably going to get more innocent people caught in its web than true criminals.
However, Bayes’ theorem paints a very different picture.
If the prevalence of wanted criminals in the population is say 1/10000 (this is hard to guess), what are the odds that a person that is flagged is a wanted criminal?
The unintuitive answer is less than 1% of the time (~0.98%) will the suspect actually be a criminal.
By far the most important term is the prevalence of the thing you are testing for in the population, in this case criminality. Any dragnet facial recognition is invariably going to get more innocent people caught in its web than true criminals.