Legal Brief: Facial Recognition Makes Steady Progress

This article originally appeared in the January 2021 issue of Security Business magazine. When sharing, don’t forget to mention @SecBusinessMag on Twitter and Security Business magazine on LinkedIn.

When I had the honor of serving as an officer in the United States Force, I was stationed in Japan for my first two years, in 1997-1999. As I traveled the country and met many wonderful people, it was not unusual to see people wearing masks from time to time. My understanding then and today was masks were worn to protect the wearer and others against contagions.

The United States does not have as long a history with masks as does Japan. Rather, widespread mask use did not occur until very recently.

While it appears that a vaccine is on its way to mass adoption, mask use in the U.S. (and certainly in Japan) will continue to some degree beyond this pandemic. It is for this reason that ongoing studies of the use of facial recognition technologies on masked faces remains of great importance to the security industry; and don’t forget that bad guys sometimes wear masks too.

In my July 2020 column (www.securityinfowatch.com/21143109), I wrote about a landmark study published by the National Institute of Standards and Technology (NIST) alleging bias in facial recognition algorithms. A second NIST study assessed the performance of facial recognition algorithms on faces partially covered by masks (www.securityinfowatch.com/21151102). The results of the second NIST study were not favorable. Error rates were high; however, among other issues with that study, test samples were pre-pandemic technologies and not every software vendor participated.

NIST recently updated the study using newer technologies and published in Dec. 2020. The “Ongoing Face Recognition Vendor Test (FRVT) Part 6B: Face Recognition Accuracy with Face Masks Using Post-COVID-19 Algorithms” test included 65 algorithms that were not a part of the original study, bringing the total number to 152.

Improvements Made by Vendors

Unsurprisingly, the latest NIST study confirms that updated technologies are much better at recognizing masked faces. The technologies tested previously were never designed to detect faces covered by masks, so developers had to adapt – and it appears they have.

The older software was designed to measure the lower areas of the face, while the newer generation of software is designed to map key points from only the eye and nose area. This has led to – in some cases – a dramatic reduction in error rates; in fact, some error rates decreased by as much as a factor of ten between the two NIST studies. The best error rates – for the most accurate technologies – ranged between 2.4% and 5%, which is comparable to tests conducted in 2017 on unmasked photos.

As with its earlier study, NIST researchers applied digitally generated face masks over photographs and performed one-to-one matching. This where images are compared to stored photos of the same people. Error rates would likely be higher if the images were compared by a one-to-many method, such as with software used for crowd surveillance. NIST intends to evaluate the algorithms’ one-to-many capabilities in future rounds of mask testing.

The study found that, when both the new image and the stored image are of masked faces, error rates run higher. Understandably, fewer facial features yield fewer points of comparison. Interestingly, the color of the masks appear to make a difference, with red and black masks yielding higher error rates than other colors (this was true in the prior NIST study as well). Unsurprisingly, larger masks that obscure more facial features result in higher error rates than smaller masks.

While NIST’s ongoing studies are of great value in progressing and measuring these technologies, not all technologies are the same. Some algorithms performed well with any combination of masked or unmasked faces, but others did not.

As an integrator, if you are considering deploying this technology, it is critical to learn how the chosen software performs in specific circumstances, and with your customer’s actual data.

Timothy J. Pastore, Esq., is a Partner in the New York office of Montgomery McCracken Walker & Rhoads LLP (www.mmwr.com), where he is Vice-Chair of the Litigation Department. Before entering private practice, Mr. Pastore was an officer and Judge Advocate General (JAG) in the U.S. Air Force and a Special Assistant U.S. Attorney with the U.S. Department of Justice. Reach him at (212) 551-7707 or by e-mail at [email protected].