By continuing to use this site, you agree to our updated Privacy Policy and Terms of Use. ×

Privacy Considerations for Choosing a Facial Recognition System

19th January, 2019 | Biometric Privacy | Entropic

Photo by Atharva Tulsi on Unsplash

While Amazon, Facebook, Apple, Google, Microsoft, and of course the Chinese and US governments tend to get the most media attention these days when it comes to facial recognition, there are actually now more than 100 companies across the globe, along with many academic institutions who are actively engaged in developing and/or selling facial recognition technologies.

If you are tasked with the responsibility of evaluating or implementing some type of facial recognition, then you most likely have heard that the U.S. Government National Institute of Standards and Technology (NIST) maintains an ongoing facial recognition vendor test for vendors that wish to benchmark their performance against their peers in the industry, and build brand visibility.

While this industry benchmark is intended to convey performance of these vendors, consumers of this information should understand that this is not a U.S. government endorsement of these vendors from a privacy or security perspective. With the slow response that governments typically have to enacting laws to protect individuals from the misuse of emerging technology, it's basically up to the entities who are developing and implementing facial recognition systems, to be responsible when handling the information of others, and to find resources to assist them in this process.

This is also why support from the greater academic community, such as the Georgetown University Center on Privacy and Technology, the Face Recognition Homepage maintained by researchers at the University of Zagreb, Croatia, and other independent groups such as the Electronic Frontier Foundation, and the Future of Privacy Forum, who regularly publish their privacy principles for facial recognition technology-in-commercial-applications guide, is so important.

Thankfully, some of these groups are taking a more balanced, holistic view of the evolution of facial recognition, vs. just from a technical/performance standpoint, so that the implications of these biometric technologies are understood, and not neglected as they evolve.

In this article we'll focus on some guidelines that cover privacy considerations when evaluating vendors facial recognition products and services, which can be a precarious exercise when considering the potentially sensitive nature of any photos that you might be arbitrarily using as an evaluator, let alone the potentially high volume of photos that are used in your final implementation.

For this reason, we decided to compile a list of privacy considerations that will hopefully help you make better decisions about selecting a facial recognition vendor.


While a greater set of evaluation guidelines will also consider factors such as recognition performance, platform requirements, cost, etc.., these guidelines focus on privacy only - preventing the unintentional loss of people's personal information while the systems are in use, including the immutable biometric data of individuals.

1 - Read the Vendor's Privacy Policy

Though this is definitely one of the less inspiring aspects of evaluating a facial recognition vendor, it can save you an awful lot of time otherwise wasted in research.

A privacy policy not only reveals how a vendor handles the biometric information you submit to them and who they share it with both internally and externally, but also how diverse their organization is, their partners and affiliates, the different types of technologies they offer, and in general their overall understanding and assumptions about biometric privacy, which can vary depending on factors such as company and national culture.

The problem is that privacy policies are inherently boring to read. So, one good strategy is to start reading it in the middle, then pick random paragraphs north and south of the document. Sounds crazy, but if the alternative is you falling asleep, or just giving up after the first few paragraphs, this approach is more likely to help you find the most useful pieces of the document faster.

Important: If you view a vendors "privacy policy" and it turns out to be an End User License Agreement, or some other unrelated legal document, then the warning bells should start sounding in your mind about their overall understanding of the meaning of privacy. Yes, these types of vendors are out there ;)

2 - Don't Use Cloud-based Facial Recognition Systems

Cloud-based facial recognition systems - ones that require an active connection to the Internet to function, are simply a bad idea, since many of them are amassing the photos and videos that are being submitted when you use the service, and these can then be unknowingly shared with, or sold off to third parties, including other companies and governments with more nefarious intentions. Additionally, the use of a centralized storage approach makes their services "high value targets" for cybercriminals and nation states.

A common response for vendors providing cloud-based services, who are questioned about the transfer of their customers data is "But, we have servers based in your country". While facial recognition vendors might claim to base their services and customer data storage in your organizations country of origin, in many cases these services merely act as a collection point for customer information, prior to transfer to the vendors home country.

As an example, Amazon S3 services are commonly exploited by some foreign companies in more geopolitically aggressive countries, as a data collection point prior to transferring data to their home country. This helps avoid being flagged by corporate firewalls, along with the resulting bad press, should their products attempt to transmit data directly back to their home country.

In a nutshell, by using a cloud-based service, you are giving the vendor the immutable biometric data of individuals, and it'll be difficult to get true visibility on what they are actually doing with this information over time.

3 - Where is the Company Based?

The government of some countries, particularly ones that have a mindset of geopolitical domination, are far more aggressive in the way that they collect and use the biometric information of individuals. Know which country the company is based in. These vendors might also be working with their local government on facial recognition projects, which increases their motivation to exfiltrate facial profiles from your on-premise installation.

Additionally, the government of this country likely has eminent domain over the data being collected and amassed by the vendor, with the ability to legitimately seize the vendors information. Submitting facial profiles and other personally identifiable information to the vendor means that you are also potentially submitting it to their countries government.

To give you an idea of the breadth of facial recognition vendors out there, below is a breakdown by country, of the approximately 60 mostly vendors & a few universities that currently participate in the NIST Ongoing Face Recognition Vendor Test (FVRT). Keep in mind that there are additional entities developing facial recognition that don't currently participate in this testing.

Breakdown By Country of Vendors Participating in NIST Ongoing Facial Recognition Testing

4 - Only Use Sample Data For Testing

When evaluating a vendors on-premise facial recognition service, it is likely not clear where the facial profiles you submit to the facial recognition vendors system will ultimately end up. For this reason, you should never use live or production facial profile photos or videos during evaluation.

Instead, use only sample facial profile photos from sources that are "In the Wild" or publicly available, until you fully understand where the facial profiles are transferred to, how they are secured, and how the vendors software updates take place. There are plenty of facial profiles out there intended for testing, some examples are listed below:

5 - Know What is Being Collected From Your On-Premise Installation

On-premise facial recognition systems allow customers to build their own private data sets of facial profiles, and query them privately to protect the privacy of their customers.

If you evaluate or deploy an on-premise facial recognition system, this should be constantly monitored to understand whether the vendor transfers any of this private biometric information back to their network. This is because once installed and configured, an on-premise facial recognition system can be modified by the vendor over time to alter what they do with submitted facial profile information.

Ideally, no traffic should be allowed between the on-premise installation, and the Internet. The vendors installation should be able to operate entirely offline, with the ability to be updated via periodic updates downloaded from a separate Internet connected system.

6 - Know How Secure The Vendor's On-Premise Installation Is

Understand how secure the vendors on-premise installation is. This includes methods of authentication, how accounts, hardware and software are secured with firewalls, malicious code scanning, and how vendor's software updates are issued, tested and managed.

Ideally, no traffic should be allowed between the on-premise installation, and the Internet. The vendors installation should be able to operate entirely offline, with the ability to be updated via periodic updates downloaded from a separate Internet connected system.

7 - Have an Organizational Data Handling Policy

If not already, your organization should develop an internal policy which includes how your customers personal information should be handled by employees with respect to privacy and the law, and the consequences when it is not. This can be included with any training that is facilitated as part of the deployment of your internal facial recognition systems.

An internal data handling policy helps to prevent the mishandling of personal information over time, through organizational structure and people changes.

An example of a policy template for law enforcement entities is provided by the US Department of Homeland Security

8 - Be Aware of Vendor Acquisitions/Mergers/Investors

Over time, it's important to monitor your vendor's acquisition, merger, and investor status. A merger, acquisition, or change in investors of a company can change the nature of it's products and services. For instance their products might be neglected, put into maintenance mode, or modified to be more aggressive, or more neglectful when handling your customer's biometric data.

In the event that your vendor undergoes a major structural change, it pays to have an alternate facial recognition vendor in mind that can continue to regard the privacy of your customers information, or more actively maintain their facial recognition offering.

Acknowledgements & Additional References:

- Privacy principles for facial recognition technology in commercial applications by FPF - Future Privacy Foundation
- The Face Recognition Homepage, maintained by Prof. Mislav Grgic, Ph.D., and Kresimir Delac, Ph.D. in Zagreb, Croatia
- Georgetown University Center on Privacy and Technology
- Electronic Frontier Foundation
- Face Recognition Vendor Test (FRVT) by NIST - National Institute of Standards and Technology
- Best Practices for Common Uses of Facial Recognition Technologies by FTC - Federal Trade Commission (2012)
- Photo by Atharva Tulsi on Unsplash