The Food of Facial Recognition: Your Regularly Updated Photos & Videos
9th January, 2019 | Biometric Privacy | Entropic
In recent years, facial recognition technology has evolved significantly, and is greatly improving the ability to efficiently detect people in still images and video. The improvements in facial recognition, along with increased availability is now empowering more and more uses of the technology across various industries.
A major benefit of facial recognition technology is about improving upon historically established use cases where people needed to be efficiently verified by another person. This manual verification process can be time consuming, mundane, and becomes increasingly difficult with shortages of staff, and where large numbers of people need to be verified in a short space of time.
A recent example of this is the US Customs and Border Protection, who are working with airports and airlines to deploy facial recognition across major airports in the US, such as Washington Dulles Airport to facilitate automated verification of travelers.
Emerging uses include adding facial recognition to existing and newly established monitoring systems to generate alerts when specific people are identified such as in public areas and at schools, and adding face recognition to existing methods of sign-in authentication, such as password-based systems and two-factor authentication.
When implemented with proper regards to the privacy of individuals they monitor, facial recognition systems provide significant benefits across many fields of use, such as:
- Identifying Unknown Persons - Helping to reveal people who are unknown. Ex: Identifying potential criminals at border entry points, or identifying missing/lost persons.
- Negative Identification - Helping to determine whether someone is masquerading as someone else, or has additional registrations under a different name. Ex: Drivers license and passport verification.
- Positive identification - Allowing people to identify themselves as physically present using their face instead of, or in addition to another form of identification such as a password. Ex: Facial authentication on your smartphone, or boarding a plane or ship.
How is Your Face Verified?
A facial recognition system needs to cross-reference, and learn from a number of existing facial profiles of an individual in order to be effective in recognizing them under varying conditions. The more the better. To this end, vendors who are developing these facial recognition systems offer several options.
Global Databases (Cloud)
Several vendors are in the process of amassing billions of profile photos of individuals that they gather from a wide variety of sources (ref: illustration below), including photos and videos from public, hacked, government sources, and from the private facial recognition queries of their own customers. It's also possible to glean multiple facial profiles of different individuals from photos and videos which capture more than one person. Videos can also yield multiple facial profiles of the same person or multiple people, which further assists the facial recognition training process.
A very important example of this is when you update your profile photo in LinkedIn, vendors can and do collect this updated photo, adding it to their repertoire as yet another facial profile that can be used to further train their facial recognition system to more accurately identify you.
These extracted facial profiles are then amassed in a global database, and used to train facial recognition systems by gleaning facial landmark information - information about the key areas on an individuals face that the system will rely upon to identify them from potentially different angles, such as in a CCTV video capture. Finally, this information might also be linked with other previously gathered personally identifiable information, including other types of biometric data.
How They Are Used
When customers want to perform a facial comparison, they can access their vendor's global facial recognition service using an Application Programming Interface (API), which allows them to perform a query by face - cross-referencing a selected facial profile against the vendor's facial landmark data stored in their global database in the cloud.
In this process, the photo or video that the customer submits for recognition can also be collected and stored by the vendor.
A very clear example of this from one Chinese vendor's privacy policy, who are by no means alone in this practice is "...we will collect and save the original photo securely for our research purpose, not share with 3rd party". The Chinese government, in this instance is not considered a 3rd party.
On-Site Databases
Some vendors allow their customers to build their own on-premise database of facial profiles, which is hosted in the customer's own network, allowing them to keep their relevant facial profiles more private within their own network, and improve performance.
How They Are Used
When their customers want to perform a facial comparison, they use their vendor's local on-premise facial recognition service which cross-references facial landmark data stored in their local on-premise database within the confines of their own network. In addition, it may also cross-reference their global database, if it exists. Whether or not the submitted photos are also being transferred back to the vendor's global database, is something that should be very clearly understood, and continuously verified and controlled as needed.
How Effective Is Facial Recognition?
To understand how effective the most current facial recognition vendor offerings are, the US Government has established a regular Face Recognition Vendor Test (FRVT) of some participating commercial & university facial recognition systems.
While this benchmark provided by NIST - the National Institute of Standards and Technology, focuses on measuring how accurate and efficient facial recognition systems are, it should not be construed as an evaluation of how good the privacy controls are with any given system, neither should it be construed as a U.S. Government endorsement of these services from a security perspective.
Privacy Considerations
When posting a photo or video that includes yourself or others, for instance to social media, it might seem harmless enough. Increasingly however, what you post online to "the wild" becomes subject to collection by governments, universities and the private sector who are developing and tuning facial recognition systems to an expanding set of use cases that are still being defined.
If you are evaluating, implementing or using facial recognition systems, you are entrusting the unique and immutable biometric information of individuals who have entrusted you with their information, to a third party. Once this biometric signature information is submitted to this third party, it is pretty much impossible to alter or reign in.
Anyone using, evaluating, or implementing specific facial recognition services should understand the privacy implications of using these services, before using them.
An example consideration is to recognize the home country of the vendor providing the service, since the government of this country likely has eminent domain over the data being collected and amassed by the vendor. Submitting facial profiles and other personally identifiable information to the vendor means that you are also potentially submitting it to their countries government. Refer to our vendor privacy considerations for more information.
Below is a breakdown by country, of the approximately 60 vendors & universities that currently participate in the NIST Ongoing Face Recognition Vendor Test (FVRT). Keep in mind that there are additional entities developing facial recognition that don't currently participate in this testing.
The Saving Grace
Over time a person's face can undergo changes due to factors such as aging, injury, surgery, sun damage, and receding hairline. From a privacy perspective this is their saving grace, and means that it is not enough for a facial recognition vendor to collect one photo of you at one time only. Facial recognition vendors must not only build their database of facial profiles, but they must also refresh these facial profiles at regular intervals or their ability to confidently recognize your face might diminish over time.
Conclusion
While facial recognition will offer us significant benefits in the future, as individuals we need to be aware that any photo or video we post becomes a potential facial profile that can be gleaned by governments, and the private sector for the purposes of better identifying us in a plethora of use cases that are still being defined.
Those who are evaluating and using facial recognition technologies need to look to the privacy implications of each service - something that is not presently the focus of the NIST testing. Law enforcement, homeland security, and other organizations who have large quantities of sensitive facial profiles at their disposal need to be aware of the privacy ramifications of evaluating and using facial recognition services.
At a time when facial recognition is coming of age, the concern for protecting our privacy will induce more innovation in the field of biometric privacy. Some examples on how emerging Blockchain technologies can help us to protect our privacy is discussed in this article by Alex Moskov, originally posted on CoinCentral.com.
If you have any feedback, questions, or suggestions, please let us know.
Acknowledgements:
Photo by Shamim Nakhaei on Unsplash
Photo by Jurica Koletic on Unsplash
Photo by Pawel Czerwinski on Unsplash
Photo by Thomas Lefebvre on Unsplash
Photo by Michal Kubalczyk on Unsplash
Photo by Imgix on Unsplash
Photo by Markus Spiske on Unsplash
Photo by Omar Lopez on Unsplash
Photo by Ashton Bingham on Unsplash
Face Recognition Vendor Test (FRVT) by NIST - National Institute of Standards and Technology