Big tech is rethinking its facial recognition technology business. IBM, Microsoft and Amazon all made pledges earlier this summer to stop selling the technology to law enforcement in the wake of protests calling for social justice, with IBM going so far as to move away from the tech altogether.

“IBM firmly opposes and will not condone uses of any technology, including facial recognition technology offered by other vendors, for mass surveillance, racial profiling, violations of basic human rights and freedoms, or any purpose which is not consistent with our values and Principles of Trust and Transparency,” IBM CEO Arvind Krishna said in a letter to Congress in June. “We believe now is the time to begin a national dialogue on whether and how facial recognition technology should be employed by domestic law enforcement agencies.”

Big tech’s exodus from the facial recognition space comes as financial institutions are starting to adopt the technology. Its use will only grow from here, according to Kjell Carlsson, principal analyst at Forrester Research, and it will be hard to move away from, once implemented.

The challenge, according to experts, will be to navigate best practices with a technology that has faced scrutiny for discrimination and errors within other verticals. Although many institutions say they have yet to run into these issues, problems could arise as lenders expand their application of the technology beyond simple login functions. 

Face the future

JPMorgan Chase, Wells Fargo, USAA and Navy Federal Credit Union all use facial recognition for login purposes and to verify customers before certain transactions. According to Mike Slaugh, executive director of fraud prevention at USAA, security is a key reason.

“USAA designs its products and solutions in ways that are inclusive to all our members,” said Slaugh. “Facial recognition and other biometrics provide a high level of security at the onset of an interaction and minimizes the need to interrupt the customer in the middle of a transaction by asking for additional authentication.”

While financial institutions say they have avoided discrimination problems so far, they also have yet to use the technology to its full potential. With a seemingly insatiable need to push innovation, financial institutions will need to navigate a delicate balance. If the technology goes too far, it could make consumers feel uneasy. 

Consumers might be fine with an opt-in process that uses facial recognition technology to identify high net worth clients as they enter a branch, Carlsson said. They would be less fine, he added, if banks use the technology to identify annoyed customers or push advertisements without consumer consent.

Facial recognition software takes multiple images of a face or a photo and then takes several measurements, like the distance between the eyes, width of the mouth and length of the nose, to associate with the closest face in its database. 

The technology measures pixel lengths, curves and sizes, or pixel value to pick out the closest resembling face, according to Joseph Robinson, a researcher at SMILe Lab, a Northeastern University project dedicated to applied machine learning, human-computer interaction and high-level image and video understanding. These measurements can be discriminatory if the algorithm is not fed enough diverse data sets to be accurate across all spectrums of age, race and gender, Robinson explained.

According to research from the National Institute of Standards and Technology, facial recognition tech has inherent shortfalls. Patrick Grother, a computer scientist at the National Institute of Standards and Technology, assembled a team to look further into the discriminatory nature of the tech. Grother and his team studied mugshots, passport and visa photos, immigration photos, and driver’s license photos of 8.49 million people, collecting 18.27 million images and running them through 189 facial recognition algorithms available in the market. 

The study results, published in December 2019, showed that false-positive rates for Black and Asian faces were higher than for Caucasian faces, meaning that two different faces were more frequently mistakenly matched due to their similarities. “The differentials often ranged from a factor of 10 to 100 times, depending on the individual algorithm,” according to the report. False positives for women were higher than men across all spectrums. 

Grother and his fellow researchers did not draw a conclusion as to why the technology misidentified certain demographics more than others.

Financial institutions overseas have already started expanding facial recognition beyond login purposes. Ping An, one of the largest insurance and financial service providers in China, uses the tech on potential customers to assess risk, according to the Wall Street Journal.

Forrester’s Carlsson said banks could run into problems if they use facial recognition to assess risk, as in looking for signs that a consumer appears worried or anxious. The technology could feel invasive to customers, and those who are rejected might view the system as one that is flawed as it could punish them if their faces show concern when applying for a loan. Plus, the technology itself isn’t quite there.

“There’s going to be a lot of false positives and false negatives,” Carlsson said.

An uneasy feeling

If banks continue to broaden their use of facial recognition technology, they will have to contend with the negative image that some agencies, law enforcement for example, have given the tech.

The Surveillance Technology Oversight Project (STOP), a New York-based activist group, has been working to regulate the use of facial technology without public consent by law enforcement and other government agencies. STOP filed a lawsuit against the NYPD in July for disclosing the use of facial recognition scans completed on people in Times Square.

“NYPD used facial recognition systems nearly 10,000 times last year and they use it in ways that are completely unscientific to include photoshopped images and alter them prior to running scans, creating a huge risk of a false positive,” said Alex Cahn, executive director of STOP. “And they use facial recognition technology that has been shown to be biased against communities of color.”

Another risk associated with the technology is fraud. 

“If you were to go back a long while previously, you could spoof a facial recognition system by just holding up a picture of a person,” Carlsson said. “Then there were ways in which you could spoof the system by having a 3D mask of a person.”

But with technological advancements, it’s getting more difficult to trick facial recognition systems, Carlsson said. Developers have taught algorithms to look for human-like behavior not present in masks or static photos, like facial twitches. Colin Whitmore, senior analyst at Aite Group, agreed that it is difficult to fool facial recognition technology because it can now decipher whether it’s looking at a live person. But that could change over time as fraudsters find ways to bypass legacy biometrics technology used by financial institutions, Whitmore added.

LexisNexis Risk Solutions, which provides risk mitigating tools to financial institutions, uses a liveliness test that asks users to blink and move in real time during the facial recognition authentication, said Chris Schnieper, an expert in emerging authentication and digital identity at the company. He said facial recognition technology is secure for now, but it may not be secure forever because fraudsters “have an enormous amount of testing that they do every day.” 

See also: FICO fights fraud through selfie-based authentication tools

As fraudsters and banks race to evolve security tech to stay one step ahead of each other, banks will need to navigate the technology’s inherent risk of discrimination and external risks of criminals targeting synthetic ID fraud.

“At the end of the day, you never want to 100% rely on [facial recognition] as it’s always best to use this as a … multifactor authentication,” Carlsson said. “I am sure there are institutions out there who are using outdated technology, where it will be possible to bypass this.”

Bank Innovation Build, which takes place Sept. 9-10 as a virtual experience, is a must-attend industry event for professionals overseeing financial technologies, product experiences and services. Register here.