Computer-generated children’s voices so realistic they fool their own parents. Masks created with photos from social media that can penetrate a system protected by face ID. They sound like the stuff of science fiction, but these techniques are already available to criminals preying on everyday consumers.
The proliferation of scam tech has alarmed regulators, the police and people at the highest levels of the financial industry. Artificial intelligence (AI), in particular, is being used to “turbocharge” fraud, US Federal Trade Commission chairman Lina Khan warned recently, calling for increased vigilance from law enforcement.
Even before AI broke loose and became available to anyone with an Internet connection, the world was struggling to contain an explosion in financial fraud. In the United States alone, consumers lost almost US$8.8 billion (S$12 billion) in 2022, up 44 per cent from 2021, despite record investment in detection and prevention.
Financial crime experts at major banks, including Wells Fargo & Co and Deutsche Bank, say the fraud boom on the horizon is one of the biggest threats facing their industry. On top of paying the cost of fighting scams, the financial industry risks losing the faith of affected customers. “It’s an arms race,” said Mr James Roberts, who heads up fraud management at the Commonwealth Bank of Australia, the country’s biggest bank. “It would be a stretch to say that we’re winning.”
Scammers use data that you post online
The AI explosion offers not only new tools but also the potential for life-changing financial loss. And the increased sophistication and novelty of the technology mean that everyone, not just the credulous, is a potential victim.
Some of Mr Roberts’ team of more than 500 spend their days eavesdropping on cons to hear first-hand how AI is reshaping their battle. A fake request for money from a loved one isn’t new. But now parents get calls that clone their child’s voice with AI to sound indistinguishable from the real thing. These tricks, known as social engineering scams, tend to have the highest hit rates and generate some of the quickest returns for fraudsters.
Cloning a person’s voice is increasingly easy. Once a scammer downloads a short sample from an audio clip from someone’s social media or voicemail message – it can be as short as 30 seconds – he can use AI voice-synthesising tools readily available online to create the content he needs.
Public social media accounts make it easy to figure out who a person’s relatives and friends are, not to mention where they live and work and other vital information. Bank bosses stress that scammers, running their operations like businesses, are prepared to be patient, sometimes planning attacks for months.
How banks try to detect scams
To give a sense of the challenge facing banks, Mr Roberts said that right now, the Commonwealth Bank of Australia is tracking about 85 million events a day through a network of surveillance tools. That’s in a country with a population of just 26 million.
The industry hopes to fight back by educating consumers about the risks and increasing investment in defensive technology. New software alerts the bank when customers use their computer mouse in an unusual way during a transaction – a red flag for a possible scam. Anything suspicious, including the destination of an order and how the purchase is processed, can alert staff in as little as 30 milliseconds, allowing them to block the transaction.
At Deutsche Bank, computer engineers have recently rebuilt their suspicious transaction detection system – called Black Forest – using the latest natural language processing models, according to Mr Thomas Graf, a senior machine learning engineer there. The tool looks at transaction criteria such as volume, currency and destination and automatically learns from reams of data which patterns suggest fraud.
The model can be used on both retail and corporate transactions and has already unearthed several cases, including one involving organised crime, money laundering and tax evasion.
Wells Fargo has overhauled tech systems to counter the risk of AI-generated videos and voices. But the system needs to keep evolving to keep up with the criminals who try to find a workaround.
Voice deepfakes are coming for your bank balance
Scammers use deepfakes to create voice recordings and videos to trick victims’ family, friends
For example, some US banks require customers to upload a photo of an ID document when signing up for an account. Scammers are now buying stolen data on the Dark Web, finding photos of their victims from social media, and 3D-printing masks to create fake IDs with the stolen information.
“And these can look like everything from what you get at a Halloween shop to an extremely lifelike silicone mask of Hollywood standards,” said Mr Alain Meier, head of identity at Plaid, which helps banks, financial technology companies and other businesses battle fraud with its ID verification software.
Plaid analyses skin texture and translucency to make sure the person in the photo looks real. Mr Meier, who has dedicated his career to detecting fraud, said the best fraudsters, those running their schemes as a business, build scamming software and package it up to sell on the Dark Web. Prices can range from US$20 to thousands of dollars. “For example, it could be a Chrome extension to help you bypass fingerprinting, or tools that can help you generate synthetic images,” he said.
Financial industry leaders around the world are also trying to push a share of the responsibility onto tech firms. The fastest-growing scam category is investment fraud, often introduced to victims through search engines, where scammers can easily buy sponsored advertising spots. When would-be investors click through, they often find realistic prospectuses and other financial data. Once they transfer their money, it can take months, if not years, to realise they have been swindled when they try to cash in on their “investment”.
In June, a group of 30 lenders in Britain sent a letter to Prime Minister Rishi Sunak asking that tech companies contribute to refunds for victims of fraud stemming from their platforms. The government says it is planning new legislation and other measures to crack down on online financial scams.
The banking industry is lobbying to spread the responsibility more widely, in part because costs appear to be going up. Once again, a familiar problem from economics applies in the scam economy, too. Like pollution from a factory, new technology is creating an externality, or a cost imposed on others. In this case, it is heightened reach and risk for scams. Neither banks nor consumers want to be the only ones forced to pay the price.
Mr Chris Sheehan spent almost three decades with the country’s police force before joining National Australia Bank, where he heads investigations and fraud. He has added about 40 people to his team in the past year with constant investment by the bank. When he adds up all the staff and tech costs, “it scares me how big the number is”, he said.
“I am hopeful, because there are technological solutions, but you never completely solve the problem,” he said. It reminds him of his time fighting drug gangs as a cop. Framing it as a war on drugs was “a big mistake”, he said. “I will never phrase it in that framework – of a war on scams – because the implication is a war is winnable. This is not winnable.” BLOOMBERG
Can you trust your ears? AI voice scams rattle the US
Invest microsite: Get more investment and career tips
Join ST’s Telegram channel and get the latest breaking news delivered to you.
p.st_telegram_boilerplate:before {
display: inline-block;
content: ” “;
border-radius: 6px;
height: 6px;
width: 6px;
background-color: #12239a;
margin-left: 0px;
margin-right: 13px;
}
a.st_boilerplate {
font-family: “SelaneWebSTForty”, Georgia, “Times New Roman”, Times, serif;
}