Do you think you are an alert and rational person? Have you never been tricked by crude and false fraud messages or never clicked unknown suspicious links? If you receive a call from your relatives at three o’clock in the morning, saying that they are in a car accident on the highway or were kidnapped and in urgent need of money. When you are facing an anxious face and a familiar voice on the other side of the line, could you accurately and rapidly recognize whether it is your relative——or AI-generated fraud?
In today’s society, with the development of AI technology, more and more information and images generated by AI have emerged, which makes the boundary between individual creators and AI technology blurry, and the difficulty of distinguishing between real and fake information has also increased. We can see a large number of AI-generated images online, they have been used in various industries, such as taking the place of individual designers to accomplish simple poster tasks, or virtual AI characters that could make a conversation with users.
AI-generated information can be seen everywhere on the internet, and contemporary AI technology takes the fraud aspect to another level. By synthesizing and imitating faces and voices through algorithms and personal data, the technology called “deepfakes” or AI-generated media has been used to defraud. Bray (2023) conducted an experiment to assess the ability of people to distinguish real human face images and deepfake human face images, based on the conclusion of the experiment, we can see that the whole accuracy rate of participants was 62%, but the accuracy rate of images in the range of 30% to 85%, every five images on average has the accuracy rate below 50%. In the experiment, most of the participants were confident that they could distinguish the images easily. It implies the potential threats to people posed by AI-generated deepfake images.
The kidnapping case of deepfake-voice
The New Yorker has reported a recent fraud case that happened in Brooklyn, a woman named Robin was woken up by a phone call, she heard the voices of her father-in-law and mother-in-law crying for help, urging her to find her husband Steve. Robin gave the phone to Steve, Steve heard a man’s voice telling him that his parents had been kidnapped, demanding him to transfer five hundred dollars to a specified account, or he would shoot his parents. Steve sought help from his colleague who has experience in hostage negotiation, at the same time he continued to deal with the “kidnapper”. When Steve requested to hear his parents’ voices again, the man refused him and showed impatient as well as threatened him. Steve did as the man said, transferred five hundred dollars then two hundred fifty dollars to the specified account. After everything finished, the man hung up the phone. Steve then called his parents to confirm their safety, but his parents sounded confused and puzzled, they said that they were just sleeping as usual, and asked Steve what happened to him. Obviously, Steve has been deceived, it was a fraud of AI-generated voice.
In this scam, the criminal stole the number of Steve’s mother, obtained the family members’ information and their relationship, as well as synthesized his parents’ voice. The last aspect is the most important element of this AI fraud. If there were only the first two pieces of information involved, maybe Steve would not have been deceived, but hearing family members crying for help would easily lead people to panic and lose their rational minds. If he called his father before he transferred the ransom money, he would have realized that his parents were safe, but the sense of reality conveyed by the AI-generated voice caused him to completely trust this fraud.
Other similar AI frauds also happened around the world, a businessman in Fuzhou, China, received a video call from a friend on WeChat, when he saw the familiar face and voice of the friend, he did not suspect him and transferred 4.3 million RMB to the specified account as requested, then he sent a message to the real friend but received a deny reply. Only then did he realize that it was a scam of AI synthesizing faces and voices. Based on the monitoring and report of NewsGurad, which is an organization that tracks and exposes fake news sites, it reveals a large number of fake news generated by AI on the websites without human supervision, the fake news covers different languages and includes various aspects such as politics, entertainment, technology and so on, some of them also forge the speeches of politicians and the death of celebrities. Some of the fake news are widely spread, and people trust them and tell them to friends and family, causing further misunderstandings in this process. The fake news would have a negative impact on individuals and society, leading to rumors and panic, as well as inciting people’s emotions.
How does AI generate our voice and face?
Based on the argument of Sample (2020), the process of AI face-swapping analyzes the similarity between two faces through an encoder algorithm, simplifies them to a common feature, and then compresses images, finally using decoder algorithms to decode two faces, combine one person’s face with another person’s expression, consequently achieving AI face swapping. This technique would often be used to create pornographic and scam content.
Our society is now in the era of the big data revolution, based on the argument of Flew (2021), the big data revolution refers to the ability of modern society to collect a large amount of personal data and apply data analytics techniques to process and utilize this data for various purposes, and the three core elements of the big data revolution are datafication, dataism, and dataveillance. Dataveillance plays the most important role in user data collection, which continuously tracks and monitors citizens based on their online data, the surveillance beyond individuals and permeates the whole society. It also leads to the surveillance capitalism, the algorithm on the platform makes the collection of user information easy. The platform processes a large amount of user data, from personal information that is protected by privacy law, including name, address, age, position, and ID number, to information that could not be directly linked to individuals, such as browsing history and search history. The current regulation of data by platform is not transparent enough, and it may lead to some potential issues and risks. The news of user information leaks is common, whether due to the platform illegally selling data for profit or the hacker attack resulting from technique issues, and many frauds arise from them.
Many people upload their photos on social platforms such as TikTok, Instagram, and Facebook, they also share their location and selfies, and their facial information could be collected by algorithm. There are many AI face-swapping apps available online, users can upload their photos and import their faces into movies or onto celebrities for entertainment which seems authentic. Many beauty cameras also developed the function of AI face-swapping. AI face-swapping seems to become effortless, and people are usually interested in putting their faces onto celebrities, but uploading their facial information to the platform is not a perfectly safe thing, especially in a situation where privacy law is still incomplete.
The collection of user voice data is also easy, just needs to capture a few seconds of people’s voices posted on the platform, then synthesize and imitate the person’s voice through AI technique. If you have not uploaded your voice on a public platform before, the criminal just needs to make a harassing call to you, capture your voice, and then synthesize it, it seems to be a fraud method that is difficult to accurately regulate and hard to self-protect.
What can we do to avoid being deceived?
The self-protection of individuals
In the aspect of synthesizing eyes and fingers, AI tends to make mistakes easily, when we see a human photo with unreasonable eye orientation and weird twisted fingers, it is more likely to be an AI-generated image. We can also pay attention to whether the light and color look real, and how often people blink in the video. The AI-generated video seems could not blink normally because the training data of real human faces images always keep their eyes open (Sample, 2020).
When we receive a video call from our relatives or friends, be careful to observe whether they blink, their tooth, their skin, and the shape of their mouths when they speak are normal, and whether their movements are smooth, asking them to turn their heads or making a specific gesture. You can also make a secret signal that only both of you know in daily life. When receiving the video call, ask them to say the secret signal to prove their identity. The most important thing is to keep calm and contact your relatives in person and call the police as soon as possible.
To protect our personal privacy from leaking and being used to deceive our relatives, we should avoid posting our clear and full-face photos on public platforms, refuse to expose our voice when receiving a suspicious call, and avoid exposing our personal information such as address and position on public platforms. We should also avoid clicking unknown links, which could be phishing links that aim to obtain our private information.
The duty of the platforms
Platforms should be responsible for the personal information that user provides, they should improve the transparency of their algorithm, allowing users to understand and check the mechanics behind the algorithm. The platforms should not ask some unnecessary permissions from users, such as camera permission for a music app, or position permission for an online film app.
The supervision of government
It is unfair to just ask users to stop sharing their daily photos online. Freedom of expression and communication is the right of everyone on the internet, so the regulation and supervision of platforms and fraud from the government is especially important. The government should publish more comprehensive and complete laws and punishment mechanics of algorithm privacy that are adapted to the current society. And to reduce AI scams and protect user’s privacy. It is necessary for the government to restrict the platforms from the transaction of user data.
Conclusion
The development of AI technology is currently in an emerging stage, which is inevitable and unstoppable. It has a significant impact on various industries and has led to many new fraud methods. We can distinguish current AI-generated video by observing human movements and skin details, but the accuracy is not high enough. With the further improvement of AI technology, it may develop AI-generated video that is no different from real people. In that case, if you fall into the trap of fraud or slander, how to prove the person with the same voice and appearance as you in the video is not you? When you receive a call from relatives for help, do you have enough ability and a calm mind to distinguish? Individual power has limited function when faced with the development of technology. Instead of making people live in fear and nervous about distinguishing real and fake, it is more necessary for governments to publish relevant regulations and laws of AI techniques.
Reference
Bethea, C. (2024, March 7). The Terrifying A.I. Scam That Uses Your Loved One’s Voice. The New Yorker. https://www.newyorker.com/science/annals-of-artificial-intelligence/the-terrifying-ai-scam-that-uses-your-loved-ones-voice
Bray, S. D., Johnson, S. D., & Kleinberg, B. (2023). Testing human ability to detect “deepfake” images of human faces. Journal of Cybersecurity, 9(1). https://doi.org/10.1093/cybsec/tyad011
Flew, T. (2021). Regulating Platforms (pp. 79–86). Polity Press.
GrumpyBeere. (2024). man and robot . https://pixabay.com/illustrations/ai-generated-man-robot-synthetic-8597468/
Karimi, F. (2023, April 29). “Mom, these bad men have me”: She believes scammers cloned her daughter’s voice in a fake kidnapping. CNN. https://edition.cnn.com/2023/04/29/us/ai-scam-calls-kidnapping-cec/index.html
Sample, I. (2020, January 13). What are deepfakes – and how can you spot them? The Guardian. https://www.theguardian.com/technology/2020/jan/13/what-are-deepfakes-and-how-can-you-spot-them\
Tracking AI-enabled Misinformation: Over 300 “Unreliable AI-Generated News” Websites (and Counting), Plus the Top False Narratives Generated by Artificial Intelligence Tools. (n.d.). NewsGuard. https://www.newsguardtech.com/special-reports/ai-tracking-center/
Wu, J. (2023, May 24). Face-swapping fraud sparks AI-powered crime fears in China · TechNode. TechNode. https://technode.com/2023/05/24/face-swapping-fraud-sparks-ai-powered-crime-fears-in-china/
Be the first to comment