Generative AI

2024

Security

IBM Security researchers recently discovered a “surprisingly and scarily easy” technique to hijack and manipulate live conversations using artificial intelligence (AI).
The attack, called “audio-jacking,” relies on generative AI — a class of AI that includes OpenAI’s ChatGPT and Meta’s Llama-2 — and deepfake audio technology.
<h2 id="audio-jacking">Audio jacking</h2>
In the experiment, researchers instructed the AI to process audio from two sources in a live communication — such as a phone conversation. Upon hearing a specific keyword or phrase, the AI is further instructed to intercept the related audio and manipulate it before sending it on to the intended recipient.
According to a blog post from IBM Security, the experiment <a href="https://securityintelligence.com/posts/using-generative-ai-distort-live-audio-transactions/" target="_blank" rel="noopener nofollow">ended</a> with the AI successfully intercepting a speaker’s audio when they were prompted by the other human speaker to give their bank account information. The AI then replaced the authentic voice with deepfake audio, giving a different account number. The attack was undetected by the “victims” in the experiment.
<figure data-ct-non-breakable="true">
 <img src="https://s3.cointelegraph.com/uploads/2024-02/cba19e43-3fe9-43d7-8486-978fe10ab97a.png" alt="" title="">
 <figcaption style="text-align: center;">
 Source: <a href="https://securityintelligence.com/posts/using-generative-ai-distort-live-audio-transactions/" target="_blank" rel="noopener nofollow">IBM Security</a>
 </figcaption>
</figure>
<h2 data-ct-non-breakable="true" id="generative-ai">Generative AI</h2>
The blog points out that, while executing the attack would require some level of social engineering or phishing, developing the AI system itself posed little challenge:
<blockquote data-ct-non-breakable="true">
 “Building this PoC [proof-of-concept] was surprisingly and scarily easy. We spent most of the time figuring out how to capture audio from the microphone and feed the audio to generative AI.”
</blockquote>
Traditionally, building a system to autonomously intercept specific audio strings and replace them with audio files generated on the fly would have required a multi-disciplinary computer science effort.
But modern generative AI does the heavy lifting itself. “We only need three seconds of an individual’s voice to clone it,” reads the blog, adding that, nowadays, these kinds of deepfakes are done via API.
Related: <a href="https://cointelegraph.com/news/ai-artificial-intelligence-deepfakes-fool-voters-politicians-2024-us-presidential-elections">AI deepfakes fool voters and politicians ahead of 2024 US elections — ‘I thought it was real’</a>
The threat of audio jacking goes beyond tricking unwitting victims into depositing funds into the wrong account. The researchers also point out that it could function as an invisible form of censorship, with the potential to change the content of live news broadcasts or political speeches in real time.

Security researchers unveil deepfake AI audio attack that hijacks live conversations

목차

Cointelegraph

2024년 2월 5일에 게재된 다른 기사

Will Gary Gensler continue as SEC Chair if Trump comes into power?

US Government Officials Publish a Report on Cryptocurrency Mining and Electricity Consumption

Institutional Interest in Cryptocurrency Markets

Binance hits out against FUD after users claim KYC leaks on dark web

Heroes of Mavia Surpasses 1 Million Downloads, Dominates Global App Store Rankings Before Token L...

LPL Financial to evaluate spot Bitcoin ETFs ‘three months’ before joining the market

무료 뉴스레터에 가입하여 매일 최신 암호화폐 소식을 받아보세요!