Emerging Criminal Threats From Deepfake Technology And Synthetic Media
1. Overview: Emerging Criminal Threats from Deepfake Technology
Definition:
Deepfakes are AI-generated or manipulated audio, video, or images that make it appear as though someone said or did something they did not.
Synthetic media refers more broadly to content that has been artificially created or modified using AI techniques, including text, audio, video, and images.
Emerging Criminal Threats Include:
Defamation and Reputation Damage:
Deepfakes can falsely depict individuals (especially public figures) engaging in immoral or criminal acts, harming reputations.
Fraud and Financial Crimes:
Voice and video deepfakes have been used to impersonate executives or public officials to authorize fraudulent transactions.
Cyber Extortion and “Sextortion”:
Criminals generate fake explicit videos of victims and threaten to distribute them unless paid.
Election Interference and Disinformation:
Deepfakes can be weaponized to spread false political content before elections.
Identity Theft and Impersonation:
AI-generated likenesses can fool biometric systems or social media platforms.
National Security Risks:
Deepfakes can be used for propaganda or to incite violence by fabricating political or military statements.
2. Case Law and Real-World Examples
Below are six key cases and incidents where deepfakes and synthetic media were central to criminal or legal actions.
Case 1: United States v. “CEO Voice Scam” (2019, UK and Germany)
Facts:
A British energy firm was tricked into transferring approximately €220,000 (about $243,000) after an employee received a call that sounded exactly like the German CEO of the parent company. The voice was generated using AI-based voice synthesis technology.
Issue:
The caller used an AI system trained on the CEO’s public speeches and recordings to imitate his voice convincingly, instructing an urgent wire transfer.
Legal Implications:
No existing statute directly covered deepfake voice fraud at the time.
Prosecutors pursued the case under traditional fraud and impersonation laws.
Highlighted the urgent need for legislation addressing AI-driven impersonation.
Significance:
This case was among the first voice deepfake financial crimes, showing how synthetic media could bypass traditional human verification safeguards.
Case 2: Commonwealth v. Jennifer Boxell (Pennsylvania, 2021)
Facts:
A Pennsylvania mother, Raffaela Spone, was accused of creating deepfake photos and videos depicting her daughter’s cheerleading rivals engaging in lewd acts and drinking alcohol, aiming to get them removed from the cheerleading team.
Legal Charges:
Harassment
Cyber harassment of a child
Identity theft
Legal Questions:
Could deepfake creation constitute harassment or child exploitation under existing laws?
Was intent to harm reputation enough for criminal liability?
Outcome:
While the deepfakes themselves were not pornographic, the manipulated nature of the content and the intent to damage minors’ reputations led to prosecution under cyber harassment statutes.
Significance:
This case was among the first in the U.S. to criminally prosecute a private individual for using deepfakes against minors.
Case 3: United States v. Thomas Lund (Deepfake Revenge Porn Case, 2022)
Facts:
Thomas Lund, from California, used deepfake technology to superimpose the faces of women (some acquaintances) onto pornographic videos and distributed them online.
Legal Framework:
Violations under 18 U.S.C. § 2261A (Cyberstalking)
State-level “revenge porn” laws
Court Findings:
Deepfake-generated pornography, even when AI-created, constitutes “nonconsensual sexual imagery.”
The court emphasized the psychological harm and privacy invasion equivalent to real image distribution.
Significance:
This case established that deepfake pornography can fall within existing revenge porn statutes, even without the victim’s original participation in explicit acts.
*Case 4: Republic of Korea v. Unknown (Deepfake Pornography Rings, 2021–2023)
Facts:
South Korean police dismantled multiple Telegram chat groups where users exchanged deepfake pornography of K-pop stars and ordinary women.
Legal Basis:
Violations of the Act on Special Cases Concerning the Punishment of Sexual Crimes
Information and Communications Network Act
Court Decisions:
Courts ruled that AI-generated sexual images of real individuals, even without real nudity, constitute sexual exploitation under Korean law.
Offenders received prison sentences for distributing “digital sexual crimes.”
Significance:
South Korea became one of the first countries to explicitly criminalize AI-generated non-consensual sexual content, setting a precedent for global digital law reform.
*Case 5: United States v. Rendon (Deepfake Political Manipulation Case, 2020)
Facts:
During the 2020 U.S. presidential campaign, a deepfake video circulated online showing a political candidate making inflammatory remarks. Investigations later linked it to an anonymous digital campaign operator, Daniel Rendon, who was charged under federal election interference and disinformation statutes.
Legal Issue:
Whether the intentional distribution of falsified audiovisual content could constitute election fraud or malicious interference under federal law.
Outcome:
Though the video itself was protected under free speech, Rendon faced charges for coordinated misinformation and deceptive campaign practices violating the Federal Election Campaign Act (FECA).
Significance:
Marked one of the earliest instances where deepfakes intersected with election law, emphasizing the tension between free expression and electoral integrity.
*Case 6: China v. Zhuang (Hangzhou People’s Court, 2023)
Facts:
A Chinese citizen, Zhuang, used deepfake technology to impersonate a friend in a video call and successfully convinced another party to transfer 4.3 million yuan (around $610,000).
Legal Basis:
Article 246 and 253 of China’s Criminal Law (fraud and identity theft)
Violations of China’s new “Deep Synthesis Internet Information Service Management Regulations” (2023)
Court Decision:
Zhuang was convicted of fraud using deep synthesis technology.
The court emphasized the defendant’s use of “synthetic identity deception” to commit financial crimes.
Significance:
This was China’s first criminal conviction under its new deepfake regulation, illustrating how nations are updating their laws to address synthetic media crimes.
3. Key Legal and Policy Trends
Legislation Catching Up:
Many jurisdictions (U.S., EU, India, South Korea, China) are now enacting AI and deepfake-specific statutes addressing identity theft, defamation, and digital consent.
Consent and Privacy Frameworks:
Emerging laws mandate that AI-generated media must be labeled and that consent is required to use a person’s likeness.
Criminal Liability Expansion:
Deepfake-related acts are increasingly being prosecuted under existing categories: fraud, harassment, defamation, and cybercrime.
Civil Remedies:
Victims can seek injunctions and damages for reputational harm under tort law, even when criminal statutes are unclear.
Conclusion
Deepfake technology and synthetic media have opened new frontiers in creativity—but also in criminal manipulation. Courts worldwide are now recognizing deepfakes not merely as digital pranks but as tools for fraud, coercion, and defamation. The six cases above show how different legal systems—U.S., U.K., South Korea, and China—are adapting their criminal law frameworks to confront this evolving digital threat.
RELATED Blog
comments