Synthetic human-like fakes - Revision history

Juho Kunsola: /* Real-time digital look-and-sound-alike fraud in 2024 */ + '''''Arup lost $25mn in Hong Kong deepfake video conference scam''''' at ft.com] reporting by Cheng Leng and Chan Ho-him 2025-05-17

2025-08-22T11:49:29Z

Real-time digital look-and-sound-alike fraud in 2024: + '''''Arup lost $25mn in Hong Kong deepfake video conference scam''''' at ft.com] reporting by Cheng Leng and Chan Ho-him 2025-05-17

← Older revision		Revision as of 11:49, 22 August 2025
Line 376:		Line 376:

	=== Real-time digital look-and-sound-alike fraud in 2024 ===		=== Real-time digital look-and-sound-alike fraud in 2024 ===
	Reporting
			'''Reporting'''
			* [https://www.ft.com/content/b977e8d4-664c-4ae4-8a8e-eb93bdf785ea '''''Arup lost $25mn in Hong Kong deepfake video conference scam''''' at ft.com] reporting by Cheng Leng and Chan Ho-him 2025-05-17
	* [https://edition.cnn.com/2024/02/04/asia/deepfake-cfo-scam-hong-kong-intl-hnk/index.html '''''Finance worker pays out $25 million after video call with deepfake "chief financial officer"''''' at edition.cnn.com], February 2024 reporting by Heather Chen and Kathleen Magramo, CNN		* [https://edition.cnn.com/2024/02/04/asia/deepfake-cfo-scam-hong-kong-intl-hnk/index.html '''''Finance worker pays out $25 million after video call with deepfake "chief financial officer"''''' at edition.cnn.com], February 2024 reporting by Heather Chen and Kathleen Magramo, CNN
	* [https://edition.cnn.com/2024/05/16/tech/arup-deepfake-scam-loss-hong-kong-intl-hnk/index.html '''''British engineering giant Arup revealed as $25 million deepfake scam victim''''' at edition.cnn.com] May 2024 reporting by Kathleen Magramo, CNN		* [https://edition.cnn.com/2024/05/16/tech/arup-deepfake-scam-loss-hong-kong-intl-hnk/index.html '''''British engineering giant Arup revealed as $25 million deepfake scam victim''''' at edition.cnn.com] May 2024 reporting by Kathleen Magramo, CNN

Juho Kunsola: /* Real-time digital look-and-sound-alike fraud in 2024 */

2025-04-26T17:20:40Z

Real-time digital look-and-sound-alike fraud in 2024

← Older revision		Revision as of 17:20, 26 April 2025
Line 378:		Line 378:
	Reporting		Reporting
	* [https://edition.cnn.com/2024/02/04/asia/deepfake-cfo-scam-hong-kong-intl-hnk/index.html '''''Finance worker pays out $25 million after video call with deepfake "chief financial officer"''''' at edition.cnn.com], February 2024 reporting by Heather Chen and Kathleen Magramo, CNN		* [https://edition.cnn.com/2024/02/04/asia/deepfake-cfo-scam-hong-kong-intl-hnk/index.html '''''Finance worker pays out $25 million after video call with deepfake "chief financial officer"''''' at edition.cnn.com], February 2024 reporting by Heather Chen and Kathleen Magramo, CNN
	* [https://edition.cnn.com/2024/05/16/tech/arup-deepfake-scam-loss-hong-kong-intl-hnk/index.html '''''British engineering giant Arup revealed as $25 million deepfake scam victim''''' at edition.cnn.com] by Kathleen Magramo, CNN		* [https://edition.cnn.com/2024/05/16/tech/arup-deepfake-scam-loss-hong-kong-intl-hnk/index.html '''''British engineering giant Arup revealed as $25 million deepfake scam victim''''' at edition.cnn.com] May 2024 reporting by Kathleen Magramo, CNN
	----		----

Juho Kunsola: /* Real-time digital look-and-sound-alike fraud in 2024 */ + '''''British engineering giant Arup revealed as $25 million deepfake scam victim''''' at edition.cnn.com by Kathleen Magramo, CNN

2025-04-26T17:19:37Z

Real-time digital look-and-sound-alike fraud in 2024: + '''''British engineering giant Arup revealed as $25 million deepfake scam victim''''' at edition.cnn.com by Kathleen Magramo, CNN

← Older revision		Revision as of 17:19, 26 April 2025
Line 378:		Line 378:
	Reporting		Reporting
	* [https://edition.cnn.com/2024/02/04/asia/deepfake-cfo-scam-hong-kong-intl-hnk/index.html '''''Finance worker pays out $25 million after video call with deepfake "chief financial officer"''''' at edition.cnn.com], February 2024 reporting by Heather Chen and Kathleen Magramo, CNN		* [https://edition.cnn.com/2024/02/04/asia/deepfake-cfo-scam-hong-kong-intl-hnk/index.html '''''Finance worker pays out $25 million after video call with deepfake "chief financial officer"''''' at edition.cnn.com], February 2024 reporting by Heather Chen and Kathleen Magramo, CNN
			* [https://edition.cnn.com/2024/05/16/tech/arup-deepfake-scam-loss-hong-kong-intl-hnk/index.html '''''British engineering giant Arup revealed as $25 million deepfake scam victim''''' at edition.cnn.com] by Kathleen Magramo, CNN
			----

	~~----~~
	== Text syntheses ==		== Text syntheses ==
	[[w:Chatbot]]s and [[w:spamming]] have existed for a longer time, but only now armed with AI they are becoming more deceiving.		[[w:Chatbot]]s and [[w:spamming]] have existed for a longer time, but only now armed with AI they are becoming more deceiving.

Juho Kunsola: + w:Sora (text-to-video model), a w:text-to-video model developed by w:OpenAI, that has worrying levels of realism was published in Dec 2024

2025-02-23T11:37:53Z

+ w:Sora (text-to-video model), a w:text-to-video model developed by w:OpenAI, that has worrying levels of realism was published in Dec 2024

← Older revision		Revision as of 11:37, 23 February 2025
Line 492:		Line 492:

	== 2020's synthetic human-like fakes ==		== 2020's synthetic human-like fakes ==
			* '''2024''' \| '''<font color="red">text-to-video model</font>''' \| '''[[w:Sora (text-to-video model)]]''', a [[w:text-to-video model]] developed by [[w:OpenAI]], that has worrying levels of realism was published in 2024. It was released to subscription paying users of ChatGPT in December 2024.

	* '''2023''' \| '''<font color="orange">Real-time digital look-and-sound-alike crime</font>''' \| In April a man in northern China was defrauded of 4.3 million yuan by a criminal employing a digital look-and-sound-alike pretending to be his friend on a video call made with a stolen messaging service account.<ref name="Reuters real-time digital look-and-sound-alike crime 2023"/>		* '''2023''' \| '''<font color="orange">Real-time digital look-and-sound-alike crime</font>''' \| In April a man in northern China was defrauded of 4.3 million yuan by a criminal employing a digital look-and-sound-alike pretending to be his friend on a video call made with a stolen messaging service account.<ref name="Reuters real-time digital look-and-sound-alike crime 2023"/>

Juho Kunsola: + == Digital look-and-sound-alikes == + === Real-time digital look-and-sound-alike fraud in 2024 ===

2024-12-26T17:26:24Z

+ == Digital look-and-sound-alikes == + === Real-time digital look-and-sound-alike fraud in 2024 ===

@@ Line 359: / Line 359: @@
 It is high time to act and to '''[[Law proposals to ban covert modeling|criminalize the covert modeling of human voice!]]'''
 == Text syntheses ==
 [[w:Chatbot]]s and [[w:spamming]] have existed for a longer time, but only now armed with AI they are becoming more deceiving.

Juho Kunsola: /* On known history of digital sound-alikes */

2024-12-24T19:40:11Z

On known history of digital sound-alikes

← Older revision		Revision as of 19:40, 24 December 2024
Line 212:		Line 212:

	In November 2024, Nvidia researchers announced they have made and trained a [https://fugatto.github.io/ Foundational Generative Audio Transformer (Opus 1) at fugatto.github.io] or Fugatto for short.		In November 2024, Nvidia researchers announced they have made and trained a [https://fugatto.github.io/ Foundational Generative Audio Transformer (Opus 1) at fugatto.github.io] or Fugatto for short.

			The researchers state ''Fugatto is a versatile audio synthesis and transformation model capable of following
			free-form text instructions with optional audio inputs. ''<ref>https://research.nvidia.com/publication/2024-11_fugatto-1-foundational-generative-audio-transformer-opus-1</ref>

	=== Documented crimes with digital sound-alikes ===		=== Documented crimes with digital sound-alikes ===

Juho Kunsola: + In November 2024, Nvidia researchers announced they have made and trained a Foundational Generative Audio Transformer

2024-12-24T19:37:34Z

+ In November 2024, Nvidia researchers announced they have made and trained a Foundational Generative Audio Transformer

← Older revision		Revision as of 19:37, 24 December 2024
Line 210:		Line 210:

	{{#ev:youtube\|0sR1rU3gLzQ\|640px\|right\|Video [https://www.youtube.com/watch?v=0sR1rU3gLzQ video 'This AI Clones Your Voice After Listening for 5 Seconds' by '2 minute papers' at YouTube] describes the voice thieving machine by Google Research in [[w:NeurIPS\|w:NeurIPS]] 2018.}}		{{#ev:youtube\|0sR1rU3gLzQ\|640px\|right\|Video [https://www.youtube.com/watch?v=0sR1rU3gLzQ video 'This AI Clones Your Voice After Listening for 5 Seconds' by '2 minute papers' at YouTube] describes the voice thieving machine by Google Research in [[w:NeurIPS\|w:NeurIPS]] 2018.}}

			In November 2024, Nvidia researchers announced they have made and trained a [https://fugatto.github.io/ Foundational Generative Audio Transformer (Opus 1) at fugatto.github.io] or Fugatto for short.

	=== Documented crimes with digital sound-alikes ===		=== Documented crimes with digital sound-alikes ===

Juho Kunsola: language

2024-10-19T21:00:44Z

language

← Older revision		Revision as of 21:00, 19 October 2024
Line 293:		Line 293:
	==== 2021 digital sound-alike enabled fraud ====		==== 2021 digital sound-alike enabled fraud ====

	<section begin=2021 digital sound-alike enabled fraud />The 2nd publicly known fraud done with a digital sound-alike<ref group="1st seen in" name="2021 digital sound-alike fraud case">https://www.reddit.com/r/VocalSynthesis/</ref> took place on Friday 2021-01-15. A bank in Hong Kong was manipulated to wire money to numerous bank accounts by using a voice stolen from one of the their client company's directors. They managed to defraud $35 million of the U.A.E. based company's money.<ref name="Forbes reporting on 2021 digital sound-alike fraud">https://www.forbes.com/sites/thomasbrewster/2021/10/14/huge-bank-fraud-uses-deep-fake-voice-tech-to-steal-millions/</ref>. This case came into light when Forbes saw [https://www.documentcloud.org/documents/21085009-hackers-use-deep-voice-tech-in-400k-theft a document] where the U.A.E. financial authorities were seeking administrative assistance from the US authorities towards ~~the end of~~ recovering a small portion of the defrauded money that had been sent to bank accounts in the USA.<ref name="Forbes reporting on 2021 digital sound-alike fraud" />		<section begin=2021 digital sound-alike enabled fraud />The 2nd publicly known fraud done with a digital sound-alike<ref group="1st seen in" name="2021 digital sound-alike fraud case">https://www.reddit.com/r/VocalSynthesis/</ref> took place on Friday 2021-01-15. A bank in Hong Kong was manipulated to wire money to numerous bank accounts by using a voice stolen from one of the their client company's directors. They managed to defraud $35 million of the U.A.E. based company's money.<ref name="Forbes reporting on 2021 digital sound-alike fraud">https://www.forbes.com/sites/thomasbrewster/2021/10/14/huge-bank-fraud-uses-deep-fake-voice-tech-to-steal-millions/</ref>. This case came into light when Forbes saw [https://www.documentcloud.org/documents/21085009-hackers-use-deep-voice-tech-in-400k-theft a document] where the U.A.E. financial authorities were seeking administrative assistance from the US authorities towards recovering a small portion of the defrauded money that had been sent to bank accounts in the USA.<ref name="Forbes reporting on 2021 digital sound-alike fraud" />

	'''Reporting on the 2021 digital sound-alike enabled fraud'''		'''Reporting on the 2021 digital sound-alike enabled fraud'''

Juho Kunsola: housekeeping

2024-10-19T20:49:32Z

housekeeping

← Older revision		Revision as of 20:49, 19 October 2024
Line 101:		Line 101:
	<small>[[:File:Deb-2000-reflectance-separation.png\|Original picture]] by [[w:Paul Debevec]] et al. - Copyright ACM 2000 https://dl.acm.org/citation.cfm?doid=311779.344855</small>]]		<small>[[:File:Deb-2000-reflectance-separation.png\|Original picture]] by [[w:Paul Debevec]] et al. - Copyright ACM 2000 https://dl.acm.org/citation.cfm?doid=311779.344855</small>]]

	In the cinemas we have seen digital look-alikes for 20 years. These digital look-alikes have "clothing" (a simulation of clothing is not clothing) or "superhero costumes" and "superbaddie costumes", and they don't need to care about the laws of physics, let alone laws of physiology. It is generally accepted that digital look-alikes made their public debut in the sequels of The Matrix i.e. [[w:The Matrix Reloaded]] and [[w:The Matrix Revolutions]] released in 2003. It can be considered almost certain, that it was not possible to make these before the year 1999, as the final piece of the puzzle to make a (still) digital look-alike that passes human testing, the [[Glossary#Reflectance capture\|reflectance capture]] over the human face, was made for the first time in 1999 at the [[w:University of Southern California]] and was presented to the crème de la crème		In the cinemas we have seen digital look-alikes for over 20 years. These digital look-alikes have "clothing" (a simulation of clothing is not clothing) or "superhero costumes" and "superbaddie costumes", and they don't need to care about the laws of physics, let alone laws of physiology. It is generally accepted that digital look-alikes made their public debut in the sequels of The Matrix i.e. [[w:The Matrix Reloaded]] and [[w:The Matrix Revolutions]] released in 2003. It can be considered almost certain, that it was not possible to make these before the year 1999, as the final piece of the puzzle to make a (still) digital look-alike that passes human testing, the [[Glossary#Reflectance capture\|reflectance capture]] over the human face, was made for the first time in 1999 at the [[w:University of Southern California]] and was presented to the crème de la crème
	of the computer graphics field in their annual gathering SIGGRAPH 2000.<ref name="Deb2000">		of the computer graphics field in their annual gathering SIGGRAPH 2000.<ref name="Deb2000">
	{{cite book		{{cite book

Juho Kunsola: language

2024-10-19T20:48:14Z

language

← Older revision		Revision as of 20:48, 19 October 2024
Line 411:		Line 411:
	If the handwriting-like synthesis passes human and media forensics testing, it is a '''digital handwrite-alike'''.		If the handwriting-like synthesis passes human and media forensics testing, it is a '''digital handwrite-alike'''.

	Here we find a '''risk''' similar to that which ~~realized~~ when the '''[[w:speaker recognition]] systems''' turned out to be instrumental in the development of '''[[#Digital sound-alikes\|digital sound-alikes]]'''. After the knowledge needed to recognize a speaker was [[w:Transfer learning\|w:transferred]] into a generative task in 2018 by Google researchers, we no longer cannot effectively determine for English speakers which recording is human in origin and which is from a machine origin.		Here we find a possible '''risk''' similar to that which became a reality, when the '''[[w:speaker recognition]] systems''' turned out to be instrumental in the development of '''[[#Digital sound-alikes\|digital sound-alikes]]'''. After the knowledge needed to recognize a speaker was [[w:Transfer learning\|w:transferred]] into a generative task in 2018 by Google researchers, we no longer cannot effectively determine for English speakers which recording is human in origin and which is from a machine origin.

	'''Handwriting-like syntheses''':		'''Handwriting-like syntheses''':