Synthetic human-like fakes

Definitions

When the camera does not exist, but the subject being imaged with a simulation of a (movie) camera deceives the watcher to believe it is some living or dead person it is a digital look-alike.

In 2017-2018 this started to be referred to as w:deepfake, even though altering video footage of humans with a computer with a deceiving effect is actually 20 yrs older than the name "deep fakes" or "deepfakes".

When it cannot be determined by human testing or media forensics whether some fake voice is a synthetic fake of some person's voice, or is it an actual recording made of that person's actual real voice, it is a pre-recorded digital sound-alike. This is now commonly referred to as w:audio deepfake.

Real-time digital look-and-sound-alike in a video call was used to defraud a substantial amount of money in 2023.

Read more about synthetic human-like fakes, see and support organizations and events against synthetic human-like fakes and what they are doing, what kinds of Laws against synthesis and other related crimes have been formulated, examine the SSFWIKI timeline of synthetic human-like fakes or view the Mediatheque.

This is not a picture of Obama, because it is not Obama in the video that this screenshot is from, but a synthetic human-like fake, more precisely a pre-recorded digital look-alike.

Click on the picture or Obama's appearance thieved - a public service announcement digital look-alike by Monkeypaw Productions and Buzzfeed to view an April 2018 public service announcement moving digital look-alike made to appear Obama-like. The video is accompanied with imitator sound-alike, and was made by w:Monkeypaw Productions (.com) in conjunction with w:BuzzFeed (.com). You can also View the same video at YouTube.com.

Image 2 (low resolution rip) shows a 1999 technique for sculpting a morphable model, till it matches the target's appearance.
(1) Sculpting a morphable model to one single picture
(2) Produces 3D approximation
(4) Texture capture
(3) The 3D model is rendered back to the image with weight gain
(5) With weight loss
(6) Looking annoyed
(7) Forced to smile Image 2 by Blanz and Vettel – Copyright ACM 1999 – http://dl.acm.org/citation.cfm?doid=311535.311556 – Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.

Digital look-alikes

It is recommended that you watch In Event of Moon Disaster - FULL FILM (2020) at the moondisaster.org project website (where it has interactive portions) by the Center for Advanced Virtuality of the w:MIT

Introduction to digital look-alikes

Image 1: Separating specular and diffuse reflected light

(a) Normal image in dot lighting

(b) Image of the diffuse reflection which is caught by placing a vertical polarizer in front of the light source and a horizontal in the front the camera

(c) Image of the highlight specular reflection which is caught by placing both polarizers vertically

(d) Subtraction of c from b, which yields the specular component

Images are scaled to seem to be the same luminosity.

Original image by Debevec et al. – Copyright ACM 2000 – https://dl.acm.org/citation.cfm?doid=311779.344855 – Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.

Subtraction of the diffuse reflection from the specular reflection yields the specular component of the model's reflectance.

Original picture by w:Paul Debevec et al. - Copyright ACM 2000 https://dl.acm.org/citation.cfm?doid=311779.344855

In the cinemas we have seen digital look-alikes for over 20 years. These digital look-alikes have "clothing" (a simulation of clothing is not clothing) or "superhero costumes" and "superbaddie costumes", and they don't need to care about the laws of physics, let alone laws of physiology. It is generally accepted that digital look-alikes made their public debut in the sequels of The Matrix i.e. w:The Matrix Reloaded and w:The Matrix Revolutions released in 2003. It can be considered almost certain, that it was not possible to make these before the year 1999, as the final piece of the puzzle to make a (still) digital look-alike that passes human testing, the reflectance capture over the human face, was made for the first time in 1999 at the w:University of Southern California and was presented to the crème de la crème of the computer graphics field in their annual gathering SIGGRAPH 2000.

“Do you think that was w:Hugo Weaving's left cheekbone that w:Keanu Reeves punched in with his right fist?”

~ Trad on The Matrix Revolutions

The problems with digital look-alikes

Extremely unfortunately for the humankind, organized criminal leagues, that posses the weapons capability of making believable looking synthetic pornography, are producing on industrial production pipelines terroristic synthetic pornography by animating digital look-alikes and distributing it in the murky Internet in exchange for money stacks that are getting thinner and thinner as time goes by.

These industrially produced pornographic delusions are causing great human suffering, especially in their direct victims, but they are also tearing our communities and societies apart, sowing blind rage, perceptions of deepening chaos, feelings of powerlessness and provoke violence.

These kinds of hate illustration increases and strengthens hate feeling, hate thinking, hate speech and hate crimes and tears our fragile social constructions apart and with time perverts humankind's view of humankind into an almost unrecognizable shape, unless we interfere with resolve.

Children-like sexual abuse images

Sadly by 2023 there is a market for synthetic human-like sexual abuse material that looks like children. See 'Illegal trade in AI child sex abuse images exposed' at bbc.com 2023-06-28 reports w:Stable Diffusion being abused to produce this kind of images. The w:Internet Watch Foundation also reports on the alarming existence of production of synthetic human-like sex abuse material portraying minors. See 'Prime Minister must act on threat of AI as IWF ‘sounds alarm’ on first confirmed AI-generated images of child sexual abuse' at iwf.org.uk (2023-08-18)

Fixing the problems from digital look-alikes

We need to move on 3 fields: legal, technological and cultural.

Technological: Computer vision system like FacePinPoint.com for seeking unauthorized pornography / nudes used to exist 2017-2021 and could be revived if funding is found. It was a service practically identical with SSFWIKI original concept Adequate Porn Watcher AI (concept).

Legal: Legislators around the planet have been waking up to this reality that not everything that seems a video of people is a video of people and various laws have been passed to protect humans and humanity from the menaces of synthetic human-like fakes, mostly digital look-alikes so far, but hopefully humans will be protected also fro other aspects of synthetic human-like fakes by laws. See Laws against synthesis and other related crimes

Age analysis and rejuvenating and aging syntheses

• 'An Overview of Two Age Synthesis and Estimation Techniques' at arxiv.org (.pdf), submitted for review on 2020-01-26
• 'Dual Reference Age Synthesis' at sciencedirect.com (preprint at arxiv.org) published on 2020-10-21 in w:Neurocomputing (journal)
• 'A simple automatic facial aging/rejuvenating synthesis method' at ieeexplore.ieee.org read free at researchgate.net, published at the proceedings of the 2011 IEEE International Conference on Systems, Man and Cybernetics
• 'Age Synthesis and Estimation via Faces: A Survey' at ieeexplore.ieee.org (paywall) at researchgate.net published November 2010

Temporal limit of digital look-alikes

A picture of the 1895 w:Cinematograph

w:History of film technology has information about where the border is.

Digital look-alikes cannot be used to attack people who existed before the technological invention of film. For moving pictures the breakthrough is attributed to w:Auguste and Louis Lumière's w:Cinematograph premiered in Paris on 28 December 1895, though this was only the commercial and popular breakthrough, as even earlier moving pictures exist. (adapted from w:History of film)

The w:Kinetoscope is an even earlier motion picture exhibition device. A prototype for the Kinetoscope was shown to a convention of the National Federation of Women's Clubs on May 20, 1891. The first public demonstration of the Kinetoscope was held at the Brooklyn Institute of Arts and Sciences on May 9, 1893. (Wikipedia)

---

Digital sound-alikes

University of Florida published an antidote to synthetic human-like fake voices in 2022

2022 saw a brilliant counter-measure presented to peers at the 31st w:USENIX Security Symposium 10-12 August 2022 by w:University of Florida Detecting deep-fake audio through vocal tract reconstruction.

The university's foundation has applied for a patent and let us hope that they will w:copyleft the patent as this protective method needs to be rolled out to protect the humanity.

Below transcluded from the article

Detecting deep-fake audio through vocal tract reconstruction is an epic scientific work, against fake human-like voices, from the w:University of Florida in published to peers in August 2022.

University of Florida, the home of Florida Institute for Cybersecurity Research (FICS) at fics.institute.ufl.edu, that brought us the Detecting deep-fake audio through vocal tract reconstruction system to hunt down the fake human-like voices.

The Office of Naval Research (ONR) at nre.navy.mil of the USA funded this breakthrough science.

The work Who Are You (I Really Wanna Know)? Detecting Audio DeepFakes Through Vocal Tract Reconstruction at usenix.org, presentation page, version included in the proceedings and slides from researchers of the Florida Institute for Cybersecurity Research (FICS) at fics.institute.ufl.edu in the w:University of Florida received funding from the w:Office of Naval Research and was presented on 2022-08-11 at the 31st w:USENIX Security Symposium.

This work was done by PhD student Logan Blue, Kevin Warren, Hadi Abdullah, Cassidy Gibson, Luis Vargas, Jessica O’Dell, Kevin Butler and Professor Patrick Traynor.

The University of Florida Research Foundation Inc has filed for and received an US patent titled 'Detecting deep-fake audio through vocal tract reconstruction' registration number US20220036904A1 (link to patents.google.com) with 20 claims. The patent application was published on Thursday 2022-02-03. The patent application was approved on 2023-07-04 and has an adjusted expiration date of Sunday 2041-12-29. PhD student Logan Blue and professor Patrick Traynor wrote an article for the general public on the work titled Deepfake audio has a tell – researchers use fluid dynamics to spot artificial imposter voices at theconversation.com that was published Tuesday 2022-09-20 and permanently w:copylefted it under Creative Commons Attribution-NoDerivatives (CC-BY-ND).

This new counter-measure needs to be rolled out to humans to protect humans against the fake human-like voices.

Below is an exact copy of the original article from the SSF! wordpress titled "Amazing method and results from University of Florida scientists in 2022 against the menaces of digital sound-alikes / audio deepfakes". Thank you to the original writers for having the wisdom of licensing the article under CC-BY-ND.

On known history of digital sound-alikes

A picture of a cut-away titled "Voice-terrorist could mimic a leader" from a 2012 w:Helsingin Sanomat warning that the sound-like-anyone machines are approaching. Thank you to homie Prof. David Martin Howard of the w:University of York, UK and the anonymous editor for the heads-up.

The first English speaking digital sound-alikes were first introduced in 2016 by Adobe and Deepmind, but neither of them were made publicly available.

Then in 2018 at the w:Conference on Neural Information Processing Systems (NeurIPS) the work 'Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis' (at arXiv.org) was presented. The pre-trained model is able to steal voices from a sample of only 5 seconds with almost convincing results

The Iframe below is transcluded from 'Audio samples from "Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis"' at google.gituhub.io, the audio samples of a sound-like-anyone machine presented as at the 2018 w:NeurIPS conference by Google researchers.

Have a listen.

Observe how good the "VCTK p240" system is at deceiving to think that it is a person that is doing the talking.

Reporting on the sound-like-anyone-machines

• "Artificial Intelligence Can Now Copy Your Voice: What Does That Mean For Humans?" May 2019 reporting at forbes.com on w:Baidu Research'es attempt at the sound-like-anyone-machine demonstrated at the 2018 w:NeurIPS conference.

The to the right video 'This AI Clones Your Voice After Listening for 5 Seconds' by '2 minute papers' at YouTube describes the voice thieving machine presented by Google Research in w:NeurIPS 2018.

Video video 'This AI Clones Your Voice After Listening for 5 Seconds' by '2 minute papers' at YouTube describes the voice thieving machine by Google Research in w:NeurIPS 2018.

In November 2024, Nvidia researchers announced they have made and trained a Foundational Generative Audio Transformer (Opus 1) at fugatto.github.io or Fugatto for short.

The researchers state Fugatto is a versatile audio synthesis and transformation model capable of following free-form text instructions with optional audio inputs.

Documented crimes with digital sound-alikes

In 2019 reports of crimes being committed with digital sound-alikes started surfacing. As of Jan 2022 no reports of other types of attack than fraud have been found.

2019 digital sound-alike enabled fraud

By 2019 digital sound-alike anyone technology found its way to the hands of criminals. In 2019 Symantec researchers knew of 3 cases where digital sound-alike technology had been used for w:crime.

Of these crimes the most publicized was a fraud case in March 2019 where 220,000€ were defrauded with the use of a real-time digital sound-alike. The company that was the victim of this fraud had bought some kind of cyberscam insurance from French insurer w:Euler Hermes and the case came to light when Mr. Rüdiger Kirsch of Euler Hermes informed w:The Wall Street Journal about it.

Reporting on the 2019 digital sound-alike enabled fraud

• Fraudsters Used AI to Mimic CEO’s Voice in Unusual Cybercrime Case at wsj.com original reporting, date unknown, updated 2019-08-30

• "Fake voices 'help cyber-crooks steal cash'" at bbc.com July 2019 reporting
• "An artificial-intelligence first: Voice-mimicking software reportedly used in a major theft" at washingtonpost.com documents a w:fraud committed with digital sound-like-anyone-machine, July 2019 reporting.
• A Voice Deepfake Was Used To Scam A CEO Out Of $243,000 at forbes.com, 2019-09-03 reporting

2020 digital sound-alike fraud attempt

In June 2020 fraud was attempted with a poor quality pre-recorded digital sound-alike with delivery method was voicemail. (Listen to a redacted clip at soundcloud.com) The recipient in a tech company didn't believe the voicemail to be real and alerted the company and they realized that someone tried to scam them. The company called in Nisos to investigate the issue. Nisos analyzed the evidence and they were certain it was a fake, but had aspects of a cut-and-paste job to it. Nisos prepared a report titled "The Rise of Synthetic Audio Deepfakes" at nisos.com on the issue and shared it with Motherboard, part of w:Vice (magazine) prior to its release.

2021 digital sound-alike enabled fraud

The 2nd publicly known fraud done with a digital sound-alike took place on Friday 2021-01-15. A bank in Hong Kong was manipulated to wire money to numerous bank accounts by using a voice stolen from one of the their client company's directors. They managed to defraud $35 million of the U.A.E. based company's money.. This case came into light when Forbes saw a document where the U.A.E. financial authorities were seeking administrative assistance from the US authorities towards recovering a small portion of the defrauded money that had been sent to bank accounts in the USA.

Reporting on the 2021 digital sound-alike enabled fraud

• Fraudsters Cloned Company Director’s Voice In $35 Million Bank Heist, Police Find at forbes.com 2021-10-14 original reporting
• Deepfaked Voice Enabled $35 Million Bank Heist in 2020 at unite.ai reporting updated on 2021-10-15
• USD 35m voice cloning heist at aiaaic.org, October 2021 AIAAIC repository entry

More fraud cases with digital sound-alikes

• They thought loved ones were calling for help. It was an AI scam. at washingtonpost.com, March 2023 reporting

Example of a hypothetical 4-victim digital sound-alike attack

A very simple example of a digital sound-alike attack is as follows:

Someone puts a digital sound-alike to call somebody's voicemail from an unknown number and to speak for example illegal threats. In this example there are at least two victims:

1. Victim #1 - The person whose voice has been stolen into a covert model and a digital sound-alike made from it to frame them for crimes
2. Victim #2 - The person to whom the illegal threat is presented in a recorded form by a digital sound-alike that deceptively sounds like victim #1
3. Victim #3 - It could also be viewed that victim #3 is our law enforcement systems as they are put to chase after and interrogate the innocent victim #1
4. Victim #4 - Our judiciary which prosecutes and possibly convicts the innocent victim #1.

Examples of speech synthesis software not quite able to fool a human yet

Some other contenders to create digital sound-alikes are though, as of 2019, their speech synthesis in most use scenarios does not yet fool a human because the results contain tell tale signs that give it away as a speech synthesizer.

• Lyrebird.ai (listen)
• CandyVoice.com (test with your choice of text)
• Merlin, a w:neural network based speech synthesis system by the Centre for Speech Technology Research at the w:University of Edinburgh
• 'Neural Voice Cloning with a Few Samples at papers.nips.cc, w:Baidu Research'es shot at sound-like-anyone-machine did not convince in 2018

Temporal limit of digital sound-alikes

w:Thomas Edison and his early w:phonograph. Cropped from w:Library of Congress copy, ca. 1877, (probably 18 April 1878)

The temporal limit of whom, dead or living, the digital sound-alikes can attack is defined by the w:history of sound recording.

The article starts by mentioning that the invention of the w:phonograph by w:Thomas Edison in 1877 is considered the start of sound recording.

The phonautograph is the earliest known device for recording w:sound. Previously, tracings had been obtained of the sound-producing vibratory motions of w:tuning forks and other objects by physical contact with them, but not of actual sound waves as they propagated through air or other media. Invented by Frenchman W:Édouard-Léon Scott de Martinville, it was patented on March 25, 1857.

Apparently, it did not occur to anyone before the 1870s that the recordings, called phonautograms, contained enough information about the sound that they could, in theory, be used to recreate it. Because the phonautogram tracing was an insubstantial two-dimensional line, direct physical playback was impossible in any case. Several phonautograms recorded before 1861 were successfully played as sound in 2008 by optically scanning them and using a computer to process the scans into digital audio files. (Wikipedia)

A w:spectrogram of a male voice saying 'nineteenth century'

What should we do about digital sound-alikes?

Living people can defend themselves against digital sound-alike by denying the things the digital sound-alike says if they are presented to the target, but dead people cannot. Digital sound-alikes offer criminals new disinformation attack vectors and wreak havoc on provability.

For these reasons the bannable raw materials i.e. covert voice models should be prohibited by law in order to protect humans from abuse by criminal parties.

It is high time to act and to criminalize the covert modeling of human voice!

Digital look-and-sound-alikes

Real-time digital look-and-sound-alike fraud in 2023

Real-time digital look-and-sound-alike in a video call was used to defraud a substantial amount of money in 2023.

Real-time digital look-and-sound-alike fraud in 2024

Reporting

• Finance worker pays out $25 million after video call with deepfake "chief financial officer" at edition.cnn.com, February 2024 reporting by Heather Chen and Kathleen Magramo, CNN

---

Text syntheses

w:Chatbots and w:spamming have existed for a longer time, but only now armed with AI they are becoming more deceiving.

In w:natural language processing development in w:natural-language understanding leads to more cunning w:natural-language generation AI.

w:Large language models (LLM) are very large w:language models consisting of a w:neural network with many parameters.

w:OpenAI's w:Generative Pre-trained Transformer (GPT) is a left-to-right w:transformer (machine learning model)-based text generation model succeeded by w:GPT-2 and w:GPT-3

November 2022 saw the publication of OpenAI's w:ChatGPT, a conversational artificial intelligence.

w:Bard (chatbot) is a conversational w:generative artificial intelligence w:chatbot developed by w:Google, based on the w:LaMDA family of w:large language models. It was developed as a direct response to the rise of w:OpenAI's w:ChatGPT, and was released in March 2023. (Wikipedia)

Reporting / announcements (in reverse chronology)

• Reinventing search with a new AI-powered Microsoft Bing and Edge, your copilot for the web at blogs.microsoft.com February 2023 (2023-02-07). The new improved Bing, available only in Microsoft's Edge browser is reportedly based on a language model refined from GPT 3.5.

• New AI classifier for indicating AI-written text at openai.com, a January 2023 blog post about OpenAI's AI classifier for detecting AI-written texts.

• Introducing ChatGPT at openai.com November 2022 (2022-11-30)

• 'A college kid’s fake, AI-generated blog fooled tens of thousands. This is how he made it.' at technologyreview.com August 2020 reporting in the w:MIT Technology Review by Karen Hao about GPT-3.

• 'OpenAI’s latest AI text generator GPT-3 amazes early adopters' at siliconangle.com July 2020 reporting on GPT-3

• OpenAI releases the full version of GPT-2 at openai.com in August 2019

• 'OpenAI releases curtailed version of GPT-2 language model' at venturebeat.com, August 2019 reporting on the original release of of the curtailed version of GPT-2

External links

• "Detection of Fake and False News (Text Analysis): Approaches and CNN as Deep Learning Model" at analyticsteps.com, a 2019 summmary written by Shubham Panth.

Detectors for synthesized texts

Introduction of w:ChatGPT by OpenAI brought the need for software to detect machine-generated texts.

Try AI plagiarism detection for free

• AI Content Detector at contentdetector.ai- AI Content Detector - Detect ChatGPT Plagiarism (try for free)
• AI Text Classifier at platform.openai.com- The AI Text Classifier is a fine-tuned GPT model that predicts how likely it is that a piece of text was generated by AI from a variety of sources, such as ChatGPT. (free account required)
• GPT Radar at gptradar.com - AI text detector app (try for free)
• GPTZero at gptzero.me - The World's #1 AI Detector with over 1 Million Users (try for free)
• Plagiarism Checker at copyleaks.com- Plagiarism Checker by Copyleaks (try for free)
• https://gowinston.ai/ - The most powerful AI content detection solution (free-tier available)
• ZeroGPT at zerogpt.com - GPT-4 And ChatGPT detector by ZeroGPT: detect OpenAI text - ZeroGPT the most Advanced and Reliable Chat GPT and GPT-4 detector tool (try for free)

For-a-fee AI plagiarism detection tools

• https://originality.ai/ - The Most Accurate AI Content Detector and Plagiarism Checker Built for Serious Content Publishers
• https://www.turnitin.com/ - Empower students to do their best, original work

Handwriting syntheses

Handwriting syntheses could be used

1. Defensively, to hide one's handwriting style from public view
2. Offensively, to thieve somebody else's handwriting style

If the handwriting-like synthesis passes human and media forensics testing, it is a digital handwrite-alike.

Here we find a possible risk similar to that which became a reality, when the w:speaker recognition systems turned out to be instrumental in the development of digital sound-alikes. After the knowledge needed to recognize a speaker was w:transferred into a generative task in 2018 by Google researchers, we no longer cannot effectively determine for English speakers which recording is human in origin and which is from a machine origin.

Handwriting-like syntheses: w:Recurrent neural networks (RNN) seem are a popular choice for this task.

• GitHub topic handwriting-synthesis has 29 public repositories as of September 2021.
• GitHub topic handwriting-generation has 21 public repositories as of September 2021.

• Deep imitator: Handwriting calligraphy imitation via deep attention networks' at sciencedirect.com, published in w:Pattern Recognition (journal) in August 2020.

• Scribe - Generating Realistic Handwriting with TensorFlow at greydanus.github.io blog post published on 2016-08-21. Scribe code at github.com

• My Text in Your Handwriting at dl.acm.org, a system from w:University College London published on 2016-05-18 in w:ACM Transactions on Graphics.

• Generating Sequences With Recurrent Neural Networks at arxiv.org by Alex Graves published on 2013-08-04 in Neural and Evolutionary Computing.

1. Recurrent neural network handwriting generation demo at cs.toronto.edu is a demonstration site for publication
2. Calligrapher.ai - Realistic computer-generated handwriting - The user may control parameters: speed, legibility, stroke width and style. The domain is registered by some organization in Iceland and the website offers no about-page. According to this reddit post Calligrapher.ai is based on Graves' 2013 work, but "adds an w:inference model to allow for sampling latent style vectors (similar to the VAE model used by SketchRNN)".

Handwriting recognition

• w:Handwriting recognition (HWR), also known as Handwritten Text Recognition (HTR), is the ability of a computer to receive and interpret intelligible w:handwritten input (Wikipedia)
• w:Intelligent word recognition, or IWR, is the recognition of unconstrained handwritten words. (Wikipedia)

• GitHub topic handwriting-recognition contains 238 repositories as of September 2021.

Singing syntheses

As of 2020 the digital sing-alikes may not yet be here, but when we hear a faked singing voice and we cannot hear that it is fake, then we will know. An ability to sing does not seem to add much hostile capabilities compared to the ability to thieve spoken word.

• 'Fast and High-Quality Singing Voice Synthesis System based on Convolutional Neural Networks' at arxiv.org, a 2019 singing voice synthesis technique using w:convolutional neural networks (CNN). Accepted into the 2020 International Conference on Acoustics, Speech, and Signal Processing (ICASSP).
• 'State of art of real-time singing voice synthesis' at compmus.ime.usp.br presented at the 2019 17th Brazilian Symposium on Computer Music
• 'Synthesis and expressive transformation of singing voice' at theses.fr as .pdf a 2017 doctorate thesis by Luc Ardaillon
• 'Synthesis of the Singing Voice by Performance Sampling and Spectral Models' at mtg.upf.edu, a 2007 journal article in the w:IEEE Signal Processing Society's Signal Processing Magazine
• 'Speech-to-Singing Synthesis: Converting Speaking Voices to Singing Voices by Controlling Acoustic Features Unique to Singing Voices' at researchgate.net, a November 2007 paper published in the IEEE conference on Applications of Signal Processing to Audio and Acoustics

• w:Category:Singing software synthesizers

---

Timeline of synthetic human-like fakes

See the #SSFWIKI Mediatheque for viewing media that is or is probably to do with synthetic human-like fakes.

2020's synthetic human-like fakes

• 2023 | Real-time digital look-and-sound-alike crime | In April a man in northern China was defrauded of 4.3 million yuan by a criminal employing a digital look-and-sound-alike pretending to be his friend on a video call made with a stolen messaging service account.

• 2023 | Election meddling with digital look-alikes | The w:2023 Turkish presidential election saw numerous deepfake controversies.

• • "Ahead of the election in Turkey, President Recep Tayyip Erdogan showed a video linking his main challenger Kemal Kilicdaroglu to the militant Kurdish organization PKK." [...] "Research by DW's fact-checking team in cooperation with DW's Turkish service shows that the video at the campaign rally was manipulated by combining two separate videos with totally different backgrounds and content." reports dw.com

• 2023 | March 7 th | science / demonstration | Microsoft researchers submitted a paper for publication outlining their Cross-lingual neural codec language modeling system at arxiv.org dubbed VALL-E X at microsoft.com, that extends upon VALL-E's capabilities to be cross-lingual and also maintaining the same "emotional tone" from sample to fake.

• 2023 | January 5th | science / demonstration | Microsoft researchers announced VALL-E - Neural Codec Language Models are Zero-Shot Text to Speech Synthesizers (at microsoft.com) that is able to thieve a voice from only 3 seconds of sample and it is also able to mimic the "emotional tone" of the sample the synthesis if produced of.

• 2023 | January 1st | Law | Law on sexual offences in Finland 2023 is found in Chapter 20 of the Finnish Criminal Code titled "Seksuaalirikoksista" ("Sexual offences") and came into effect on Sunday 2023-01-01.

The new law in Finland protects adults against sexual image based abuse be it real or synthetic in origin.

Other countries have also woken up to the problems of synthesis crime and have legislated laws against synthesis and other related crimes.

Relevant sections of Chapter 20

• 7 § Non-consensual dissemination of a sexual image criminalizes distribution of unauthorized real and synthetic sexual images without permission. (7 § Seksuaalisen kuvan luvaton levittäminen)
• 19 § Distribution of an image depicting a child in a sexual manner criminalizes the distribution of real and synthetic child sexual abuse material (CSAM). Attempting this crime is also punishable. (19 § Lasta seksuaalisesti esittävän kuvan levittäminen)
• 20 § Aggravated distribution of an image depicting a child in a sexual manner defines the parameters for aggravated form of the crime of making CSAM available. (20 § Törkeä lasta seksuaalisesti esittävän kuvan levittäminen)
• 21 § Possession of an image depicting a child in a sexual manner criminalizes the possession of CSAM and acquiring access for the intent to access CSAM. (21 § Lasta seksuaalisesti esittävän kuvan hallussapito))

This 2023 upgrade and gather-together of the Finnish Criminal Code on sexual offences was made upon the initiative of the 2019-2023 w:Marin Cabinet, was voted into law by the w:Members of the Parliament of Finland, 2019–2023 and it came into effect on Sunday 2023-01-01.

Translation to English by the Ministry of Justice: Criminal Code (39/1889) - Chapter 20 - Sexual offences (translation) as .pdf at oikeusministerio.fi (subject to possible revisions)

• 2022 | science and demonstration | w:OpenAI(.com) published w:ChatGPT, a discutational AI accessible with a free account at chat.openai.com. Initial version was published on 2022-11-30.

• 2022 | brief report of counter-measures | Protecting world leaders against deep fakes using facial, gestural, and vocal mannerisms is a brief report by Matyáš Boháček and Hany Farid on their recent work published on Wednesday 2022-11-23 in w:Proceedings of the National Academy of Sciences of the United States of America (PNAS). 'Protecting world leaders against deep fakes using facial, gestural, and vocal mannerisms' at pnas.org Publication date 2022-11-23.

• 2022 | counter-measure | Detecting deep-fake audio through vocal tract reconstruction is an epic scientific work, against fake human-like voices, from the w:University of Florida in published to peers in August 2022.

University of Florida, the home of Florida Institute for Cybersecurity Research (FICS) at fics.institute.ufl.edu, that brought us the Detecting deep-fake audio through vocal tract reconstruction system to hunt down the fake human-like voices.

The Office of Naval Research (ONR) at nre.navy.mil of the USA funded this breakthrough science.

The work Who Are You (I Really Wanna Know)? Detecting Audio DeepFakes Through Vocal Tract Reconstruction at usenix.org, presentation page, version included in the proceedings and slides from researchers of the Florida Institute for Cybersecurity Research (FICS) at fics.institute.ufl.edu in the w:University of Florida received funding from the w:Office of Naval Research and was presented on 2022-08-11 at the 31st w:USENIX Security Symposium.

This work was done by PhD student Logan Blue, Kevin Warren, Hadi Abdullah, Cassidy Gibson, Luis Vargas, Jessica O’Dell, Kevin Butler and Professor Patrick Traynor.

The University of Florida Research Foundation Inc has filed for and received an US patent titled 'Detecting deep-fake audio through vocal tract reconstruction' registration number US20220036904A1 (link to patents.google.com) with 20 claims. The patent application was published on Thursday 2022-02-03. The patent application was approved on 2023-07-04 and has an adjusted expiration date of Sunday 2041-12-29.

PhD student Logan Blue and professor Patrick Traynor wrote an article for the general public on the work titled Deepfake audio has a tell – researchers use fluid dynamics to spot artificial imposter voices at theconversation.com that was published Tuesday 2022-09-20 and permanently w:copylefted it under Creative Commons Attribution-NoDerivatives (CC-BY-ND).. Presented to peers in August 2022 and to the general public in September 2022.

• 2022 | disinformation attack | In June 2022 a fake digital look-and-sound-alike in the appearance and voice of w:Vitali Klitschko, mayor of w:Kyiv, held fake video phone calls with several European mayors. The Germans determined that the video phone call was fake by contacting the Ukrainian officials. This attempt at covert disinformation attack was originally reported by w:Der Spiegel.

• 2022 | science | w:DALL-E 2, a successor designed to generate more realistic images at higher resolutions that "can combine concepts, attributes, and styles" was published in April 2022. (Wikipedia)

• 2022 | counter-measure | Protecting President Zelenskyy against deep fakes 'Protecting President Zelenskyy against Deep Fakes' at arxiv.org by Matyáš Boháček of Johannes Kepler Gymnasium and w:Hany Farid, the dean and head of of w:Berkeley School of Information at the University of California, Berkeley. This brief paper describes their automated digital look-alike detection system and evaluate its efficacy and reliability in comparison to humans with untrained eyes. Their work provides automated evaluation tools to catch so called "deep fakes" and their motivation seems to have been to find automation armor against disinformation warfare against humans and the humanity. Automated digital media forensics is a very good idea explored by many. Boháček and Farid 2022 detection system works by evaluating both facial mannerisms as well as gestural mannerisms to detect the non-human ones from the ones that are human in origin. Preprint published in February 2022 and submitted to w:arXiv in June 2022

• 2022 | science / review of counter-measures | 'A Review of Modern Audio Deepfake Detection Methods: Challenges and Future Directions' at mdpi.com, a review of audio deepfake detection methods by researchers Zaynab Almutairi and Hebah Elgibreen of the w:King Saud University, Saudi Arabia published in w:Algorithms (journal) on Wednesday 2022-05-04 published by the w:MDPI (Multidisciplinary Digital Publishing Institute). This article belongs to the Special Issue Commemorative Special Issue: Adversarial and Federated Machine Learning: State of the Art and New Perspectives at mdpi.com

• 2022 | science / counter-measure | 'Attacker Attribution of Audio Deepfakes' at arxiv.org, a pre-print was presented at the Interspeech 2022 conference organized by w:International Speech Communication Association in Korea September 18-22 2022.

• 2021 | Science and demonstration | In the NeurIPS 2021 held virtually in December researchers from Nvidia and w:Aalto University present their paper Alias-Free Generative Adversarial Networks (StyleGAN3) at nvlabs.github.io and associated implementation in w:PyTorch and the results are deceivingly human-like in appearance. StyleGAN3 paper as .pdf at nvlabs-fi-cdn.nvidia.com

• 2021 | Entertainment | The Swedish pop band w:ABBA published an album in September and will be performing shows where the music is live and real, but the visuals will be rejuvenated digital look-alikes of the band members displayed to the fans with w:holography technology. ABBA used w:Industrial Light & Magic as the purveyor of technology. w:Industrial Light & Magic was acquired by w:The Walt Disney Company in 2012 as part of their acquisition w:Lucasfilm.

• 2021 | Controversy | July 2021 saw the release of w:Roadrunner: A Film About Anthony Bourdain and soon controversy arose, as the director w:Morgan Neville admitted to w:Helen Rosner, a food writer for w:The New Yorker that he had contracted an AI company to thieve w:Anthony Bourdain's voice and used it to insert audio that sounded like him, without declaring it as faked.

• 2021 | Science | Voice Cloning: a Multi-Speaker Text-to-Speech Synthesis Approach based on Transfer Learning .pdf at arxiv.org, a paper submitted in Feb 2021 by researchers from the w:University of Turin.

• 2021 | crime / fraud | The 2nd publicly known fraud done with a digital sound-alike took place on Friday 2021-01-15. A bank in Hong Kong was manipulated to wire money to numerous bank accounts by using a voice stolen from one of the their client company's directors. They managed to defraud $35 million of the U.A.E. based company's money.. This case came into light when Forbes saw a document where the U.A.E. financial authorities were seeking administrative assistance from the US authorities towards recovering a small portion of the defrauded money that had been sent to bank accounts in the USA.

Reporting on the 2021 digital sound-alike enabled fraud

• Fraudsters Cloned Company Director’s Voice In $35 Million Bank Heist, Police Find at forbes.com 2021-10-14 original reporting
• Deepfaked Voice Enabled $35 Million Bank Heist in 2020 at unite.ai reporting updated on 2021-10-15
• USD 35m voice cloning heist at aiaaic.org, October 2021 AIAAIC repository entry

• 2021 | science and demonstration | DALL-E, a w:deep learning model developed by w:OpenAI to generate digital images from w:natural language descriptions, called "prompts" was published in January 2021. DALL-E uses a version of w:GPT-3 modified to generate images. (Adapted from Wikipedia)

• 2020 | counter-measure | The 'AI Incident Database' at incidentdatabase.ai was introduced on 2020-11-18 by the w:Partnership on AI.

In Dec 2020 Channel 4 aired a Queen-like fake i.e. they had thieved the appearance of Queen Elizabeth II using deepfake methods.

• 2020 | Controversy / Public service announcement | Channel 4 thieved the appearance of Queen Elizabeth II using deepfake methods. The product of synthetic human-like fakery originally aired on Channel 4 on 25 December at 15:25 GMT. View in YouTube

• 2020 | reporting | "Deepfake porn is now mainstream. And major sites are cashing in" at wired.co.uk by Matt Burgess. Published August 2020.

• 2020 | demonstration | Moondisaster.org (full film embedded in website) project by the Center for Advanced Virtuality of the w:MIT published in July 2020, makes use of various methods of making a synthetic human-like fake. Alternative place to watch: In Event of Moon Disaster - FULL FILM at youtube.com
  • Cnet.com July 2020 reporting MIT releases deepfake video of 'Nixon' announcing NASA Apollo 11 disaster

• 2020 | US state law |
  

  Homie w:Marc Berman, a righteous fighter for our human rights in this age of industrial disinformation filth and an Assemblymember of the w:California State Assembly, most loved for authoring AB-602 - Depiction of individual using digital or electronic technology: sexually explicit material: cause of action, which came into effect on Jan 1 2020, banning both the manufacturing and w:digital distribution of synthetic pornography without the w:consent of the people depicted.
  January 1 2020 the w:California w:US state law "AB-602 Depiction of individual using digital or electronic technology: sexually explicit material: cause of action." came into effect in the civil code of the w:California Codes banning the manufacturing and w:digital distribution of synthetic pornography without the w:consent of the people depicted. AB-602 provides victims of synthetic pornography with w:injunctive relief and poses legal threats of w:statutory and w:punitive damages on w:criminals making or distributing synthetic pornography without consent. The bill AB-602 was signed into law by California w:Governor w:Gavin Newsom on October 3 2019 and was authored by w:California State Assemblymember w:Marc Berman and an identical Senate bill was coauthored by w:California Senator w:Connie Leyva. AB602 at trackbill.com

• 2020 | Chinese legislation | On Wednesday January 1 2020 Chinese law requiring that synthetically faked footage should bear a clear notice about its fakeness came into effect. Failure to comply could be considered a w:crime the w:Cyberspace Administration of China (cac.gov.cn) stated on its website. China announced this new law in November 2019. The Chinese government seems to be reserving the right to prosecute both users and w:online video platforms failing to abide by the rules.

2010's synthetic human-like fakes

• 2019 | science and demonstration | At the December 2019 NeurIPS conference, a novel method for making animated fakes of anything with AI First Order Motion Model for Image Animation (website at aliaksandrsiarohin.github.io), (paper) (github) was presented.
  • Reporting Memers are making deepfakes, and things are getting weird at technologyreview.com, 2020-08-28 by Karen Hao.
• 2019 | demonstration | In September 2019 w:Yle, the Finnish w:public broadcasting company, aired a result of experimental w:journalism, a deepfake of the President in office w:Sauli Niinistö in its main news broadcast for the purpose of highlighting the advancing disinformation technology and problems that arise from it.
• 2019 | US state law | On September 1 2019 w:Texas Senate bill SB 751 - Relating to the creation of a criminal offense for fabricating a deceptive video with intent to influence the outcome of an election w:amendments to the election code came into effect in the w:Law of Texas, giving w:candidates in w:elections a 30-day protection period to the elections during which making and distributing digital look-alikes or synthetic fakes of the candidates is an offense. The law text defines the subject of the law as "a video, created with the intent to deceive, that appears to depict a real person performing an action that did not occur in reality" SB 751 was introduced to the Senate by w:Bryan Hughes (politician).

• 2019 | US state law |
  

  Homie w:Marcus Simon (marcussimon.com) is a Member of the w:Virginia House of Delegates and a true pioneer in legislating against synthetic filth.
  Since July 1 2019 w:Virginia w:has criminalized the sale and dissemination of unauthorized synthetic pornography, but not the manufacture., as section § 18.2-386.2 titled 'Unlawful dissemination or sale of images of another; penalty.' became part of the w:Code of Virginia.

Code of Virginia (TOC) » Title 18.2. Crimes and Offenses Generally » Chapter 8. Crimes Involving Morals and Decency » Article 5. Obscenity and Related Offenses » Section § 18.2-386.2. Unlawful dissemination or sale of images of another; penalty

The section § 18.2-386.2. Unlawful dissemination or sale of images of another; penalty. of Virginia is as follows:

A. Any w:person who, with the w:intent to w:coerce, w:harass, or w:intimidate, w:maliciously w:disseminates or w:sells any videographic or still image created by any means whatsoever that w:depicts another person who is totally w:nude, or in a state of undress so as to expose the w:genitals, pubic area, w:buttocks, or female w:breast, where such person knows or has reason to know that he is not w:licensed or w:authorized to disseminate or sell such w:videographic or w:still image is w:guilty of a Class 1 w:misdemeanor.

For purposes of this subsection, "another person" includes a person whose image was used in creating, adapting, or modifying a videographic or still image with the intent to depict an actual person and who is recognizable as an actual person by the person's w:face, w:likeness, or other distinguishing characteristic.

B. If a person uses w:services of an w:Internet service provider, an electronic mail service provider, or any other information service, system, or access software provider that provides or enables computer access by multiple users to a computer server in committing acts prohibited under this section, such provider shall not be held responsible for violating this section for content provided by another person.

C. Venue for a prosecution under this section may lie in the w:jurisdiction where the unlawful act occurs or where any videographic or still image created by any means whatsoever is produced, reproduced, found, stored, received, or possessed in violation of this section.

D. The provisions of this section shall not preclude prosecution under any other w:statute.

The identical bills were House Bill 2678 presented by w:Delegate w:Marcus Simon to the w:Virginia House of Delegates on January 14 2019 and three day later an identical Senate bill 1736 was introduced to the w:Senate of Virginia by Senator w:Adam Ebbin.

• 2019 | Science | Sample Efficient Adaptive Text-to-Speech .pdf at arxiv.org, a 2019 paper from Google researchers, published as a conference paper at w:International Conference on Learning Representations (ICLR)

• 2019 | science and demonstration | 'Speech2Face: Learning the Face Behind a Voice' at arXiv.org a system for generating likely facial features based on the voice of a person, presented by the w:MIT Computer Science and Artificial Intelligence Laboratory at the 2019 w:CVPR. Speech2Face at github.com This may develop to something that really causes problems. "Speech2Face: Neural Network Predicts the Face Behind a Voice" reporing at neurohive.io, "Speech2Face Sees Voices and Hears Faces: Dreams Come True with AI" reporting at belitsoft.com

• 2019 | crime | w:Fraud with digital sound-alike technology surfaced in 2019. See 'An artificial-intelligence first: Voice-mimicking software reportedly used in a major theft', a 2019 Washington Post article or 'A Voice Deepfake Was Used To Scam A CEO Out Of $243,000' at Forbes.com (2019-09-03)

• 2019 | demonstration | 'Which Face is real?' at whichfaceisreal.com is an easily unnerving game by Carl Bergstrom and Jevin West where you need to try to distinguish from a pair of photos which is real and which is not. A part of the "tools" of the Calling Bullshit course taught at the w:University of Washington. Relevancy: certain

• 2019 | demonstration | 'Thispersondoesnotexist.com' (since February 2019) by Philip Wang. It showcases a w:StyleGAN at the task of making an endless stream of pictures that look like no-one in particular, but are eerily human-like. Relevancy: certain

• 2019 | action | w:Nvidia w:open sources w:StyleGAN, a novel w:generative adversarial network.

• 2018 | counter-measure | In September 2018 Google added “involuntary synthetic pornographic imagery” to its ban list, allowing anyone to request the search engine block results that falsely depict them as “nude or in a sexually explicit situation.” Information on removing involuntary fake pornography from Google at support.google.com if it shows up in Google and the form to request removing involuntary fake pornography at support.google.com, select "I want to remove: A fake nude or sexually explicit picture or video of myself"

w:Google's logo. Google Research demonstrated their sound-like-anyone-machine at the 2018 w:Conference on Neural Information Processing Systems (NeurIPS). It requires only 5 seconds of sample to steal a voice.

• 2018 | science and demonstration | The work 'Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis' (at arXiv.org) was presented at the 2018 w:Conference on Neural Information Processing Systems (NeurIPS). The pre-trained model is able to steal voices from a sample of only 5 seconds with almost convincing results.

• 2018 | science | Progressive Growing of GANs for Improved Quality, Stability, and Variation at arxiv.org (.pdf), colloquially known as ProGANs were presented by Nvidia researchers at the 2018 ICLR. w:International Conference on Learning Representations

• 2018 | demonstration | At the 2018 w:World Internet Conference in w:Wuzhen the w:Xinhua News Agency presented two digital look-alikes made to the resemblance of its real news anchors Qiu Hao (w:Chinese language) and Zhang Zhao (w:English language). The digital look-alikes were made in conjunction with w:Sogou. Neither the w:speech synthesis used nor the gesturing of the digital look-alike anchors were good enough to deceive the watcher to mistake them for real humans imaged with a TV camera.

• 2018 | action | Deep Fakes letter to the Office of the Director of National Intelligence at schiff.house.gov, a letter sent to the w:Director of National Intelligence on 2018-09-13 by congresspeople w:Adam Schiff, w:Stephanie Murphy and w:Carlos Curbelo requesting a report be compiled on the synthetic human-like fakes situation and what are the threats and what could be the solutions.

• 2018 | controversy / demonstration | The w:deepfakes controversy surfaces where porn videos were doctored utilizing w:deep machine learning so that the face of the actress was replaced by the software's opinion of what another persons face would look like in the same pose and lighting.

• 2017 | science | 'Synthesizing Obama: Learning Lip Sync from Audio' at grail.cs.washington.edu. In SIGGRAPH 2017 by Supasorn Suwajanakorn et al. of the w:University of Washington presented an audio driven digital look-alike of upper torso of Barack Obama. It was driven only by a voice track as source data for the animation after the training phase to acquire w:lip sync and wider facial information from w:training material consisting 2D videos with audio had been completed. Relevancy: certain

• 2016 | movie | w:Rogue One is a Star Wars film for which digital look-alikes of actors w:Peter Cushing and w:Carrie Fisher were made. In the film their appearance would appear to be of same age as the actors were during the filming of the original 1977 w:Star Wars (film) film.

• 2016 | science / demonstration | w:DeepMind's w:WaveNet owned by w:Google also demonstrated ability to steal people's voices

w:Adobe Inc.'s logo. We can thank Adobe for publicly demonstrating their sound-like-anyone-machine in 2016 before an implementation was sold to criminal organizations.

#w:Adobe Voco. Adobe Audio Manipulator Sneak Peak with w:Jordan Peele (at Youtube.com). November 2016 demonstration of a Adobe's unreleased sound-like-anyone-machine, the w:Adobe Voco at the w:Adobe MAX 2016 event in w:San Diego, w:California. The original Adobe Voco required 20 minutes of sample to thieve a voice.

• 2016 | science and demonstration | w:Adobe Inc. publicly demonstrates w:Adobe Voco, a sound-like-anyone machine '#VoCo. Adobe Audio Manipulator Sneak Peak with Jordan Peele | Adobe Creative Cloud' on Youtube. THe original Adobe Voco required 20 minutes of sample to thieve a voice. Relevancy: certain.

• 2016 | science | 'Face2Face: Real-time Face Capture and Reenactment of RGB Videos' at Niessnerlab.org A paper (with videos) on the semi-real-time 2D video manipulation with gesture forcing and lip sync forcing synthesis by Thies et al, Stanford. Relevancy: certain

• 2016 | music video | 'Plug' by Kube at youtube.com - A 2016 music video by w:Kube (rapper) (w:fi:Kube), that shows deepfake-like technology this early. Video was uploaded on 2016-09-15 and is directed by Faruk Nazeri.

• 2015 | Science | 'Deep Learning Face Attributes in the Wild' at arxiv.org presented at the 2015 w:International Conference on Computer Vision

• 2015 | movie | In the w:Furious 7 a digital look-alike made of the actor w:Paul Walker who died in an accident during the filming was done by w:Weta Digital to enable the completion of the film.

• 2014 | science | w:Ian Goodfellow et al. presented the principles of a w:generative adversarial network. GANs made the headlines in early 2018 with the w:deepfakes controversies.

• 2013 | demonstration | At the 2013 SIGGGRAPH w:Activision and USC presented a w:real time computing "Digital Ira" a digital face look-alike of Ari Shapiro, an ICT USC research scientist, utilizing the USC light stage X by Ghosh et al. for both reflectance field and motion capture. The end result both precomputed and real-time rendering with the modernest game w:GPU shown here and looks fairly realistic.

• 2013 | demonstration | A 'Scanning and Printing a 3D Portrait of President Barack Obama' at ict.usc.edu. A 7D model and a 3D bust was made of President Obama with his consent. Relevancy: Relevancy: certain

• 2011 | Law in Finland | Distribution and attempt of distribution and also possession of synthetic CSAM was criminalized on Wednesday 2011-06-01, upon the initiative of the w:Vanhanen II Cabinet. These protections against CSAM were moved into 19 §, 20 § and 21 § of Chapter 20 when the Law on sexual offences in Finland 2023 was improved and gathered into Chapter 20 upon the initiative of the w:Marin Cabinet.

2000's synthetic human-like fakes

• 2010 | movie | w:Walt Disney Pictures released a sci-fi sequel entitled w:Tron: Legacy with a digitally rejuvenated digital look-alike made of the actor w:Jeff Bridges playing the w:antagonist w:CLU.

• 2009 | movie | A digital look-alike of a younger w:Arnold Schwarzenegger was made for the movie w:Terminator Salvation though the end result was critiqued as unconvincing. Facial geometry was acquired from a 1984 mold of Schwarzenegger.

• 2009 | demonstration | Paul Debevec: 'Animating a photo-realistic face' at ted.com Debevec et al. presented new digital likenesses, made by w:Image Metrics, this time of actress w:Emily O'Brien whose reflectance was captured with the USC light stage 5. At 00:04:59 you can see two clips, one with the real Emily shot with a real camera and one with a digital look-alike of Emily, shot with a simulation of a camera - Which is which is difficult to tell. Bruce Lawmen was scanned using USC light stage 6 in still position and also recorded running there on a w:treadmill. Many, many digital look-alikes of Bruce are seen running fluently and natural looking at the ending sequence of the TED talk video. Motion looks fairly convincing contrasted to the clunky run in the w:Animatrix: Final Flight of the Osiris which was w:state-of-the-art in 2003 if photorealism was the intention of the w:animators.

• 2004 | movie | The w:Spider-man 2 (and w:Spider-man 3, 2007) films. Relevancy: The films include a digital look-alike made of actor w:Tobey Maguire by w:Sony Pictures Imageworks.

• 2003 | short film | w:The Animatrix: Final Flight of the Osiris a w:state-of-the-art want-to-be human likenesses not quite fooling the watcher made by w:Square Pictures.

Logo of the w:The Matrix (franchise)

Traditional w:BRDF vs. subsurface scattering inclusive BSSRDF i.e. w:Bidirectional scattering-surface reflectance distribution function.

An analytical BRDF must take into account the subsurface scattering, or the end result will not pass human testing.

• 2003 | movie(s) | The w:Matrix Reloaded and w:Matrix Revolutions films. Relevancy: First public display of digital look-alikes that are virtually indistinguishable from the real actors. 'Universal Capture - Image-based Facial Animation for "The Matrix Reloaded"' at researchgate.net (2003)

Music video for Bullet by w:Covenant from 2002. Here you can observe the classic "skin looks like cardboard"-bug that stopped the pre-reflectance capture era versions from passing human testing.

• 2002 | music video | 'Bullet' by Covenant on Youtube by w:Covenant (band) from their album w:Northern Light (Covenant album). Relevancy: Contains the best upper-torso digital look-alike of Eskil Simonsson (vocalist) that their organization could procure at the time. Here you can observe the classic "skin looks like cardboard"-bug (assuming this was not intended) that thwarted efforts to make digital look-alikes that pass human testing before the reflectance capture and dissection in 1999 by w:Paul Debevec et al. at the w:University of Southern California and subsequent development of the "Analytical w:BRDF" (quote-unquote) by ESC Entertainment, a company set up for the sole purpose of making the cinematography for the 2003 films Matrix Reloaded and Matrix Revolutions possible, lead by George Borshukov.

1990's synthetic human-like fakes

Logo of the w:Institute for Creative Technologies founded in 1999 in the w:University of Southern California by the w:United States Army

• 1999 | science | 'Acquiring the reflectance field of a human face' paper at dl.acm.org w:Paul Debevec et al. of w:USC did the first known reflectance capture over the human face with their extremely simple w:light stage. They presented their method and results in w:SIGGRAPH 2000. The scientific breakthrough required finding the w:subsurface light component (the simulation models are glowing from within slightly) which can be found using knowledge that light that is reflected from the oil-to-air layer retains its w:Polarization (waves) and the subsurface light loses its polarization. So equipped only with a movable light source, movable video camera, 2 polarizers and a computer program doing extremely simple math and the last piece required to reach photorealism was acquired.

• 1999 | institute founded | The w:Institute for Creative Technologies was founded by the w:United States Army in the w:University of Southern California. It collaborates with the w:United States Army Futures Command, w:United States Army Combat Capabilities Development Command, w:Combat Capabilities Development Command Soldier Center and w:United States Army Research Laboratory.. In 2016 w:Hao Li was appointed to direct the institute.

• 1997 | technology / science | 'Video rewrite: Driving visual speech with audio' at www2.eecs.berkeley.edu Christoph Breigler, Michelle Covell and Malcom Slaney presented their work at the ACM SIGGRAPH 1997. Download video evidence of Video rewrite: Driving visual speech with audio Bregler et al 1997 from dropbox.com, view author's site at chris.bregler.com, paper at dl.acm.org paper at researchgate.net

• 1994 | movie | w:The Crow (1994 film) was the first film production to make use of w:digital compositing of a computer simulated representation of a face onto scenes filmed using a w:body double. Necessity was the muse as the actor w:Brandon Lee portraying the protagonist was tragically killed accidentally on-stage.

1970's synthetic human-like fakes

w:A Computer Animated Hand is a 1972 short film by w:Edwin Catmull and w:Fred Parke. This was the first time that w:computer-generated imagery was used in film to animate likenesses of moving human appearance.

• 1976 | movie | w:Futureworld reused parts of A Computer Animated Hand on the big screen.

• 1972 | entertainment | 'A Computer Animated Hand' on Vimeo. w:A Computer Animated Hand by w:Edwin Catmull and w:Fred Parke. Relevancy: This was the first time that w:computer-generated imagery was used in film to animate moving human-like appearance.

• 1971 | science | 'Images de synthèse : palme de la longévité pour l’ombrage de Gouraud' (still photos). w:Henri Gouraud (computer scientist) made the first w:Computer graphics w:geometry w:digitization and representation of a human face. Modeling was his wife Sylvie Gouraud. The 3D model was a simple w:wire-frame model and he applied w:Gouraud shading to produce the first known representation of human-likeness on computer.

1960's synthetic human-like fakes

• 1961 | demonstration | The first singing by a computer was performed by an w:IBM 704 and the song was w:Daisy Bell, written in 1892 by British songwriter w:Harry Dacre. Go to Mediatheque#1961 to view.

1930's synthetic human-like fakes

w:Voder demonstration pavillion at the w:1939 New York World's Fair

• 1939 | demonstration | w:Voder (Voice Operating Demonstrator) from the w:Bell Telephone Laboratory was the first time that w:speech synthesis was done electronically by breaking it down into its acoustic components. It was invented by w:Homer Dudley in 1937–1938 and developed on his earlier work on the w:vocoder. (Wikipedia)

1770's synthetic human-like fakes

A replica of w:Wolfgang von Kempelen's w:Wolfgang von Kempelen's Speaking Machine, built 2007–09 at the Department of w:Phonetics, w:Saarland University, w:Saarbrücken, Germany. This machine added models of the tongue and lips, enabling it to produce w:consonants as well as w:vowels

• 1791 | science | w:Wolfgang von Kempelen's Speaking Machine of w:Wolfgang von Kempelen of w:Pressburg, w:Hungary, described in a 1791 paper was w:bellows-operated. This machine added models of the tongue and lips, enabling it to produce w:consonants as well as w:vowels. (based on w:Speech synthesis#History)

• 1779 | science / discovery | w:Christian Gottlieb Kratzenstein won the first prize in a competition announced by the w:Russian Academy of Sciences for models he built of the human w:vocal tract that could produce the five long w:vowel sounds. (Based on w:Speech synthesis#History)

---

Footnotes

Contact information of organizations

Please contact these organizations and tell them to work harder against the disinformation weapons