Editing Synthetic human-like fakes

<section begin=definitions-of-synthetic-human-like-fakes />
When the '''[[Glossary#No camera|camera does not exist]]''', but the subject being imaged with a simulation of a (movie) camera deceives the watcher to believe it is some living or dead person it is a '''[[#Digital look-alikes|digital look-alike]]'''.

When it cannot be determined by human testing or media forensics whether some fake voice is a synthetic fake of some person's voice, or is it an actual recording made of that person's actual real voice, it is a pre-recorded '''[[#Digital sound-alikes|digital sound-alike]]'''. 
<section end=definitions-of-synthetic-human-like-fakes />

[[File:BlV1999-morphable-model-till-match-low-res-rip.png|thumb|left|460px|Image 2 (low resolution rip) 
<br/>(1) Sculpting a morphable model to one single picture 
<br/>(2) Produces 3D approximation 
<br/>(4) Texture capture 
<br/>(3) The 3D model is rendered back to the image with weight gain 
<br/>(5) With weight loss 
<br/>(6) Looking annoyed 
<br/>(7) Forced to smile 

<small>Image 2 by Blanz and Vettel – Copyright ACM 1999 – http://dl.acm.org/citation.cfm?doid=311535.311556 – Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.</small>]]

[[File:Saint John on Patmos.jpg|thumb|right|360px|link=Biblical explanation - The books of Daniel and Revelation|See <big>'''[[Biblical explanation - The books of Daniel and Revelation]]'''</big> to see the advance warning for our time that we were given in 6th century BC and then again in 1st century.
<br/><br/>
'Saint John on Patmos' pictures [[w:John of Patmos]] on [[w:Patmos]] writing down the visions to make the [[w:Book of Revelation]]. Picture from folio 17 of the [[w:Très Riches Heures du Duc de Berry]] (1412-1416) by the [[w:Limbourg brothers]]. Currently located at the [[w:Musée Condé]] 40km north of Paris, France.]]

== Digital look-alikes ==

{{#ev:youtube|LWLadJFI8Pk|400px|right|It is recommended that you watch ''In Event of Moon Disaster - FULL FILM'' (2020) at the '''[https://moondisaster.org/ moondisaster.org''' project website] (where it has interactive portions) by the [https://virtuality.mit.edu/ Center for Advanced Virtuality] of the [[w:Massachusetts Institute of Technology|w:MIT]]}}


=== Introduction to digital look-alikes ===
[[File:Deb-2000-reflectance-separation.png|thumb|360px|right|Image 1: Separating specular and diffuse reflected light

<br/> <br />

(a) Normal image in dot lighting
<br/><br/> 
(b) Image of the diffuse reflection which is caught by placing a vertical polarizer in front of the light source and a horizontal in the front the camera
<br/><br/> 
(c) Image of the highlight specular reflection which is caught by placing both polarizers vertically
<br/><br/> 
(d) Subtraction of c from b, which yields the specular component
<br/><br/> 
Images are scaled to seem to be the same luminosity.
<br/><br/> 
<small>Original image by Debevec et al. – Copyright ACM 2000 – https://dl.acm.org/citation.cfm?doid=311779.344855 – Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.</small>]]

[[File:The-diffuse-reflection-deducted-from-the-specular-reflection-Debevec-2000.png|thumb|left|260px|Subtraction of the diffuse reflection from the specular reflection yields the specular component of the model's reflectance.  
<br /><br />
<small>[[:File:Deb-2000-reflectance-separation.png|Original picture]]  by [[w:Paul Debevec]] et al. - Copyright ACM 2000 https://dl.acm.org/citation.cfm?doid=311779.344855</small>]]

In the cinemas we have seen digital look-alikes for over 15 years. These digital look-alikes have "clothing" (a simulation of clothing is not clothing) or "superhero costumes" and "superbaddie costumes", and they don't need to care about the laws of physics, let alone laws of physiology. It is generally accepted that digital look-alikes made their public debut in the sequels of The Matrix i.e. [[w:The Matrix Reloaded]] and [[w:The Matrix Revolutions]] released in 2003. It can be considered almost certain, that it was not possible to make these before the year 1999, as the final piece of the puzzle to make a (still) digital look-alike that passes human testing, the [[Glossary#Reflectance capture|reflectance capture]] over the human face, was made for the first time in 1999 at the [[w:University of Southern California]] and was presented to the crème de la crème 
of the computer graphics field in their annual gathering SIGGRAPH 2000.<ref name="Deb2000">
{{cite book
| last = Debevec
| first = Paul
| author-link = Paul Debevec
| chapter = Acquiring the reflectance field of a human face
| journal =
| pages = 145–156
| publisher = ACM
| year = 2000
| chapter-url = http://dl.acm.org/citation.cfm?id=344855 
| chapter-format =
| doi = 10.1145/344779.344855
| accessdate =  2020-06-27
| title = Proceedings of the 27th annual conference on Computer graphics and interactive techniques - SIGGRAPH '00
| isbn = 978-1581132083
}}</ref>

{{#lst:Quotes|MatrixTrad}}

=== The problems with digital look-alikes ===

Extremely unfortunately for the humankind, organized criminal leagues, that posses the '''weapons capability''' of making believable looking '''synthetic pornography''', are producing on industrial production pipelines '''synthetic terror porn'''<ref group="footnote" name="About the term synthetic terror porn">It is terminologically more precise, more inclusive and more useful to talk about 'synthetic terror porn', if we want to talk about things with their real names, than 'synthetic rape porn', because also synthesizing recordings of consentual looking sex scenes can be terroristic in intent.</ref> by animating digital look-alikes and distributing it in the murky Internet in exchange for money stacks that are getting thinner and thinner as time goes by.

These industrially produced pornographic delusions are causing great humane suffering, especially in their direct victims, but they are also tearing our communities and societies apart, sowing blind rage, perceptions of deepening chaos, feelings of powerlessness and provoke violence. This '''hate illustration''' increases and strengthens hate thinking, hate speech, hate crimes and tears our fragile social constructions apart and with time perverts humankind's view of humankind into an almost unrecognizable shape, unless we interfere with resolve.

=== List of possible naked digital look-alike attacks ===

* The classic "''portrayal of as if in involuntary sex''"-attack. (Digital look-alike "cries")
* "''Sexual preference alteration''"-attack. (Digital look-alike "smiles")
* "''Cutting / beating''"-attack  (Constructs a deceptive history for genuine scars) 
* "''Mutilation''"-attack (Digital look-alike "dies")
* "''Unconscious and injected''"-attack (Digital look-alike gets "disease")

=== Age analysis and rejuvenating and aging syntheses ===

*  [https://arxiv.org/abs/2002.03750 ''''''An Overview of Two Age Synthesis and Estimation Techniques'''''' at arxiv.org] [https://arxiv.org/pdf/2002.03750.pdf (.pdf)], submitted for review on 2020-01-26
* [https://www.sciencedirect.com/science/article/abs/pii/S0925231220309942 ''''''Dual Reference Age Synthesis'''''' at sciencedirect.com] [https://arxiv.org/pdf/1908.02671.pdf (preprint at arxiv.org)] published on 2020-10-21 in [[w:Neurocomputing (journal)]]
* [https://ieeexplore.ieee.org/document/6084154 ''''''A simple automatic facial aging/rejuvenating synthesis method'''''' at ieeexplore.ieee.org] [https://www.researchgate.net/publication/220755281_Automatic_Facial_AgingRejuvenating_Synthesis_Method read free at researchgate.net], published at the proceedings  of the 2011 IEEE International Conference on Systems, Man and Cybernetics 
* [https://ieeexplore.ieee.org/document/5406526 ''''''Age Synthesis and Estimation via Faces: A Survey'''''' at ieeexplore.ieee.org] (paywall) [https://www.researchgate.net/publication/46288561_Age_Synthesis_and_Estimation_via_Faces_A_Survey at researchgate.net] published November 2010

=== Temporal limit of digital look-alikes ===
[[File:Institut Lumière - CINEMATOGRAPHE Camera.jpg|thumb|left|120px|A picture of the 1895 [[w:Cinematograph]]]]

[[w:History of film technology]] has information about where the border is.

Digital look-alikes cannot be used to attack people who existed before the technological invention of film. For moving pictures the breakthrough is attributed to [[w:Auguste and Louis Lumière]]'s [[w:Cinematograph]] premiered in Paris on 28 December '''1895''', though this was only the commercial and popular breakthrough, as even earlier moving pictures exist. (adapted from [[w:History of film]])

The '''[[w:Kinetoscope]]''' is an even earlier motion picture exhibition device. A prototype for the Kinetoscope was shown to a convention of the National Federation of Women's Clubs on May 20, 1891.<ref name="memory.loc.gov">

{{cite web
|publisher=[[w:Library of Congress]]
|website=Memory.loc.gov
|url=http://memory.loc.gov/ammem/edhtml/edmvhist.html
|title=Inventing Entertainment: The Early Motion Pictures and Sound Recordings of the Edison Companies
|access-date=2020-12-09
}}

</ref> The first public demonstration of the Kinetoscope was held at the [[w:Brooklyn Museum|Brooklyn Institute of Arts and Sciences]] on '''May 9''', '''1893'''. ([[w:Kinetoscope|Wikipedia]])<ref name="memory.loc.gov"/>


----

== Digital sound-alikes ==
[[File:Helsingin-Sanomat-2012-David-Martin-Howard-of-University-of-York-on-apporaching-digital-sound-alikes.jpg|right|thumb|338px|A picture of a cut-away titled "''Voice-terrorist could mimic a leader''" from a 2012 [[w:Helsingin Sanomat]] warning that the sound-like-anyone machines are approaching. Thank you to homie [https://pure.york.ac.uk/portal/en/researchers/david-martin-howard(ecfa9e9e-1290-464f-981a-0c70a534609e).html Prof. David Martin Howard] of the [[w:University of York]], UK and the anonymous editor for the heads-up.]]

Living people can defend<ref group="footnote" name="judiciary maybe not aware">Whether a suspect can defend against faked synthetic speech that sounds like him/her depends on how up-to-date the judiciary is. If no information and instructions about digital sound-alikes have been given to the judiciary, they likely will not believe the defense of denying that the recording is of the suspect's voice.</ref> themselves against digital sound-alike by denying the things the digital sound-alike says if they are presented to the target, but dead people cannot. Digital sound-alikes offer criminals new disinformation attack vectors and wreak havoc on provability. 

For these reasons the bannable '''raw materials''' i.e. covert voice models '''[[Law proposals to ban covert modeling|should be prohibited by law]]''' in order to protect humans from abuse by criminal parties.

=== Documented digital sound-alike attacks ===
* Sound like anyone technology found its way to the hands of criminals as in '''2019''' [[w:NortonLifeLock|Symantec]] researchers knew of 3 cases where technology has been used for '''[[w:crime]]'''
** [https://www.bbc.com/news/technology-48908736 '''"Fake voices 'help cyber-crooks steal cash''''" at bbc.com] July 2019 reporting <ref name="BBC2019">
{{cite web
 |url= https://www.bbc.com/news/technology-48908736
 |title= Fake voices 'help cyber-crooks steal cash'
 |last=
 |first= 
 |date= 2019-07-08
 |website= [[w:bbc.com]]
 |publisher= [[w:BBC]]
 |access-date= 2020-07-22
 |quote= }}
</ref>
** [https://www.washingtonpost.com/technology/2019/09/04/an-artificial-intelligence-first-voice-mimicking-software-reportedly-used-major-theft/ '''"An artificial-intelligence first: Voice-mimicking software reportedly used in a major theft"''' at washingtonpost.com] documents a [[w:fraud]] committed with digital sound-like-anyone-machine, July 2019 reporting.<ref name="WaPo2019">
{{cite web
 |url= https://www.washingtonpost.com/technology/2019/09/04/an-artificial-intelligence-first-voice-mimicking-software-reportedly-used-major-theft/
 |title= An artificial-intelligence first: Voice-mimicking software reportedly used in a major theft
 |last= Drew
 |first= Harwell
 |date= 2020-04-16
 |website= [[w:washingtonpost.com]]
 |publisher= [[w:Washington Post]]
 |access-date= 2019-07-22
 |quote= }}
</ref>
----

=== 'Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis' 2018 by Google Research (external transclusion) ===
<section begin=GoogleTransferLearning2018 />
* In the '''2018''' at the '''[[w:Conference on Neural Information Processing Systems]]''' (NeurIPS) the work [http://papers.nips.cc/paper/7700-transfer-learning-from-speaker-verification-to-multispeaker-text-to-speech-synthesis 'Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis'] ([https://arxiv.org/abs/1806.04558 at arXiv.org]) was presented. The pre-trained model is able to steal voices from a sample of only '''5 seconds''' with almost convincing results

Observe how good the "VCTK p240" system is at deceiving to think that it is a person that is doing the talking.

{{#Widget:Iframe - Audio samples from Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis by Google Research}}

The Iframe above is transcluded from [https://google.github.io/tacotron/publications/speaker_adaptation/ 'Audio samples from "Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis"' at google.gituhub.io], the audio samples of a sound-like-anyone machine presented as at the 2018 [[w:NeurIPS]] conference by Google researchers.
<section end=GoogleTransferLearning2018 />

=== Digital sing-alikes ===
The to the right [https://www.youtube.com/watch?v=0sR1rU3gLzQ video 'This AI Clones Your Voice After Listening for 5 Seconds' by '2 minute papers' at YouTube] describes the voice thieving machine presented by Google Research in [[w:NeurIPS|w:NeurIPS]] 2018.
 
{{#ev:youtube|0sR1rU3gLzQ|640px|right|Video [https://www.youtube.com/watch?v=0sR1rU3gLzQ video 'This AI Clones Your Voice After Listening for 5 Seconds' by '2 minute papers' at YouTube] describes the voice thieving machine by Google Research in [[w:NeurIPS|w:NeurIPS]] 2018.}}

As of 2020 the digital sing-alikes may not yet be here, but when we hear a faked singing voice and we cannot hear that it is fake, then we will know. An ability to sing does not seem to add much hostile capabilities compared to the ability to thieve spoken word.

* [https://arxiv.org/abs/1910.11690 ''''''Fast and High-Quality Singing Voice Synthesis System based on Convolutional Neural Networks'''''' at arxiv.org], a 2019 singing voice synthesis technique using [[w:convolutional neural network|w:convolutional neural networks (CNN)]]. Accepted into the 2020 [[w:International Conference on Acoustics, Speech, and Signal Processing|International Conference on Acoustics, Speech, and Signal Processing (ICASSP)]].
* [http://compmus.ime.usp.br/sbcm/2019/papers/sbcm-2019-7.pdf ''''''State of art of real-time singing voice synthesis'''''' at compmus.ime.usp.br] presented at the 2019 [http://compmus.ime.usp.br/sbcm/2019/program/ 17th Brazilian Symposium on Computer Music]
* [http://theses.fr/2017PA066511 ''''''Synthesis and expressive transformation of singing voice'''''' at theses.fr] [https://www.theses.fr/2017PA066511.pdf as .pdf] a 2017 doctorate thesis by [http://theses.fr/227185943 Luc Ardaillon]
* [http://mtg.upf.edu/node/512 ''''''Synthesis of the Singing Voice by Performance Sampling and Spectral Models'''''' at mtg.upf.edu], a 2007 journal article in the [[w:IEEE Signal Processing Society]]'s Signal Processing Magazine
* [https://www.researchgate.net/publication/4295714_Speech-to-Singing_Synthesis_Converting_Speaking_Voices_to_Singing_Voices_by_Controlling_Acoustic_Features_Unique_to_Singing_Voices ''''''Speech-to-Singing Synthesis: Converting Speaking Voices to Singing Voices by Controlling Acoustic Features Unique to Singing Voices'''''' at researchgate.net], a November 2007 paper published in the IEEE conference on Applications of Signal Processing to Audio and Acoustics


* [[w:Category:Singing software synthesizers]]

----

=== Example of a hypothetical 4-victim digital sound-alike attack ===
A very simple example of a digital sound-alike attack is as follows: 

Someone puts a digital sound-alike to call somebody's voicemail from an unknown number and to speak for example illegal threats. In this example there are at least two victims:

# Victim #1 - The person whose voice has been stolen into a covert model and a digital sound-alike made from it to frame them for crimes
# Victim #2 - The person to whom the illegal threat is presented in a recorded form by a digital sound-alike that deceptively sounds like victim #1
# Victim #3 - It could also be viewed that victim #3 is our law enforcement systems as they are put to chase after and interrogate the innocent victim #1
# Victim #4 - Our judiciary which prosecutes and possibly convicts the innocent victim #1.

Thus it is high time to act and to '''[[Law proposals to ban covert modeling|criminalize the covert modeling of human voice!]]'''

=== Examples of speech synthesis software not quite able to fool a human yet ===
Some other contenders to create digital sound-alikes are though, as of 2019, their speech synthesis in most use scenarios does not yet fool a human because the results contain tell tale signs that give it away as a speech synthesizer.  

* '''[https://lyrebird.ai/ Lyrebird.ai]''' [https://www.youtube.com/watch?v=xxDBlZu__Xk (listen)] 
* '''[https://candyvoice.com/ CandyVoice.com]''' [https://candyvoice.com/demos/voice-conversion (test with your choice of text)] 
* '''[https://cstr-edinburgh.github.io/merlin/ Merlin]''', a [[w:neural network]] based speech synthesis system by the Centre for Speech Technology Research at the [[w:University of Edinburgh]]
* [https://papers.nips.cc/paper/8206-neural-voice-cloning-with-a-few-samples ''''Neural Voice Cloning with a Few Samples''' at papers.nips.cc], [[w:Baidu Research]]'es shot at sound-like-anyone-machine did not convince in '''2018'''

=== Reporting on the sound-like-anyone-machines ===
* [https://www.forbes.com/sites/bernardmarr/2019/05/06/artificial-intelligence-can-now-copy-your-voice-what-does-that-mean-for-humans/#617f6d872a2a '''"Artificial Intelligence Can Now Copy Your Voice: What Does That Mean For Humans?"''' May 2019 reporting at forbes.com] on [[w:Baidu Research]]'es attempt at the sound-like-anyone-machine demonstrated at the 2018 [[w:NeurIPS]] conference.

=== Temporal limit of digital sound-alikes ===
[[File:Edison_and_phonograph_edit1.jpg|thumb|left|210px|[[w:Thomas Edison]] and his early [[w:phonograph]]. Cropped from [[w:Library of Congress]] copy, ca. 1877,  (probably 18 April 1878)]]

The temporal limit of whom, dead or living, the digital sound-alikes can attack is defined by the '''[[w:history of sound recording]]'''. 

The article starts by mentioning that the invention of the [[w:phonograph]] by [[w:Thomas Edison]] in '''1877''' is considered the start of sound recording. 

The '''phonautograph''' is the earliest known device for recording [[w:sound]]. Previously, tracings had been obtained of the sound-producing vibratory motions of [[w:tuning forks]] and other objects by physical contact with them, but not of actual sound waves as they propagated through air or other media. Invented by Frenchman [[W:Édouard-Léon Scott de Martinville]], it was patented on March 25, '''1857'''.<ref name="NPR-Phonautograph">

{{Cite news
|url=https://www.npr.org/templates/story/story.php?storyId=89380697
|title=1860 'Phonautograph' Is Earliest Known Recording
|last=Flatow
|first=Ira|date=April 4, 2008|work=NPR
|access-date=2012-12-09
|language=en}}

</ref> 

Apparently, it did not occur to anyone before the 1870s that the recordings, called '''phonautograms''', contained enough information about the sound that they could, in theory, be '''used to recreate it'''. Because the phonautogram tracing was an insubstantial two-dimensional line, direct physical playback was impossible in any case. Several phonautograms recorded '''before 1861''' were successfully played as sound in '''2008''' by optically scanning them and using a computer to process the scans into digital audio files. ([[w:Phonautograph|Wikipedia]])

[[File:Spectrogram-19thC.png|thumb|right|640px|A [[w:spectrogram]] of a male voice saying 'nineteenth century']]

== Text syntheses ==
[[w:Chatbot]]s have existed for a longer time, but only now armed with AI they are becoming more deceiving. 

In [[w:natural language processing]] development in [[w:natural-language understanding]] leads to more cunning [[w:natural-language generation]] AI.

[[w:OpenAI]]'s [[w:OpenAI#GPT|w:Generative Pre-trained Transformer]] ('''GPT''') is a left-to-right [[w:transformer (machine learning model)]]-based [[w:Natural-language generation|text generation]] model succeeded by [[w:OpenAI#GPT-2|w:GPT-2]] and [[w:OpenAI#GPT-3|w:GPT-3]]

''' Reporting / announcements '''

* [https://www.technologyreview.com/2020/08/14/1006780/ai-gpt-3-fake-blog-reached-top-of-hacker-news/ ''''A college kid’s fake, AI-generated blog fooled tens of thousands. This is how he made it.'''' at technologyreview.com] '''August 2020''' reporting in the [[w:MIT Technology Review]] by Karen Hao about GPT-3.

* [https://siliconangle.com/2020/07/19/openais-latest-ai-text-generator-gpt-3-amazes-early-adopters/ ''''OpenAI’s latest AI text generator GPT-3 amazes early adopters'''' at siliconangle.com] '''July 2020''' reporting on GPT-3

* [https://openai.com/blog/gpt-2-6-month-follow-up/ '''OpenAI releases the full version of GPT-2''' at openai.com]  in '''August 2019'''

* [https://venturebeat.com/2019/08/20/openai-releases-curtailed-version-of-gpt-2-language-model/ ''''OpenAI releases curtailed version of GPT-2 language model'''' at venturebeat.com], '''August 2019''' reporting on the original release of of the curtailed version of GPT-2

''' External links '''
* [https://analyticssteps.com/blogs/detection-fake-and-false-news-text-analysis-approaches-and-cnn-deep-learning-model '''"Detection of Fake and False News (Text Analysis): Approaches and CNN as Deep Learning Model"''' at analyticsteps.com], a 2019 summmary written by Shubham Panth.

== Countermeasures against synthetic human-like fakes ==

<section begin=APW_AI-transclusion />
=== Organizations against synthetic human-like fakes ===

[[File:DARPA_Logo.jpg|thumb|left|240px|The Defense Advanced Research Projects Agency, better known as [[w:DARPA]] has been active in the field of countering synthetic fake video for longer than the public has been aware of the problems existing.]]

* '''[[w:DARPA]]''' [https://www.darpa.mil/program/media-forensics '''DARPA program''': ''''Media Forensics'''' ('''MediFor''') at darpa.mil] aims to develop technologies for the automated assessment of the integrity of an image or video and integrating these in an end-to-end media forensics platform. Archive.org first crawled their homepage in [https://web.archive.org/web/20160630154819/https://www.darpa.mil/program/media-forensics June '''2016''']<ref name="IA-MediFor-2016-crawl">https://web.archive.org/web/20160630154819/https://www.darpa.mil/program/media-forensics</ref>.

* [https://www.darpa.mil/program/semantic-forensics '''DARPA program''': ''''Semantic Forensics'''' ('''SemaFor''') at darpa.mil] aims to counter synthetic disinformation by developing systems for detecting semantic inconsistencies in forged media. They state that they hope to create technologies that "will help identify, deter, and understand adversary disinformation campaigns". More information at [[w:Duke University]]'s [https://researchfunding.duke.edu/semantic-forensics-semafor '''Research Funding database: Semantic Forensics (SemaFor)''' at researchfunding.duke.edu] and some at [https://www.grants.gov/web/grants/view-opportunity.html?oppId=319894 '''Semantic Forensics grant opportunity''' (closed Nov 2019) at grants.gov]. Archive.org first cralwed their website in [https://web.archive.org/web/20191108090036/https://www.darpa.mil/program/semantic-forensics November '''2019''']<ref name="IA-SemaFor-2019-crawl">https://web.archive.org/web/20191108090036/https://www.darpa.mil/program/semantic-forensics November</ref>

* '''[[w:University of Colorado Denver]]''' is the home of the [https://artsandmedia.ucdenver.edu/areas-of-study/national-center-for-media-forensics/about-the-national-center-for-media-forensics '''National Center for Media Forensics''' at artsandmedia.ucdenver.edu] at the [[w:University of Colorado Denver]] offers a [https://artsandmedia.ucdenver.edu/areas-of-study/national-center-for-media-forensics/media-forensics-graduate-program Master's degree program], [https://artsandmedia.ucdenver.edu/areas-of-study/national-center-for-media-forensics/training-courses training courses] and [https://artsandmedia.ucdenver.edu/areas-of-study/national-center-for-media-forensics/national-center-for-media-forensics-research scientific basic and applied research]. [https://artsandmedia.ucdenver.edu/areas-of-study/national-center-for-media-forensics/faculty-staff Faculty staff at the NCMF]

[[File:Connie Leyva 2015.jpg|thumb|left|240px|[[w:California]] [[w:California State Senate|w:Senator]] [[w:Connie Leyva]] introduced [https://leginfo.legislature.ca.gov/faces/billNavClient.xhtml?bill_id=201920200SB564 California Senate Bill SB 564] in Feb '''2019'''. It has been [https://www.sagaftra.org/action-alert-support-california-bill-end-deepfake-porn endorsed by SAG-AFTRA], but has not yet passed.]]

* '''[[w:SAG-AFTRA]]''' [https://www.sagaftra.org/action-alert-support-california-bill-end-deepfake-porn SAG-AFTRA ACTION ALERT: '''"Support California Bill to End Deepfake Porn"''' at sagaftra.org '''endorses'''] [https://leginfo.legislature.ca.gov/faces/billNavClient.xhtml?bill_id=201920200SB564 California Senate Bill SB 564] introduced to the  [[w:California State Senate]] by [[w:California]] [[w:Connie Leyva|w:Senator Connie Leyva]] in Feb '''2019'''.

=== Organizations possibly against synthetic human-like fakes ===

Originally harvested from the study [https://www.europarl.europa.eu/RegData/etudes/STUD/2020/634452/EPRS_STU(2020)634452_EN.pdf The ethics of artificial intelligence: Issues and initiatives (.pdf)]  by the [[w:European Parliamentary Research Service]], published on the [[w:Europa (web portal)]] in March 2020.<ref group="1st seen in" name="EU-Parl-Ethical-AI-Study-2020">

{{cite web
 |url= https://www.europarl.europa.eu/RegData/etudes/STUD/2020/634452/EPRS_STU(2020)634452_EN.pdf
 |title= The ethics of artificial intelligence: Issues and initiatives
 |last= 
 |first=
 |date= March 2020
 |website= [[w:Europa (web portal)]]
 |publisher=[[w:European Parliamentary Research Service]]
 |access-date=2021-02-17
 |quote=This study deals with the ethical implications and moral questions that arise from the development and implementation of artificial intelligence (AI) technologies.}}

</ref>

* [https://ieai.mcts.tum.de/ '''INSTITUTE FOR ETHICS IN ARTIFICIAL INTELLIGENCE''' at ieai.mcts.tum.de] received initial funding from [[w:Facebook]] in 2019.<ref group="1st seen in" name="EU-Parl-Ethical-AI-Study-2020"/>
* [https://ethical.institute/ '''The Institute for Ethical AI & Machine Learning''' at ethical.institute]<ref group="1st seen in" name="EU-Parl-Ethical-AI-Study-2020"/>
* [https://www.buckingham.ac.uk/research-the-institute-for-ethical-ai-in-education/ '''The Institute for Ethical AI in Education''' at buckingham.ac.uk]<ref group="1st seen in" name="EU-Parl-Ethical-AI-Study-2020"/>
* [https://futureoflife.org/ '''Future of Life Institute''' at futureoflife.org] received funding from private donors.<ref group="1st seen in" name="EU-Parl-Ethical-AI-Study-2020"/>
* [https://www.ai-gakkai.or.jp/ '''The Japanese Society for Artificial Intelligence''' ('''JSAI''') at ai-gakkai.or.jp]. Publication: Ethical guidelines.<ref group="1st seen in" name="EU-Parl-Ethical-AI-Study-2020"/>
* [https://ai-4-all.org/ '''AI4All''' at ai-4-all.org] funded by [[w:Google]]<ref group="1st seen in" name="EU-Parl-Ethical-AI-Study-2020"/>
* [https://thefuturesociety.org/ '''The Future Society''' at thefuturesociety.org]<ref group="1st seen in" name="EU-Parl-Ethical-AI-Study-2020"/>
* [https://ainowinstitute.org/ '''The Ai Now Institute''' at ainowinstitute.org] at [[w:New York University]]<ref group="1st seen in" name="EU-Parl-Ethical-AI-Study-2020"/>
* [https://www.partnershiponai.org/ '''Partnership on AI''' at partnershiponai.org] is based in the USA and funded by technology companies
* [https://responsiblerobotics.org/ '''The Foundation for Responsible Robotics''' at responsiblerobotics.org] is based in [[w:Netherlands]].<ref group="1st seen in" name="EU-Parl-Ethical-AI-Study-2020"/>
* [https://ai4people.eu/ '''AI4People''' at ai4people.eu] is based in [[w:Belgium]] is a multi-stakeholder forum.<ref group="1st seen in" name="EU-Parl-Ethical-AI-Study-2020"/>
* [https://aiethicsinitiative.org/ '''The Ethics and Governance of Artificial Intelligence Initiative''' at aiethicsinitiative.org] is based in the USA.<ref group="1st seen in" name="EU-Parl-Ethical-AI-Study-2020"/>
* [ '''''' at ]<ref group="1st seen in" name="EU-Parl-Ethical-AI-Study-2020"/>
* [ '''''' at ]<ref group="1st seen in" name="EU-Parl-Ethical-AI-Study-2020"/>
* [ '''''' at ]<ref group="1st seen in" name="EU-Parl-Ethical-AI-Study-2020"/>
* [ '''''' at ]<ref group="1st seen in" name="EU-Parl-Ethical-AI-Study-2020"/>
* [ '''''' at ]<ref group="1st seen in" name="EU-Parl-Ethical-AI-Study-2020"/>
* [ '''''' at ]<ref group="1st seen in" name="EU-Parl-Ethical-AI-Study-2020"/>
* [ '''''' at ]<ref group="1st seen in" name="EU-Parl-Ethical-AI-Study-2020"/>
* [ '''''' at ]<ref group="1st seen in" name="EU-Parl-Ethical-AI-Study-2020"/>
* [ '''''' at ]<ref group="1st seen in" name="EU-Parl-Ethical-AI-Study-2020"/>
* [ '''''' at ]<ref group="1st seen in" name="EU-Parl-Ethical-AI-Study-2020"/>

=== Events against synthetic human-like fakes ===

* '''2020''' | '''[[w:Conference on Computer Vision and Pattern Recognition]] (CVPR)''' |  [https://sites.google.com/view/wmediaforensics2020/home 2020 Conference on Computer Vision and Pattern Recognition:  ''''Workshop on Media Forensics'''' at sites.google.com], a '''June 2020''' workshop at the [[w:Conference on Computer Vision and Pattern Recognition]]. 

* '''2020''' | The winners of the [https://venturebeat.com/2020/06/12/facebook-detection-challenge-winners-spot-deepfakes-with-82-accuracy/ Deepfake Detection Challenge reach 82% accuracy in detecting synthetic human-like fakes]<ref name="VentureBeat2020">https://venturebeat.com/2020/06/12/facebook-detection-challenge-winners-spot-deepfakes-with-82-accuracy/</ref>

* '''2019''' | At the annual Finnish [[w:Ministry of Defence (Finland)|w:Ministry of Defence]]'s [https://www.defmin.fi/en/frontpage/overview/ministry_of_defence/departments_and_units/defence_policy_department/scientific_advisory_board_for_defence_%28matine%29#d779859d '''Scientific Advisory Board for Defence''' ('''MATINE''')] public research seminar, a research group presented their work [https://www.defmin.fi/files/4755/1315MATINE_seminaari_21.11.pdf ''''''Synteettisen median tunnistus''''''' at defmin.fi] (Recognizing synthetic media). They developed on earlier work on how to automatically detect synthetic human-like fakes and their work was funded with a grant from MATINE.

* '''2019''' | '''[[w:Conference on Neural Information Processing Systems|w:NeurIPS]]''' | [[w:Facebook, Inc.]] [https://spectrum.ieee.org/tech-talk/artificial-intelligence/machine-learning/facebook-ai-launches-its-deepfake-detection-challenge '''"Facebook AI Launches Its Deepfake Detection Challenge"''' at spectrum.ieee.org] [[w:IEEE Spectrum]]. More reporting at [https://venturebeat.com/2019/12/11/facebook-microsoft-and-others-launch-deepfake-detection-challenge/ '''''"Facebook, Microsoft, and others launch Deepfake Detection Challenge"''''' at venturebeat.com]

* '''2019''' | '''CVPR''' | [https://sites.google.com/view/mediaforensics2019/home '''2019''' CVPR: ''''Workshop on Media Forensics'''']

* '''Annual''' (?) | '''[[w:National Institute of Standards and Technology]] (NIST)  | [https://www.nist.gov/itl/iad/mig/media-forensics-challenge NIST: 'Media Forensics Challenge'''' at nist.gov], an iterative research challenge by the [[w:National Institute of Standards and Technology]] with the ongoing challenge being the 2nd one in action. [https://www.nist.gov/itl/iad/mig/media-forensics-challenge-2019-0 the evaluation criteria for the 2019 iteration are being formed]

* '''2018''' | '''[[w:European Conference on Computer Vision|w:European Conference on Computer Vision (ECCV)]]''' [https://sites.google.com/view/wocm2018/home ECCV 2018: ''''Workshop on Objectionable Content and Misinformation'''' at sites.google.com], a workshop at the '''2018''' [[w:European Conference on Computer Vision]] in [[w:Munich]] had focus on objectionable content detection e.g. [[w:nudity]], [[w:pornography]], [[w:violence]], [[w:hate]], [[w:Child sexual abuse|w:children exploitation]] and [[w:terrorism]] among others and to address misinformation problems when people are fed [[w:disinformation]] and they punt it on as misinformation. Announced topics included [[w:Outline of forensic science|w:image/video forensics]], [[w:detection]]/[[w:analysis]]/[[w:understanding]] of [[w:Counterfeit|w:fake]] images/videos, [[w:misinformation]] detection/understanding: mono-modal and [[w:Multimodality|w:multi-modal]], adversarial technologies and detection/understanding of objectionable content

* '''2018''' | '''[[w:National Institute of Standards and Technology|w:NIST]]''' [https://www.nist.gov/itl/iad/mig/media-forensics-challenge-2018 NIST ''''Media Forensics Challenge 2018'''' at nist.gov] was the second annual evaluation to support research and help advance the state of the art for image and video forensics technologies – technologies that determine the region and type of manipulations in imagery (image/video data) and the phylogenic process that modified the imagery. 

* '''2017''' | '''[[w:National Institute of Standards and Technology|w:NIST]]'''  [https://www.nist.gov/itl/iad/mig/nimble-challenge-2017-evaluation NIST ''''Nimble Challenge 2017'''' at nist.gov]

=== Studies against synthetic human-like fakes ===

* [https://arxiv.org/abs/2001.06564 ''''Media Forensics and DeepFakes: an overview'''' at arXiv.org] [https://arxiv.org/pdf/2001.06564.pdf (as .pdf at arXiv.org)], a '''2020''' '''review''' on the subject of digital look-alikes and media forensics

* [https://scholarship.law.duke.edu/cgi/viewcontent.cgi?article=1333&context=dltr ''''DEEPFAKES: False pornography is here and the law cannot protect you'''' at scholarship.law.duke.edu] by Douglas Harris, published in [https://scholarship.law.duke.edu/dltr/vol17/iss1/ Duke Law & Technology Review - Volume 17 on '''2019'''-01-05] by [[w:Duke University]] [[w:Duke University School of Law]]

''' Search for more '''
* [[w:Law review]]
** [[w:List of law reviews in the United States]]

=== Companies against synthetic human-like fakes ===

* '''[https://cyabra.com/ Cyabra.com]''' is an AI-based system that helps organizations be on the guard against disinformation attacks<ref group="1st seen in" name="ReutersDisinfomation2020">https://www.reuters.com/article/us-cyber-deepfake-activist/deepfake-used-to-attack-activist-couple-shows-new-disinformation-frontier-idUSKCN24G15E</ref>. [https://www.reuters.com/article/us-cyber-deepfake-activist/deepfake-used-to-attack-activist-couple-shows-new-disinformation-frontier-idUSKCN24G15E Reuters.com reporting] from July 2020.

<section end=APW_AI-transclusion />

=== SSF! wiki proposed countermeasure to synthetic porn: Adequate Porn Watcher AI (transcluded) ===
Transcluded main contents from [[Adequate Porn Watcher AI (concept)]]

{{#lstx:Adequate Porn Watcher AI (concept)|See_also}}

=== Possible legal response: Outlawing digital sound-alikes (transcluded) ===
Transcluded from [[Law proposals#Law proposal to ban covert modeling of human voice|Juho's proposal on banning digital sound-alikes]]

{{#section-h:Law proposals|Law proposal to ban covert modeling of human voice}}

== Timeline  of synthetic human-like fakes ==

=== 2020's synthetic human-like fakes ===


* '''<font color="green">2020</font>''' | '''<font color="green">counter-measure</font>''' | On 2020-11-18 the [[w:Partnership on AI]] introduced the [https://incidentdatabase.ai/ ''''''AI Incident Database'''''' at incidentdatabase.ai].<ref name="PartnershipOnAI2020">https://www.partnershiponai.org/aiincidentdatabase/</ref>

* '''2020''' | reporting | [https://www.wired.co.uk/article/deepfake-porn-websites-videos-law "''Deepfake porn is now mainstream. And major sites are cashing in''" at wired.co.uk] by Matt Burgess. Published August 2020.

* '''2020''' | demonstration | '''[https://moondisaster.org/ Moondisaster.org]''' (full film embedded in website) project by the [https://virtuality.mit.edu/ Center for Advanced Virtuality] of the [[w:Massachusetts Institute of Technology|w:MIT]] published in July 2020, makes use of various methods of making a synthetic human-like fake. Alternative place to watch: [https://www.youtube.com/watch?v=LWLadJFI8Pk ''In Event of Moon Disaster - FULL FILM'' at youtube.com]  
** [https://www.cnet.com/news/mit-releases-deepfake-video-of-nixon-announcing-nasa-apollo-11-disaster/ Cnet.com July 2020 reporting ''MIT releases deepfake video of 'Nixon' announcing NASA Apollo 11 disaster'']

* '''2020''' | US state law | {{#lst:Current and possible laws and their application|California2020}}
* '''2020''' | Chinese legislation |  {{#lst:Current and possible laws and their application|China2020}}

=== 2010's synthetic human-like fakes ===

* '''2019''' | demonstration | In September 2019 [[w:Yle]], the Finnish [[w:public broadcasting company]], aired a result of experimental [[w:journalism]], [https://yle.fi/uutiset/3-10955498 '''a deepfake of the President in office'''] [[w:Sauli Niinistö]] in its main news broadcast for the purpose of highlighting the advancing disinformation technology and problems that arise from it.

* '''2019''' | US state law | {{#lst:Current and possible laws and their application|Texas2019}}

* '''2019''' | US state law | {{#lst:Current and possible laws and their application|Virginia2019}}

* '''2019''' | science and demonstration | [https://arxiv.org/pdf/1905.09773.pdf ''''Speech2Face: Learning the Face Behind a Voice'''' at arXiv.org] a system for generating likely facial features based on the voice of a person, presented by the [[w:MIT Computer Science and Artificial Intelligence Laboratory]] at the 2019 [[w:Conference on Computer Vision and Pattern Recognition|w:CVPR]]. [https://github.com/saiteja-talluri/Speech2Face Speech2Face at github.com] This may develop to something that really causes problems. [https://neurohive.io/en/news/speech2face-neural-network-predicts-the-face-behind-a-voice/ "Speech2Face: Neural Network Predicts the Face Behind a Voice" reporing at neurohive.io], [https://belitsoft.com/speech-recognition-software-development/speech2face "Speech2Face Sees Voices and Hears Faces: Dreams Come True with AI" reporting at belitsoft.com]

* '''2019''' | <font color="red">'''crime'''</font> | '''[[w:Fraud]]''' with [[#digital sound-alikes|digital sound-alike]] technology surfaced in 2019. See [https://www.washingtonpost.com/technology/2019/09/04/an-artificial-intelligence-first-voice-mimicking-software-reportedly-used-major-theft/?noredirect=on ''''''An artificial-intelligence first: Voice-mimicking software reportedly used in a major theft''''''], a 2019 Washington Post article or [https://www.forbes.com/sites/jessedamiani/2019/09/03/a-voice-deepfake-was-used-to-scam-a-ceo-out-of-243000/ ''''''A Voice Deepfake Was Used To Scam A CEO Out Of $243,000'''''' at Forbes.com] (2019-09-03)

* '''2019''' | demonstration | [http://whichfaceisreal.com/ ''''''Which Face is real?'''''' at whichfaceisreal.com] is an easily unnerving game by [http://ctbergstrom.com/ Carl Bergstrom] and [https://jevinwest.org/ Jevin West] where you need to '''try to distinguish''' from a pair of photos '''which is real and which is not'''. A part of the "tools" of the [https://callingbullshit.org/ Calling Bullshit] course taught at the [[w:University of Washington]]. <font color="green">'''Relevancy: certain'''</font>

* '''2019''' | demonstration | '''[https://www.thispersondoesnotexist.com/ 'Thispersondoesnotexist.com']''' (since February 2019) by Philip Wang. It showcases a [[w:StyleGAN]] at the task of making an endless stream of pictures that look like no-one in particular, but are eerily human-like. <font color="green">'''Relevancy: certain'''</font>

* '''2019''' | action | [[w:Nvidia]] [[w:open source]]s [[w:StyleGAN]], a novel [[w:generative adversarial network]].<ref name="Medium2019">

{{Cite web 
|url=https://medium.com/syncedreview/nvidia-open-sources-hyper-realistic-face-generator-stylegan-f346e1a73826  
|title= NVIDIA Open-Sources Hyper-Realistic Face Generator StyleGAN
|last= 
|first= 
|date= 2019-02-09
|website= [[Medium.com]]
|access-date= 2020-07-13
}}

</ref>

* '''<font color="green">2018</font>''' | '''<font color="green">counter-measure</font>''' | In September 2018 Google added “'''involuntary synthetic pornographic imagery'''” to its '''ban list''', allowing anyone to request the search engine block results that falsely depict them as “nude or in a sexually explicit situation.”<ref name="WashingtonPost2018">

{{cite web
 |url= https://www.washingtonpost.com/technology/2018/12/30/fake-porn-videos-are-being-weaponized-harass-humiliate-women-everybody-is-potential-target/
 |title= Fake-porn videos are being weaponized to harass and humiliate women: 'Everybody is a potential target'
 |last= Harwell
 |first= Drew
 |date= 2018-12-30
 |website= 
 |publisher= [[w:The Washington Post]]
 |access-date= 2020-07-13 
 |quote= In September [of 2018], Google added “involuntary synthetic pornographic imagery” to its ban list
}}

</ref> '''[https://support.google.com/websearch/answer/9116649?hl=en Information on removing involuntary fake pornography from Google at support.google.com]''' if it shows up in Google and the '''[https://support.google.com/websearch/troubleshooter/3111061#ts=2889054%2C2889099%2C2889064%2C9171203 form to request removing involuntary fake pornography at support.google.com]''', select "''I want to remove: A fake nude or sexually explicit picture or video of myself''"

[[File:GoogleLogoSept12015.png|thumb|right|300px|[[w:Google]]'s logo. Google Research demonstrated their '''[https://google.github.io/tacotron/publications/speaker_adaptation/ sound-like-anyone-machine]''' at the '''2018''' [[w:Conference on Neural Information Processing Systems|w:Conference on Neural Information Processing Systems]] (NeurIPS). It requires only 5 seconds of sample to steal a voice.]]

* '''<font color="red">2018</font>''' | <font color="red">science</font> and <font color="red">demonstration</font> | The work [http://papers.nips.cc/paper/7700-transfer-learning-from-speaker-verification-to-multispeaker-text-to-speech-synthesis ''''Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis''''] ([https://arxiv.org/abs/1806.04558 at arXiv.org]) was presented at the 2018 [[w:Conference on Neural Information Processing Systems]] ('''NeurIPS'''). The pre-trained model is able to steal voices from a sample of only '''5 seconds''' with almost convincing results.

* '''2018''' | demonstration | At the 2018 [[w:World Internet Conference]] in [[w:Wuzhen]] the [[w:Xinhua News Agency]] presented two digital look-alikes made to the resemblance of its real news anchors Qiu Hao ([[w:Chinese language]])<ref name="TheGuardian2018">
{{cite web
 | url = https://www.theguardian.com/world/2018/nov/09/worlds-first-ai-news-anchor-unveiled-in-china
 | title = World's first AI news anchor unveiled in China 
 | last = Kuo
 | first =  Lily
 | date = 2018-11-09
 | website = 
 | access-date = 2020-07-13
 | quote = }}
</ref> and Zhang Zhao ([[w:English language]]). The digital look-alikes were made in conjunction with [[w:Sogou]].<ref name="BusinessInsider2018">
{{cite web
 | url = https://businessinsider.com/ai-news-anchor-created-by-china-xinhua-news-agency-2018-11
 | title = China created what it claims is the first AI news anchor — watch it in action here 
 | last = Hamilton
 | first =  Isobel Asher
 | date = 2018-11-09
 | website = 
 | access-date = 2020-07-13
 | quote = }}
</ref> Neither the [[w:speech synthesis]] used nor the gesturing of the digital look-alike anchors were good enough to deceive the watcher to mistake them for real humans imaged with a TV camera.

* '''2018''' | controversy / demonstration | The [[w:deepfake]]s controversy surfaces where [[w:Pornographic film|porn video]]s were doctored utilizing [[w:deep learning|w:deep machine learning]] so that the face of the actress was replaced by the software's opinion of what another persons face would look like in the same pose and lighting.

* '''2017''' | science | '''[http://grail.cs.washington.edu/projects/AudioToObama/ 'Synthesizing Obama: Learning Lip Sync from Audio' at grail.cs.washington.edu]'''. In SIGGRAPH 2017 by Supasorn Suwajanakorn et al. of the [[w:University of Washington]] presented an audio driven digital look-alike of upper torso of Barack Obama. It was driven only by a voice track as source data for the animation after the training phase to acquire [[w:lip sync]] and wider facial information from [[w:training material]] consisting 2D videos with audio had been completed.<ref name="Suw2017">{{Citation
 | last = Suwajanakorn | first = Supasorn 
 | author-link = 
 | last2 = Seitz | first2 = Steven 
 | author2-link = 
 | last3 = Kemelmacher-Shlizerman | first3 = Ira 
 | author3-link = 
 | title = Synthesizing Obama: Learning Lip Sync from Audio
 | publisher = [[University of Washington]]
 | year = 2017 
 | url = http://grail.cs.washington.edu/projects/AudioToObama/
 | access-date = 2020-07-13 }}
</ref> <font color="green">'''Relevancy: certain'''</font>

* '''2016''' | movie | '''[[w:Rogue One]]''' is a Star Wars film for which digital look-alikes of actors [[w:Peter Cushing]] and [[w:Carrie Fisher]] were made. In the film their appearance would appear to be of same age as the actors were during the filming of the original 1977 ''[[w:Star Wars (film)]]'' film.

* '''2016''' | science / demonstration | [[w:DeepMind]]'s [[w:WaveNet]] owned by [[w:Google]] also demonstrated ability to steal people's voices

[[File:Adobe Corporate Logo.png|thumb|right|300px|[[w:Adobe Inc.]]'s logo. We can thank Adobe for publicly demonstrating their sound-like-anyone-machine in '''2016''' before an implementation was sold to criminal organizations.]]

{{#ev:youtube|I3l4XLZ59iw|420px|left|'''''#[[w:Adobe Voco]]. Adobe Audio Manipulator Sneak Peak with [[w:Jordan Peele]]''''' (at Youtube.com). November 2016 demonstration of a Adobe's unreleased sound-like-anyone-machine, the '''[[w:Adobe Voco]]''' at the [[w:Adobe MAX]] 2016 event in [[w:San Diego]], [[w:California]]. The original Adobe Voco '''required 20 minutes of sample''' to '''thieve a voice'''.}}
* '''<font color="red">2016</font>''' | <font color="red">science</font> and demonstration | '''[[w:Adobe Inc.]]''' publicly demonstrates '''[[w:Adobe Voco]]''', a '''sound-like-anyone machine''' [https://www.youtube.com/watch?v=I3l4XLZ59iw '#VoCo. Adobe Audio Manipulator Sneak Peak with Jordan Peele | Adobe Creative Cloud' on Youtube]. THe original Adobe Voco required '''20 minutes''' of sample '''to thieve a voice'''. <font color="green">'''Relevancy: certain'''</font>.

* '''2016''' | science | '''[http://www.niessnerlab.org/projects/thies2016face.html 'Face2Face: Real-time Face Capture and Reenactment of RGB Videos' at Niessnerlab.org]''' A paper (with videos) on the semi-real-time 2D video manipulation with gesture forcing and lip sync forcing synthesis by Thies et al, Stanford. <font color="green">'''Relevancy: certain'''</font>

* '''2016''' | music video | [https://www.youtube.com/watch?v=tMQHAy0HUDo ''''''Plug'''''' by Kube at youtube.com] - A 2016 music video by [[w:Kube (rapper)]] ([[w:fi:Kube]]), that shows deepfake-like technology this early. Video was uploaded on 2016-09-15 and is directed by Faruk Nazeri.

* '''2015''' | movie | In the ''[[w:Furious 7]]'' a digital look-alike made of the actor [[w:Paul Walker]] who died in an accident during the filming was done by [[w:Weta Digital]] to enable the completion of the film.<ref name="thr2015">
{{cite web
 | url = http://www.hollywoodreporter.com/behind-screen/furious-7-how-peter-jacksons-784157
 | title = 'Furious 7' and How Peter Jackson's Weta Created Digital Paul Walker
 | last = Giardina 
 | first =  Carolyn 
 | date = 2015-03-25
 | work=[[The Hollywood Reporter]]
 | access-date = 2020-07-13
 | quote = }}
</ref>

* '''2014''' | science | [[w:Ian Goodfellow]] et al. presented the principles of a '''[[w:generative adversarial network]]'''.  '''GAN'''s made the headlines in early 2018 with the [[w:deepfake]]s controversies.

* '''2013''' | demonstration | At the 2013 SIGGGRAPH [[w:Activision]] and USC presented a [[w:real time computing]] "Digital Ira" a digital face look-alike of Ari Shapiro, an ICT USC research scientist,<ref name="reform_youtube2015">
{{cite AV media
 | people =
 | title = ReForm - Hollywood's Creating Digital Clones
 | medium = youtube
 | publisher = The Creators Project
 | location =
 | date = 2020-07-13
 | url = https://www.youtube.com/watch?v=lTC3k9Iv4r0
}}

</ref> utilizing the USC light stage X by Ghosh et al. for both reflectance field and motion capture.<ref name="Deb2013">{{cite web
  | last = Debevec
  | first = Paul
  | title = Digital Ira SIGGRAPH 2013 Real-Time Live
  | website =
  | date =
  | url = http://gl.ict.usc.edu/Research/DigitalIra/
  | format =
  | doi =
  | accessdate =  2017-07-13}}
</ref> The end result both precomputed and real-time rendering with the modernest game [[w:Graphics processing unit|w:GPU]] shown [http://gl.ict.usc.edu/Research/DigitalIra/ here] and looks fairly realistic.

* '''2013''' | demonstration | A '''[https://ict.usc.edu/pubs/Scanning%20and%20Printing%20a%203D%20Portrait%20of%20President%20Barack%20Obama.pdf 'Scanning and Printing a 3D Portrait of President Barack Obama' at ict.usc.edu]'''.  A 7D model and a 3D bust was made of President Obama with his consent. Relevancy: <font color="green">'''Relevancy: certain'''</font>

=== 2000's synthetic human-like fakes ===

* '''2010''' | movie | [[w:Walt Disney Pictures]] released a sci-fi sequel entitled ''[[w:Tron: Legacy]]'' with a digitally rejuvenated digital look-alike made of the actor [[w:Jeff Bridges]] playing the [[w:antagonist]] [[w:List of Tron characters#CLU|w:CLU]].

* '''2009''' | movie | A digital look-alike of a younger [[w:Arnold Schwarzenegger]] was made for the movie ''[[w:Terminator Salvation]]'' though the end result was critiqued as unconvincing. Facial geometry was acquired from a 1984 mold of Schwarzenegger.

* '''2009''' | demonstration | [http://www.ted.com/talks/paul_debevec_animates_a_photo_real_digital_face.html Paul Debevec: ''''Animating a photo-realistic face'''' at ted.com] Debevec et al. presented new digital likenesses, made by [[w:Image Metrics]], this time of actress [[w:Emily O'Brien]] whose reflectance was captured with the USC light stage 5. At 00:04:59 you can see two clips, one with the real Emily shot with a real camera and one with a digital look-alike of Emily, shot with a simulation of a camera - <u>Which is which is difficult to tell</u>. Bruce Lawmen was scanned using USC light stage 6 in still position and also recorded running there on a [[w:treadmill]]. Many, many digital look-alikes of Bruce are seen running fluently and natural looking at the ending sequence of the TED talk video. <ref name="Deb2009">[http://www.ted.com/talks/paul_debevec_animates_a_photo_real_digital_face.html In this TED talk video] at 00:04:59 you can see ''two clips, one with the real Emily shot with a real camera and one with a digital look-alike of Emily, shot with a simulation of a camera - <u>Which is which is difficult to tell</u>''. Bruce Lawmen was scanned using USC light stage 6 in still position and also recorded running there on a [[w:treadmill]]. Many, many digital look-alikes of Bruce are seen running fluently and natural looking at the ending sequence of the TED talk video.</ref> Motion looks fairly convincing contrasted to the clunky run in the [[w:Animatrix#Final Flight of the Osiris|w:''Animatrix: Final Flight of the Osiris'']] which was [[w:state-of-the-art]] in 2003 if photorealism was the intention of the [[w:animators]].

* '''2004''' | movie | The '''[[w:Spider-man 2]]''' (and '''[[w:Spider-man 3]]''', 2007) films. Relevancy: The films include a [[Synthetic human-like fakes#Digital look-alike|digital look-alike]] made of actor [[w:Tobey Maguire]] by [[w:Sony Pictures Imageworks]].<ref name="Pig2005">{{cite web
  | last = Pighin
  | first = Frédéric
  | author-link =
  | title = Siggraph 2005 Digital Face Cloning Course Notes
  | website =
  | date =
  | url = http://pages.cs.wisc.edu/~lizhang/sig-course-05-face/COURSE_9_DigitalFaceCloning.pdf
  | format =
  | doi =
  | accessdate = 2020-06-26}}
</ref>

* '''2003''' | short film | [[w:The Animatrix#Final Flight of the Osiris|w:''The Animatrix: Final Flight of the Osiris'']] a [[w:state-of-the-art]] want-to-be human likenesses not quite fooling the watcher made by [[w:Square Pictures#Square Pictures|w:Square Pictures]].

[[File:The-matrix-logo.svg|thumb|left|300px|Logo of the [[w:The Matrix (franchise)]]]]

[[File:BSSDF01_400.svg|thumb|left|300px|Traditional [[w:Bidirectional reflectance distribution function|w:BRDF]] vs. [[w:subsurface scattering|subsurface scattering]] inclusive BSSRDF i.e. [[w:Bidirectional scattering distribution function#Overview of the BxDF functions|w:Bidirectional scattering-surface reflectance distribution function]]. <br/><br/>

An analytical BRDF must take into account the subsurface scattering, or the end result '''will not pass human testing'''.]]

* '''2003''' | movie(s) | The '''[[w:Matrix Reloaded]]''' and '''[[w:Matrix Revolutions]]''' films. Relevancy: '''First public display''' of '''[[synthetic human-like fakes#Digital look-alikes|digital look-alikes]]''' that are virtually '''indistinguishable from''' the '''real actors'''. [https://www.researchgate.net/publication/215518319_Universal_Capture_-_Image-based_Facial_Animation_for_The_Matrix_Reloaded ''''''Universal Capture - Image-based Facial Animation for "The Matrix Reloaded"'''''' at researchgate.net] (2003)

{{#ev:youtube|3qIXIHAmcKU|640px|right|Music video for '''''Bullet''''' by [[w:Covenant (band)|w:Covenant]] from 2002. Here you can observe the classic "''skin looks like cardboard''"-bug that stopped the pre-reflectance capture era versions from passing human testing.}}

* '''2002''' | music video | '''[https://www.youtube.com/watch?v=3qIXIHAmcKU 'Bullet' by Covenant on Youtube]''' by [[w:Covenant (band)]] from their album [[w:Northern Light (Covenant album)]]. Relevancy: Contains the best upper-torso digital look-alike of Eskil Simonsson (vocalist) that their organization could procure at the time. Here you can observe the '''classic "''skin looks like cardboard''"-bug''' (assuming this was not intended) that '''thwarted efforts to''' make digital look-alikes that '''pass human testing''' before the '''reflectance capture and dissection in 1999''' by [[w:Paul Debevec]] et al. at the [[w:University of Southern California]] and subsequent development of the '''"Analytical [[w:bidirectional reflectance distribution function|w:BRDF]]"''' (quote-unquote) by ESC Entertainment, a company set up for the '''sole purpose''' of '''making the cinematography''' for the 2003 films Matrix Reloaded and Matrix Revolutions '''possible''', lead by George Borshukov.

=== 1990's synthetic human-like fakes ===

[[File:Institute for Creative Technologies (logo).jpg|thumb|left|156px|Logo of the '''[[w:Institute for Creative Technologies]]''' founded in 1999 in the [[w:University of Southern California]] by the [[w:United States Army]]]]

* <font color="red">'''1999'''</font> | <font color="red">'''science'''</font> | '''[http://dl.acm.org/citation.cfm?id=344855 'Acquiring the reflectance field of a human face' paper at dl.acm.org ]''' [[w:Paul Debevec]] et al. of [[w:University of Southern California|w:USC]] did the '''first known reflectance capture''' over '''the human face''' with their extremely simple [[w:light stage]]. They presented their method and results in [[w:SIGGRAPH]] 2000. The scientific breakthrough required finding the [[w:subsurface scattering|w:subsurface light component]] (the simulation models are glowing from within slightly) which can be found using knowledge that light that is reflected from the oil-to-air layer retains its [[w:Polarization (waves)]] and the subsurface light loses its polarization. So equipped only with a movable light source, movable video camera, 2 polarizers and a computer program doing extremely simple math and the last piece required to reach photorealism was acquired.<ref name="Deb2000"/>

* <font color="red">'''1999'''</font> | <font color="red">'''institute founded'''</font> | The '''[[w:Institute for Creative Technologies]]''' was founded by the [[w:United States Army]] in the [[w:University of Southern California]]. It collaborates with the [[w:United States Army Futures Command]], [[w:United States Army Combat Capabilities Development Command]], [[w:Combat Capabilities Development Command Soldier Center]] and [[w:United States Army Research Laboratory]].<ref name="ICT-about">https://ict.usc.edu/about/</ref>. In 2016 [[w:Hao Li]] was appointed to direct the institute. 

* '''1994''' | movie | [[w:The Crow (1994 film)]] was the first film production to make use of [[w:digital compositing]] of a computer simulated representation of a face onto scenes filmed using a [[w:body double]]. Necessity was the muse as the actor [[w:Brandon Lee]] portraying the protagonist was tragically killed accidentally on-stage.

=== 1970's synthetic human-like fakes ===

{{#ev:vimeo|16292363|480px|right|''[[w:A Computer Animated Hand|w:A Computer Animated Hand]]'' is a 1972 short film by [[w:Edwin Catmull]] and [[w:Fred Parke]]. This was the first time that [[w:computer-generated imagery]] was used in film to animate likenesses of moving human appearance.}}

* '''1976''' | movie | ''[[w:Futureworld]]'' reused parts of ''A Computer Animated Hand'' on the big screen.

* '''1972''' | entertainment | '''[https://vimeo.com/59434349 'A Computer Animated Hand' on Vimeo]'''. [[w:A Computer Animated Hand]] by [[w:Edwin Catmull]] and [[w:Fred Parke]]. Relevancy: This was the '''first time''' that [[w:computer-generated imagery]] was used in film to '''animate''' moving '''human-like appearance'''.

* '''1971''' | science | '''[https://interstices.info/images-de-synthese-palme-de-la-longevite-pour-lombrage-de-gouraud/ 'Images de synthèse : palme de la longévité pour l’ombrage de Gouraud' (still photos)]'''. [[w:Henri Gouraud (computer scientist)]] made the first [[w:Computer graphics]] [[w:geometry]] [[w:digitization]] and representation of a human face. Modeling was his wife Sylvie Gouraud. The 3D model was a simple [[w:wire-frame model]] and he applied [[w:Gouraud shading]] to produce the '''first known representation''' of '''human-likeness''' on computer. <ref>{{cite web|title=Images de synthèse : palme de la longévité pour l'ombrage de Gouraud|url=http://interstices.info/jcms/c_25256/images-de-synthese-palme-de-la-longevite-pour-lombrage-de-gouraud}}</ref>

=== 1770's synthetic human-like fakes ===

[[File:Kempelen Speakingmachine.JPG|right|thumb|300px|A replica of [[w:Wolfgang von Kempelen]]'s [[w:Wolfgang von Kempelen's Speaking Machine]], built 2007–09 at the Department of [[w:Phonetics]], [[w:Saarland University]], [[w:Saarbrücken]], Germany. This machine added models of the tongue and lips, enabling it to produce [[w:consonant]]s as well as [[w:vowel]]s]]

* '''1791''' | science | '''[[w:Wolfgang von Kempelen's Speaking Machine]]''' of [[w:Wolfgang von Kempelen]] of [[w:Pressburg]], [[w:Hungary]], described in a 1791 paper was [[w:bellows]]-operated.<ref>''Mechanismus der menschlichen Sprache nebst der Beschreibung seiner sprechenden Maschine'' ("Mechanism of the human speech with description of its speaking machine", J. B. Degen, Wien).</ref> This machine added models of the tongue and lips, enabling it to produce [[w:consonant]]s as well as [[w:vowel]]s. (based on [[w:Speech synthesis#History]])

* '''1779''' | science / discovery | [[w:Christian Gottlieb Kratzenstein]] won the first prize in a competition announced by the [[w:Russian Academy of Sciences]] for '''models''' he built of the '''human [[w:vocal tract]]''' that could produce the five long '''[[w:vowel]]''' sounds.<ref name="Helsinki">
[http://www.acoustics.hut.fi/publications/files/theses/lemmetty_mst/chap2.html History and Development of Speech Synthesis], Helsinki University of Technology, Retrieved on November 4, 2006
</ref> (Based on [[w:Speech synthesis#History]])

----

== Footnotes ==
<references group="footnote" /> 

== 1st seen in ==
<references group="1st seen in" />

== References ==
<references />