Synthetic human-like fakes: Difference between revisions

'Saint John on Patmos' pictures [[w:John of Patmos]] on [[w:Patmos]] writing down the visions to make the [[w:Book of Revelation]]. Picture from folio 17 of the [[w:Très Riches Heures du Duc de Berry]] (1412-1416) by the [[w:Limbourg brothers]]. Currently located at the [[w:Musée Condé|Musée Condé]] 40km north of Paris, France.]]

{{#ev:youtube|LWLadJFI8Pk|640px|right|It is recommended that you watch ''In Event of Moon Disaster - FULL FILM'' (2020) at the '''[https://moondisaster.org/ moondisaster.org''' project website] (where it has interactive portions) by the [https://virtuality.mit.edu/ Center for Advanced Virtuality] of the [[w:Massachusetts Institute of Technology|MIT]]}}

<small>[[:File:Deb-2000-reflectance-separation.png|Original picture]]  by [[w:Paul Debevec|Debevec]] et al. - Copyright ACM 2000 https://dl.acm.org/citation.cfm?doid=311779.344855</small>]]

{{Q|Do you think that was [[w:Hugo Weaving|Hugo Weaving]]'s left cheekbone that [[w:Keanu Reeves|Keanu Reeves]] punched in with his right fist?|Trad|The Matrix Revolutions}}

* In the '''2018''' at the '''[[w:Conference on Neural Information Processing Systems|Conference on Neural Information Processing Systems]]''' (NeurIPS) the work [http://papers.nips.cc/paper/7700-transfer-learning-from-speaker-verification-to-multispeaker-text-to-speech-synthesis 'Transfer Learning from Speaker Verification to Multispeaker Text-To-Speech Synthesis'] ([https://arxiv.org/abs/1806.04558 at arXiv.org]) was presented. The pre-trained model is able to steal voices from a sample of only '''5 seconds''' with almost convincing results

* Sound like anyone technology found its way to the hands of criminals as in '''2019''' [[w:NortonLifeLock|Symantec]] researchers knew of 3 cases where technology has been used for '''[[w:crime|crime]]'''

The below video 'This AI Clones Your Voice After Listening for 5 Seconds' by '2 minute papers' describes the voice thieving machine presented by Google Research in [[w:NeurIPS|NeurIPS]] 2018.

{{#ev:youtube|0sR1rU3gLzQ|640px|right|Video 'This AI Clones Your Voice After Listening for 5 Seconds' by '2 minute papers' describes the voice thieving machine by Google Research in [[w:NeurIPS|NeurIPS]] 2018.}}

[[File:Spectrogram-19thC.png|thumb|right|640px|A [[w:spectrogram|spectrogram]] of a male voice saying 'nineteenth century']]

[[w:OpenAI]]'s [[w:OpenAI#GPT|Generative Pre-trained Transformer]] ('''GPT''') is a left-to-right [[w:Transformer (machine learning model)|transformer]]-based [[w:Natural-language generation|text generation]] model succeeded by [[w:OpenAI#GPT-2|GPT-2]] and [[w:OpenAI#GPT-3|GPT-3]]

[[File:DARPA_Logo.jpg|thumb|left|240px|The Defense Advanced Research Projects Agency, better known as [[w:DARPA|DARPA]] has been active in the field of countering synthetic fake video for longer than the public has been aware of the problems existing.]]

[[File:Connie Leyva 2015.jpg|thumb|left|240px|[[w:California|California]] [[w:California State Senate|Senator]] [[w:Connie Leyva|Connie Leyva]] introduced [https://leginfo.legislature.ca.gov/faces/billNavClient.xhtml?bill_id=201920200SB564 California Senate Bill SB 564] in Feb '''2019'''. It has been [https://www.sagaftra.org/action-alert-support-california-bill-end-deepfake-porn endorsed by SAG-AFTRA], but has not yet passed.]]

* '''[[w:SAG-AFTRA]]''' [https://www.sagaftra.org/action-alert-support-california-bill-end-deepfake-porn SAG-AFTRA ACTION ALERT: '''"Support California Bill to End Deepfake Porn"''' at sagaftra.org '''endorses'''] [https://leginfo.legislature.ca.gov/faces/billNavClient.xhtml?bill_id=201920200SB564 California Senate Bill SB 564] introduced to the  [[w:California State Senate|California State Senate]] by [[w:California|California]] [[w:Connie Leyva|Senator Connie Leyva]] in Feb '''2019'''.

* '''2020''' | '''[[w:Conference on Computer Vision and Pattern Recognition|CVPR]]''' |  [https://sites.google.com/view/wmediaforensics2020/home 2020 Conference on Computer Vision and Pattern Recognition:  ''''Workshop on Media Forensics'''' at sites.google.com], a '''June 2020''' workshop at the [[w:Conference on Computer Vision and Pattern Recognition]].  

* '''2019''' | '''[[w:Conference on Neural Information Processing Systems|NeurIPS]]''' | [[w:Facebook, Inc.|Facebook, Inc.]] [https://spectrum.ieee.org/tech-talk/artificial-intelligence/machine-learning/facebook-ai-launches-its-deepfake-detection-challenge '''"Facebook AI Launches Its Deepfake Detection Challenge"''' at spectrum.ieee.org]

* '''2018''' | '''[[w:European Conference on Computer Vision|ECCV]]''' [https://sites.google.com/view/wocm2018/home ECCV 2018: ''''Workshop on Objectionable Content and Misinformation'''' at sites.google.com], a workshop at the '''2018''' [[w:European Conference on Computer Vision]] in [[w:Munich]] had focus on objectionable content detection e.g. [[w:nudity]], [[w:pornography]], [[w:violence]], [[w:hate]], [[w:Child sexual abuse|children exploitation]] and [[w:terrorism]] among others and to address misinformation problems when people are fed [[w:disinformation]] and they punt it on as misinformation. Announced topics included [[w:Outline of forensic science|image/video forensics]], [[w:detection]]/[[w:analysis]]/[[w:understanding]] of [[w:Counterfeit|fake]] images/videos, [[w:misinformation]] detection/understanding: mono-modal and [[w:Multimodality|multi-modal]], adversarial technologies and detection/understanding of objectionable content

* '''2018''' | '''[[w:National Institute of Standards and Technology|NIST]]''' [https://www.nist.gov/itl/iad/mig/media-forensics-challenge-2018 NIST ''''Media Forensics Challenge 2018'''' at nist.gov] was the second annual evaluation to support research and help advance the state of the art for image and video forensics technologies – technologies that determine the region and type of manipulations in imagery (image/video data) and the phylogenic process that modified the imagery.  

* '''2017''' | '''[[w:National Institute of Standards and Technology|NIST]]'''  [https://www.nist.gov/itl/iad/mig/nimble-challenge-2017-evaluation NIST ''''Nimble Challenge 2017'''' at nist.gov]

* [https://scholarship.law.duke.edu/cgi/viewcontent.cgi?article=1333&context=dltr ''''DEEPFAKES: False pornography is here and the law cannot protect you'''' at scholarship.law.duke.edu] by Douglas Harris, published in [https://scholarship.law.duke.edu/dltr/vol17/iss1/ Duke Law & Technology Review - Volume 17 on '''2019'''-01-05] by [[w:Duke University|Duke University]] [[w:Duke University School of Law|School of Law]]

* '''2020''' | demonstration | '''[https://moondisaster.org/ Moondisaster.org]''' (full film embedded in website) project by the [https://virtuality.mit.edu/ Center for Advanced Virtuality] of the [[w:Massachusetts Institute of Technology|MIT]] published in July 2020, makes use of various methods of making a synthetic human-like fake. Alternative place to watch: [https://www.youtube.com/watch?v=LWLadJFI8Pk ''In Event of Moon Disaster - FULL FILM'' at youtube.com]   

[[File:Marc Berman.jpg|thumb|120px|left|Homie [[w:Marc Berman|Marc Berman]], a righteous fighter for our human rights in this age of industrial disinformation filth and a member of the [[w:California State Assembly]], most loved for authoring [https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=201920200AB602 AB-602], which came into effect on Jan 1 2020, banning both the manufacturing and [[w:Digital distribution|distribution]] of synthetic pornography without the [[w:consent]] of the people depicted.]]

</ref> the [[w:California]] [[w:State law (United States)|state law]] [https://leginfo.legislature.ca.gov/faces/billTextClient.xhtml?bill_id=201920200AB602 AB-602] came into effect banning the manufacturing and [[w:Digital distribution|distribution]] of synthetic pornography without the [[w:consent]] of the people depicted.  AB-602 provides victims of synthetic pornography with [[w:injunction|injunctive relief]] and poses legal threats of [[w:statutory damages|statutory]] and [[w:punitive damages]] on [[w:criminal]]s making or distributing synthetic pornography without consent. The bill AB-602 was signed into law by California [[w:Governor (United States)|Governor]] [[w:Gavin Newsom]] on October 3 2019 and was authored by [[w:California State Assembly]] member [[w:Marc Berman]].<ref name="CNET2019">

[[File:Marcus Simon.jpeg|thumb|right|108px|Homie [[w:Marcus Simon|Marcus Simon]] ([http://marcussimon.com/ marcussimon.com]) is a Member of the [[w:Virginia House of Delegates]] and a true pioneer in legislating against synthetic filth.]]

</ref> [[w:Virginia]] [[w:criminalization|has criminalized]] the sale and dissemination of unauthorized synthetic pornography, but not the manufacture.<ref name="Virginia2019Chapter515">

</ref>, as [https://law.lis.virginia.gov/vacode/18.2-386.2/ § 18.2-386.2 titled ''''''Unlawful dissemination or sale of images of another; penalty.''''''] became part of the '''[[w:Code of Virginia]]'''. The law text states: "''Any person who, with the [[w:Intention (criminal law)|intent]] to [[w:coercion|coerce]], [[w:harassment|harass]], or [[w:intimidation|intimidate]], [[w:Malice_(law)|malicious]]ly [[w:dissemination|disseminates]] or [[w:sales|sells]] any videographic or still image created by any means whatsoever that depicts another person who is totally [w:[nudity|nude]], or in a state of undress so as to expose the [[w:sex organs|genitals]], pubic area, [[w:buttocks]], or female [[w:breast]], where such person knows or has reason to know that he is not [[w:license]]d or [[w:authorization|authorized]] to disseminate or sell such videographic or still image is guilty of a Class 1 [[w:Misdemeanor#United States|misdemeanor]].''".<ref name="Virginia2019Chapter515"/> The identical bills were [https://lis.virginia.gov/cgi-bin/legp604.exe?191+sum+HB2678 House Bill 2678] presented by [[w:Delegate (American politics)|Delegate]] [[w:Marcus Simon]] to the [[w:Virginia House of Delegates]] on January 14 2019 and three day later an identical [https://lis.virginia.gov/cgi-bin/legp604.exe?191+sum+SB1736 Senate bill 1736] was introduced to the [[w:Senate of Virginia]] by Senator [[w:Adam Ebbin]].  

* '''2019''' | science and demonstration | [https://arxiv.org/pdf/1905.09773.pdf ''''Speech2Face: Learning the Face Behind a Voice'''' at arXiv.org] a system for generating likely facial features based on the voice of a person, presented by the [[w:MIT Computer Science and Artificial Intelligence Laboratory]] at the 2019 [[w:Conference on Computer Vision and Pattern Recognition|CVPR]]. [https://github.com/saiteja-talluri/Speech2Face Speech2Face at github.com] This may develop to something that really causes problems. [https://neurohive.io/en/news/speech2face-neural-network-predicts-the-face-behind-a-voice/ "Speech2Face: Neural Network Predicts the Face Behind a Voice" reporing at neurohive.io], [https://belitsoft.com/speech-recognition-software-development/speech2face "Speech2Face Sees Voices and Hears Faces: Dreams Come True with AI" reporting at belitsoft.com]

[[File:GoogleLogoSept12015.png|thumb|right|300px|[[w:Google|Google]]'s logo. Google Research demonstrated their '''[https://google.github.io/tacotron/publications/speaker_adaptation/ sound-like-anyone-machine]''' at the '''2018''' [[w:Conference on Neural Information Processing Systems|Conference on Neural Information Processing Systems]] (NeurIPS). It requires only 5 seconds of sample to steal a voice.]]

* '''2018''' | controversy / demonstration | The [[w:deepfake]]s controversy surfaces where [[w:Pornographic film|porn video]]s were doctored utilizing [[w:deep learning|deep machine learning]] so that the face of the actress was replaced by the software's opinion of what another persons face would look like in the same pose and lighting.

{{#ev:youtube|I3l4XLZ59iw|420px|left|'''''#[[w:Adobe Voco|VoCo]]. Adobe Audio Manipulator Sneak Peak with [[w:Jordan Peele|Jordan Peele]]''''' (at Youtube.com). November 2016 demonstration of a Adobe's unreleased sound-like-anyone-machine, the '''[[w:Adobe Voco|Adobe Voco]]''' at the [[w:Adobe MAX|Adobe MAX]] 2016 event in [[w:San Diego|San Diego]], [[w:California|CA]]. The original Adobe Voco '''required 20 minutes of sample''' to '''thieve a voice'''.}}

* '''2013''' | demonstration | At the 2013 SIGGGRAPH [[w:Activision]] and USC presented a [[w:real time computing|real time]] "Digital Ira" a digital face look-alike of Ari Shapiro, an ICT USC research scientist,<ref name="reform_youtube2015">

</ref> The end result both precomputed and real-time rendering with the modernest game [[w:Graphics processing unit|GPU]] shown [http://gl.ict.usc.edu/Research/DigitalIra/ here] and looks fairly realistic.

* '''2010''' | movie | [[w:Walt Disney Pictures]] released a sci-fi sequel entitled ''[[w:Tron: Legacy]]'' with a digitally rejuvenated digital look-alike made of the actor [[w:Jeff Bridges]] playing the [[w:antagonist]] [[w:List of Tron characters#CLU|CLU]].

* '''2009''' | demonstration | [http://www.ted.com/talks/paul_debevec_animates_a_photo_real_digital_face.html Paul Debevec: ''''Animating a photo-realistic face'''' at ted.com] Debevec et al. presented new digital likenesses, made by [[w:Image Metrics]], this time of actress [[w:Emily O'Brien]] whose reflectance was captured with the USC light stage 5. At 00:04:59 you can see two clips, one with the real Emily shot with a real camera and one with a digital look-alike of Emily, shot with a simulation of a camera - <u>Which is which is difficult to tell</u>. Bruce Lawmen was scanned using USC light stage 6 in still position and also recorded running there on a [[w:treadmill]]. Many, many digital look-alikes of Bruce are seen running fluently and natural looking at the ending sequence of the TED talk video. <ref name="Deb2009">[http://www.ted.com/talks/paul_debevec_animates_a_photo_real_digital_face.html In this TED talk video] at 00:04:59 you can see ''two clips, one with the real Emily shot with a real camera and one with a digital look-alike of Emily, shot with a simulation of a camera - <u>Which is which is difficult to tell</u>''. Bruce Lawmen was scanned using USC light stage 6 in still position and also recorded running there on a [[w:treadmill]]. Many, many digital look-alikes of Bruce are seen running fluently and natural looking at the ending sequence of the TED talk video.</ref> Motion looks fairly convincing contrasted to the clunky run in the [[w:Animatrix#Final Flight of the Osiris|''Animatrix: Final Flight of the Osiris'']] which was [[w:state-of-the-art]] in 2003 if photorealism was the intention of the [[w:animators]].

* '''2003''' | short film | [[w:The Animatrix#Final Flight of the Osiris|''The Animatrix: Final Flight of the Osiris'']] a [[w:state-of-the-art]] want-to-be human likenesses not quite fooling the watcher made by [[w:Square Pictures#Square Pictures|Square Pictures]].

[[File:BSSDF01_400.svg|thumb|left|300px|Traditional [[w:Bidirectional reflectance distribution function|BRDF]] vs. [[w:subsurface scattering|subsurface scattering]] inclusive BSSRDF i.e. [[w:Bidirectional scattering distribution function#Overview of the BxDF functions|Bidirectional scattering-surface reflectance distribution function]]. <br/><br/>

{{#ev:youtube|3qIXIHAmcKU|640px|right|Music video for '''''Bullet''''' by [[w:Covenant (band)|Covenant]] from 2002. Here you can observe the classic "''skin looks like cardboard''"-bug that stopped the pre-reflectance capture era versions from passing human testing.}}

* '''2002''' | music video | '''[https://www.youtube.com/watch?v=3qIXIHAmcKU 'Bullet' by Covenant on Youtube]''' by [[w:Covenant (band)]] from their album [[w:Northern Light (Covenant album)]]. Relevancy: Contains the best upper-torso digital look-alike of Eskil Simonsson (vocalist) that their organization could procure at the time. Here you can observe the '''classic "''skin looks like cardboard''"-bug''' (assuming this was not intended) that '''thwarted efforts to''' make digital look-alikes that '''pass human testing''' before the '''reflectance capture and dissection in 1999''' by [[w:Paul Debevec]] et al. at the [[w:University of Southern California]] and subsequent development of the '''"Analytical [[w:bidirectional reflectance distribution function|BRDF]]"''' (quote-unquote) by ESC Entertainment, a company set up for the '''sole purpose''' of '''making the cinematography''' for the 2003 films Matrix Reloaded and Matrix Revolutions '''possible''', lead by George Borshukov.

* <font color="red">'''1999'''</font> | <font color="red">'''science'''</font> | '''[http://dl.acm.org/citation.cfm?id=344855 'Acquiring the reflectance field of a human face' paper at dl.acm.org ]''' [[w:Paul Debevec]] et al. of [[w:University of Southern California|USC]] did the '''first known reflectance capture''' over '''the human face''' with their extremely simple [[w:light stage]]. They presented their method and results in [[w:SIGGRAPH]] 2000. The scientific breakthrough required finding the [[w:subsurface scattering|w:subsurface light component]] (the simulation models are glowing from within slightly) which can be found using knowledge that light that is reflected from the oil-to-air layer retains its [[w:Polarization (waves)]] and the subsurface light loses its polarization. So equipped only with a movable light source, movable video camera, 2 polarizers and a computer program doing extremely simple math and the last piece required to reach photorealism was acquired.<ref name="Deb2000"/>

* '''1994''' | movie | [[w:The Crow (1994 film)|The Crow]] was the first film production to make use of [[w:digital compositing]] of a computer simulated representation of a face onto scenes filmed using a [[w:body double]]. Necessity was the muse as the actor [[w:Brandon Lee]] portraying the protagonist was tragically killed accidentally on-stage.

{{#ev:vimeo|16292363|480px|right|''[[w:A Computer Animated Hand|A Computer Animated Hand]]'' is a 1972 short film by [[w:Edwin Catmull|Edwin Catmull]] and [[w:Fred Parke|Fred Parke]]. This was the first time that [[w:computer-generated imagery|computer-generated imagery]] was used in film to animate likenesses of moving human appearance.}}

* '''1972''' | entertainment | '''[https://vimeo.com/59434349 'A Computer Animated Hand' on Vimeo]'''. [[w:A Computer Animated Hand]] by [[w:Edwin Catmull]] and [[w:Fred Parke]]. Relevancy: This was the '''first time''' that [[w:computer-generated imagery|computer-generated imagery]] was used in film to '''animate''' moving '''human-like appearance'''.

[[File:Kempelen Speakingmachine.JPG|right|thumb|300px|A replica of [[w:Wolfgang von Kempelen|Kempelen]]'s [[w:Wolfgang von Kempelen's Speaking Machine|speaking machine]], built 2007–09 at the Department of [[w:Phonetics|Phonetics]], [[w:Saarland University|Saarland University]], [[w:Saarbrücken|Saarbrücken]], Germany. This machine added models of the tongue and lips, enabling it to produce [[w:consonant|consonant]]s as well as [[w:vowel|vowel]]s]]

[[Biblical explanation - The books of Daniel and Revelation#Daniel 7|Daniel 7]], Daniel's vision of the three beasts ^{[[Biblical explanation - The books of Daniel and Revelation#Daniel 7:1-6 - Three beasts|Dan 7:1-6]]} and the fourth beast ^{[[Biblical explanation - The books of Daniel and Revelation#Daniel 7:7-8 - The fourth beast|Dan 7:7-8]]} from the sea and the [[w:Ancient of Days|Ancient of Days]]^{[[Biblical explanation - The books of Daniel and Revelation#The Ancient of Days|Dan 7:9-10]]}]]