3,838
edits
Synthetic human-like fakes: Difference between revisions
Jump to navigation
Jump to search
→Events against synthetic human-like fakes: + '''CVPR 2021 research areas visualization by Joshua Preston''' at public.tableau.com + tweaks
Juho Kunsola (talk | contribs) (→Handwriting syntheses: + https://github.com/topics/handwriting-synthesis) |
Juho Kunsola (talk | contribs) (→Events against synthetic human-like fakes: + '''CVPR 2021 research areas visualization by Joshua Preston''' at public.tableau.com + tweaks) |
||
| (19 intermediate revisions by the same user not shown) | |||
| Line 2: | Line 2: | ||
When the '''[[Glossary#No camera|camera does not exist]]''', but the subject being imaged with a simulation of a (movie) camera deceives the watcher to believe it is some living or dead person it is a '''[[#Digital look-alikes|digital look-alike]]'''. | When the '''[[Glossary#No camera|camera does not exist]]''', but the subject being imaged with a simulation of a (movie) camera deceives the watcher to believe it is some living or dead person it is a '''[[#Digital look-alikes|digital look-alike]]'''. | ||
When it cannot be determined by human testing or media forensics whether some fake voice is a synthetic fake of some person's voice, or is it an actual recording made of that person's actual real voice, it is a pre-recorded '''[[#Digital sound-alikes|digital sound-alike]]'''. | When it cannot be determined by human testing or media forensics whether some fake voice is a synthetic fake of some person's voice, or is it an actual recording made of that person's actual real voice, it is a pre-recorded '''[[#Digital sound-alikes|digital sound-alike]]'''. | [[Synthetic human-like fakes|Read more about synthetic human-like fakes]], [[Synthetic human-like fakes#Timeline of synthetic human-like fakes|examine timeline of synthetic human-like fakes]] or [[Mediatheque|view Mediatheque]] | ||
<section end=definitions-of-synthetic-human-like-fakes /> | <section end=definitions-of-synthetic-human-like-fakes /> | ||
[[File:BlV1999-morphable-model-till-match-low-res-rip.png|thumb| | [[File:Screenshot at 27s of a moving digital-look-alike made to appear Obama-like by Monkeypaw Productions and Buzzfeed 2018.png|thumb|right|480px|link=Mediatheque/2018/Obama's appearance thieved - a public service announcement digital look-alike by Monkeypaw Productions and Buzzfeed|{{#lst:Mediatheque|Obama-like-fake-2018}}]] | ||
[[File:BlV1999-morphable-model-till-match-low-res-rip.png|thumb|right|460px|Image 2 (low resolution rip) shows a 1999 technique for sculpting a morphable model, till it matches the target's appearance. | |||
<br/>(1) Sculpting a morphable model to one single picture | <br/>(1) Sculpting a morphable model to one single picture | ||
<br/>(2) Produces 3D approximation | <br/>(2) Produces 3D approximation | ||
| Line 15: | Line 17: | ||
<small>Image 2 by Blanz and Vettel – Copyright ACM 1999 – http://dl.acm.org/citation.cfm?doid=311535.311556 – Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.</small>]] | <small>Image 2 by Blanz and Vettel – Copyright ACM 1999 – http://dl.acm.org/citation.cfm?doid=311535.311556 – Permission to make digital or hard copies of all or part of this work for personal or classroom use is granted without fee provided that copies are not made or distributed for profit or commercial advantage and that copies bear this notice and the full citation on the first page.</small>]] | ||
== Digital look-alikes == | == Digital look-alikes == | ||
| Line 156: | Line 154: | ||
<section end=GoogleTransferLearning2018 /> | <section end=GoogleTransferLearning2018 /> | ||
The to the right [https://www.youtube.com/watch?v=0sR1rU3gLzQ video 'This AI Clones Your Voice After Listening for 5 Seconds' by '2 minute papers' at YouTube] describes the voice thieving machine presented by Google Research in [[w:NeurIPS|w:NeurIPS]] 2018. | The to the right [https://www.youtube.com/watch?v=0sR1rU3gLzQ video 'This AI Clones Your Voice After Listening for 5 Seconds' by '2 minute papers' at YouTube] describes the voice thieving machine presented by Google Research in [[w:NeurIPS|w:NeurIPS]] 2018. | ||
{{#ev:youtube|0sR1rU3gLzQ|640px|right|Video [https://www.youtube.com/watch?v=0sR1rU3gLzQ video 'This AI Clones Your Voice After Listening for 5 Seconds' by '2 minute papers' at YouTube] describes the voice thieving machine by Google Research in [[w:NeurIPS|w:NeurIPS]] 2018.}} | {{#ev:youtube|0sR1rU3gLzQ|640px|right|Video [https://www.youtube.com/watch?v=0sR1rU3gLzQ video 'This AI Clones Your Voice After Listening for 5 Seconds' by '2 minute papers' at YouTube] describes the voice thieving machine by Google Research in [[w:NeurIPS|w:NeurIPS]] 2018.}} | ||
---- | ---- | ||
| Line 219: | Line 205: | ||
[[File:Spectrogram-19thC.png|thumb|right|640px|A [[w:spectrogram]] of a male voice saying 'nineteenth century']] | [[File:Spectrogram-19thC.png|thumb|right|640px|A [[w:spectrogram]] of a male voice saying 'nineteenth century']] | ||
== Singing syntheses == | |||
As of 2020 the '''digital sing-alikes''' may not yet be here, but when we hear a faked singing voice and we cannot hear that it is fake, then we will know. An ability to sing does not seem to add much hostile capabilities compared to the ability to thieve spoken word. | |||
* [https://arxiv.org/abs/1910.11690 ''''''Fast and High-Quality Singing Voice Synthesis System based on Convolutional Neural Networks'''''' at arxiv.org], a 2019 singing voice synthesis technique using [[w:convolutional neural network|w:convolutional neural networks (CNN)]]. Accepted into the 2020 [[w:International Conference on Acoustics, Speech, and Signal Processing|International Conference on Acoustics, Speech, and Signal Processing (ICASSP)]]. | |||
* [http://compmus.ime.usp.br/sbcm/2019/papers/sbcm-2019-7.pdf ''''''State of art of real-time singing voice synthesis'''''' at compmus.ime.usp.br] presented at the 2019 [http://compmus.ime.usp.br/sbcm/2019/program/ 17th Brazilian Symposium on Computer Music] | |||
* [http://theses.fr/2017PA066511 ''''''Synthesis and expressive transformation of singing voice'''''' at theses.fr] [https://www.theses.fr/2017PA066511.pdf as .pdf] a 2017 doctorate thesis by [http://theses.fr/227185943 Luc Ardaillon] | |||
* [http://mtg.upf.edu/node/512 ''''''Synthesis of the Singing Voice by Performance Sampling and Spectral Models'''''' at mtg.upf.edu], a 2007 journal article in the [[w:IEEE Signal Processing Society]]'s Signal Processing Magazine | |||
* [https://www.researchgate.net/publication/4295714_Speech-to-Singing_Synthesis_Converting_Speaking_Voices_to_Singing_Voices_by_Controlling_Acoustic_Features_Unique_to_Singing_Voices ''''''Speech-to-Singing Synthesis: Converting Speaking Voices to Singing Voices by Controlling Acoustic Features Unique to Singing Voices'''''' at researchgate.net], a November 2007 paper published in the IEEE conference on Applications of Signal Processing to Audio and Acoustics | |||
* [[w:Category:Singing software synthesizers]] | |||
== Text syntheses == | == Text syntheses == | ||
| Line 246: | Line 245: | ||
# Offensively, to thieve somebody else's handwriting style | # Offensively, to thieve somebody else's handwriting style | ||
Here we find a similar | If the handwriting-like synthesis passes human and media forensics testing, it is a '''digital handwrite-alike'''. | ||
Here we find a '''risk''' similar to that which realized when the '''[[w:speaker recognition]] systems''' turned out to be instrumental in the development of '''[[#Digital sound-alikes|digital sound-alikes]]'''. After the knowledge needed to recognize a speaker was [[w:Transfer learning|w:transferred]] into a generative task in 2018 by Google researchers, we no longer cannot effectively determine for English speakers which recording is human in origin and which is from a machine origin. | |||
'''Handwriting-like syntheses''': | |||
[[w:Recurrent neural network]]s (RNN) seem are a popular choice for this task. | |||
* [https://github.com/topics/handwriting-synthesis GitHub topic '''handwriting-synthesis'''] has 29 public repositories as of September 2021. | |||
* [https://github.com/topics/handwriting-generation GitHub topic '''handwriting-generation'''] has 21 public repositories as of September 2021. | * [https://github.com/topics/handwriting-generation GitHub topic '''handwriting-generation'''] has 21 public repositories as of September 2021. | ||
'''Handwriting | * [https://www.sciencedirect.com/science/article/abs/pii/S0031320319303814 '''''Deep imitator: Handwriting calligraphy imitation via deep attention networks'''' at sciencedirect.com], published in [[w:Pattern Recognition (journal)]] in August '''2020'''. | ||
* [https://greydanus.github.io/2016/08/21/handwriting/ '''Scribe''' - ''Generating Realistic Handwriting with TensorFlow'' at greydanus.github.io] blog post published on '''2016'''-08-21. [https://github.com/greydanus/scribe '''Scribe code''' at github.com] | * [https://greydanus.github.io/2016/08/21/handwriting/ '''Scribe''' - ''Generating Realistic Handwriting with TensorFlow'' at greydanus.github.io] blog post published on '''2016'''-08-21. [https://github.com/greydanus/scribe '''Scribe code''' at github.com] | ||
* [https://dl.acm.org/doi/10.1145/2886099 '''''My Text in Your Handwriting''''' at dl.acm.org], a system from [[w:University College London]] published on '''2016'''-05-18 in [[w:ACM Transactions on Graphics]].<ref group="1st seen in">https://www.ucl.ac.uk/news/2016/aug/new-computer-programme-replicates-handwriting via Google search for "ai handwriting generator"</ref> | |||
* [https://arxiv.org/abs/1308.0850 '''''Generating Sequences With Recurrent Neural Networks''''' at arxiv.org] by Alex Graves published on '''2013'''-08-04 in Neural and Evolutionary Computing. | |||
:#[https://www.cs.toronto.edu/~graves/handwriting.html '''''Recurrent neural network handwriting generation demo''''' at cs.toronto.edu] is a demonstration site for publication | |||
:# [https://www.calligrapher.ai/ '''Calligrapher.ai''' - ''Realistic computer-generated handwriting''] - The user may control parameters: speed, legibility, stroke width and style. The domain is registered by some organization in Iceland and the website offers no about-page<ref group="note">https://seanvasquez.com/handwriting-generation redirects to Calligrapher.ai - seen in https://www.reddit.com/r/MachineLearning/comments/gh9cbg/p_generate_handwriting_with_an_inbrowser/</ref>. According to [https://www.reddit.com/r/MachineLearning/comments/gh9cbg/p_generate_handwriting_with_an_inbrowser/ this reddit post] Calligrapher.ai is based on Graves' 2013 work, but "''adds an [[w:inference]] model to allow for sampling latent style vectors (similar to the VAE model used by SketchRNN)''".<ref>https://www.reddit.com/r/MachineLearning/comments/gh9cbg/p_generate_handwriting_with_an_inbrowser/</ref> | |||
''' Handwriting recognition ''' | |||
* '''[[w:Handwriting recognition]]''' ('''HWR'''), also known as '''Handwritten Text Recognition''' ('''HTR'''), is the ability of a computer to receive and interpret intelligible [[w:handwriting|w:handwritten]] input (Wikipedia) | |||
* '''[[w:Intelligent word recognition]]''', or '''IWR''', is the recognition of unconstrained handwritten words.<ref> | |||
{{Cite web | |||
|url=https://www.efilecabinet.com/what-is-iwr-intelligent-word-recognition-how-is-it-related-to-document-management/ | |||
|title=What is IWR? (Intelligent Word Recognition) | |||
|date=2016-01-04 | |||
|website=eFileCabinet | |||
|language=en-US | |||
|access-date=2021-09-21 | |||
}} | |||
</ref> (Wikipedia) | |||
* [https://github.com/topics/handwriting-recognition GitHub topic '''handwriting-recognition'''] contains 238 repositories as of September 2021. | |||
== Countermeasures against synthetic human-like fakes == | == Countermeasures against synthetic human-like fakes == | ||
| Line 636: | Line 661: | ||
</ref> - ''Open Media Forensics Challenge Evaluation (OpenMFC) is an open evaluation series organized by the NIST to assess and measure the capability of media forensic algorithms and systems.''<ref>https://www.nist.gov/itl/iad/mig/open-media-forensics-challenge</ref> | </ref> - ''Open Media Forensics Challenge Evaluation (OpenMFC) is an open evaluation series organized by the NIST to assess and measure the capability of media forensic algorithms and systems.''<ref>https://www.nist.gov/itl/iad/mig/open-media-forensics-challenge</ref> | ||
* '''2021''' | '''[[w:Conference on Computer Vision and Pattern Recognition]] (CVPR)''' 2021 | * '''2021''' | '''[[w:Conference on Computer Vision and Pattern Recognition]] (CVPR)''' 2021 | ||
** [https://public.tableau.com/app/profile/joshpreston/viz/CVPR2021/Dashboard1 '''CVPR 2021 research areas visualization by Joshua Preston''' at public.tableau.com] | |||
** [https://sites.google.com/view/mediaforensics2021 2021 ''''Workshop on Media Forensics'''' in CVPR 2021 at sites.google.com], a '''June 2021''' workshop at the Conference on Computer Vision and Pattern Recognition. | |||
* '''2020''' | [http://cvpr2020.thecvf.com/ '''CVPR''' 2020] | [https://sites.google.com/view/wmediaforensics2020/home 2020 Conference on Computer Vision and Pattern Recognition: ''''Workshop on Media Forensics'''' at sites.google.com], a '''June 2020''' workshop at the Conference on Computer Vision and Pattern Recognition. | * '''2020''' | [http://cvpr2020.thecvf.com/ '''CVPR''' 2020] | [https://sites.google.com/view/wmediaforensics2020/home 2020 Conference on Computer Vision and Pattern Recognition: ''''Workshop on Media Forensics'''' at sites.google.com], a '''June 2020''' workshop at the Conference on Computer Vision and Pattern Recognition. | ||