ECK, Douglas, Generative Recurrent Neural Nets, Google Brain, Nov 17, 2017
<p class="MsoNormal" >ECK, Douglas, Generative Recurrent Neural Nets, Google Brain, Nov 17, 2017
this is a Stfd FriSem in Psych Dept MBC on Friday at 3:15PM.
<b>Magenta Project... </b>music, video images and text by ML.
full title is <b>generative models for Music and Art....</b>
shows 3 overlapping cirlces of Music/ Art and ML = macine learning and
human creativity and feedback.
this is open source... and engages creative coders, artists, and musicians
see g.co/magenta...
our rsch is on SOTA generative models for audio, images, and text.
tensor lab is like MatLab for ML... run them at scale on a cloud.
to write big ML models.
is this a cat or a dog...
shows Deep Neural Net layers going up... but
here (in this talk) he wants to go DOWN from top.
can I use gradient descent to spit out tokens in another language.
go eg from Japanese to Korean.
job of NN is to go from one language to another.
it has an INTERLINGUA.
english to chinese is harder then going to Spanish.
this is much better than phrase-based models using HMMs.
shows Hemingway... Snows of Kilimanjaro...
we see much better translation of
What was the leopard seeking at high altitude.
we used to hand write a note of thanks to your grandma...
for gratitude and authenticity...that may come back.
shows his kids are in Japan at a restaurant...
he has teens. what can ML add (to mobile, that the kids are always on)
<b>shows QuickDraw </b>against the computer.... draw a bear.
machine model tries to parse and ID your drawing.
use autoencoder... try to reconstruct the input as an output.
from CAT to encoder to Z and then out to Decoder and to a new stroke-based CAT.
this is a recurrent NN = neural net.
the encoder is a both forward and backward RNN.
train it to reproduce the input drawing...
want to reproduce the drawing...
using RNN = recurrent NN + mixture of Gaussians.
these drawings are stroke-based. (eg a CAT face with whiskers and ears and mouth)
step 1 is an encoder using fwd and back RNN;
next step is embedding with a latent vector... with input + noise;
next step is RNN with mixture of Gaussians.
he shows a generative model for vector images... above..
the models learn what is important.
note: input cat has 5 whiskers... only 2 on one side
and output cat has 3 on each side * 2 = 6.
input is pen up, pen down and X/Y positions...
shows "low temperature" vs "hot temp" (like Hot Yoga, he jokes)
toggles variability.
L2 = Euclidean distance and use variational loss.
shows 4 drawings by David... human drawn and
he shows the computer gen faces that
reproduce them.
latent faces are intermediate btwn these.
he shows PREDICTIVE computer model...
it anticipates the mosquito drawing...
it's several steps ahead of you.
this is sketch-rnn mosquito predictor.
give it an 8 legged pig and it gives you back a 4 legged pig....
it draws a pig/truck... v amusing...
and it gets rid of the third eye in a 3 eye cat and converts it to 2 eyes.
variational loss... well behaved.
shows vector image analogies...
take a pig body and add (cat face - cat body) = pig face.
describes DOOM game...
this is just vector arithmetic as in WordToVec (with words).
David draws cartoons and they are passed thru encoder to get
to latent spaces.
difference btwn Asian chair in 2d versus Western chair in 3d...
MUSIC
*
he worked on this since 2002...
tried to make music with NNs (even Turing tried this)
model: what is happening at time t and then predict t+1.
polyphonic is v difficult.
we can advance the clock in arbitrary units...
do SoftMax...
train models on scores... but they overfit quickly...
Yamaha DiskClavier piano... records hammer velocity and timing.
</p>