the role and specifically we’re going to
talk about the question it wasn’t me to
take a girl at work and asking to make
some music or some art and so first
let’s talk about machine learning one of
the line how many of you here have show
hands okay so that work is a multi-layer
the amateurs are seen here to see some
black box those we call their eyes by
analogy to the brain and they’re
connected together by those white lines
which are the axons of neurons together
in the brain but it’s really only an
anomaly
they spectators that each of these
neurons performs a tiny tiny tiny bit
more maybe it sums up some numbers it
doesn’t multiplication and in doing that
across numerous neurons in a state of
Caracalla for translation you might try
that about millions and girls and now
we’re talking about models that contain
billions of neurons and being able to
train their model top GPUs and the way
that training works is that those white
lines are actually wasting their numbers
may be between minus 1 and 1 and there
are rules training rules that allow
those numbers to be slowly adapted to
solve a problem so for example
I see a picture of something in the
laundry basket
and it’s trying to figure out what’s a
teflon basket and in fact it is a wig
and about I can tell you if it guesses
wrong it’s a little bit embarrassing
gradually over millions and billions of
trials it gradually learns to identify
it and the cool part about it is it does
so in a way that is flexible in general
so they learn little filters like the
tech initiative detecting little packets
of light and it will figure out what a
dog is or a cat is not by they 12 rules
what that dollar cad is by figuring out
for itself so the take home message on
this is if artificial intelligence is
the job of building smart computers or
clever computers then machine learning
is building computers that can learn to
be better and we’ve seen some huge huge
gains in a big role of heart problems
using Mishima one of the most recent is
a problem with translation and some work
that actually happened in Google
before other words is learning how to
translate from one language to another
by simply showing their neural network
match sentences so the neural network
would see is input a sentence in English
and then it would know what language was
trying to translate it to and it would
try to translate that sentence and the
background will get some error and it
learns to train all the neurons in the
network with the ways to make the right
version and the quality of this is
nothing last my other than shopping
revolutionary so we look at this came
from an article written by a Japanese
researcher who was looking at
translation and expressiveness quality
translation and we see in our old model
a translation of a line that came back
when the Japanese Akane back to English
from anyway whether the leopard have
what the demand had that altitude
there’s no that nobody can explain
captured the gist of what’s happening
but it definitely is not food the new
system that uses the neural networking
translates asks no one can explain what
leopard was seeking at that altitude it
missed the mark a blue there for you but
it suggests to me that these models are
getting good enough and they’re getting
good enough because they’re big
data so we can start to ask questions
about can we do some things with these
models that are genuinely expressive
that allows us to create something new
can we generate new paragraphs can you
generate music imaginary part and what
does that mean for us
not with terms of creativity so we start
here we start with something as simple
as Google in boxes smart black it gives
you a sentence like I’ve got no plans
yet or just something to you and we’re
hoping to move from there towards more
expressive uses of this technology not
only music and drawing but also in
language and elsewhere by the way that’s
that that statistics should be
surprising to you I think they
surprising to me the 12% of responses
that I hope are cyclic and all basic
suggestions that’s a huge number
okay let’s hit reset I’m a gun parent or
my daughter Olivia here was thank God
she graduated to help out now my son Sam
is at JLS now because one was for
anything so it was one person one of
these kisses and I put their picture up
here not because I wanted to show off
these kids but because I wanted to make
a comment now moving from translation
towards using technology to communicate
I’m here to talk about art and music but
I think what makes are important is that
it is an artist communicating with her
audience it’s about having something to
say and we’ve always use technology to
support this in the arms we have paint
we have no cameras we are publishing an
avatar and etc and so the reason I put
this slide up here is to connect back to
sort of fundamentally what is happening
with this revolution of having own
devices and smarter and smarter software
it is that people are finding new ways
to campaign maybe it’s with snapchat and
Instagram and this is really directed
for the parents in the room I can’t see
you but I’ve ever there I think there’s
a huge there’s a huge risk with these
smartphones sort of taking over his
lives here similarly we were in Japan
and you see the devices prevalence is
everywhere sitting next to Robin but at
the same time I think it opens up new
and very interesting opportunities for
this next generation to communicate in a
way that opens up you know more very
interesting ways to exploit other’s
lives and I think there’s an aspect of
this is very possible if we’re careful
with what we do and so I want you to see
this connection between art and music
and communication and try to make this
overarching connection back to machine
translation and ask the question can she
wearing
give us something new something we don’t
have right now give lasts a new form
that’s recitative
and I hope as I drag you through two
examples you’ll see that using art
pretty cool places to to explore this so
we’re going to number one at number one
is about the wall we’re going to talk
about how to Train
learning algorithms neural networks to
draw pictures now we’re going to use the
dataset that came from a game both quick
drive I’m a people here for a quick drop
definitely see a few hands up it was a
game a playing pictionary against the
computer and the computer punchers are
drawing and if you get it right it says
oh you drove it we have not collected
over 1 billion drawings with this game
it’s a shocking number and we we saved
all of the strokes so you see that
beautiful banner on the right we can
actually animate out the drawing of that
bear it’s not just the fixes that we
have we’re going to see what we can
offer and so what we’re going to do what
we did was we decided to learn to draw
based upon these so these small drawings
and I want you to come away I’m gonna
make one point about the team I think is
important for this ok they’re about you
know if we had time we would go on to
three or four principles of machine
learning they’re important for our
hearts but that that principle is that
the memorization is a bad idea
memorization is a bad idea and the
recent memorization is a bad idea is
that a model that you just would
memorize the data the disease never
learns what’s important property we want
to understand the gist of drawing the
cat the gist of Jonica there may seem
like a silly thing to do but
and so take a close look at what we have
here on the screen we have three funny
words encoder receive and decode urgent
the job of the encoder is to take the
drawing and represent it in some way
that it can be understood by the machine
money having its encoding the drawing
with some blue space on the other side
the decoders job is to take advantage of
whatever information is in that space
and recreate the job okay now if you may
see you can think of as kind of the
memory how much how much juice does this
model have you may see huge and what’s
it gonna do simply finishes some
memorize all strokes because it has more
capacity so we use those make C small
means of America noise to see now that
that that brings up a nice I think nice
and fascinating effect which is that the
model is not able to reproduce better
than disease and if while they’ve been
talking about looking at these two
drawings the input was drawn by David
hot primary researcher on the project
the one that’s decoded the elephant one
doesn’t look exactly the same right
asbestos whiskers not five and that’s
because this model doesn’t know how to
represent a fire cat doesn’t see enough
of them and so he’d seen if I was their
cat and the best that he can do is make
mistakes was recap I think there’s
something very very suggestive about
that in turn four
these are some yoga poses that were
dropped from a model train on yoga you
can see the machine learning model can
capture lots and lots of bearings while
still not being able to memorize the
data so none of those poses exist in a
building except the ones that are trend
up by the machine and we have ways to
make the drawings a little bit riskier a
little crazier so the ones on the bottom
are crazier Hot Yoga bad for your health
don’t see that are you the ones at the
furthermore because we have the strokes
in place that see the dogs is there a
way we can start a drawing and let them
she finished applause and I think this
ability to sort of take a part of an
idea from us and expand upon it is
really interesting too so here is David
trying to learn how to draw vestido and
fullcycle and he actually said you know
I can’t have drawn the speed up for
doing this project and that drive ahead
look to your right you see nine
different ways finally best yet and so
this kind of playfulness and ability to
explore that I think is really wonderful
finally because these models cannot
never I say that because sometimes
uncover interesting maybe funny aspects
about our everyday world on the Left we
have a bunch of cats are pretty normal
cat dries and they get more or less
reproduced by the honorable fast cast on
the right of England you Pig it gets
turned into a four-legged thing
and my favorite my favorite favorite
drawing the world with this project is
that truck which is put into a model of
and specifically I’m going to talk about
teaching the neural network hops to a
plaintiff in this case we’re going to
talk about playing piano music talk
about classroom I’ve been working in
this space for 25 years I can tell you
there have been many many attempts to
take a bunch of beautiful compositions
by a famous composers and then teach
them their own network how to make a new
one here’s 100 rock compositions let’s
make 130 and by and large the results
have been at best interesting in this
case we changed the rules of the game
and instead of trying to predict the
next note composition instead we’re
gonna work with performances real fun
and this data came out from Google this
came from Yamaha optician every
Republican competition and they reduce
the audio name of these committee so but
have another keyword yeah so what you
see now is we’re going to train on it’s
not just a beautiful score which is
pristine
it’s the don’t exactly where they need
to be there’s no noise at all and
instead we’re going to be much messier
data we’re gonna get a real performance
with decisions like I’m going to pin
louder under my software under under
this low and compared to what we can do
with the scores I can tell you that the
outputs that we’re getting these
performances are very what you’re
hearing now is the output of a neural
network the these compositions where
these performances come would want
because their model of playing they are
not similar to any of the training set
so it’s not just you also should expect
five minute sonata but I hope it appear
is by picking up on D the timing and the
loudness changes the real performance we
give this neural network something
that’s quite interesting to learn from
it by fishing for us to see the musical
there’s no need to alarm these models
are quite that good take a look at the
this is show Penn State student majoring
on top printer that is probably animals
you see on the right which is one means
that is we can move around that stay so
you could explore what’s near that one
food truck and what other trucks we can
generate near that picture or what’s
near this one cache because the space
that we have to play with is about their
number sensing and so we kind of move
around America and we’ve been working
with that of states of music as well and
what we’ve modeled now is not cold piano
performances in the classical world but
pop music’s pacifically loop based
problems and the model that we though
important attendance really only see one
mentor who’s at that time and that
allows us to build a model where we can
move around that space so what you’re
gonna hear is a demo where we start with
private and they don’t clear the last
better and then you’re gonna hear the
computer the neural network try to find
[Music]
[Applause]
[Music]
[Applause]
[Music]
[Music]
[Applause]
[Music]
[Applause]
[Music]
[Applause]
[Music]
we now do the same thing but we have
drama space and melody and I’m changing
these measures together doing a
composition and this little thing right
here the neural network generated all
three parts
and I think it’s pretty interesting how
because they’re coming from the same see
that little compressed world that they
kind of communicate to each other so the
all right they’ll be dealing so
supposing these are two guitar pedals
your electric guitar took what we target
us and you can have to drive fast and we
think of them as a goal for us because
part of my potential we write a lot of
software
I write a lot of code and open-source
noon but it’s much harder to actually
get to the point where we have tools
that artists musicians can use coders
and so we’re thinking all the time
what’s the analogy of something that is
so perfect some useful as guitar fell
and if you’re not seeing the analogy
that point out that these guitar post
moments simulated using your laptop you
pretty much replaced that with code but
nobody hires to stand on stage it’s not
work so we need to play by the rules
community and so we’re we’re working
class we’re working towards that in our
project if you want to learn more check
us out we’re gtfo magenta is a
completely open-source project we have a
blog and a specialist specifically
because we’re here I’m here we’re always
looking for our people or collaboration
points working really hard
and experimenting new musicians artists
and we’d love to hear from you you can
reach us via our specialists I discuss my work there and
talk about the question it wasn’t me to
take a girl at work and asking to make
some music or some art and so first
let’s talk about machine learning one of
the line how many of you here have show
hands okay so that work is a multi-layer
the amateurs are seen here to see some
black box those we call their eyes by
analogy to the brain and they’re
connected together by those white lines
which are the axons of neurons together
in the brain but it’s really only an
anomaly
they spectators that each of these
neurons performs a tiny tiny tiny bit
more maybe it sums up some numbers it
doesn’t multiplication and in doing that
across numerous neurons in a state of
Caracalla for translation you might try
that about millions and girls and now
we’re talking about models that contain
billions of neurons and being able to
train their model top GPUs and the way
that training works is that those white
lines are actually wasting their numbers
may be between minus 1 and 1 and there
are rules training rules that allow
those numbers to be slowly adapted to
solve a problem so for example
I see a picture of something in the
laundry basket
and it’s trying to figure out what’s a
teflon basket and in fact it is a wig
and about I can tell you if it guesses
wrong it’s a little bit embarrassing
gradually over millions and billions of
trials it gradually learns to identify
it and the cool part about it is it does
so in a way that is flexible in general
so they learn little filters like the
tech initiative detecting little packets
of light and it will figure out what a
dog is or a cat is not by they 12 rules
what that dollar cad is by figuring out
for itself so the take home message on
this is if artificial intelligence is
the job of building smart computers or
clever computers then machine learning
is building computers that can learn to
be better and we’ve seen some huge huge
gains in a big role of heart problems
using Mishima one of the most recent is
a problem with translation and some work
that actually happened in Google
before other words is learning how to
translate from one language to another
by simply showing their neural network
match sentences so the neural network
would see is input a sentence in English
and then it would know what language was
trying to translate it to and it would
try to translate that sentence and the
background will get some error and it
learns to train all the neurons in the
network with the ways to make the right
version and the quality of this is
nothing last my other than shopping
revolutionary so we look at this came
from an article written by a Japanese
researcher who was looking at
translation and expressiveness quality
translation and we see in our old model
a translation of a line that came back
when the Japanese Akane back to English
from anyway whether the leopard have
what the demand had that altitude
there’s no that nobody can explain
captured the gist of what’s happening
but it definitely is not food the new
system that uses the neural networking
translates asks no one can explain what
leopard was seeking at that altitude it
missed the mark a blue there for you but
it suggests to me that these models are
getting good enough and they’re getting
good enough because they’re big
data so we can start to ask questions
about can we do some things with these
models that are genuinely expressive
that allows us to create something new
can we generate new paragraphs can you
generate music imaginary part and what
does that mean for us
not with terms of creativity so we start
here we start with something as simple
as Google in boxes smart black it gives
you a sentence like I’ve got no plans
yet or just something to you and we’re
hoping to move from there towards more
expressive uses of this technology not
only music and drawing but also in
language and elsewhere by the way that’s
that that statistics should be
surprising to you I think they
surprising to me the 12% of responses
that I hope are cyclic and all basic
suggestions that’s a huge number
okay let’s hit reset I’m a gun parent or
my daughter Olivia here was thank God
she graduated to help out now my son Sam
is at JLS now because one was for
anything so it was one person one of
these kisses and I put their picture up
here not because I wanted to show off
these kids but because I wanted to make
a comment now moving from translation
towards using technology to communicate
I’m here to talk about art and music but
I think what makes are important is that
it is an artist communicating with her
audience it’s about having something to
say and we’ve always use technology to
support this in the arms we have paint
we have no cameras we are publishing an
avatar and etc and so the reason I put
this slide up here is to connect back to
sort of fundamentally what is happening
with this revolution of having own
devices and smarter and smarter software
it is that people are finding new ways
to campaign maybe it’s with snapchat and
Instagram and this is really directed
for the parents in the room I can’t see
you but I’ve ever there I think there’s
a huge there’s a huge risk with these
smartphones sort of taking over his
lives here similarly we were in Japan
and you see the devices prevalence is
everywhere sitting next to Robin but at
the same time I think it opens up new
and very interesting opportunities for
this next generation to communicate in a
way that opens up you know more very
interesting ways to exploit other’s
lives and I think there’s an aspect of
this is very possible if we’re careful
with what we do and so I want you to see
this connection between art and music
and communication and try to make this
overarching connection back to machine
translation and ask the question can she
wearing
give us something new something we don’t
have right now give lasts a new form
that’s recitative
and I hope as I drag you through two
examples you’ll see that using art
pretty cool places to to explore this so
we’re going to number one at number one
is about the wall we’re going to talk
about how to Train
learning algorithms neural networks to
draw pictures now we’re going to use the
dataset that came from a game both quick
drive I’m a people here for a quick drop
definitely see a few hands up it was a
game a playing pictionary against the
computer and the computer punchers are
drawing and if you get it right it says
oh you drove it we have not collected
over 1 billion drawings with this game
it’s a shocking number and we we saved
all of the strokes so you see that
beautiful banner on the right we can
actually animate out the drawing of that
bear it’s not just the fixes that we
have we’re going to see what we can
offer and so what we’re going to do what
we did was we decided to learn to draw
based upon these so these small drawings
and I want you to come away I’m gonna
make one point about the team I think is
important for this ok they’re about you
know if we had time we would go on to
three or four principles of machine
learning they’re important for our
hearts but that that principle is that
the memorization is a bad idea
memorization is a bad idea and the
recent memorization is a bad idea is
that a model that you just would
memorize the data the disease never
learns what’s important property we want
to understand the gist of drawing the
cat the gist of Jonica there may seem
like a silly thing to do but
and so take a close look at what we have
here on the screen we have three funny
words encoder receive and decode urgent
the job of the encoder is to take the
drawing and represent it in some way
that it can be understood by the machine
money having its encoding the drawing
with some blue space on the other side
the decoders job is to take advantage of
whatever information is in that space
and recreate the job okay now if you may
see you can think of as kind of the
memory how much how much juice does this
model have you may see huge and what’s
it gonna do simply finishes some
memorize all strokes because it has more
capacity so we use those make C small
means of America noise to see now that
that that brings up a nice I think nice
and fascinating effect which is that the
model is not able to reproduce better
than disease and if while they’ve been
talking about looking at these two
drawings the input was drawn by David
hot primary researcher on the project
the one that’s decoded the elephant one
doesn’t look exactly the same right
asbestos whiskers not five and that’s
because this model doesn’t know how to
represent a fire cat doesn’t see enough
of them and so he’d seen if I was their
cat and the best that he can do is make
mistakes was recap I think there’s
something very very suggestive about
that in turn four
these are some yoga poses that were
dropped from a model train on yoga you
can see the machine learning model can
capture lots and lots of bearings while
still not being able to memorize the
data so none of those poses exist in a
building except the ones that are trend
up by the machine and we have ways to
make the drawings a little bit riskier a
little crazier so the ones on the bottom
are crazier Hot Yoga bad for your health
don’t see that are you the ones at the
furthermore because we have the strokes
in place that see the dogs is there a
way we can start a drawing and let them
she finished applause and I think this
ability to sort of take a part of an
idea from us and expand upon it is
really interesting too so here is David
trying to learn how to draw vestido and
fullcycle and he actually said you know
I can’t have drawn the speed up for
doing this project and that drive ahead
look to your right you see nine
different ways finally best yet and so
this kind of playfulness and ability to
explore that I think is really wonderful
finally because these models cannot
never I say that because sometimes
uncover interesting maybe funny aspects
about our everyday world on the Left we
have a bunch of cats are pretty normal
cat dries and they get more or less
reproduced by the honorable fast cast on
the right of England you Pig it gets
turned into a four-legged thing
and my favorite my favorite favorite
drawing the world with this project is
that truck which is put into a model of
and specifically I’m going to talk about
teaching the neural network hops to a
plaintiff in this case we’re going to
talk about playing piano music talk
about classroom I’ve been working in
this space for 25 years I can tell you
there have been many many attempts to
take a bunch of beautiful compositions
by a famous composers and then teach
them their own network how to make a new
one here’s 100 rock compositions let’s
make 130 and by and large the results
have been at best interesting in this
case we changed the rules of the game
and instead of trying to predict the
next note composition instead we’re
gonna work with performances real fun
and this data came out from Google this
came from Yamaha optician every
Republican competition and they reduce
the audio name of these committee so but
have another keyword yeah so what you
see now is we’re going to train on it’s
not just a beautiful score which is
pristine
it’s the don’t exactly where they need
to be there’s no noise at all and
instead we’re going to be much messier
data we’re gonna get a real performance
with decisions like I’m going to pin
louder under my software under under
this low and compared to what we can do
with the scores I can tell you that the
outputs that we’re getting these
performances are very what you’re
hearing now is the output of a neural
network the these compositions where
these performances come would want
because their model of playing they are
not similar to any of the training set
so it’s not just you also should expect
five minute sonata but I hope it appear
is by picking up on D the timing and the
loudness changes the real performance we
give this neural network something
that’s quite interesting to learn from
it by fishing for us to see the musical
there’s no need to alarm these models
are quite that good take a look at the
this is show Penn State student majoring
on top printer that is probably animals
you see on the right which is one means
that is we can move around that stay so
you could explore what’s near that one
food truck and what other trucks we can
generate near that picture or what’s
near this one cache because the space
that we have to play with is about their
number sensing and so we kind of move
around America and we’ve been working
with that of states of music as well and
what we’ve modeled now is not cold piano
performances in the classical world but
pop music’s pacifically loop based
problems and the model that we though
important attendance really only see one
mentor who’s at that time and that
allows us to build a model where we can
move around that space so what you’re
gonna hear is a demo where we start with
private and they don’t clear the last
better and then you’re gonna hear the
computer the neural network try to find
[Music]
[Applause]
[Music]
[Applause]
[Music]
[Music]
[Applause]
[Music]
[Applause]
[Music]
[Applause]
[Music]
we now do the same thing but we have
drama space and melody and I’m changing
these measures together doing a
composition and this little thing right
here the neural network generated all
three parts
and I think it’s pretty interesting how
because they’re coming from the same see
that little compressed world that they
kind of communicate to each other so the
all right they’ll be dealing so
supposing these are two guitar pedals
your electric guitar took what we target
us and you can have to drive fast and we
think of them as a goal for us because
part of my potential we write a lot of
software
I write a lot of code and open-source
noon but it’s much harder to actually
get to the point where we have tools
that artists musicians can use coders
and so we’re thinking all the time
what’s the analogy of something that is
so perfect some useful as guitar fell
and if you’re not seeing the analogy
that point out that these guitar post
moments simulated using your laptop you
pretty much replaced that with code but
nobody hires to stand on stage it’s not
work so we need to play by the rules
community and so we’re we’re working
class we’re working towards that in our
project if you want to learn more check
us out we’re gtfo magenta is a
completely open-source project we have a
blog and a specialist specifically
because we’re here I’m here we’re always
looking for our people or collaboration
points working really hard
and experimenting new musicians artists
and we’d love to hear from you you can
reach us via our specialists I discuss my work there and
Please follow and like us: