the great magic to be here and telling
you a little bit about what they do so
one story is that I grew up in Romania
back in the days when it was a communist
country so things were pretty dreary and
as a result I know being rebellious
teenagers we did very radical things
like watching pirated sci-fi movies it’s
also very much else radical to do and so
I’m not looking at androids like beta
and robots and I thought you know this
is really cool I really like about
something like that
so it really inspired me to actually go
inside the computer science of the
University and one of the first sort of
significant assignments that I had there
was to build a program that could play
chess now just as a game that people
consider hallmark of intelligence I
thought okay this is pretty good we’re
gonna actually make some set to build
some robots another that because my
assignment and these people said well
you know think about how do you play
chess and then make those rules and
programs I know did some I’m really a
lousy chess player so all I could do is
basically tell the program some very
very basic things about how the pieces
move it wasn’t really very good now I
decided to not give up and so I went to
grad school and at the beginning of my
grad school actually a watershed moment
for artificial intelligence happened
this is a picture that was in the news
at the time this is Gary Kasparov who
was the world’s chess champion beaten by
a computer and the computer was blue was
a creation of IBM and really all of us
were studying AI and we’re very inspired
by this we thought okay now this is true
intelligence computer it’s better than a
person again
really consider to be very significant
and so no very enthusiastically we
welcome to read the papers that describe
the technology and then he realized that
actually this was an amazing feat but it
was essentially a human feet there was a
computer that was purpose-built to play
chess that’s all it could do it was
really really didn’t playing chess but
it was very special and couldn’t do much
of anything else
also people sort of thought very
carefully about how masters played the
game and they built this capability into
the computer so it was amazing but it
wasn’t quite what I was looking for now
yeah we are 20 years later and actually
things have moved quite a bit so these
are some pictures taken from the games
of Lisa goal and kg who played also
computer program this time at the game
of God now go is an ancient game that is
considered significantly harder than
chess if they’re all strategy as well as
tactics and it is sort of considerably
more difficult also because there’s many
many many possible moves and people have
to think that very close okay capture
territory how do I defend and so on and
it turns out that this program called
alpha del and it’s sort of next version
called alpha zero actually was really
really good to the point that it is now
essentially the world’s best goalie and
it made the cover of nature and a lot of
us got very excited about it and we now
have in fact a much more interesting way
of doing this kind of a feat which is
not so much engineering so this is what
I want to tell you about how do these
programs actually play the game it turns
out that they’re not so much programs as
they’d learn how to do it now when you
think about learning kids learning
comes to mind is going to school okay so
go to school you have a teacher then eat
your selfie various things you kind of
ingest on the menu learn how to do stuff
that’s one way to do learn in in
computers it’s called supervised
learning so you can have for example a
computer that wants to learn how to
label images how do we do this what will
show the computer a lot of images where
somebody has gone in and actually
labeled the things that are interesting
so in this case we’re on the computer to
learn how to label faces and so
somebody’s going via mentioned drawn
some little squares around the faces and
I’m going to show this information to
the computer the image is just a set of
pixels and we know what we’re looking
for in the image and so now we can show
the computer lots and lots and lots of
images millions of images as it turns
out and to learn how to recognize faces
now what does the computer have inside
it has a little brain for the artificial
neural network or a big network so in
the picture over there it really has
some very small units that can do very
simple things okay like compute a
certain line function or maybe like a
little squiggly line function so each of
the circuit is very very simple kind of
like the human neuron the power comes
from the connections the connections
have numbers on them and these numbers
get trained some of them get Kai out
some of them get low and each of these
little units that’s a very specialized
kind of thing so for example some of
them are going to just simply learn to
recognize edges or lines
some of them are going to actually
recognize certain kinds of faces turns
out some of these names actually learn
how to recognize cats okay why well
because a lot of damages on the internet
are cats and dogs and so on and so we
now have these wonderful artificial
brains that are really really good at
recognizing cats from all angles and all
different breeds so this is interesting
this is part of the way how alpha bill
does things in fact okay so what I mean
by that initially after though estranged
by looking at a lot of games that people
play good Grandmaster level players and
essentially thinking the move that they
did was the right to move okay
and so the program learns how to refuse
it use these moves now this is great
it’s very interesting it’s a very
powerful program but there’s one catch
here which is this requires a lot of
data that is provided by people okay
with the images we need millions of
images labeled by people now people get
bored very quickly of this kind of task
and then subjects is like a mechanical
there actually aren’t that many games
that are played by grandmasters that we
can look at for information so now if we
think about how people learn yes you
learn things in school but actually
you’re going types of stuff even before
you go yourself you learn how to
recognize objects and what colors are
you don’t have to count and you learn to
interact through the world in various
ways and so that’s the kind of process
we would like to mimic in a computer
it’s a different kind of learning called
reinforcement learning and it turns out
that this is not even just something
that people do but animals do it it’s
very widespread in the natural world and
so in one picture there you have a build
Mouse
that’s learning a task the task is
essentially cupressus sequence of levers
in order to get some coolant and the
mouth just kind of dust things up random
stumbles upon believers when it gets the
right femur as it gets the food and
that’s its reward okay that feels good
so no smells it’s happy and after what
kind of triggers a path of learning that
makes it remember what were these things
I get that like me today reward
it turns out we can train computers in
much the same way through reinforcement
for it so you think of the computer
program now as being exposed to the
world it watches the world it can do
things it’s an outreach servant and as a
result of what it does it may receive a
reward now this could be a positive
reward number and number and the whole
the computer is to find actions that
essentially give it as large a number as
possible
that’s a bias that we put it but it’s
never told the actual thing that it
should do it’s allowed to experiment
the other interesting thing is that
these numbers might come much later than
why the action has happened for those of
you who are doing the University you
probably you know expect some reward at
the end of the day when you get your
degree if it’s going to take some years
to get there and so it won’t anybody
make it somehow they still be enforce
our actions so that’s in fact how
alphago learns it looks at the board
it tries something that’s an actions
there’s no feedback until the end of the
game yeah
it observes whether it’s one big a
monstrosity and if it’s won the game
it’s going to make all of the actions
that happen during that game a little
bit more likely because they obviously
let’s till the dolls come and if the
game will stop all those actions are
going to become a little bit less likely
so that’s the basic principle and not
the program is just going to sit there
in its happy computer corner and just
blame millions and millions of games
bites up it turns out that this leads
the best program in the world not only
that but because the program is not
really told what’s the right thing to do
because it learns by experimentation it
can in fact invent new ways of playing
that are different from what people
believe so this is a graph that’s
showing after those year or learning
performance is high the higher the
better so this is it’s it’s renting and
after three days it actually surpasses
the ability of our goalie this is the
version that we lease at all they’ll
strength much by supervised learning and
then after 21 days it actually reaches
the level of master which is the one
that defeated key G and at the end of
the training after 40 days it becomes so
good that it beats the previous version
a hundred games to nothing so it’s
absolutely the best go player in the
world what else is interesting about
this program is that now there’s no
special purpose hardware ok turns on the
regular computer and the algorithm is
actually very general there’s nothing in
the algorithm that told the computer
that is specific
though we just know that you should
prefer to win and now go up and display
out the games and figure it out so
that’s a very powerful concept because
it generalizes to many many different
situations now
of course I’m very happy with games and
many of us computer scientists are very
happy with games but there’s actually
real applications to this and so in our
lab we look at all kinds of ways that
artificial intelligence and these kinds
of learning methods can be applied one
of the its pictured here is the project
we have going on at the Children’s
Hospital which is looking at very
premature babies so in this case the
babies are born they’re very small they
cannot breathe on their own and so
they’re hooked up to a ventilator and
now we would like to figure out when
they are they actually ready to be taken
off of that ventilator machine that’s a
decision done by a doctor but the doctor
needs help because actually they’re not
always there to observe the baby and so
they have limited amounts of information
and also there’s a lot of noise
that’s a thankful setting the babies get
picked up and put down the sensors lose
contact and so on so what we do is we
gather data in the hospital setting and
then we want a computer to essentially
predict their risk that the baby would
have incurred if it were to be extubated
right now and in this case the computer
is not actually experimenting with
taking actions
it’s simply learning to predict the
right outcome and it’s being rewarded
for predicting the right now there’s
another application that I’ll show you
here which is we have some data image
data that’s very corrupted I’m not sure
if you can tell what’s in this image
okay it’s a little bit hard to tell but
if I let the computer do its job okay
you see some digits up here
image now this is a task that’s called
fixing corrupting data our data
imputation in statistical terms it’s a
problem where you have to look at the
data that’s there in order to kind of
get us but what’s not there and people
are good at this putting a little bit of
a guys missing but they’re not so good
at when a lot of data is missing and you
can kind of see that it’s going to be
hard for you to recognize these images
have to kind of fill filling the gaps
without any help so what the computer
does is it looks at the image it looks
at the context of what’s missing and can
decide to fill in a pixel or not and it
kind of does this over time little by
little and again it’s rewarded by
obtaining at the end the right answer
but once it’s trained it can do this on
any other kinds of images so in the
context of this presentation of giving
voice I think it’s really important for
us to think of how we’re going to use
these very powerful tools now in our
lives they are available some of you
probably use them every day if you
interact with with your phone you talk
to your phone your phone that some voice
recognition if you do translation google
translate you’re going to interact with
these kinds of methods how are you going
to make use of this so that this
technology can actually help us improve
and I hope that all of you can think of
some useful ways it
[Applause]
you a little bit about what they do so
one story is that I grew up in Romania
back in the days when it was a communist
country so things were pretty dreary and
as a result I know being rebellious
teenagers we did very radical things
like watching pirated sci-fi movies it’s
also very much else radical to do and so
I’m not looking at androids like beta
and robots and I thought you know this
is really cool I really like about
something like that
so it really inspired me to actually go
inside the computer science of the
University and one of the first sort of
significant assignments that I had there
was to build a program that could play
chess now just as a game that people
consider hallmark of intelligence I
thought okay this is pretty good we’re
gonna actually make some set to build
some robots another that because my
assignment and these people said well
you know think about how do you play
chess and then make those rules and
programs I know did some I’m really a
lousy chess player so all I could do is
basically tell the program some very
very basic things about how the pieces
move it wasn’t really very good now I
decided to not give up and so I went to
grad school and at the beginning of my
grad school actually a watershed moment
for artificial intelligence happened
this is a picture that was in the news
at the time this is Gary Kasparov who
was the world’s chess champion beaten by
a computer and the computer was blue was
a creation of IBM and really all of us
were studying AI and we’re very inspired
by this we thought okay now this is true
intelligence computer it’s better than a
person again
really consider to be very significant
and so no very enthusiastically we
welcome to read the papers that describe
the technology and then he realized that
actually this was an amazing feat but it
was essentially a human feet there was a
computer that was purpose-built to play
chess that’s all it could do it was
really really didn’t playing chess but
it was very special and couldn’t do much
of anything else
also people sort of thought very
carefully about how masters played the
game and they built this capability into
the computer so it was amazing but it
wasn’t quite what I was looking for now
yeah we are 20 years later and actually
things have moved quite a bit so these
are some pictures taken from the games
of Lisa goal and kg who played also
computer program this time at the game
of God now go is an ancient game that is
considered significantly harder than
chess if they’re all strategy as well as
tactics and it is sort of considerably
more difficult also because there’s many
many many possible moves and people have
to think that very close okay capture
territory how do I defend and so on and
it turns out that this program called
alpha del and it’s sort of next version
called alpha zero actually was really
really good to the point that it is now
essentially the world’s best goalie and
it made the cover of nature and a lot of
us got very excited about it and we now
have in fact a much more interesting way
of doing this kind of a feat which is
not so much engineering so this is what
I want to tell you about how do these
programs actually play the game it turns
out that they’re not so much programs as
they’d learn how to do it now when you
think about learning kids learning
comes to mind is going to school okay so
go to school you have a teacher then eat
your selfie various things you kind of
ingest on the menu learn how to do stuff
that’s one way to do learn in in
computers it’s called supervised
learning so you can have for example a
computer that wants to learn how to
label images how do we do this what will
show the computer a lot of images where
somebody has gone in and actually
labeled the things that are interesting
so in this case we’re on the computer to
learn how to label faces and so
somebody’s going via mentioned drawn
some little squares around the faces and
I’m going to show this information to
the computer the image is just a set of
pixels and we know what we’re looking
for in the image and so now we can show
the computer lots and lots and lots of
images millions of images as it turns
out and to learn how to recognize faces
now what does the computer have inside
it has a little brain for the artificial
neural network or a big network so in
the picture over there it really has
some very small units that can do very
simple things okay like compute a
certain line function or maybe like a
little squiggly line function so each of
the circuit is very very simple kind of
like the human neuron the power comes
from the connections the connections
have numbers on them and these numbers
get trained some of them get Kai out
some of them get low and each of these
little units that’s a very specialized
kind of thing so for example some of
them are going to just simply learn to
recognize edges or lines
some of them are going to actually
recognize certain kinds of faces turns
out some of these names actually learn
how to recognize cats okay why well
because a lot of damages on the internet
are cats and dogs and so on and so we
now have these wonderful artificial
brains that are really really good at
recognizing cats from all angles and all
different breeds so this is interesting
this is part of the way how alpha bill
does things in fact okay so what I mean
by that initially after though estranged
by looking at a lot of games that people
play good Grandmaster level players and
essentially thinking the move that they
did was the right to move okay
and so the program learns how to refuse
it use these moves now this is great
it’s very interesting it’s a very
powerful program but there’s one catch
here which is this requires a lot of
data that is provided by people okay
with the images we need millions of
images labeled by people now people get
bored very quickly of this kind of task
and then subjects is like a mechanical
there actually aren’t that many games
that are played by grandmasters that we
can look at for information so now if we
think about how people learn yes you
learn things in school but actually
you’re going types of stuff even before
you go yourself you learn how to
recognize objects and what colors are
you don’t have to count and you learn to
interact through the world in various
ways and so that’s the kind of process
we would like to mimic in a computer
it’s a different kind of learning called
reinforcement learning and it turns out
that this is not even just something
that people do but animals do it it’s
very widespread in the natural world and
so in one picture there you have a build
Mouse
that’s learning a task the task is
essentially cupressus sequence of levers
in order to get some coolant and the
mouth just kind of dust things up random
stumbles upon believers when it gets the
right femur as it gets the food and
that’s its reward okay that feels good
so no smells it’s happy and after what
kind of triggers a path of learning that
makes it remember what were these things
I get that like me today reward
it turns out we can train computers in
much the same way through reinforcement
for it so you think of the computer
program now as being exposed to the
world it watches the world it can do
things it’s an outreach servant and as a
result of what it does it may receive a
reward now this could be a positive
reward number and number and the whole
the computer is to find actions that
essentially give it as large a number as
possible
that’s a bias that we put it but it’s
never told the actual thing that it
should do it’s allowed to experiment
the other interesting thing is that
these numbers might come much later than
why the action has happened for those of
you who are doing the University you
probably you know expect some reward at
the end of the day when you get your
degree if it’s going to take some years
to get there and so it won’t anybody
make it somehow they still be enforce
our actions so that’s in fact how
alphago learns it looks at the board
it tries something that’s an actions
there’s no feedback until the end of the
game yeah
it observes whether it’s one big a
monstrosity and if it’s won the game
it’s going to make all of the actions
that happen during that game a little
bit more likely because they obviously
let’s till the dolls come and if the
game will stop all those actions are
going to become a little bit less likely
so that’s the basic principle and not
the program is just going to sit there
in its happy computer corner and just
blame millions and millions of games
bites up it turns out that this leads
the best program in the world not only
that but because the program is not
really told what’s the right thing to do
because it learns by experimentation it
can in fact invent new ways of playing
that are different from what people
believe so this is a graph that’s
showing after those year or learning
performance is high the higher the
better so this is it’s it’s renting and
after three days it actually surpasses
the ability of our goalie this is the
version that we lease at all they’ll
strength much by supervised learning and
then after 21 days it actually reaches
the level of master which is the one
that defeated key G and at the end of
the training after 40 days it becomes so
good that it beats the previous version
a hundred games to nothing so it’s
absolutely the best go player in the
world what else is interesting about
this program is that now there’s no
special purpose hardware ok turns on the
regular computer and the algorithm is
actually very general there’s nothing in
the algorithm that told the computer
that is specific
though we just know that you should
prefer to win and now go up and display
out the games and figure it out so
that’s a very powerful concept because
it generalizes to many many different
situations now
of course I’m very happy with games and
many of us computer scientists are very
happy with games but there’s actually
real applications to this and so in our
lab we look at all kinds of ways that
artificial intelligence and these kinds
of learning methods can be applied one
of the its pictured here is the project
we have going on at the Children’s
Hospital which is looking at very
premature babies so in this case the
babies are born they’re very small they
cannot breathe on their own and so
they’re hooked up to a ventilator and
now we would like to figure out when
they are they actually ready to be taken
off of that ventilator machine that’s a
decision done by a doctor but the doctor
needs help because actually they’re not
always there to observe the baby and so
they have limited amounts of information
and also there’s a lot of noise
that’s a thankful setting the babies get
picked up and put down the sensors lose
contact and so on so what we do is we
gather data in the hospital setting and
then we want a computer to essentially
predict their risk that the baby would
have incurred if it were to be extubated
right now and in this case the computer
is not actually experimenting with
taking actions
it’s simply learning to predict the
right outcome and it’s being rewarded
for predicting the right now there’s
another application that I’ll show you
here which is we have some data image
data that’s very corrupted I’m not sure
if you can tell what’s in this image
okay it’s a little bit hard to tell but
if I let the computer do its job okay
you see some digits up here
image now this is a task that’s called
fixing corrupting data our data
imputation in statistical terms it’s a
problem where you have to look at the
data that’s there in order to kind of
get us but what’s not there and people
are good at this putting a little bit of
a guys missing but they’re not so good
at when a lot of data is missing and you
can kind of see that it’s going to be
hard for you to recognize these images
have to kind of fill filling the gaps
without any help so what the computer
does is it looks at the image it looks
at the context of what’s missing and can
decide to fill in a pixel or not and it
kind of does this over time little by
little and again it’s rewarded by
obtaining at the end the right answer
but once it’s trained it can do this on
any other kinds of images so in the
context of this presentation of giving
voice I think it’s really important for
us to think of how we’re going to use
these very powerful tools now in our
lives they are available some of you
probably use them every day if you
interact with with your phone you talk
to your phone your phone that some voice
recognition if you do translation google
translate you’re going to interact with
these kinds of methods how are you going
to make use of this so that this
technology can actually help us improve
and I hope that all of you can think of
some useful ways it
[Applause]
Please follow and like us: