hello
about 60 years ago the field of
artificial intelligence was created and
right from the beginning the goal was to
build software that is more intelligent
than humans are and since then we have
been asking the question is AI smarter
than humans well I want to pass that
question on to you for a second everyone
who thinks that you are still smarter
than AI raise your hand great everyone
who thinks that the AI is smarter than
you raise your hand
see this is exactly what I expected the
opinions are very divided and that’s for
a good reason because recently we’ve
seen many applications that just
outperform human performance but yet
still we don’t see robots walking around
being as smart as we are we don’t have
AI that understands humor and there are
many other things that remain unsolved
so what I’m trying to attempt with this
TED talk is give you a more decisive
answer than well it’s not a yes and not
a no to this question let’s start by
looking at some things where AI actually
happens to be superhuman this is the
game of Go a game that we humans have
been playing since 2,500 years and a
game that since 2016 a AI is better at
but it’s not like this is a simple game
this is actually a very complex game
that was a main challenge of AI for
decades before and in fact 2014 in 2014
the programmer of the AI that was best
that go at that time thought it would
take another 10 years but yet in 2016
alphago developed by google deepmind
beat the world champion and what i find
most fascinating about this example is
that in 2017 only one year later we
actually found that version 2 of this
same algorithm beat the version won by
an even larger margin than the version 1
beat the world champion by so this thing
is getting better very quickly and
that’s not the only example games are
the only example see some computer
vision tasks such as single object
classification which means you get an
image and you say what the prominent the
most prominent object in that images are
also performed better than AI by AI than
by humans and in fact that works with up
to a thousand categories of images which
is pretty same a pretty sick I mean
since 2015
computer vision or the sub K sub task of
single object classification AI is
better at it well that’s so much for
super human but one thing that I find
particularly interesting about the state
of AI is that AI can actually speak a
single wavenet can capture the
characteristics of many different
speakers with equal fidelity now it’s
fast see this speech isn’t created by
vocal cords it’s created by google’s
wavenet version – this is completely
simulated but that’s 48,000 samples per
second that sound pretty realistic
coming from nowhere not from vocal cords
so let’s get back to our question is AI
super human well yes we clearly see that
in some applications it is but in some
it’s just not and to get a feeling for
why that is let me show you what AI
actually is and how it works this is a
neural network that’s the core
technology that’s at the bottom of
almost all innovations that happen in
the field of machine learning and
artificial intelligence today and
contrary to what most of you might think
know that core technology is not complex
it’s in fact super simple it’s composed
of inputs which could be the pixels of
an image could also be anything else the
only thing that matters is that it’s a
number
it has outputs which could be
probabilities say in the case of image
classification what you do is you’d feed
the pixels of the image in and out you
would get the probability for each
object that the neural network knows to
exist so if there is a dog a cat in the
car there would be a probability for
each of these objects the things in
between are sort of like synapses and
the neurons in our brain or at least
they try to
be that the synapses the lines
connecting the neurons are the actual
things that we can learn this is what
has changed
when the neural network learns when it
when it trains you have also just
numbers still there is nothing else than
numbers in this picture you have numbers
that we change during training when we
want to adapt the output depending on a
certain input and the notes in between
all of these connections our functions
very simple functions I mean look at
this this is simple what they do is they
sum up all the neurons from the previous
layer multiplied by the weights
connecting that layer to the node to the
neuron sum that up so take the sum of
all of this and push it through this
simple function if it’s negative it
becomes zero if it’s positive it stays
the same so and if you run this
computation for an input run it through
the whole network then you have an
output that’s how a neural network
predicts something it’s fairly simple
right but there is one question
remaining how do we train that thing
like how does it learn actually well for
explanitory purposes let us build a
simple self-driving car this thing well
you get is you input images from a
camera say like you have a self-driving
car that has five cameras and you you
input all of the images into the neural
network and what you want to get out is
the angle for the steering wheel and the
speed that the car should go with okay
that’s a use case great how do we train
it one way to train it is by using
professional drivers say we have a
couple people that we know are very good
at driving and we just let them drive
that same car that the neural network
should drive soon we would record all
the sensor data we would record all the
camera footage right while the person is
driving and we would also record the
actions that he decided to take so if he
decided to go fast and steer left given
a certain image or given a certain
situation we record that and by that we
are building a data set of input and
output pairs and now comes the trick
right we give the neural network that
same situation that the human was
trained was facing observe what it would
have done by running that whole
calculation that I showed you before
and then we compare the two well
optimally they would be equal right
that’s what we want to achieve with this
how do we get to that well obviously as
you remember when you change the weights
or
the calculation of the neural network
depends on the weights so if we change
the weights the output changes and
that’s what we want to do we want to
change the output into a particular
direction so we simply change the
weights a little bit right we just add
or subtract a bit from these numbers and
by doing that we get the output of the
neural network closer to what the human
did if we do it in the right direction
and that’s easily to be done with a
mathematical procedure called gradient
descent that’s super easy calculus
actually if you’re into math look into
it pretty easy but not for an 18 minute
TED talk yeah but there is one problem
with this probably some of you see it
might not be the best driver right might
not be perfect still causing accidents
we wouldn’t really want to learn from
something that’s not perfect so can we
fix this can we fix the fact that our
training data might not be perfect and
in this case that would be quite bad if
we couldn’t because then our neural
network would have learned from someone
holding a smartphone while they’re
driving not a good example right so we
want to fix this how can we fix this
well we need to somehow get rid of the
expert data and that’s done by using a
learning procedure called reinforcement
learning if you sort of want to draw a
comparison to humans what I’ve showed
you before it’s sort of like what babies
do with their parents they think their
parents are perfect and they simply try
to imitate everything that they do right
but as they grow older they discover
that that might be that might not be
perfect and they start to explore they
do things that well their data didn’t
give them and they see how well they
perform they get the feedback from the
world and if it was good they try to do
that more often than if it was bad they
don’t do it anymore
that’s reinforcement learning in a
nutshell so what you do is you take the
outputs from the neural network actually
execute them in the real world and now
comes the point we have to define what
good and what bad means so we have to
define that for a car for example
reaching your destination is good
driving over some person is bad and
somehow turned this into a mathematical
function that outputs something we call
we can call reward that’s the number
still like just good or badness
basically so that’s it we try to learn
from that we try
to learn from our defined notion of what
good what’s good and what’s bad we don’t
learn from a strategy that was given by
human examples but since trying random
things in the real world usually doesn’t
result in happiness in case of
self-driving cars because that would
probably cost a couple lives we usually
do these things in simulation and that’s
why reinforcement learning is focused on
games so much that’s why there are so
many papers published that just care
about games today
games have these very great features for
AI for example that score is very
well-defined there is a very clear
notion of reward that’s a philosophical
problem how you define that with games
that’s sort of inherent and you have a
very clear observation with cars you’d
usually have your sensor data right
you’d have lidar sensors and camera
sensors and all these complex things
with games you have a game screen and
that’s it so we can focus on the AI and
also the buttons it’s just like some
couple of buttons that you could press
so that’s easy good but how does it
learn let me get to that again the graph
is actually simpler than one before we
take the screen run the calculation
through the neural network see what it
would do right just the action that the
neural network would take executes
direction in the environment and that
actually let pac-man move right really
do it and then observe the reward
observe the score now comes the thing
instead of changing the weights so that
the actions get closer to what a human
would have done we try to change the
weights so that the actions that happen
to give us a higher reward than the
average become more likely so view this
as probabilities we want to increase the
probability of actions that were more
rewarding than the average right that
makes sense and we want to decrease of
course the probability of actions that
turn out to be less rewarding than the
average good actions bad actions pretty
simple so we again change the weights
change this mathematical formula and
neural network is nothing but that to
output a higher probability for good
actions pretty simple run this for a
couple million iterations and you
actually have a program that plays
pac-man better than any human code and
that worked for go that’s why the go
program that Google built alphago is so
much better than humans because it
didn’t
learn or diversion to didn’t learn from
human examples that’s D contrast between
the two versions version 1 learn from
human examples version 2 learnt this way
but wait there are two players in go one
thing is missing right pac-man is a
single-player game how could we
incorporate that how could we include
multiple players in this system
well it turns out to be super simple you
just train them against each other and
in the case of opening eyes recent
research it looks like this and what you
see is quite interesting like first of
all of course because the physics engine
isn’t perfect they move a bit weird but
what you’ll notice is they actually sort
of they don’t give away what they will
do right they wait until their opponent
does something and only then do they
communicate what direction will be so
they they really exploit each other’s
behaviors they really try to sort of
develop robust policies that play well
no matter whether their opponent is
stupid or not pretty much and by
learning by learning these systems keep
improving parallel with each other they
always have a competitor that’s on the
same level and because you always have a
competitor on the same level you can
keep getting better and better by trying
to see what wins against that competitor
on the same level so you get better and
better that’s why alphago version two
alphago zero it’s called was so much
better so still
why isn’t AI super human done right
we’ve just seen like so many cool things
and in fact the first procedure I showed
you this supervised learning thing well
you where you learn from data that’s not
a joke it’s actually used for skin
cancer detection and because they only
use the lab results for the data it’s
better than what human doctors do and
that that has huge implications for us
for as for us as a society but still the
question remains why don’t we have super
smart robots that could deliver this
speech for example I don’t know I can’t
I’m gonna call myself super smart that’s
not what I intended whatever so like
really items I just make this up yeah ok
so why don’t we have superhuman AI look
at this image what do you see well you
see a projector with an image on it if I
give this to an AI
these nine numbers zero for each field
if there is nothing in it a one if there
is a circle in it and a two if there is
an ax in it its output would have to be
one of nine actions put your X into this
field this field this field this field
or whatever right how would this thing
ever develop a complex understanding of
the real world of the connections
between different tasks and different
problems right it will always just
understand this problem so it’s
inherently needed to present our AI
algorithms with very complex and general
problems for them to develop very
complex and general understanding of the
world if we look at as humans we have
that right we always perform muscle
movements I can’t actually speak words
what I can do is I can move my muscles
so that my vocal cords vibrate in a
certain way to generate sounds with
various frequencies that you perceive as
words that’s what my body is able to do
and I’m not able to move my arm from A
to B I am able to contract muscles and
not contract muscles
that’s the actress set that I can
perform it no matter whether I play
chess or whether I run I still have the
same muscles and I have to use the same
muscles so we always have the same way
to act we also always have the same
observation type we see with the same
eyes we have the same resolution that
doesn’t change that doesn’t improve for
a game if we like discover that one game
is better viewed as in this way and one
is better viewed in this way no we see
that way we hear we listen we smell we
taste we feel always the same input
signal no matter what problem we try to
solve and I believe this is one of the
key reasons the fact that we couldn’t
include this in our current AI systems
that we don’t have general super human
AI but we are moving into that direction
and one such moving into that direction
step is opening eyes gym retro dataset
that’s a data set of a thousand games
while they always while they all play on
the same console they are quite
different in terms of mechanics in terms
of goals that have to be achieved and
what they want to do with this
dataset is they want to encourage
researchers to try to train on say 900
of these games and to test your
performance on the other 1000 so you’d
like to have a general understanding in
the algorithm of what an objective in
the game is and how to efficiently learn
to achieve it because that’s what we do
if we are presented with a new problem
we don’t need to train from scratch we
don’t need to train a specific strategy
for that game only we can use our past
knowledge to apply it to problems we
serve we solve today and that’s what
they want to encourage and I think
that’s one of the main reasons why we
are actually on a good path to the
technology that AI research wanted to
form in the beginning 60 years ago
intelligence that’s actually smarter
than we are but one last thing and I
want to close off with that AI and
especially general superhuman AI has
lots of implications it’s important to
understand the principles of it and to
understand where it would be coming from
and to understand what it is for you to
be able to participate in decisions that
are that hopefully are democratic in the
future that the reason these decisions
are important that decisions will be how
this is used first of all why this is
used and where it is developed right now
a AGI is the major research field in
many corporates and many large
corporates in many large firms and that
possesses some risks because if you have
profit as your main goal as a researcher
you tend to neglect safety and with
intelligence that multiplies as quickly
as alpha girls intelligence did that
might be a problem and that’s also a
reason why open AI is such a great into
an institution they focus on safety and
I think safety decisions should not be
made by a few selected researchers and
today by the way these safety decisions
are not yet really important but they
will be soon I think and for you to
participate in this safety decisions
security ai safety it is important that
you understand the principles of AI and
that’s precisely the reason why I
smuggled the bit of AI education into
this today and I hope you enjoyed it thank you
about 60 years ago the field of
artificial intelligence was created and
right from the beginning the goal was to
build software that is more intelligent
than humans are and since then we have
been asking the question is AI smarter
than humans well I want to pass that
question on to you for a second everyone
who thinks that you are still smarter
than AI raise your hand great everyone
who thinks that the AI is smarter than
you raise your hand
see this is exactly what I expected the
opinions are very divided and that’s for
a good reason because recently we’ve
seen many applications that just
outperform human performance but yet
still we don’t see robots walking around
being as smart as we are we don’t have
AI that understands humor and there are
many other things that remain unsolved
so what I’m trying to attempt with this
TED talk is give you a more decisive
answer than well it’s not a yes and not
a no to this question let’s start by
looking at some things where AI actually
happens to be superhuman this is the
game of Go a game that we humans have
been playing since 2,500 years and a
game that since 2016 a AI is better at
but it’s not like this is a simple game
this is actually a very complex game
that was a main challenge of AI for
decades before and in fact 2014 in 2014
the programmer of the AI that was best
that go at that time thought it would
take another 10 years but yet in 2016
alphago developed by google deepmind
beat the world champion and what i find
most fascinating about this example is
that in 2017 only one year later we
actually found that version 2 of this
same algorithm beat the version won by
an even larger margin than the version 1
beat the world champion by so this thing
is getting better very quickly and
that’s not the only example games are
the only example see some computer
vision tasks such as single object
classification which means you get an
image and you say what the prominent the
most prominent object in that images are
also performed better than AI by AI than
by humans and in fact that works with up
to a thousand categories of images which
is pretty same a pretty sick I mean
since 2015
computer vision or the sub K sub task of
single object classification AI is
better at it well that’s so much for
super human but one thing that I find
particularly interesting about the state
of AI is that AI can actually speak a
single wavenet can capture the
characteristics of many different
speakers with equal fidelity now it’s
fast see this speech isn’t created by
vocal cords it’s created by google’s
wavenet version – this is completely
simulated but that’s 48,000 samples per
second that sound pretty realistic
coming from nowhere not from vocal cords
so let’s get back to our question is AI
super human well yes we clearly see that
in some applications it is but in some
it’s just not and to get a feeling for
why that is let me show you what AI
actually is and how it works this is a
neural network that’s the core
technology that’s at the bottom of
almost all innovations that happen in
the field of machine learning and
artificial intelligence today and
contrary to what most of you might think
know that core technology is not complex
it’s in fact super simple it’s composed
of inputs which could be the pixels of
an image could also be anything else the
only thing that matters is that it’s a
number
it has outputs which could be
probabilities say in the case of image
classification what you do is you’d feed
the pixels of the image in and out you
would get the probability for each
object that the neural network knows to
exist so if there is a dog a cat in the
car there would be a probability for
each of these objects the things in
between are sort of like synapses and
the neurons in our brain or at least
they try to
be that the synapses the lines
connecting the neurons are the actual
things that we can learn this is what
has changed
when the neural network learns when it
when it trains you have also just
numbers still there is nothing else than
numbers in this picture you have numbers
that we change during training when we
want to adapt the output depending on a
certain input and the notes in between
all of these connections our functions
very simple functions I mean look at
this this is simple what they do is they
sum up all the neurons from the previous
layer multiplied by the weights
connecting that layer to the node to the
neuron sum that up so take the sum of
all of this and push it through this
simple function if it’s negative it
becomes zero if it’s positive it stays
the same so and if you run this
computation for an input run it through
the whole network then you have an
output that’s how a neural network
predicts something it’s fairly simple
right but there is one question
remaining how do we train that thing
like how does it learn actually well for
explanitory purposes let us build a
simple self-driving car this thing well
you get is you input images from a
camera say like you have a self-driving
car that has five cameras and you you
input all of the images into the neural
network and what you want to get out is
the angle for the steering wheel and the
speed that the car should go with okay
that’s a use case great how do we train
it one way to train it is by using
professional drivers say we have a
couple people that we know are very good
at driving and we just let them drive
that same car that the neural network
should drive soon we would record all
the sensor data we would record all the
camera footage right while the person is
driving and we would also record the
actions that he decided to take so if he
decided to go fast and steer left given
a certain image or given a certain
situation we record that and by that we
are building a data set of input and
output pairs and now comes the trick
right we give the neural network that
same situation that the human was
trained was facing observe what it would
have done by running that whole
calculation that I showed you before
and then we compare the two well
optimally they would be equal right
that’s what we want to achieve with this
how do we get to that well obviously as
you remember when you change the weights
or
the calculation of the neural network
depends on the weights so if we change
the weights the output changes and
that’s what we want to do we want to
change the output into a particular
direction so we simply change the
weights a little bit right we just add
or subtract a bit from these numbers and
by doing that we get the output of the
neural network closer to what the human
did if we do it in the right direction
and that’s easily to be done with a
mathematical procedure called gradient
descent that’s super easy calculus
actually if you’re into math look into
it pretty easy but not for an 18 minute
TED talk yeah but there is one problem
with this probably some of you see it
might not be the best driver right might
not be perfect still causing accidents
we wouldn’t really want to learn from
something that’s not perfect so can we
fix this can we fix the fact that our
training data might not be perfect and
in this case that would be quite bad if
we couldn’t because then our neural
network would have learned from someone
holding a smartphone while they’re
driving not a good example right so we
want to fix this how can we fix this
well we need to somehow get rid of the
expert data and that’s done by using a
learning procedure called reinforcement
learning if you sort of want to draw a
comparison to humans what I’ve showed
you before it’s sort of like what babies
do with their parents they think their
parents are perfect and they simply try
to imitate everything that they do right
but as they grow older they discover
that that might be that might not be
perfect and they start to explore they
do things that well their data didn’t
give them and they see how well they
perform they get the feedback from the
world and if it was good they try to do
that more often than if it was bad they
don’t do it anymore
that’s reinforcement learning in a
nutshell so what you do is you take the
outputs from the neural network actually
execute them in the real world and now
comes the point we have to define what
good and what bad means so we have to
define that for a car for example
reaching your destination is good
driving over some person is bad and
somehow turned this into a mathematical
function that outputs something we call
we can call reward that’s the number
still like just good or badness
basically so that’s it we try to learn
from that we try
to learn from our defined notion of what
good what’s good and what’s bad we don’t
learn from a strategy that was given by
human examples but since trying random
things in the real world usually doesn’t
result in happiness in case of
self-driving cars because that would
probably cost a couple lives we usually
do these things in simulation and that’s
why reinforcement learning is focused on
games so much that’s why there are so
many papers published that just care
about games today
games have these very great features for
AI for example that score is very
well-defined there is a very clear
notion of reward that’s a philosophical
problem how you define that with games
that’s sort of inherent and you have a
very clear observation with cars you’d
usually have your sensor data right
you’d have lidar sensors and camera
sensors and all these complex things
with games you have a game screen and
that’s it so we can focus on the AI and
also the buttons it’s just like some
couple of buttons that you could press
so that’s easy good but how does it
learn let me get to that again the graph
is actually simpler than one before we
take the screen run the calculation
through the neural network see what it
would do right just the action that the
neural network would take executes
direction in the environment and that
actually let pac-man move right really
do it and then observe the reward
observe the score now comes the thing
instead of changing the weights so that
the actions get closer to what a human
would have done we try to change the
weights so that the actions that happen
to give us a higher reward than the
average become more likely so view this
as probabilities we want to increase the
probability of actions that were more
rewarding than the average right that
makes sense and we want to decrease of
course the probability of actions that
turn out to be less rewarding than the
average good actions bad actions pretty
simple so we again change the weights
change this mathematical formula and
neural network is nothing but that to
output a higher probability for good
actions pretty simple run this for a
couple million iterations and you
actually have a program that plays
pac-man better than any human code and
that worked for go that’s why the go
program that Google built alphago is so
much better than humans because it
didn’t
learn or diversion to didn’t learn from
human examples that’s D contrast between
the two versions version 1 learn from
human examples version 2 learnt this way
but wait there are two players in go one
thing is missing right pac-man is a
single-player game how could we
incorporate that how could we include
multiple players in this system
well it turns out to be super simple you
just train them against each other and
in the case of opening eyes recent
research it looks like this and what you
see is quite interesting like first of
all of course because the physics engine
isn’t perfect they move a bit weird but
what you’ll notice is they actually sort
of they don’t give away what they will
do right they wait until their opponent
does something and only then do they
communicate what direction will be so
they they really exploit each other’s
behaviors they really try to sort of
develop robust policies that play well
no matter whether their opponent is
stupid or not pretty much and by
learning by learning these systems keep
improving parallel with each other they
always have a competitor that’s on the
same level and because you always have a
competitor on the same level you can
keep getting better and better by trying
to see what wins against that competitor
on the same level so you get better and
better that’s why alphago version two
alphago zero it’s called was so much
better so still
why isn’t AI super human done right
we’ve just seen like so many cool things
and in fact the first procedure I showed
you this supervised learning thing well
you where you learn from data that’s not
a joke it’s actually used for skin
cancer detection and because they only
use the lab results for the data it’s
better than what human doctors do and
that that has huge implications for us
for as for us as a society but still the
question remains why don’t we have super
smart robots that could deliver this
speech for example I don’t know I can’t
I’m gonna call myself super smart that’s
not what I intended whatever so like
really items I just make this up yeah ok
so why don’t we have superhuman AI look
at this image what do you see well you
see a projector with an image on it if I
give this to an AI
these nine numbers zero for each field
if there is nothing in it a one if there
is a circle in it and a two if there is
an ax in it its output would have to be
one of nine actions put your X into this
field this field this field this field
or whatever right how would this thing
ever develop a complex understanding of
the real world of the connections
between different tasks and different
problems right it will always just
understand this problem so it’s
inherently needed to present our AI
algorithms with very complex and general
problems for them to develop very
complex and general understanding of the
world if we look at as humans we have
that right we always perform muscle
movements I can’t actually speak words
what I can do is I can move my muscles
so that my vocal cords vibrate in a
certain way to generate sounds with
various frequencies that you perceive as
words that’s what my body is able to do
and I’m not able to move my arm from A
to B I am able to contract muscles and
not contract muscles
that’s the actress set that I can
perform it no matter whether I play
chess or whether I run I still have the
same muscles and I have to use the same
muscles so we always have the same way
to act we also always have the same
observation type we see with the same
eyes we have the same resolution that
doesn’t change that doesn’t improve for
a game if we like discover that one game
is better viewed as in this way and one
is better viewed in this way no we see
that way we hear we listen we smell we
taste we feel always the same input
signal no matter what problem we try to
solve and I believe this is one of the
key reasons the fact that we couldn’t
include this in our current AI systems
that we don’t have general super human
AI but we are moving into that direction
and one such moving into that direction
step is opening eyes gym retro dataset
that’s a data set of a thousand games
while they always while they all play on
the same console they are quite
different in terms of mechanics in terms
of goals that have to be achieved and
what they want to do with this
dataset is they want to encourage
researchers to try to train on say 900
of these games and to test your
performance on the other 1000 so you’d
like to have a general understanding in
the algorithm of what an objective in
the game is and how to efficiently learn
to achieve it because that’s what we do
if we are presented with a new problem
we don’t need to train from scratch we
don’t need to train a specific strategy
for that game only we can use our past
knowledge to apply it to problems we
serve we solve today and that’s what
they want to encourage and I think
that’s one of the main reasons why we
are actually on a good path to the
technology that AI research wanted to
form in the beginning 60 years ago
intelligence that’s actually smarter
than we are but one last thing and I
want to close off with that AI and
especially general superhuman AI has
lots of implications it’s important to
understand the principles of it and to
understand where it would be coming from
and to understand what it is for you to
be able to participate in decisions that
are that hopefully are democratic in the
future that the reason these decisions
are important that decisions will be how
this is used first of all why this is
used and where it is developed right now
a AGI is the major research field in
many corporates and many large
corporates in many large firms and that
possesses some risks because if you have
profit as your main goal as a researcher
you tend to neglect safety and with
intelligence that multiplies as quickly
as alpha girls intelligence did that
might be a problem and that’s also a
reason why open AI is such a great into
an institution they focus on safety and
I think safety decisions should not be
made by a few selected researchers and
today by the way these safety decisions
are not yet really important but they
will be soon I think and for you to
participate in this safety decisions
security ai safety it is important that
you understand the principles of AI and
that’s precisely the reason why I
smuggled the bit of AI education into
this today and I hope you enjoyed it thank you
Please follow and like us: