good morning my name is Josh Packer I’m
a senior here at the University of
Rochester studying computer science and
philosophy my senior year project has
been on the topic of ethical decision
making and autonomous vehicles and
specifically this idea of how we might
go about mitigating bias when
programming autonomous vehicles so you
guys have probably heard of self-driving
cars they sometimes look like this and
more than you know the self-driving
technology in the cars that you buy
today and this trend will only continue
as we move forward the technology is
amazing and that it enables new
efficiencies and optimizations but we
also have to do a lot of work to make
sure that this technology is safe so
that people want to use it and feel
comfortable having this as an everyday
experience so what do I mean when I talk
about safety and autonomous vehicles
well some of you may have seen this
problem in a different kind of context
this is a famous ethical issue that
several decades old called the trolley
problem and for those of you who don’t
know the idea is that you as a bystander
or witnessing a trolley that has
unfortunately lost its brakes sort of
hurtle down the track at five people who
are stuck therefore facing their
imminent demise and you are watching
this horrific situation and have the
choice to pull a lever that’s just in
front of you and if you do so you’ll
divert the trolley onto a separate track
just ahead and instead of killing the
five people you’ll kill one so this is a
classic problem people have gone back
and forth for as long as the problem has
been around and they’ll probably
continue to do that for a lot longer as
it’s already been several decades more
importantly than which you choose in
this scenario is why you chose it of
course the question is what are our
ethics what should we do which is where
it becomes interesting in the context
again of self-driving cars a few years
ago a group at MIT Media Lab made this
project called that mighty moral machine
and it’s a website you can log on to and
choose between these trolly like
scenarios and they sort of extrapolated
from the trolley problem with several
different factors besides just five and
one and it’s really interesting and
they’ve obviously collected a ton of
data on what people tend to do but
sometimes what people tend to do isn’t
enough and it certainly isn’t enough to
know what we should do because what
people think and what’s right is often
totally different so we see
went into a pretty important problem
which is that when we program autonomous
vehicles we can’t simply instill our own
moral fuey because we don’t know it well
enough and if someone thinks they know
it they’re probably wrong or someone
will disagree with them so it’s a big
problem what we do know is that
autonomous vehicles should be you should
do the following things well they should
align with our values and advocate as
our proxies they should be able to
perform even better than humans because
even the best human drivers aren’t good
enough for our self-driving cars and
lastly they should use machine learning
to get there which is one of the key
enabling features of modern artificial
intelligence for those of you who aren’t
as familiar with machine learning the
idea is that you can apply it to solving
abstract problems by using logic math
and statistics so if there’s some goal
we want to accomplish but we don’t know
exactly how to get there we can use
machine learning to help us there’s a
few different kinds of machine learning
that I’m going to talk about that are
pretty important to the front of this
problem of self-driving cars I think one
sort of option is supervised learning
it’s a pretty common approach and the
idea is that we provide a huge data set
and we provide all this data and we go
through this process of training and in
doing so we demonstrate how we might go
about doing some behavior and then
through this process of training the
algorithm will learn to predict our
behavior so classic example of this is
image classification so we might have a
lot of images of cats and dogs and along
with that we’ll have these sort of
ground truth labels of cat and dog that
we supply along with the images and
it’ll certain you’ll start to learn that
a cat image looks like this and a dog
image looks like this by extrapolating
abstract patterns and eventually we’ll
be able to feed it a new image and it’ll
say oh that’s a cat but that’s a dog and
the problem with this approach is that
we need to be able to provide this
massive set of data and if we don’t feel
comfortable providing this massive set
of data then we maybe shouldn’t take
this approach reinforcement learning is
another option and the idea here is that
instead of providing a huge amount of
data we can provide some sense of value
or reward and then by through the
process of training the algorithm can
learn to maximize that reward and make
decisions that help align with that
award reward and they can choose to not
make decisions that are against that
the problem here in terms of bias is
that we have to feel comfortable
specifying rewards so in this case we
might have a mouse robot or a mouse AI
navigating a maze that starts randomly
by trial and error and we specify
through the maze some negative utilities
and some positive utilities and
eventually it’ll get burned by the
negative utilities and choose good the
good path instead so we have to feel
comfortable supplying these rewards in
any setting where we want to use
reinforcement learning more complex
example of these seemingly simple
approaches would be something like this
and for those of you who don’t recognize
this this is an ancient Chinese board
game called go and it’s become the gold
standard for a lot of artificial
intelligence research it’s really
complex
it’s a 19 by 19 board and there’s more
possible configurations of the game than
there are particles in the universe so
needless to say it’s really complicated
and it’s also why it’s a great way to
test our artificial intelligence so
there’s a group called deep mind which
is now Google that’s been doing these
really interesting AI is called alphago
which are a eyes that play this game and
they beat the best human players out
there the most recent one and arguably
most interesting for our purposes is
alphago zero which as the name suggests
starts from a really low level of
intelligence as you can see we give it
very limited information upfront we
don’t provide a huge data set we might
provide the rules of the game and how
you might generate the following legal
move and that’s about it
and from that it learns to play
tremendously well and if you look back
to the graph it actually learns to play
better than any other version of go the
dashed line of the graph there
represents the benchmark of human play
human expert play from another version
called alpha goalie and you’ll see that
a supervised learning approach and purpl
doesn’t quite pass it because it never
can because as we said supervised
learning is just predicting our behavior
but alphago 0 and blue there is able to
surpass the previous previously
unsurpassable standard before too much
time is even necessary in training and
this means it can innovate in really
clever ways and play really clever moves
and one of the versions of alphago that
used the same kind of approach it played
a move against the world champion go
player that no one expected and thought
was a missed everyone thought was a
mistake but it turns out it won the game
a few moves later and it’s
incredible discourse amongst going thews
iasts so that’s a really cool board game
but self-driving cars are really
complicated and more so than that they
also matter in a way that board games
don’t because if a self-driving car
messes up someone could die so we want
to pay a little more attention to what
we how we program those cars what are
the differences then between a really
clever approach like alphago zero and
what we would need to change in order to
get something like autonomous cars to
work as we want them to well for one the
rules of alphago zero and we’re ago I
really defined quite well but the rules
so to speak of the problem of
self-driving cars or not I mean these
are the moral rules we were talking
about before that are so elusive and we
can’t possibly be the ones to state them
explicitly and pin them down because if
we’re wrong the consequences would be
catastrophic there’s also this other
issue that the following states the sort
of what could happen next is not as
clearly defined even though an NGO
there’s many different options you could
technically enumerate them if you wanted
to and in the real world that’s just not
the way it is something crazy could
happen the Millenium Falcon could fall
from the sky and we don’t know so how do
we sort of solve this well just I think
to solve the first problem this idea of
not having rules we have to create
something that we give this algorithm
innately I say that we have to create
some notion of primitive value something
that’s fundamental and unwavering some
kind of moral truth that most people
would not being trying to disagree with
this might be something like all
sentient life or all human life has
intrinsic moral worth something like
that and then from there we would let
the machine do more of the work instead
of us providing these explicit rules and
risking getting it wrong what I mean by
that is with an approach like alphago
zeroes you can provide something based
like this like that like the rule of the
game and have it figure out through the
trial and error process of searching and
playing itself what decisions lead to
good moves that are aligned with these
values and what decisions lead to really
terrible moves that aren’t aligned with
these values so it will abstract and
create higher-level moral truths without
us having to do it and to solve the
problem of not knowing a following state
we could have a separate system that has
some knowledge of physical laws of
course this is
immensely complex but it’s not
unthinkable car manufacturers and
automakers have tons of data on car
crashes and what people generally do
wrong so we already have a good sense of
what might not be a good idea to do or
what might lead to a crash let’s return
to this notion of primitive value and no
that’s not a mistake what this means is
that may be raising our robots is not so
different from raising our children in
some sense we don’t tell our children
everything or make them carbon copies of
us because it’s impossible we can’t we
might try but we can’t instead I think
what it might be like is a child with a
natural curiosity who’s exploring the
world but has this underlying innate
machinery for example we all have a
sense of pain doesn’t mean we know what
causes pain or we know what not to do we
have to find that out through messing up
but a child might touch a really hot
stove and then afterwards probably never
do it again because it really hurt and
that process is crucial that’s their
exploration and it’s the very same thing
as alphago zero or what I’m suggesting
here we have to supply something
innately but we can’t supply everything
innately that has to be done through
self play so what this means is that um
we have this sort of evolutionary
approach and an evolutionary approach
means that we sort of provide something
and then we allow for a machine to
figure out more on its own and you might
be thinking well that that sounds good
but it’s not really enough is it enough
to just provide some really small bit of
formative moral truth and trust that the
machine gets it of course not
it’s not gonna cut it and it might go
terribly wrong so we don’t want to just
do that what we can do instead is we can
start with that and then through that
process of training we can make
checkpoints along the way where we say
here’s a scenario what are you going to
do in this scenario and we can see if it
makes sense and if people agree that it
makes sense or if it’s a terrible idea
and if it’s a terrible idea we can
provide that feedback if it’s a terrible
idea we can say no and instead of
specifying a structural change
explicitly ourselves or going and
manually and changing some code we can
have it do the machine do it itself
it can say since it’s just a bunch of
probabilities and numbers anyway this
was what I was most certain about and
that was my number one choice that was
my decision but if we say it wasn’t a
good idea
it can choose this number two choice or
number three and it can do this in a
bunch of different configurations
because it’s obviously very complex and
there’s many different nodes so through
this process we can continue this ad
nauseam and come up with a solution that
actually ends up being exactly where we
want it to be in terms of our values we
can have a system that aligns with our
values because we’ve tested it
rigorously but we don’t have to
explicitly define our moral truths in
order to get there if you consider the
enforcement learning like we talked
about earlier we don’t know our truths
well enough to just simplify them in a
cost function it would be a disaster
instead we can meet the Machine halfway
and allow it to do some of the work as
well I know it seems counterintuitive to
think that the safest option is actually
to take this middle path but I really
think it is the case there incredible
benefits of self-driving cars and we
want to be able to reap those benefits
of efficiency safety and so much more
and I think that the only clear way to
do that properly is to allow the machine
to have more freedom than we would have
initially thought and make ourselves the
tool of verification as a standards
board and make sure that nothing bad
will ever fall into the hands of
consumers or those who wish to engage in
the use of self-driving cars this will
enable a self-driving future and make
our lives a whole lot easier thank you [Applause]
a senior here at the University of
Rochester studying computer science and
philosophy my senior year project has
been on the topic of ethical decision
making and autonomous vehicles and
specifically this idea of how we might
go about mitigating bias when
programming autonomous vehicles so you
guys have probably heard of self-driving
cars they sometimes look like this and
more than you know the self-driving
technology in the cars that you buy
today and this trend will only continue
as we move forward the technology is
amazing and that it enables new
efficiencies and optimizations but we
also have to do a lot of work to make
sure that this technology is safe so
that people want to use it and feel
comfortable having this as an everyday
experience so what do I mean when I talk
about safety and autonomous vehicles
well some of you may have seen this
problem in a different kind of context
this is a famous ethical issue that
several decades old called the trolley
problem and for those of you who don’t
know the idea is that you as a bystander
or witnessing a trolley that has
unfortunately lost its brakes sort of
hurtle down the track at five people who
are stuck therefore facing their
imminent demise and you are watching
this horrific situation and have the
choice to pull a lever that’s just in
front of you and if you do so you’ll
divert the trolley onto a separate track
just ahead and instead of killing the
five people you’ll kill one so this is a
classic problem people have gone back
and forth for as long as the problem has
been around and they’ll probably
continue to do that for a lot longer as
it’s already been several decades more
importantly than which you choose in
this scenario is why you chose it of
course the question is what are our
ethics what should we do which is where
it becomes interesting in the context
again of self-driving cars a few years
ago a group at MIT Media Lab made this
project called that mighty moral machine
and it’s a website you can log on to and
choose between these trolly like
scenarios and they sort of extrapolated
from the trolley problem with several
different factors besides just five and
one and it’s really interesting and
they’ve obviously collected a ton of
data on what people tend to do but
sometimes what people tend to do isn’t
enough and it certainly isn’t enough to
know what we should do because what
people think and what’s right is often
totally different so we see
went into a pretty important problem
which is that when we program autonomous
vehicles we can’t simply instill our own
moral fuey because we don’t know it well
enough and if someone thinks they know
it they’re probably wrong or someone
will disagree with them so it’s a big
problem what we do know is that
autonomous vehicles should be you should
do the following things well they should
align with our values and advocate as
our proxies they should be able to
perform even better than humans because
even the best human drivers aren’t good
enough for our self-driving cars and
lastly they should use machine learning
to get there which is one of the key
enabling features of modern artificial
intelligence for those of you who aren’t
as familiar with machine learning the
idea is that you can apply it to solving
abstract problems by using logic math
and statistics so if there’s some goal
we want to accomplish but we don’t know
exactly how to get there we can use
machine learning to help us there’s a
few different kinds of machine learning
that I’m going to talk about that are
pretty important to the front of this
problem of self-driving cars I think one
sort of option is supervised learning
it’s a pretty common approach and the
idea is that we provide a huge data set
and we provide all this data and we go
through this process of training and in
doing so we demonstrate how we might go
about doing some behavior and then
through this process of training the
algorithm will learn to predict our
behavior so classic example of this is
image classification so we might have a
lot of images of cats and dogs and along
with that we’ll have these sort of
ground truth labels of cat and dog that
we supply along with the images and
it’ll certain you’ll start to learn that
a cat image looks like this and a dog
image looks like this by extrapolating
abstract patterns and eventually we’ll
be able to feed it a new image and it’ll
say oh that’s a cat but that’s a dog and
the problem with this approach is that
we need to be able to provide this
massive set of data and if we don’t feel
comfortable providing this massive set
of data then we maybe shouldn’t take
this approach reinforcement learning is
another option and the idea here is that
instead of providing a huge amount of
data we can provide some sense of value
or reward and then by through the
process of training the algorithm can
learn to maximize that reward and make
decisions that help align with that
award reward and they can choose to not
make decisions that are against that
the problem here in terms of bias is
that we have to feel comfortable
specifying rewards so in this case we
might have a mouse robot or a mouse AI
navigating a maze that starts randomly
by trial and error and we specify
through the maze some negative utilities
and some positive utilities and
eventually it’ll get burned by the
negative utilities and choose good the
good path instead so we have to feel
comfortable supplying these rewards in
any setting where we want to use
reinforcement learning more complex
example of these seemingly simple
approaches would be something like this
and for those of you who don’t recognize
this this is an ancient Chinese board
game called go and it’s become the gold
standard for a lot of artificial
intelligence research it’s really
complex
it’s a 19 by 19 board and there’s more
possible configurations of the game than
there are particles in the universe so
needless to say it’s really complicated
and it’s also why it’s a great way to
test our artificial intelligence so
there’s a group called deep mind which
is now Google that’s been doing these
really interesting AI is called alphago
which are a eyes that play this game and
they beat the best human players out
there the most recent one and arguably
most interesting for our purposes is
alphago zero which as the name suggests
starts from a really low level of
intelligence as you can see we give it
very limited information upfront we
don’t provide a huge data set we might
provide the rules of the game and how
you might generate the following legal
move and that’s about it
and from that it learns to play
tremendously well and if you look back
to the graph it actually learns to play
better than any other version of go the
dashed line of the graph there
represents the benchmark of human play
human expert play from another version
called alpha goalie and you’ll see that
a supervised learning approach and purpl
doesn’t quite pass it because it never
can because as we said supervised
learning is just predicting our behavior
but alphago 0 and blue there is able to
surpass the previous previously
unsurpassable standard before too much
time is even necessary in training and
this means it can innovate in really
clever ways and play really clever moves
and one of the versions of alphago that
used the same kind of approach it played
a move against the world champion go
player that no one expected and thought
was a missed everyone thought was a
mistake but it turns out it won the game
a few moves later and it’s
incredible discourse amongst going thews
iasts so that’s a really cool board game
but self-driving cars are really
complicated and more so than that they
also matter in a way that board games
don’t because if a self-driving car
messes up someone could die so we want
to pay a little more attention to what
we how we program those cars what are
the differences then between a really
clever approach like alphago zero and
what we would need to change in order to
get something like autonomous cars to
work as we want them to well for one the
rules of alphago zero and we’re ago I
really defined quite well but the rules
so to speak of the problem of
self-driving cars or not I mean these
are the moral rules we were talking
about before that are so elusive and we
can’t possibly be the ones to state them
explicitly and pin them down because if
we’re wrong the consequences would be
catastrophic there’s also this other
issue that the following states the sort
of what could happen next is not as
clearly defined even though an NGO
there’s many different options you could
technically enumerate them if you wanted
to and in the real world that’s just not
the way it is something crazy could
happen the Millenium Falcon could fall
from the sky and we don’t know so how do
we sort of solve this well just I think
to solve the first problem this idea of
not having rules we have to create
something that we give this algorithm
innately I say that we have to create
some notion of primitive value something
that’s fundamental and unwavering some
kind of moral truth that most people
would not being trying to disagree with
this might be something like all
sentient life or all human life has
intrinsic moral worth something like
that and then from there we would let
the machine do more of the work instead
of us providing these explicit rules and
risking getting it wrong what I mean by
that is with an approach like alphago
zeroes you can provide something based
like this like that like the rule of the
game and have it figure out through the
trial and error process of searching and
playing itself what decisions lead to
good moves that are aligned with these
values and what decisions lead to really
terrible moves that aren’t aligned with
these values so it will abstract and
create higher-level moral truths without
us having to do it and to solve the
problem of not knowing a following state
we could have a separate system that has
some knowledge of physical laws of
course this is
immensely complex but it’s not
unthinkable car manufacturers and
automakers have tons of data on car
crashes and what people generally do
wrong so we already have a good sense of
what might not be a good idea to do or
what might lead to a crash let’s return
to this notion of primitive value and no
that’s not a mistake what this means is
that may be raising our robots is not so
different from raising our children in
some sense we don’t tell our children
everything or make them carbon copies of
us because it’s impossible we can’t we
might try but we can’t instead I think
what it might be like is a child with a
natural curiosity who’s exploring the
world but has this underlying innate
machinery for example we all have a
sense of pain doesn’t mean we know what
causes pain or we know what not to do we
have to find that out through messing up
but a child might touch a really hot
stove and then afterwards probably never
do it again because it really hurt and
that process is crucial that’s their
exploration and it’s the very same thing
as alphago zero or what I’m suggesting
here we have to supply something
innately but we can’t supply everything
innately that has to be done through
self play so what this means is that um
we have this sort of evolutionary
approach and an evolutionary approach
means that we sort of provide something
and then we allow for a machine to
figure out more on its own and you might
be thinking well that that sounds good
but it’s not really enough is it enough
to just provide some really small bit of
formative moral truth and trust that the
machine gets it of course not
it’s not gonna cut it and it might go
terribly wrong so we don’t want to just
do that what we can do instead is we can
start with that and then through that
process of training we can make
checkpoints along the way where we say
here’s a scenario what are you going to
do in this scenario and we can see if it
makes sense and if people agree that it
makes sense or if it’s a terrible idea
and if it’s a terrible idea we can
provide that feedback if it’s a terrible
idea we can say no and instead of
specifying a structural change
explicitly ourselves or going and
manually and changing some code we can
have it do the machine do it itself
it can say since it’s just a bunch of
probabilities and numbers anyway this
was what I was most certain about and
that was my number one choice that was
my decision but if we say it wasn’t a
good idea
it can choose this number two choice or
number three and it can do this in a
bunch of different configurations
because it’s obviously very complex and
there’s many different nodes so through
this process we can continue this ad
nauseam and come up with a solution that
actually ends up being exactly where we
want it to be in terms of our values we
can have a system that aligns with our
values because we’ve tested it
rigorously but we don’t have to
explicitly define our moral truths in
order to get there if you consider the
enforcement learning like we talked
about earlier we don’t know our truths
well enough to just simplify them in a
cost function it would be a disaster
instead we can meet the Machine halfway
and allow it to do some of the work as
well I know it seems counterintuitive to
think that the safest option is actually
to take this middle path but I really
think it is the case there incredible
benefits of self-driving cars and we
want to be able to reap those benefits
of efficiency safety and so much more
and I think that the only clear way to
do that properly is to allow the machine
to have more freedom than we would have
initially thought and make ourselves the
tool of verification as a standards
board and make sure that nothing bad
will ever fall into the hands of
consumers or those who wish to engage in
the use of self-driving cars this will
enable a self-driving future and make
our lives a whole lot easier thank you [Applause]
Please follow and like us: