the rise of the machines who here is
scared of killer robots
I am I used to work in UAVs unmanned
aerial vehicles and all I could think
seeing these things is that someday
somebody is gonna strap a machine gun to
these things and they’re gonna hunt me
down in swarms I work in robotics at
Brown University and I’m scared of
robots actually I’m kind of terrified
but can you blame me ever since I was a
kid all I’ve seen are movies that
portrayed the ascendance of artificial
intelligence and our inevitable conflict
with it 2001 a Space Odyssey The
Terminator
The Matrix and the stories they tell are
pretty scary
rogue bands of humans running away from
super intelligent machines that scares
me from the hands it seems like it
scares you as well I know it is scary
too Elon Musk but you know we have a
little bit of time before the robots
rise up robots like the pr2 that I have
my initiative they can’t even open the
door yet so in my mind this discussion
of super intelligent robots is a little
bit of a distraction from something far
more insidious that is going on with AI
systems across the country you see right
now there are people doctors judges
accountants who are getting information
from an AI system and treating it as if
it was information from a trusted
colleague
it’s this trust that bothers me not
because of how often AI gets it wrong
AI researchers pride themselves in
accuracy on results
it’s how badly it gets it wrong when it
makes a mistake that has me worried
these systems do not fail gracefully so
let’s take a look at what this looks
like this is a dog that has been
misidentified as a wolf by an AI
algorithm the researchers want to know
why did this particular husky get
misidentified as a wolf
so they rewrote the algorithm to explain
to them the parts the picture it was
paying attention to when the AI
algorithm made its decision in this
picture what do you think it paid
attention to what would you pay
attention to maybe the eyes maybe the
ears the snout this is what it paid
attention to mostly the snow and the
background of the picture you see there
was bias in the data set that was fed to
this algorithm most of the pictures of
wolves were in snow so the AI algorithm
conflated the presence of absence of
snow for the presidents that are absence
of a wolf the scary thing about this is
the researchers had no idea this was
happening until they rewrote the
algorithm to explain itself and that’s
the thing with AI algorithms deep
learning machine learning even the
developers who work on this stuff have
no idea what it’s doing so that might be
a great example from research but was
this mean in the real world
the compass criminal sentencing
algorithm is used in 13 States to
determine criminal recidivism or the
risk of committing a crime again after
you’re released
ProPublica found that if you’re African
ameri-ghen compass was 77 percent more
likely to qualify you as a potentially
violent offender than if you’re a
Caucasian this is a real system being
used in the real world by real judges to
make decisions about real people’s lives
why would the judges trust it if it
seems to exhibit bias well the reason
they use compass is because it is a
model for efficiency compass lets them
go through caseloads
much faster in a backlogged criminal
justice system why would they question
their own software it’s been
requisitioned by the state approved by
their ID Department why would they
question it well the people sentenced by
compass have questioned it and their
lawsuits should kill us all
the Wisconsin State Supreme Court ruled
that compass did not deny a defendant
due process provide it was used properly
in the same set of rulings they ruled
that defendant could not inspect the
source code of compass it has to be used
properly but you can’t inspect the
source code this is a disturbing set of
rulings when taken together for anyone
facing criminal sentencing now you may
not care about this because you’re not
facing criminal sentencing but what if I
told you that black box AI algorithms
like this are being used to decide
whether or not you can get a loan for
your house whether you get a job
interview whether you get Medicaid and
are even driving cars and trucks down
the highway would you want the public to
be able to inspect the algorithm that’s
trying to make a decision between a
shopping cart and a baby carriage in a
self-driving truck the same way the dog
wolf algorithm was trying to decide
between a dog or a wolf are you
potentially a metaphorical dog who’s
been misidentified as a wolf by
somebody’s AI algorithm considering the
complexity of people it’s possible is
there anything you can
do about it now probably not and that’s
what we need to focus on we need to
demand standards of accountability
transparency and recourse in AI systems
ISO the international standards
organization just formed a committee to
make decisions about what to do for AI
standards they’re about five years out
from coming up with a standard these
systems are being used now not just in
loans but they’re being used in vehicles
like I was saying they’re being used in
things like cooperative adaptive cruise
control it’s funny that they call that
cruise control because the type of
controller used in cruise control a PID
controller was used for 30 years in
chemical plants before it ever made into
a car the type of controller that’s used
to drive a self-driving car and machine
learning that’s only been used in
research since 2007 these are new
technologies we need to demand the
standards and we need to demand
regulation so that we don’t get snake
oil in the marketplace and we also have
to have a little bit of skepticism the
experiments in authority done by Stanley
Milgram after World War two showed that
your average person would follow an
authority figures orders even if it
meant harming their fellow citizen in
this experiment every day Americans
would shock an actor past the point of
him complaining about her trouble past
the point of him screaming in pain past
the point of him going silent and
simulated death all because somebody
with no credentials in a lab coat was
saying some variation of the phrase the
experiment MA
to continue in AI we have Milgram’s
ultimate authority figure we have a
dispassionate system that cannot reflect
that cannot make another decision that
there is no recourse to that will always
say the system or the process must
continue now I’m going to tell you a
little story it’s about a car trip they
took driving across country and I was
coming in to Salt Lake City and it
started raining as I climbed the
mountains that rain turned into snow and
pretty soon that snow was whiteout I
couldn’t see the taillights of the car
in front of me
I started skidding I went 361 way I went
360 the other way I went off the highway
mud coated my windows I couldn’t see a
thing I was terrified some car was gonna
come crashing into me now I’m telling
you this story to get you thinking about
how something small and seemingly
mundane like a little bit precipitation
can easily grow into something very
dangerous we are driving in the rain
with AI right now and that rain will
turn to snow and that snow could become
a blizzard we need to pause check the
conditions put in place safety standards
and ask ourselves how far do we want to
go because the economic incentives for
AI and automation to replace human labor
will be beyond anything we have seen
since the Industrial Revolution human
salary demands cannot compete with the
base cost of electricity
a eyes and robots will replace fry cooks
and fast-food joints and radiologists
and hospitals someday an AI will
diagnose your cancer and a robot will
perform the surgery only a healthy
skepticism of these systems is going to
help keep people in the loop and I’m
confident if we can keep people in the
loop if we can build transparent AI
systems like the dog wolf example where
the AI explained what it was doing to
people and people were able to spot
check it we can create new jobs for
people
partnering with AI if we work together
with AI we will probably be able to
solve some of our greatest challenges
but to do that we need to lead and not
follow we need to choose to be less like
robots and we need to build the robots
to be more like people because
ultimately the only thing we need to
fear is not killer robots it’s our own
intellectual laziness the only thing we
need to fear is ourselves
thank you
you
scared of killer robots
I am I used to work in UAVs unmanned
aerial vehicles and all I could think
seeing these things is that someday
somebody is gonna strap a machine gun to
these things and they’re gonna hunt me
down in swarms I work in robotics at
Brown University and I’m scared of
robots actually I’m kind of terrified
but can you blame me ever since I was a
kid all I’ve seen are movies that
portrayed the ascendance of artificial
intelligence and our inevitable conflict
with it 2001 a Space Odyssey The
Terminator
The Matrix and the stories they tell are
pretty scary
rogue bands of humans running away from
super intelligent machines that scares
me from the hands it seems like it
scares you as well I know it is scary
too Elon Musk but you know we have a
little bit of time before the robots
rise up robots like the pr2 that I have
my initiative they can’t even open the
door yet so in my mind this discussion
of super intelligent robots is a little
bit of a distraction from something far
more insidious that is going on with AI
systems across the country you see right
now there are people doctors judges
accountants who are getting information
from an AI system and treating it as if
it was information from a trusted
colleague
it’s this trust that bothers me not
because of how often AI gets it wrong
AI researchers pride themselves in
accuracy on results
it’s how badly it gets it wrong when it
makes a mistake that has me worried
these systems do not fail gracefully so
let’s take a look at what this looks
like this is a dog that has been
misidentified as a wolf by an AI
algorithm the researchers want to know
why did this particular husky get
misidentified as a wolf
so they rewrote the algorithm to explain
to them the parts the picture it was
paying attention to when the AI
algorithm made its decision in this
picture what do you think it paid
attention to what would you pay
attention to maybe the eyes maybe the
ears the snout this is what it paid
attention to mostly the snow and the
background of the picture you see there
was bias in the data set that was fed to
this algorithm most of the pictures of
wolves were in snow so the AI algorithm
conflated the presence of absence of
snow for the presidents that are absence
of a wolf the scary thing about this is
the researchers had no idea this was
happening until they rewrote the
algorithm to explain itself and that’s
the thing with AI algorithms deep
learning machine learning even the
developers who work on this stuff have
no idea what it’s doing so that might be
a great example from research but was
this mean in the real world
the compass criminal sentencing
algorithm is used in 13 States to
determine criminal recidivism or the
risk of committing a crime again after
you’re released
ProPublica found that if you’re African
ameri-ghen compass was 77 percent more
likely to qualify you as a potentially
violent offender than if you’re a
Caucasian this is a real system being
used in the real world by real judges to
make decisions about real people’s lives
why would the judges trust it if it
seems to exhibit bias well the reason
they use compass is because it is a
model for efficiency compass lets them
go through caseloads
much faster in a backlogged criminal
justice system why would they question
their own software it’s been
requisitioned by the state approved by
their ID Department why would they
question it well the people sentenced by
compass have questioned it and their
lawsuits should kill us all
the Wisconsin State Supreme Court ruled
that compass did not deny a defendant
due process provide it was used properly
in the same set of rulings they ruled
that defendant could not inspect the
source code of compass it has to be used
properly but you can’t inspect the
source code this is a disturbing set of
rulings when taken together for anyone
facing criminal sentencing now you may
not care about this because you’re not
facing criminal sentencing but what if I
told you that black box AI algorithms
like this are being used to decide
whether or not you can get a loan for
your house whether you get a job
interview whether you get Medicaid and
are even driving cars and trucks down
the highway would you want the public to
be able to inspect the algorithm that’s
trying to make a decision between a
shopping cart and a baby carriage in a
self-driving truck the same way the dog
wolf algorithm was trying to decide
between a dog or a wolf are you
potentially a metaphorical dog who’s
been misidentified as a wolf by
somebody’s AI algorithm considering the
complexity of people it’s possible is
there anything you can
do about it now probably not and that’s
what we need to focus on we need to
demand standards of accountability
transparency and recourse in AI systems
ISO the international standards
organization just formed a committee to
make decisions about what to do for AI
standards they’re about five years out
from coming up with a standard these
systems are being used now not just in
loans but they’re being used in vehicles
like I was saying they’re being used in
things like cooperative adaptive cruise
control it’s funny that they call that
cruise control because the type of
controller used in cruise control a PID
controller was used for 30 years in
chemical plants before it ever made into
a car the type of controller that’s used
to drive a self-driving car and machine
learning that’s only been used in
research since 2007 these are new
technologies we need to demand the
standards and we need to demand
regulation so that we don’t get snake
oil in the marketplace and we also have
to have a little bit of skepticism the
experiments in authority done by Stanley
Milgram after World War two showed that
your average person would follow an
authority figures orders even if it
meant harming their fellow citizen in
this experiment every day Americans
would shock an actor past the point of
him complaining about her trouble past
the point of him screaming in pain past
the point of him going silent and
simulated death all because somebody
with no credentials in a lab coat was
saying some variation of the phrase the
experiment MA
to continue in AI we have Milgram’s
ultimate authority figure we have a
dispassionate system that cannot reflect
that cannot make another decision that
there is no recourse to that will always
say the system or the process must
continue now I’m going to tell you a
little story it’s about a car trip they
took driving across country and I was
coming in to Salt Lake City and it
started raining as I climbed the
mountains that rain turned into snow and
pretty soon that snow was whiteout I
couldn’t see the taillights of the car
in front of me
I started skidding I went 361 way I went
360 the other way I went off the highway
mud coated my windows I couldn’t see a
thing I was terrified some car was gonna
come crashing into me now I’m telling
you this story to get you thinking about
how something small and seemingly
mundane like a little bit precipitation
can easily grow into something very
dangerous we are driving in the rain
with AI right now and that rain will
turn to snow and that snow could become
a blizzard we need to pause check the
conditions put in place safety standards
and ask ourselves how far do we want to
go because the economic incentives for
AI and automation to replace human labor
will be beyond anything we have seen
since the Industrial Revolution human
salary demands cannot compete with the
base cost of electricity
a eyes and robots will replace fry cooks
and fast-food joints and radiologists
and hospitals someday an AI will
diagnose your cancer and a robot will
perform the surgery only a healthy
skepticism of these systems is going to
help keep people in the loop and I’m
confident if we can keep people in the
loop if we can build transparent AI
systems like the dog wolf example where
the AI explained what it was doing to
people and people were able to spot
check it we can create new jobs for
people
partnering with AI if we work together
with AI we will probably be able to
solve some of our greatest challenges
but to do that we need to lead and not
follow we need to choose to be less like
robots and we need to build the robots
to be more like people because
ultimately the only thing we need to
fear is not killer robots it’s our own
intellectual laziness the only thing we
need to fear is ourselves
thank you
you
Please follow and like us: