[Applause]
today I want to talk about something
that’s been in the news almost every day
probably multiple times and it’s
artificial intelligence or AI so I’m
going to use this little cartoon that I
drew of a little robot I’m going to use
this guy to depict an AI which would be
a robot or a driverless car or just a an
algorithm running somewhere allocating
resources and I’m going to ask the
question of how do we handle the
challenges that come with this
technology so on one hand a AI promises
many benefits from better
recommendations for restaurants and
movies and even romantic partners to
safer cars to more accurate medical
diagnosis and a whole bunch of other
benefits but I also raises many
challenges that people are talking about
very frequently in the media from
algorithms that filter news putting us
in filter bubbles to unfair matching in
markets to mass unemployment of course
and a whole bunch of other challenges so
how do we balance the benefits and the
risks now one idea that is commonly
discussed is that we always need a human
in the loop of these systems so it would
look something like this
you’ve got an AI that is performing a
function let’s say it’s flying a plane
you always need to put a human in the
loop like a captain in the cockpit and
this person would ensure that the system
behaves in accordance with our plans and
desires and also is able to intervene in
case something goes wrong and maybe more
importantly we have somebody to blame
something goes wrong now an interesting
thing to notice about this picture is
that everybody here has the same goal
so most passengers I would hope would
require getting from A to B safely maybe
there are exceptions but in many cases
our systems with
perform functions in which there are
multiple goals and these goals may be in
conflict so somebody may be interested
in fairness so another person cares more
about efficiency and yet another person
cares more about safety and these goals
may not be simultaneously achievable for
whatever reason now so how do we agree
on these goals first before we tell the
Machine what to do and have the
human-in-the-loop intervene so this is
less like a human in the loop challenge
and more like a society in the loop
challenge whereby society in the loop
here I mean human in the loop plus a
social contract between the stakeholders
who care about different things and the
social contract is meant to provide the
agreement in which into which people
would opt in in order to cooperate to
their collective benefit it’s a
necessity and the social contract is not
really a new idea it’s been around for a
while at least 350 years next year this
is the book Leviathan by political
philosopher and economist Thomas Hobbes
and Thomas Hobbes is innovation was to
identify this notion of legitimacy of
the sovereign power that doesn’t come
from above it doesn’t come out of
nothing but it comes out of the mutual
consent of those who are governed so if
you zoom in on this sovereign this
all-powerful sovereign looking over the
land here you see that it’s made up of
hundreds of little people who have opted
into the social contract by giving up
some of their individual rights and
freedoms in order to maintain this
sovereign power to enforce property
rights as well as other rights and of
course if these people would opt out of
this contract this very powerful thing
would immediately cease to exist
now Hobbes made this sovereign very
powerful he said that this sovereign
should have almost all power short of
taking people’s lives but then we spent
many decades actually centuries taming
this Leviathan and we did this by
introducing many checks and balances on
its power introducing fundamental rights
that it cannot
violate and a whole bunch of other
political developments since then and
today as we think about technology it’s
important to realize that we’ve created
and maintained and tamed the Leviathan
through a social contract so today as we
build a technology on a kind of cyborg
sovereign that is made up of humans as
well as algorithms making decisions that
impact us we also need a kind of social
contract that is also algorithmic and I
want to talk a little bit about the
nature of this thing now how do and how
do we tame it so before we discuss this
we need to talk about what things
regulates and I can’t think of a better
source of or classification of things
that regulate then Larry Lessig’s
regulation architecture so there is well
Larry in the middle he’s being regulated
and he’s regulated by four major forces
this is from his book code and other
laws of cyberspace which was very
influential in the 90s as the internet
was starting so law is one of the main
regulators this is what governments do
but also social norms markets that set
prices on different behaviors and
objects but also architecture and by
this he means both the physical
architecture the built architecture but
also the information architecture things
like the design of cyberspace so how is
algorithmic social contract changing
this picture I think on one hand it’s
introducing a new kind of actor that
requires regulation so we are humans
still but we’re regulating new kinds of
actors so instead of Larry in the middle
here we have our little robot and this
robot is being regulated by similar
forces this could be for example a bunch
of programs trading in an algorithmic
trading market or it could be a
driverless car that is moving around
urban spaces and so on now on the other
hand we have now a I driven regulation
of human behavior as well
cause again going back to this picture
and putting Larry again in the middle
the law is now becoming AI Augmented
predictive policing is telling police
where two police judges are using
algorithms to inform therefore all
decisions and so on the markets are now
using algorithms to allocate buyers and
sellers for instance in peer-to-peer
markets or ride-sharing markets and the
information architecture is being
mediated by algorithms as well deciding
what information we see out of the sea
of of information out there and even
norms are being altered by these
algorithms for instance think of dating
websites and how they alter the kinds of
relationships with humans form so I
think it’s important to think as as we
think about how this picture is changing
is important to think about what has
remained the same and what has changed
because I think things that remain the
same means mean that we can still use
some of our old tools to regulate this
new environment so one of the things
that remains the same is opacity and
think consider for example teaching a
machine right from wrong like what to do
of course you know on one hand we have
difficulty articulating our goals to
machines but we’ve always had difficulty
articulating goals to humans we also had
difficulty trusting other humans and now
we have difficulty trusting other
machines to comply with these goals and
constraints and of course even when even
well-intentioned goals can sometimes go
out of control you know you could tell a
robot to minimize misery and it kills
all people because that’s one way of
achieving that goal now so there is
something wrong with this picture
of requiring programmers to ensure that
their machines are ethical before they
let them into the wild and I think it’s
because this goal is actually quite a
tall order to ask of programmers you
don’t ask people to open up other
people’s brains to check if they are
ethical by looking at their brain
architecture this is just unrealistic
instead what we do is we
observe people’s behavior and then we
hold it up we evaluated against set
standards of conduct the second thing
that hasn’t really changed much is the
presence of unsolvable dilemmas so very
common one that is I’ve been partly
responsible for popularizing is the
ethical dilemma that a an autonomous car
may face so the story goes the car is
somehow lost control is going to kill
some pedestrians but it’s able to swerve
and kill maybe its own passengers in
order to save more pedestrians again
many people think that you know AI
should solve this problem but this
problem is fundamentally unsolvable
that’s that’s exactly why it’s called a
dilemma it’s by definition unsolvable
and it’s really a tall order to expect
machines to solve a problem that defied
the best legal minds and ethical
philosophical minds for thousands of
years before we add up the technology
but that is not to say that we can solve
these problems we can’t have a measured
or a reasonable way forward for instance
just think of these moose bar dilemma
right I’ll call it which is this
metallic bar that is placed in the front
of the car if you remove these bars this
most parts also called the kangaroo bar
or a rule bar in Australia depends on
what kind of animal you might hit and
these things removing them would reduce
the risk of injury by 21% to pedestrians
and the risk of death by 6% which is why
they are banned in many parts of Europe
they’re banned in Australia but they’re
not banned in the u.s. so there we are
constantly making trade-offs you know
about the risks that we’re willing to
put up with relative risks in this case
that are defined by some kind of social
contract so we can continue to do the
same with machines that are intelligent
a third thing that is the same is our
own psychological bias you know we’re
we’re still human
we still reason about the world in
particular ways and so recently in our
research with problem
research that is trying to overcome
these kinds of dilemmas so we’ve shown
people their lifetime odds of dying from
being in a regular car versus an
autonomous car as you know according to
the projections of the safety benefits
of those cars and people say okay that’s
good III like that but that does nothing
to alleviate people’s concern about
being sacrificed by the car okay and and
these concerns alone despite of the
absolute reduction in risk may dissipate
people from adopting the cars and we’ve
seen this before right the lifetime odds
of dying from a car accident are
astronomically higher than dying from a
plane crash yet many more people are
afraid of flying air travel compared to
getting in a car and again we need to
educate people about reasoning about
risk in order to overcome these kinds of
psychological biases so these are things
that are similar that are not new I
think so what’s different one thing that
is different is the theory of mind so in
in the current regulation in the current
social contract we have we’re regulating
entities that are humans we have some
sort of theory of mind of how they might
behave what they’re interested in and
and what they dislike and like and we do
the same thing with those who govern but
with machines we have a new kind of
agent that is slightly different so
psychologists gray and colleagues have
done a research on how we perceive other
minds including the minds of humans the
minds of pets and the minds of robots
and as you can see here we perceive
humans as being high on agency the
ability to make autonomous decisions as
well as high on experience the ability
to experience pain pleasure and so on
but you can see here that the robots are
high on agency to some degree and
actually moving further along the x-axis
here but they’re very low on experience
we don’t know how to reason about these
kinds of agents as we regulate them for
one
we can’t punish them because they don’t
care about pain and I think this is a
fundamental shift that we have to think
about as we form this new social
contract a second thing that is
different is adaptation beyond the
initial intent in a completely
unpredictable manner so for example the
US Securities and Exchange Commission
defines market manipulation as an
intentional conduct designed to deceive
investors but attribute an intent to a
synthesized strategy that includes
machine learning that adapts the
strategy over time it’s an extremely
difficult thing to do which makes it
very difficult for us to point to the
cause behind or the intent behind some
behaviors a third thing that is
different is speed so as systems become
faster and faster for instance
algorithmic trading systems the speed-up
can generate new entirely new entirely
new behavioral regimes for example much
more frequent flash crashes and this
happens precisely at the point where
these where humans lose the ability to
intervene in real time in these kinds of
systems which brings up a whole lot of
new challenges so we could slow down the
machines somehow find ways to to keep
them running at the speed that we can
keep keep up with but obviously this
type of solution means that we have to
slow down our ability to benefit from
those machines as well and if we were if
we took this to the extreme apply this
precautionary principle I would be
sitting here and we will all be cavemen
women and I’ll be telling you about the
catastrophic consequences of using fire
holding holding civilization obviously
from progressing forward so one way of
handling this challenge of speed is to
use other programs to use programs to
watch other programs so obviously
instead of having a human here in the
loop watching the algorithm or the AI we
can have another AI watching
this is what some people have referred
to as oversight programs which i think
is going to be increasingly common of
course there are always limits to this
because then who watches the algorithm
who is certifying the first algorithm
and obviously this can go only so far
until you run out of material to build
more powerful machines but I think it
gives us some room to move so just kind
of wrapping up the algorithmic social
contract is going to change the our
picture of regulation in two ways it’s
going to change the kinds of actors that
we regulate and is going to change the
way that we regulate humans as well so
the picture looks something like this
where you have algorithmic actors that
are being regulated by algorithm
Augmented regulatory forces which kind
of put brings a question of what is left
for us to do but that’s what a different
talk now one more thing super
intelligence is this challenge that some
people are talking about well what if
machines become just so vastly more
intelligent that all of humans combined
how do we handle these types of
challenges and is this the most urgent
catastrophic risk that could end the
human civilization altogether I think
these kinds of challenges are not insane
they are real but I think they’re very
hard to predict and prepare for but I do
think that by building technologies and
institutions for creating an algorithmic
social contract now we’re actually in a
better position to be ready for for the
broader challenge the longer-term
challenge so thank you
you
[Applause]
today I want to talk about something
that’s been in the news almost every day
probably multiple times and it’s
artificial intelligence or AI so I’m
going to use this little cartoon that I
drew of a little robot I’m going to use
this guy to depict an AI which would be
a robot or a driverless car or just a an
algorithm running somewhere allocating
resources and I’m going to ask the
question of how do we handle the
challenges that come with this
technology so on one hand a AI promises
many benefits from better
recommendations for restaurants and
movies and even romantic partners to
safer cars to more accurate medical
diagnosis and a whole bunch of other
benefits but I also raises many
challenges that people are talking about
very frequently in the media from
algorithms that filter news putting us
in filter bubbles to unfair matching in
markets to mass unemployment of course
and a whole bunch of other challenges so
how do we balance the benefits and the
risks now one idea that is commonly
discussed is that we always need a human
in the loop of these systems so it would
look something like this
you’ve got an AI that is performing a
function let’s say it’s flying a plane
you always need to put a human in the
loop like a captain in the cockpit and
this person would ensure that the system
behaves in accordance with our plans and
desires and also is able to intervene in
case something goes wrong and maybe more
importantly we have somebody to blame
something goes wrong now an interesting
thing to notice about this picture is
that everybody here has the same goal
so most passengers I would hope would
require getting from A to B safely maybe
there are exceptions but in many cases
our systems with
perform functions in which there are
multiple goals and these goals may be in
conflict so somebody may be interested
in fairness so another person cares more
about efficiency and yet another person
cares more about safety and these goals
may not be simultaneously achievable for
whatever reason now so how do we agree
on these goals first before we tell the
Machine what to do and have the
human-in-the-loop intervene so this is
less like a human in the loop challenge
and more like a society in the loop
challenge whereby society in the loop
here I mean human in the loop plus a
social contract between the stakeholders
who care about different things and the
social contract is meant to provide the
agreement in which into which people
would opt in in order to cooperate to
their collective benefit it’s a
necessity and the social contract is not
really a new idea it’s been around for a
while at least 350 years next year this
is the book Leviathan by political
philosopher and economist Thomas Hobbes
and Thomas Hobbes is innovation was to
identify this notion of legitimacy of
the sovereign power that doesn’t come
from above it doesn’t come out of
nothing but it comes out of the mutual
consent of those who are governed so if
you zoom in on this sovereign this
all-powerful sovereign looking over the
land here you see that it’s made up of
hundreds of little people who have opted
into the social contract by giving up
some of their individual rights and
freedoms in order to maintain this
sovereign power to enforce property
rights as well as other rights and of
course if these people would opt out of
this contract this very powerful thing
would immediately cease to exist
now Hobbes made this sovereign very
powerful he said that this sovereign
should have almost all power short of
taking people’s lives but then we spent
many decades actually centuries taming
this Leviathan and we did this by
introducing many checks and balances on
its power introducing fundamental rights
that it cannot
violate and a whole bunch of other
political developments since then and
today as we think about technology it’s
important to realize that we’ve created
and maintained and tamed the Leviathan
through a social contract so today as we
build a technology on a kind of cyborg
sovereign that is made up of humans as
well as algorithms making decisions that
impact us we also need a kind of social
contract that is also algorithmic and I
want to talk a little bit about the
nature of this thing now how do and how
do we tame it so before we discuss this
we need to talk about what things
regulates and I can’t think of a better
source of or classification of things
that regulate then Larry Lessig’s
regulation architecture so there is well
Larry in the middle he’s being regulated
and he’s regulated by four major forces
this is from his book code and other
laws of cyberspace which was very
influential in the 90s as the internet
was starting so law is one of the main
regulators this is what governments do
but also social norms markets that set
prices on different behaviors and
objects but also architecture and by
this he means both the physical
architecture the built architecture but
also the information architecture things
like the design of cyberspace so how is
algorithmic social contract changing
this picture I think on one hand it’s
introducing a new kind of actor that
requires regulation so we are humans
still but we’re regulating new kinds of
actors so instead of Larry in the middle
here we have our little robot and this
robot is being regulated by similar
forces this could be for example a bunch
of programs trading in an algorithmic
trading market or it could be a
driverless car that is moving around
urban spaces and so on now on the other
hand we have now a I driven regulation
of human behavior as well
cause again going back to this picture
and putting Larry again in the middle
the law is now becoming AI Augmented
predictive policing is telling police
where two police judges are using
algorithms to inform therefore all
decisions and so on the markets are now
using algorithms to allocate buyers and
sellers for instance in peer-to-peer
markets or ride-sharing markets and the
information architecture is being
mediated by algorithms as well deciding
what information we see out of the sea
of of information out there and even
norms are being altered by these
algorithms for instance think of dating
websites and how they alter the kinds of
relationships with humans form so I
think it’s important to think as as we
think about how this picture is changing
is important to think about what has
remained the same and what has changed
because I think things that remain the
same means mean that we can still use
some of our old tools to regulate this
new environment so one of the things
that remains the same is opacity and
think consider for example teaching a
machine right from wrong like what to do
of course you know on one hand we have
difficulty articulating our goals to
machines but we’ve always had difficulty
articulating goals to humans we also had
difficulty trusting other humans and now
we have difficulty trusting other
machines to comply with these goals and
constraints and of course even when even
well-intentioned goals can sometimes go
out of control you know you could tell a
robot to minimize misery and it kills
all people because that’s one way of
achieving that goal now so there is
something wrong with this picture
of requiring programmers to ensure that
their machines are ethical before they
let them into the wild and I think it’s
because this goal is actually quite a
tall order to ask of programmers you
don’t ask people to open up other
people’s brains to check if they are
ethical by looking at their brain
architecture this is just unrealistic
instead what we do is we
observe people’s behavior and then we
hold it up we evaluated against set
standards of conduct the second thing
that hasn’t really changed much is the
presence of unsolvable dilemmas so very
common one that is I’ve been partly
responsible for popularizing is the
ethical dilemma that a an autonomous car
may face so the story goes the car is
somehow lost control is going to kill
some pedestrians but it’s able to swerve
and kill maybe its own passengers in
order to save more pedestrians again
many people think that you know AI
should solve this problem but this
problem is fundamentally unsolvable
that’s that’s exactly why it’s called a
dilemma it’s by definition unsolvable
and it’s really a tall order to expect
machines to solve a problem that defied
the best legal minds and ethical
philosophical minds for thousands of
years before we add up the technology
but that is not to say that we can solve
these problems we can’t have a measured
or a reasonable way forward for instance
just think of these moose bar dilemma
right I’ll call it which is this
metallic bar that is placed in the front
of the car if you remove these bars this
most parts also called the kangaroo bar
or a rule bar in Australia depends on
what kind of animal you might hit and
these things removing them would reduce
the risk of injury by 21% to pedestrians
and the risk of death by 6% which is why
they are banned in many parts of Europe
they’re banned in Australia but they’re
not banned in the u.s. so there we are
constantly making trade-offs you know
about the risks that we’re willing to
put up with relative risks in this case
that are defined by some kind of social
contract so we can continue to do the
same with machines that are intelligent
a third thing that is the same is our
own psychological bias you know we’re
we’re still human
we still reason about the world in
particular ways and so recently in our
research with problem
research that is trying to overcome
these kinds of dilemmas so we’ve shown
people their lifetime odds of dying from
being in a regular car versus an
autonomous car as you know according to
the projections of the safety benefits
of those cars and people say okay that’s
good III like that but that does nothing
to alleviate people’s concern about
being sacrificed by the car okay and and
these concerns alone despite of the
absolute reduction in risk may dissipate
people from adopting the cars and we’ve
seen this before right the lifetime odds
of dying from a car accident are
astronomically higher than dying from a
plane crash yet many more people are
afraid of flying air travel compared to
getting in a car and again we need to
educate people about reasoning about
risk in order to overcome these kinds of
psychological biases so these are things
that are similar that are not new I
think so what’s different one thing that
is different is the theory of mind so in
in the current regulation in the current
social contract we have we’re regulating
entities that are humans we have some
sort of theory of mind of how they might
behave what they’re interested in and
and what they dislike and like and we do
the same thing with those who govern but
with machines we have a new kind of
agent that is slightly different so
psychologists gray and colleagues have
done a research on how we perceive other
minds including the minds of humans the
minds of pets and the minds of robots
and as you can see here we perceive
humans as being high on agency the
ability to make autonomous decisions as
well as high on experience the ability
to experience pain pleasure and so on
but you can see here that the robots are
high on agency to some degree and
actually moving further along the x-axis
here but they’re very low on experience
we don’t know how to reason about these
kinds of agents as we regulate them for
one
we can’t punish them because they don’t
care about pain and I think this is a
fundamental shift that we have to think
about as we form this new social
contract a second thing that is
different is adaptation beyond the
initial intent in a completely
unpredictable manner so for example the
US Securities and Exchange Commission
defines market manipulation as an
intentional conduct designed to deceive
investors but attribute an intent to a
synthesized strategy that includes
machine learning that adapts the
strategy over time it’s an extremely
difficult thing to do which makes it
very difficult for us to point to the
cause behind or the intent behind some
behaviors a third thing that is
different is speed so as systems become
faster and faster for instance
algorithmic trading systems the speed-up
can generate new entirely new entirely
new behavioral regimes for example much
more frequent flash crashes and this
happens precisely at the point where
these where humans lose the ability to
intervene in real time in these kinds of
systems which brings up a whole lot of
new challenges so we could slow down the
machines somehow find ways to to keep
them running at the speed that we can
keep keep up with but obviously this
type of solution means that we have to
slow down our ability to benefit from
those machines as well and if we were if
we took this to the extreme apply this
precautionary principle I would be
sitting here and we will all be cavemen
women and I’ll be telling you about the
catastrophic consequences of using fire
holding holding civilization obviously
from progressing forward so one way of
handling this challenge of speed is to
use other programs to use programs to
watch other programs so obviously
instead of having a human here in the
loop watching the algorithm or the AI we
can have another AI watching
this is what some people have referred
to as oversight programs which i think
is going to be increasingly common of
course there are always limits to this
because then who watches the algorithm
who is certifying the first algorithm
and obviously this can go only so far
until you run out of material to build
more powerful machines but I think it
gives us some room to move so just kind
of wrapping up the algorithmic social
contract is going to change the our
picture of regulation in two ways it’s
going to change the kinds of actors that
we regulate and is going to change the
way that we regulate humans as well so
the picture looks something like this
where you have algorithmic actors that
are being regulated by algorithm
Augmented regulatory forces which kind
of put brings a question of what is left
for us to do but that’s what a different
talk now one more thing super
intelligence is this challenge that some
people are talking about well what if
machines become just so vastly more
intelligent that all of humans combined
how do we handle these types of
challenges and is this the most urgent
catastrophic risk that could end the
human civilization altogether I think
these kinds of challenges are not insane
they are real but I think they’re very
hard to predict and prepare for but I do
think that by building technologies and
institutions for creating an algorithmic
social contract now we’re actually in a
better position to be ready for for the
broader challenge the longer-term
challenge so thank you
you
[Applause]
Please follow and like us: