
this is all of history on one graph on
the x-axis we have time collapsed
logarithmic ly so we can see it all
together on the y-axis is the logarithm
of growth rates and you can see history
is well summarized as a sequence of
exponential growth modes during each era
the growth rate was steady and then
there were sudden jumps where the growth
rate grew by a factor of 50 to 200
during the forager era humans doubled
roughly every quarter million years for
about two million years ago ten thousand
years there was a sudden transitions to
the farming revolution where farmers
started doubling roughly a thousand
years a few hundred years ago there was
a sudden transition to the Industrial
Revolution where industry has been
doubling roughly every 15 years if this
pattern were to continue then roughly
sometime in the next century or so there
would be a transition that took ten
years or less after which the economy
would be doubling every month and that
would last for only a year or two until
something else happened that would be
the straightforward projection of
history following this pattern what
could possibly cause such an enormous
change one of the most common
speculations is robots as smartest
people there are several ways that might
conceivably happen probably the most
straightforward way is that we keep
accumulating software the way we’ve been
doing for the last 70 years I used to be
an AI researcher for nine years and I’ve
been in the habit of talking to a our
researchers who’ve been in their field
for at least twenty years such as our
last speaker and asking them the
question how far have we come in the
last 20 years as a percentage of the
distance to human level abilities and
they usually tell me five to ten percent
of the way noticeable acceleration and
at that rate we’re talking two to four
centuries but that would eventually
happen some people think I was described
in a previous video
that it’s gonna happen a lot faster
because we’re gonna discover some great
grand theories of intelligence or
learning i’m skeptical but i’m gonna
talk in this here about a third scenario
it’s not guaranteed to happen but it’s
plausible and enough to be worth
exploring the scenario is two-port
software so today if you have an old
machine writing software you like and
you want software like that running on a
new machine you could try to stare at
this software guess how it works and
then write software in the new machine
that works how you think it works on the
old machine or you can write what’s
called an emulator on the new machine
that makes the new one looked like the
old one to the software if you can write
an emulator you just move the software
right over and it works without you
having to understand it so the idea is
to try to do that for the human brain to
make that work we’re gonna have three
technologies that aren’t at the right
levels yet but when they are emulations
become possible one is we need lots of
cheap fast parallel computers we don’t
have them yet second we need to take
individual brains scan them and find
spatial and chemical detail to see
exactly what’s we’re connected to what
of what type third we need computer
models of each kind of cell in the brain
a model of how it takes signals in
changes internal states and signals out
if you have good enough models for all
the kinds of cells a good enough scan of
a particular brain then you can make a
model of that entire brain an emulation
of that brain and if the models and
parameters are good enough then it will
have the same input-output behavior as
the original brain that is you could
hook it up with artificial eyes ears
hands mouth you could talk to it it
would talk back you could ask to do jobs
it might do them if it was cheap it
would change everything
now I’ve been around people talking
about this idea for many decades
it’s a favorite topic college
late-night conversations and the usual
topics that come up are is it even
possible to make an emulation of a human
a machine that emulates a real
flesh-and-blood human if you made one
would it be conscious or is it just an
empty machine if you made one of me is
it me or is it someone else
these are all fascinating questions that
I’m going to ignore because I thought
the neglected question has been what
would actually happen so I wrote this
book to try to say of what I’m gonna
explain my method on this slide and then
from here we’re
say will all the conclusions about this
world my method is first I’m trying not
to be creative or original other than by
asking this unusual question just
straightforwardly apply academic
consensus secondly I’m gonna not say
what should be I’m gonna just try to say
what’s likely to happen if we did the
least to avoid it you don’t have to like
it not my job
third I’m gonna focus on the robots
because that’s where most of the action
is but we will talk about humans I’m
gonna try to talk about the next era
after ours it’s as big as different from
ours as these previous eras I’m not
gonna tell you about the history of the
universe for the next trillion years
after this error other things happen I
don’t know what I’m gonna focus on after
a transition things are easier to
predict when it’s an equilibrium during
transitions things are unstable I’m
gonna use our standard first cut tool of
economists which is called supply and
demand which implicitly assumes there’s
lots of buyers and sellers for
everything not much regulation finally
I’m gonna soon the emulations themselves
are as simple as they can be they’re
opaque you made this emulation of
someone it’s a black box you can turn it
on turn it off
erase it copy it run it fast run it slow
that’s it
can’t do anything else all right those
are the rules and now let’s hear about
the age of em first of all I can tell
you a number of things that are true
just because Em’s are robots they would
be true for any world dominated by
robots robots can be represented as
computer files so they can be immortal
now your houses and cars are in
principle immortal because you could
keep repairing them and they could last
forever but that doesn’t mean you do in
principle immortality doesn’t mean
actual immortality it’s a possibility
you could send computer files around the
world at the speed of light so robots
can move around the world at the speed
of light as a file you know that if we
accidentally kill nature you will die
you are afraid rightly so robots know
that if they are made in factories out
of things dug up in mines if they
accidentally kill nature they do not die
they are less it’s scared of killing
nature finally you can make copies of
files and that has enormous implications
first of all wages very quickly fall to
subsistence levels because the
population of robots can expand so
quickly that makes the economy grow much
faster it can plausibly double every
month that’s
rate forward implication of standard
economic theories if it takes a year to
get to Mars and the economy doubles
every month nobody’s gonna be very
interested in going to Mars during this
period humans collectively must retire
all at once for good now humans
collectively owned all of the capital in
this world and it economy doubles every
month wealth doubles every month so
collectively humans are getting rich but
individual humans may not know much
besides their ability to work so if they
don’t get sufficient sharing and
insurance arrangements they’re in
trouble
I predict that will vary some some will
and some won’t
all humans start out as simulations so
their mental characteristics are within
the human range they are recognisably
human in fact when you turn one on you
have to convince it it’s now the
emulation because a moment ago it
remembered being the human but they are
not typical humans the emulation economy
is very competitive and it selects the
humans who are most productive in the
emulation world and makes billions or
trillions of copies of them so most
simulations are copies of the few
hundred most productive humans that
makes the typical emulation as
productive as a typical Nobel Prize
winner Olympic gold medalist head of
state their elite better than average
and they know it if they they may look
on humans with nostalgia and gratitude
but not so much respect which is if you
think about it how you think about your
ancestors now we know a lot of things
about how humans vary in terms of who’s
more productive we could just use those
things to predict features of Em’s
they’ll be smarter more conscientious
hard work or coholic scraps married
religious because these are things that
correlate with productivity among humans
most jobs and in advance the economy or
desk jobs and there’s since emulations
are already on a computer there’s no
point in building a little desk for them
they might as well sit in virtual
reality and do their desk job and in
virtual reality it’s very cheap to give
them a luxurious environment that’s
beautiful and expensive and their bodies
can be virtual and gorgeous they never
need to feel pain hunger disease crime
it’s a comfortable beautiful world where
these are all desks they’re working all
the time
but still beautifully now the most
traumatic thing that ever happened to
humans arguably was a transition from
foraging to farming foragers are in
rough equilibrium with their environment
when they do what they do what feels
natural it’s usually the right thing to
do humans were able to become farmers
only because we have enough cultural
plasticity to change and have strong
norms that tell farmers that they have
to do something different than what felt
natural to a forger and that became
possible with religion and strong
conformity pressures but in the last few
hundred years we’ve gotten rich and most
of the major social trends over the last
few hundred years can be understood as
us drifting back to forager attitudes as
we get rich the social pressures that
turned foragers into farmers no longer
feel compelling when we’re rich this can
continue to be true for the humans but
no longer true for the emulations they
have need of the social pressures as
farmers did so emulations go back to
more farmer style attitudes it’s no
longer a Star Trek future for them it’s
more something else again you don’t have
to like it I’m just telling you my best
guess of what’s likely to happen this is
your life you start and then you end
this is the life of an emulation who
everyday splits off a small number of
short copies who do short-term tasks and
then end or retire this is much cheaper
so the main line copy has to rest
through most of the day and that only
works a part of the day each of these
short-term copies they’re all work we’ll
talk more about that in a minute this is
an emulation who’s opportunistic they
make more copies of the versions of
themselves which are in more demand less
of others they don’t know which versions
will be popular and where their future
is going to go this is an emulation
designer they conceive of a large system
when one copy and then they split into
copies to elaborate the design in the
various parts all the way down until
they’ve implemented the entire design in
this way a single emulation it can
implement conceive and implement a much
larger more coherent design than humans
can today where one person designs and
other people implement this is an
emulation plumber who remembers that for
20 years everyday they only ever work
two hours a day a life of leisure what
actually happened is that every day they
split into a thousand copies each of
whom did a two hour plumbing job and
only one of them in
on to the next day objectively they are
working well over 99% of the time
subjectively they remember a life of
leisure we know of many systems like
software and the human brain which have
the feature that as they adapt to
circumstances they become fragile and
harder to readapt to new circumstances
humans start out with fluid intelligence
when you’re young like you later on we
have crystallized intelligence where we
know a lot but we hard find it harder to
learn new tricks
this plausibly means that emulations
even if they’re immortal they have a
limited career length after which they
need to retire and then they are
replaced by younger versions of
themselves who have been training and
slightly newer ways so emulations see
their future in great detail around them
they see slightly older versions of
themselves so they know where they’re
gonna work and live who they’ll marry so
they know their marriages are gonna work
out because they see the other version
working out now again this is you start
an un this could be you at the beginning
of party where you take a drug at the
beginning of a party which means you
won’t remember that party the next day
or ever after I’m told some people do
that my question for you is at the end
of this party will you say to yourself
I’m about to die I was created as a new
peach creature at the beginning of this
party the other person who lives on
tomorrow that’s not me because they
don’t remember me I hate a being about
to die why did I get myself into this
you could have that attitude or you
could say to yourself that person
tomorrow that’s me I will go on but I
just won’t remember what I did tonight
both of those are valid attitudes now
this is the same structure an emulation
splits off a short-term copy to stand in
line at the DMV or whatever and then
ends they can think of this new copy as
a new creature who has a short life and
they hate that or they could think of it
as another part of themselves that they
won’t remember I predict I’ll have that
second attitude not because it’s
philosophically correct it’s just the
attitude you need to have to get along
in this world that’s a very productive
advantage if you have that attitude
today it’s hard to meet celebrities
their time is very valuable they’re rare
for the emulations if you think about it
it’s easy to meet celebrities they just
spit off another copy and you interact
with that other copy and then that copy
ends the hard thing to do is to get a
celebrity to remember you because
remember that copy that
doesn’t remember the other parts so
emulations can use this to an advantage
however so today if the president your
leader tells you we must invade Iraq and
you say why and they say can’t tell you
state secrets you don’t know whether to
trust them for an emulation however the
President and you can both make a copy
who go inside is safe and inside the
safe they could explain all their secret
reasons do you and at the end you decide
if you’re convinced or not you send one
bit out back to your original version
was I convinced so your original version
can hear yes there’s a good reason I
don’t know what it is but apparently I
was just convinced a few moments ago
that there’s a good reason for this so
leaders can be more trusted today in our
live we have three stages of a life we
train and then we work and then we
retire and they have the same three
stages but they can train a small number
of copies and then pick the very best of
those and then make many copies who work
so they can invest a lot more resources
and training it’s training as much
cheaper for them and later on when they
retire if they’re not rich enough to
just go on retiring the way they were
living then they can just run slow and
retire slow so I told you they could run
at different speeds and cost is
proportional to speed so if you only
have one hundreds as much resources it
takes to go on at a regular speed while
you just go down to one hundredth of
speed and you can continue on and retire
at a slow speed now emulations can run
up many different speeds and they would
probably clump in to speed so they can
interact with others at the same speed
and that is true over a very wide range
that they could spend more and then run
faster so they can possibly run up to a
million times faster than humans or down
to a billion times slower than humans
this creates an enormous class hierarchy
hi class Thames at the top are really
our higher status they have many of them
markers that we treat it as high status
in our world and at the bottom they are
lower status and in fact lowers slow
ends are kind of like ghosts in our
literature if you recall ghosts are
these creatures who are all around
they’re obsessed with the past they
can’t influence once if you talk to them
they don’t know much so what’s the point
there’s ghosts but you might as well
ignore them in our world we clumped
together in cities but we aren’t all
together in one big city because as we
have bigger cities they start to have
larger and larger costs of congestion it
takes longer and more expensive to get
around emulations however in a large
city they
can interact with others using virtual
reality without actually sending their
mind or body and that means it’s cheap
for them to interact across a wide scope
and their cities can therefore be much
larger and more dense so emulations
probably do concentrate into a small
number of very dense cities for us
because the speeds we run we have
roughly a tenth of per second second
delay in our reaction time and that’s
enough time for the light signal to go
across the entire earth
so in virtual reality if you’re
interacting with somebody else you don’t
actually know where they are on the
earth you feel fast enough signal but
emulations who run faster than humans
for example if they’re running sixteen
times faster than humans that need to be
within roughly a thousand kilometers
before they won’t notice where the other
one is for a signal to light and if they
run even faster the circle gets even
narrower or how close they have to be on
the other hand if they run too slow and
they train for a career then their
career will basically be over too fast
because the world changes so by doubling
every month so the sweet spot on the
trade-off between these two factors is
roughly a thousand times human speed so
I predict humans run roughly at a
thousand times human speed and that
means from their point of view their
world is more stable and changing less
than your world is for you this is my
last slide I know from previous
experience that you guys are eager to
evaluate this world and decide whether
you like it or not but think your
ancestors from a thousand or more years
ago if they had heard about your world
they would have loved it or hated it
depending on the first few things they
heard because your world is really just
weird
so maybe you should pause before you
evaluate this else think about it maybe
even read a book on it and but I’m gonna
summarize for the moment some of the
major pluses and minuses of this world
if you want to evaluate it first they
spend most time working and leisure in
virtual reality where there’s no pain
hunger disease grime their bodies are
always beautiful there’s a vast
population finding it life worth living
have you ever said I couldn’t live in a
small town because there’s not enough
going on there that’s what they say
about your cities because there’s a much
larger more intricate larger economy can
afford better art music and all sorts of
things on the other hand most of them
are living at Malthusian subsistence
wages which is how pretty much all
humans ever lived until a few hundred
years ago and how pretty much all
animals have ever lived so it’s not
strange hypothetical it’s the usual case
history there’s more inequality in this
world not just of wealth but also of
class and speed they probably have
larger more bureaucratic firms less
nature perhaps less space travel less
democracy more religion this is the age
of em remember it’s not my job to make
you love it or hate it just to tell you
what I think’s most likely to happen if
we did the least to avoid it if you
don’t like this world do something to
change it thank you