accept more than one reality in February
2015
something happened it had a divisive
impact on the world
people said it challenged their
objective truth Time magazine said it
changed the course of internet history
forever I’d like to call it a fashion
disagreement but there’s something more
important about this we have a really
hard time accepting that we experience
the physical world differently many of
you remember the image of this dress a
lot of us see it differently some people
see it as white and gold some people see
it as blue and black I happen to see it
as white and gold turns out I’m wrong
the color you can go buy in the store is
actually blue and black but it’s not
about the actual color of the dress
that’s not what matters what matter is
how our brain perceives the color in
this image people have looked at things
like demographics whether that drive
these differences does it matter if
you’re a boy or a girl young or old or
what kind of lighting environments
you’ve spent time in and at the core of
all of these is that how we experience
the world your brain is constantly
integrating information about your
life’s experiences in this case about
luminance shadow and color to make
probabilistic decisions and it turns out
that we all have slightly different
information going into that equation so
I can bias you if I change the context
to see this image differently hopefully
maybe we see it differently now but I
can also take it away and hopefully now
we all see it the same these kinds of
perceptual Mally abilities happen all
the time so I’m hoping we can take some
time and think about where there’s power
and opportunity in understanding these
differences I’m going to tell you some
stories
some are personal some are about art
some are about immersive technologies
but they’re all about the power of
gaining empathy to each other’s
experiences that if we can understand
how we can see hear and feel the exact
same information in the world in
entirely different ways we have an
opportunity we have an opportunity a
chance to move away from all of the
dehumanization that people talk about
and rehumanize our interactions and the
technologies we built and used to do
this I’ve been a violinist since I was
three I was nine the first time I knew
my I heard the world a little
differently I always assumed everyone
knew my refrigerator hum to be flat
could write down the notes of a song or
was like really uncomfortable if happy
birth they didn’t start on Adi and I
have absolute pitch not particularly
unique in this way there are plenty of
us who have this we hear sound a lot
like many of us see color but it does
mean we see we hear the world
differently it’s not good or bad
it just is consequently I’ve spent my
life as a neurophysiologist as a
technologist and as a musician trying to
understand these differences in
perception in the brain processes that
drive them as scientists we spend our
lives trying to understand and look for
the ways for the most similar I truly
believe that the innovation leaps and
interaction leaps we need to make are
going to come from understanding the
ways were the most different for example
so before my mom passed away she had
macular degeneration she would say
something to me like I can see but I
can’t see so it’s kind of a nonsensical
statement what do you do with something
like that macular degeneration typically
involves deterioration of the central
part of someone’s retina people with it
experience a blind spot in their central
vision I get ocular migraines the types
of migraines I get a wave of innervation
travels through my brain and my brain
areas important for my central vision
when those cells are being innovated by
that wave they can’t respond to
information in the external world and I
develop a blind spot in my central
vision it looks a lot like this if you
try reading and tracking the yellow dot
you’ll see this is this is a lot like
the experience I have so for 15 minutes
a year when I’m having this type of
in order to see something I have to look
away from it when she says to me I
concede what I can’t see I get it
I can empathize and I change I changed
the way I think about innovations to
help her life I spent time building
contraptions that magnified what she was
looking away from instead of what she
was looking at and I fundamentally think
differently about the types of
interactions I can have with her to
connect it’s this place where a headache
and messed up vision that so many we
spend our times trying to avoid actually
gave me invaluable insight to my mom’s
world but it’s not always that easy to
see both sides
sometimes we’re doing it and we don’t
even know it unfortunately though no
truly knowing what someone is hearing
feeling or seeing is is difficult it’s
difficult to know when it’s happening
and even when it is most of us don’t
even know how to describe it but there
are people who are really good at
describing what they see and it gives us
this opportunity to peek into their
sensory world and ways we might not even
realize there’s a condition called
facial dysmorphia so people that have
facial dis Marfatia don’t see faces
naturally when they’re looking at a face
the features appear to move so
everywhere their gaze is directed they
develop a blind spot and it fills them
with nearby information this gives the
illusion of sort of movement of the
faces and sort of contorted and
distorted figures I’m going to show you
an example of that
I was only willing to do this to my face
that’s why you get to look at me but
if you stare at the little cross I may
follow the yellow crosses and you’ll
sort of get an idea of what this
experience might look like for someone
with this condition okay so you saw that
it also kind of looks like this Francis
Bacon was 20th century painter very well
known for painting abstract art the
import rates that were a contorted sort
of distorted figures it turns out
Francis Bacon had a condition that was
likely facial dysmorphia so when he was
sharing what we look at the art in this
case he was actually representing how he
truly saw the world Monet’s water lilies
so this is something I noticed when I
was looking at this painting that I
thought was interesting so if you stare
I find that when I stare at one of the
lilies and I close one I go ahead and do
that
I actually the lily seems to pop out
more almost as if it’s in 3d okay so I’m
not an art historian art critic but I am
a scientist and when I see things like
this I got ahead of hypotheses about
Monet’s vision there are different
pieces of information ways that we
accuse we take in from the world to be
able to know where an object is in space
and depth in particular some of those
cues make use of one eye we call them
monocular some of those cubes make use
of two eyes and we call them binocular
cues like color shading and occlusion
whether one objects in front of the
other all of those can we can get from
one eye but cues that require two eyes
something like relative disparity which
is actually a very important cue
required that both eyes are functioning
functioning normally so I had a
hypothesis that Monet actually couldn’t
make use of the information coming from
both eyes turns out Monet had very bad
cataracts and when you started looking
around his cataracts were so bad that
when he
painting water lilies he actually had
one of the cataracts removed and this is
a time when we didn’t trip late when
lenses weren’t replaced so money
couldn’t have made use of the
information from two eyes but the
information from one eye was likely very
developed at making use of that
information so when you look at this
painting now and you look through one
eye you really are
seeing it more like Monet there’s a
power to seeing both sides but we don’t
always get the clairvoyance of the Manta
Magritte’s painting but we can accept
and understand the power perspective has
on perception so I’m going to show you
an example where something as simple as
changing your gaze completely changes
how you experience the world I want you
all to look up to the right hand corner
and you’re gonna there’s some sunlight
in the water and you should see bubbles
that are sort of rising up to the
surface if you’re staring at that Sun
like those bubbles probably appear to
sort go up and fly off the screen does
everyone see that and they fly off the
screen kind of at an angle now shift
your gaze to where the bubbles are okay
so now you probably realize those
bubbles are going straight up you can go
back and forth this experience happens
because our brain has a trouble
resolving conflicting information about
local and global motion the bubbles have
a horizontal grating that’s moving
across them while they’re rising
vertically when your gaze is directed at
the bubbles we can separate them and we
can experience them both as they are but
when your gaze is directed away we
experience an integration of that
but of course I like to put a bigger
message on it so there are a few things
here it’s the power of perspective
something as simple as moving my gaze
completely changes how I experience
what’s in front of me and when I my gaze
when my perspective is directed in one
place or another I can’t choose to see
it differently even though I’ve had the
experience before I don’t get to I know
it’s there but I can’t experience the
same way my perspective determines how I
experience it but I do have the power to
choose to know that someone else might
be experiencing it’s that way right now
the human problem is different and it’s
there’s a lot of opacity here we don’t
always know the state of our biology and
our brains constantly change things like
hearing loss or changes in our vision
these are happening all the time to our
lower sensory processes but even things
like our environments affect our context
and affect our sensitivities whether we
grew up in a village in Afghanistan or a
town in the Midwest or in a city like
New York or Kathmandu with different
pressures neuroplasticity helps us take
the information that’s statistically
prevalent and relevant in our
environments and make use of it so that
we change to adapt to our environments
things like color shading contour all
tried this cultural language differences
and the categorical boundaries we form
can cause us to have different
experiences at the most basic level of
whether we see green or blue some are
more obvious some less so and sometimes
we actually get an opportunity to try on
each other sensory experiences we did
this with a pair of arthritis simulation
gloves what we wanted to do here was see
if we could take a group of
non-designers and give them an
empathetic experience of their target
user group to work with and
it would change how they thought about
the problem how would it change their
cognitive approach to the problem the
group with the gloves did change they
thought about simplicity in their design
they thought about space they got rid of
things like drop-down menus which are
you know completely terrible for anyone
with digit mobility issues they changed
how they thought about the problem and
they changed what they thought was
important immersive technologies like
augmented and virtual reality are really
starting to try to bridge this gap to
empathy in the case of the arthritis
gloves experience was powerful it gave
them an empathetic experience that was
actionable in what they did but we have
to go further we have to go further than
just showing experiences we have to
actually engage sensory experiences in a
way that is authentic and natural to how
our senses interact in the natural world
so let me give you an example of where I
when I realized how powerful that this
was if we did this right in we were
developing some cond some new
technologies adult imaging technologies
at my company one of the things we were
looking at was the impact of brightness
just so you realize a typical display
you would have bought three years ago
was 310 dollars per square meter the
natural moon is about one to two
thousand candelas we were looking at
luminances much brighter than that and
we were looking at fire content so we
were watching this fire content and
something happened people reacted I’ve
reacted in ways I hadn’t predicted just
by seeing an image of fire and watching
this my body started to expel heat as if
it were real we took we could take
thermal imaging cameras and measure
changes in the temperature on people’s
faces we could the screen wasn’t
changing but we were reacting to the
content in a way that was natural
in this case all my brain knew was it
had never experienced fire that was that
right and wasn’t real so it did what it
did best and just based on the luminance
reaching my retina and my knowledge of
flame my body did what it does if we are
gonna build experiences to try to bridge
and share and gain empathy to each
other’s experiences we have to build
immersive acknowledged ease that bridge
this natural sensory experience so I’ve
been talking a lot about why we should
care about empathy and why we need to
have empathy but there is one problem
what we don’t get empathy unless we make
ourselves vulnerable and we don’t like
to do that Beethoven lived in fear that
people would find out he was going deaf
and losing his hearing we have more and
more outlets which we all use where we
get to curate who we are and what people
see about us but we know from game
theory prisoner’s dilemma Nash
equilibrium that we end up in a
suboptimal state not as good as we could
do if we don’t share our vulnerabilities
so to get to the place where we want to
be we have to be able to empathize with
others vulnerabilities but we also have
to be willing to share our own so the
next time you interact with someone and
it doesn’t make sense to the way you see
the world it challenges it maybe they
challenge you rather than assuming they
have the same information to make
decisions about the world that you do
assume they don’t assume it’s different
empathy and experience are powerful and
they give us an opportunity to change we
can change our behaviors we can change
our technologies we can change our
assumptions about why someone might be
interacting with us in a way that we
don’t particularly like we need to
change our perspective if you could see
hear or feel someone else’s
alternate reality today and question
your own perceptions what will you
change thank you [Applause]