over the years we’ve been trained to
navigate to different realities from the
moment we are born we learn to explore
our world through our senses were able
to touch to see to hear we can even
taste our surrounded at the same time
most of us have been have learned how to
use computer at some point in our lives
we look at things like this and we
understand what each icon with each
glyph will do it with if we interact
with it these two worlds have coexisted
parallel to each other for a really long
time and for I guess for the most part
in this room we all have been trained in
technology after we have trained to
actually understand our surroundings
however new generations they don’t
really know this distinction unlike me
newborns I introduced the technology
very early on in their lives they’re
learning both worlds at the same time
and not necessarily in the same order
that we did this is first as a picture
of my son I capture three weeks ago when
I realized he was mesmerized just
looking at my wife – what I thought was
more interesting is there are only like
three minutes later I was able to
capture this other picture where he
seems to have this cover his hand for
the first time it was crazy he was he
was aware of the existence of an eye
from before he knew he has hands so with
evolution of technology the way we
interact with the world and the way we
interact with our digital environment is
getting closer and closer with VR for
instance when we about to cross the line
that divides or the separate is
basically the user and what they’re
interacting with we believe this trend
will continue in the future to a point
where we won’t be able to distinguish
you know what is real or what is digital
and especially because kids will learn
both worlds at the same time would
become we believe this distinction
between these two different realities
will become irrelevant so with that said
my team’s chart a microscope is to
anticipate and embrace this future
reality by underst
what pattern shape our perception of the
real world and how could we take
advantage of them today in our current
interfaces we flow in design we’re
trying to reinforce that connection that
could exist between our real world and
out to the interfaces to solve real user
experience problems like hierarchy focus
or continuity long story short we want
you to be able to use what you know from
this world when you’re using your
personal computer and this is basically
about leveraging this concept of natural
user interfaces by Bill Buxton we
believe that if we’re able to exploit
the skills that our users have acquire
in the world over a lifetime we’ll be
able to reduce the cognitive load that
takes to digest a new interface and
therefore will minimize the destruction
so with that said I would like to walk
you through three different examples
that have inspired the team to actually
reinforce this connection between these
two different worlds I’m gonna be
talking about color and deaf materials
on light and I’ll finish with sounds so
regarding colors color and depth
something that inspired a team to make
technology more natural is basically is
how we perceive the world and what
properties of that spatial understanding
could be actually leveraged to perform
better in a 2d interface so in the
through in the real world
everything is distributed in a 3d
environment right the hierarchy of the
objects around you is indicated through
properties like scale distance perceive
motion of the contextual value looking
at this picture you should be worried
about bumping in this put into this
person but you should be probably paying
more attention to the red light the
green light that is letting you know
that you can still cross the crosswalk
safely if we bring this closer to what
will be a workstation in a in the real
world if you’re like sitting working
with certain things the hierarchy of
objects is communicated via the distance
like the distance between the object and
the subject you could basically conclude
that the objects that are closer to the
subject those that are being used more
often pursues the others that are like
or in the background you know at this
point there are more like props so if we
bring this even closer to actually you
know the pattern that we’ve been trained
for the last three years of computing of
this desktop metaphor and 2d planes that
basically hosts different apps and we
place that in a 3d environment you can
see that there are certain effects that
already start to happen and we thought
that was actually pretty interesting
so in the real world distance equals
scale and the fulfill and well that is
something that we don’t even realize
that we’re scanning a room for the first
time it doesn’t really work if you put
that on a 2d interface it suddenly
creates a really a linear way to browse
and perform actions therefore it doesn’t
allow for multitasking or it doesn’t let
you be really productive and get things
done so if we take those effects out of
the equation and we change the camera
from perspective to orthographic so you
actually don’t have that distortion in
space even though it looks a bit more
familiar to what we have today there are
certain lighting cues that we believe
we’re really interesting because it
already started communicating that with
the environment so our representation of
apps in the desktop metaphor could go
from something like this which is pretty
much what we have today in every OS out
there to something closer to this how
wider variety of smart shadows the first
communicate better dev and help indicate
hierarchy and facilitate focus by
putting upfront what you’re interacting
with the most so just like in the real
world but still keeping this window
based legacy that we’ve been trained to
understand for the last 20 years another
effect that we think is really
interesting is this phenomenon called
atmospheric perspective in the real
world the illumination of a scene
captures the colors of the environment
and applies them to the objects in that
environment with a slightly different
tint you see that if you’re trying to
take a picture of a rolling hill right
if you’re trying to take a picture of a
mountain of that is really far even on a
clear day you’re gonna see this blue
tent gets applied to the object right so
this is something that happens in the
real world but it’s been used for
centuries in fine arts right in the Mona
Lisa you see that in the background
as a way to make it seen feel more
integrated and subject and make sure we
focus our attention on what matters
which is the subject in this case so if
we apply that to our 2d windows we could
go from something like this well the
first thing that we need is a background
so we could use the same one we have
before and now if we apply this tent
again you can see how objects could
start getting pushed in the distance and
again this is the dominant color of the
composition so if we change that
background to the sabia warmer data will
change if it is cooler it will change as
well we believe that by using real
visual effects that match our real-world
perception like death and life in a 2d
environment
we hope to reduce the cognitive load
that takes to digest this complex modern
UI that we exposed every day help
increased focus and help communicate a
better hierarchy the second thing that I
like to talk about is materials on light
now that we have a better understanding
of you know what this environment is
like let’s pay attention to their
objects in the environment we believe
that there’s a gigantic opportunity no
one is really taking advantage of to
liberate some of those properties that
objects have in the real world you know
to the interfaces so in the real world
every single manufacturer object have
materials and shape that communicate
intent right you understand them just by
looking at their appearance or by
touching them you could close your eyes
touch something feel that the material
feels inviting and it’s meant to be used
materials communicates so much that you
can even understand that without looking
at them if we look at the evolution of
the treatment that we’ve used you know
from the beginning of graphic user
interfaces we’ve covered a wide variety
of different execution from extremely
minimalistic to a richer set of graphics
to a point where it became his cue
morphic to a point where we actually
flipped the pendulum the other way
around and everything became flat and
now we die we get into this dimensional
UI and it feels that even though we have
explore all these variations we haven’t
even scratched the surface of what
materials could do in a digital
environment so with that in mind the
team actually started exploring
what material properties could be useful
in a in a user environment in that
either 2d or 3d environment right so
then started asking questions like you
know what truly defines the color of an
object how do different materials deal
with light and relate to each other in a
composition or how is our perception
getting distorted by different types of
materials especially if you start moving
those objects in 3d and something that
was pretty relevant to again modern UI
is how layer in and position will affect
the properties of those materials if you
cluster them on top of each other we’ll
realize that for materials to
communicate in a digital world as much
as they do in the real world we needed
to be we needed to pay attention to two
different things
the first is how our materials
represented how do they look and again
we’ve done lots of lots of different
explorations of what this pallet could
be and really only time and our users
will be able to validate certain
assumptions this is just the beginning
of the journey this is just a snapshot
in time but the idea the goal is to be
able to link these materials to UX
affordances so users can almost like
rely on muscle memory and understand
what things are meant to do just by
looking at them the second thing that we
needed to pay attention to is well it’s
not only how they look but how do they
interact with how do they feel when you
interact with them or when they interact
with other objects in the scene and you
can see for example here how it’s
impossible to really get a full idea of
what a material could be like until the
moment you see that material interacting
with for instance the environment in
this other example we call this reveal
materials reveal properties when you
actually shine line light on top of them
right so the cursor acts like a light
source and therefore only to reveal
certain properties when you’re
interacting with them we’re again which
again will declutter the UI and
facilitate focus and there’s so much we
could do in this case for instance we’re
trying to understand intent in the world
of UI normally we have been constrained
to a certain
assumptions that basically affect our
interaction so for instance we measure
things like oh you hover in this button
or you press in or you release in this
button but what if we could understand
user intent and only reveal the
properties of let’s say a button when
that button becomes relevant to the user
in the composition that way we could
actually declutter the UI and basically
help them focus on content over this
dance of cultivations competing with
each other there is so much we could do
with materials the world in the real
world materials communicate a lot
they’re extremely important and
essential to how we interact with the
real world yet when you sit down in your
computer it feels like you’re
compromising it feels that you live in a
cheap version of what that could be
the more we believe we can leverage
materials inauthentic digital way the
better we’ll serve our users and
especially as we transition to fully
immersive through the environment right
so the last thing that I’d like to talk
about is sounds in the context of
notifications sound is an extremely
important part of any design system yet
it’s normally overlooked after the
aesthetics and the behavior of it we
need sound if we want to create a
multi-sensory experience in the real
world
sound is key to locate ourselves he
provides feedback anticipation and it
gives us a spatial hierarchy it is
essential to how we navigate the world
just trying going out there with your
headphones let’s say noise cancelling
headphones you suddenly missing a
gigantic part of what it takes to what
it makes you basically replacing in this
world sound is also essential to how we
communicate with others and how we
communicate with again other parts of
the world beyond the specific sound of
words every spoken language has musical
patterns and that melody and rhythm has
an emotional meaning that has the power
to create an emotional connection so the
sound team actually worked on this idea
of analyzing the speech contour of a
different set of languages to try to
synthesize what could be a universal
pattern
that could be meaningful to everybody
beyond the culture or country of origin
or their language the hypothesis was
that if we can define the foundation of
our sound system based on the emotional
connection we have with others and with
the world we’re going to be able to
bring this gigantic piece of emotional
connection we have with the real world
in our digital interfaces and that is
really powerful we’ll be able to talk to
a global audience as well so what I want
to do now I want to share a few examples
of that this is for instance the
reminder sound on Windows already and
this is basically the sentence they
selected to come up with that sound and
we’re gonna listen to that sentence in
different languages so this is how it
sounds in English ready to go Japanese
yo edict our mantra Mahalo MA
in Spanish it’s does lista
so hopefully you were able to already
figured out the melody identify the
melody but just in case let’s play all
of them at the same time I’ll be crazy
alright so from there hopefully doesn’t
happen but from there this is the
abstract contour the team came up with
ready to go and this is how it sounds
your desktop machine
another example is new message if you
get a new message in Windows this is
basically this is how they got there so
new message in English new message
French Nouvelle massage and Russian Nova
/ Sonya and this is the abstract contour
now how it sounds in your desktop
alright so all the other examples are
for instance getting a new mail and the
center was message for you background
alerts everything is good and for
Granollers place it now so this really
all these sounds they already exist in
Windows today I encourage you to go and
experience there for yourself
so finally hold on a second what a
second I still have one minute this is
just really everything that I just
talked about those are just really a few
examples of what we’re aiming for right
we believe that by tapping into the
human innate engagement that we have
with the real world but in a digitally
authentic way we’re gonna be able to
create experiences that feel lighter
more natural and actually that leverage
our knowledge of the world again
remember since then from the moment we
were born we are hardwired to understand
everything around us we have we’re
equipped to actually explore this world
and understand what we have to do right
so I guess I’ll take away from me here
for all the creators in in the room and
that might be watching this talk later
is that it’s important that we as
creators look beyond the screen and get
back in touch with those what shapes our
perception of the world which is
ultimately what makes us human
thank you so much
you
navigate to different realities from the
moment we are born we learn to explore
our world through our senses were able
to touch to see to hear we can even
taste our surrounded at the same time
most of us have been have learned how to
use computer at some point in our lives
we look at things like this and we
understand what each icon with each
glyph will do it with if we interact
with it these two worlds have coexisted
parallel to each other for a really long
time and for I guess for the most part
in this room we all have been trained in
technology after we have trained to
actually understand our surroundings
however new generations they don’t
really know this distinction unlike me
newborns I introduced the technology
very early on in their lives they’re
learning both worlds at the same time
and not necessarily in the same order
that we did this is first as a picture
of my son I capture three weeks ago when
I realized he was mesmerized just
looking at my wife – what I thought was
more interesting is there are only like
three minutes later I was able to
capture this other picture where he
seems to have this cover his hand for
the first time it was crazy he was he
was aware of the existence of an eye
from before he knew he has hands so with
evolution of technology the way we
interact with the world and the way we
interact with our digital environment is
getting closer and closer with VR for
instance when we about to cross the line
that divides or the separate is
basically the user and what they’re
interacting with we believe this trend
will continue in the future to a point
where we won’t be able to distinguish
you know what is real or what is digital
and especially because kids will learn
both worlds at the same time would
become we believe this distinction
between these two different realities
will become irrelevant so with that said
my team’s chart a microscope is to
anticipate and embrace this future
reality by underst
what pattern shape our perception of the
real world and how could we take
advantage of them today in our current
interfaces we flow in design we’re
trying to reinforce that connection that
could exist between our real world and
out to the interfaces to solve real user
experience problems like hierarchy focus
or continuity long story short we want
you to be able to use what you know from
this world when you’re using your
personal computer and this is basically
about leveraging this concept of natural
user interfaces by Bill Buxton we
believe that if we’re able to exploit
the skills that our users have acquire
in the world over a lifetime we’ll be
able to reduce the cognitive load that
takes to digest a new interface and
therefore will minimize the destruction
so with that said I would like to walk
you through three different examples
that have inspired the team to actually
reinforce this connection between these
two different worlds I’m gonna be
talking about color and deaf materials
on light and I’ll finish with sounds so
regarding colors color and depth
something that inspired a team to make
technology more natural is basically is
how we perceive the world and what
properties of that spatial understanding
could be actually leveraged to perform
better in a 2d interface so in the
through in the real world
everything is distributed in a 3d
environment right the hierarchy of the
objects around you is indicated through
properties like scale distance perceive
motion of the contextual value looking
at this picture you should be worried
about bumping in this put into this
person but you should be probably paying
more attention to the red light the
green light that is letting you know
that you can still cross the crosswalk
safely if we bring this closer to what
will be a workstation in a in the real
world if you’re like sitting working
with certain things the hierarchy of
objects is communicated via the distance
like the distance between the object and
the subject you could basically conclude
that the objects that are closer to the
subject those that are being used more
often pursues the others that are like
or in the background you know at this
point there are more like props so if we
bring this even closer to actually you
know the pattern that we’ve been trained
for the last three years of computing of
this desktop metaphor and 2d planes that
basically hosts different apps and we
place that in a 3d environment you can
see that there are certain effects that
already start to happen and we thought
that was actually pretty interesting
so in the real world distance equals
scale and the fulfill and well that is
something that we don’t even realize
that we’re scanning a room for the first
time it doesn’t really work if you put
that on a 2d interface it suddenly
creates a really a linear way to browse
and perform actions therefore it doesn’t
allow for multitasking or it doesn’t let
you be really productive and get things
done so if we take those effects out of
the equation and we change the camera
from perspective to orthographic so you
actually don’t have that distortion in
space even though it looks a bit more
familiar to what we have today there are
certain lighting cues that we believe
we’re really interesting because it
already started communicating that with
the environment so our representation of
apps in the desktop metaphor could go
from something like this which is pretty
much what we have today in every OS out
there to something closer to this how
wider variety of smart shadows the first
communicate better dev and help indicate
hierarchy and facilitate focus by
putting upfront what you’re interacting
with the most so just like in the real
world but still keeping this window
based legacy that we’ve been trained to
understand for the last 20 years another
effect that we think is really
interesting is this phenomenon called
atmospheric perspective in the real
world the illumination of a scene
captures the colors of the environment
and applies them to the objects in that
environment with a slightly different
tint you see that if you’re trying to
take a picture of a rolling hill right
if you’re trying to take a picture of a
mountain of that is really far even on a
clear day you’re gonna see this blue
tent gets applied to the object right so
this is something that happens in the
real world but it’s been used for
centuries in fine arts right in the Mona
Lisa you see that in the background
as a way to make it seen feel more
integrated and subject and make sure we
focus our attention on what matters
which is the subject in this case so if
we apply that to our 2d windows we could
go from something like this well the
first thing that we need is a background
so we could use the same one we have
before and now if we apply this tent
again you can see how objects could
start getting pushed in the distance and
again this is the dominant color of the
composition so if we change that
background to the sabia warmer data will
change if it is cooler it will change as
well we believe that by using real
visual effects that match our real-world
perception like death and life in a 2d
environment
we hope to reduce the cognitive load
that takes to digest this complex modern
UI that we exposed every day help
increased focus and help communicate a
better hierarchy the second thing that I
like to talk about is materials on light
now that we have a better understanding
of you know what this environment is
like let’s pay attention to their
objects in the environment we believe
that there’s a gigantic opportunity no
one is really taking advantage of to
liberate some of those properties that
objects have in the real world you know
to the interfaces so in the real world
every single manufacturer object have
materials and shape that communicate
intent right you understand them just by
looking at their appearance or by
touching them you could close your eyes
touch something feel that the material
feels inviting and it’s meant to be used
materials communicates so much that you
can even understand that without looking
at them if we look at the evolution of
the treatment that we’ve used you know
from the beginning of graphic user
interfaces we’ve covered a wide variety
of different execution from extremely
minimalistic to a richer set of graphics
to a point where it became his cue
morphic to a point where we actually
flipped the pendulum the other way
around and everything became flat and
now we die we get into this dimensional
UI and it feels that even though we have
explore all these variations we haven’t
even scratched the surface of what
materials could do in a digital
environment so with that in mind the
team actually started exploring
what material properties could be useful
in a in a user environment in that
either 2d or 3d environment right so
then started asking questions like you
know what truly defines the color of an
object how do different materials deal
with light and relate to each other in a
composition or how is our perception
getting distorted by different types of
materials especially if you start moving
those objects in 3d and something that
was pretty relevant to again modern UI
is how layer in and position will affect
the properties of those materials if you
cluster them on top of each other we’ll
realize that for materials to
communicate in a digital world as much
as they do in the real world we needed
to be we needed to pay attention to two
different things
the first is how our materials
represented how do they look and again
we’ve done lots of lots of different
explorations of what this pallet could
be and really only time and our users
will be able to validate certain
assumptions this is just the beginning
of the journey this is just a snapshot
in time but the idea the goal is to be
able to link these materials to UX
affordances so users can almost like
rely on muscle memory and understand
what things are meant to do just by
looking at them the second thing that we
needed to pay attention to is well it’s
not only how they look but how do they
interact with how do they feel when you
interact with them or when they interact
with other objects in the scene and you
can see for example here how it’s
impossible to really get a full idea of
what a material could be like until the
moment you see that material interacting
with for instance the environment in
this other example we call this reveal
materials reveal properties when you
actually shine line light on top of them
right so the cursor acts like a light
source and therefore only to reveal
certain properties when you’re
interacting with them we’re again which
again will declutter the UI and
facilitate focus and there’s so much we
could do in this case for instance we’re
trying to understand intent in the world
of UI normally we have been constrained
to a certain
assumptions that basically affect our
interaction so for instance we measure
things like oh you hover in this button
or you press in or you release in this
button but what if we could understand
user intent and only reveal the
properties of let’s say a button when
that button becomes relevant to the user
in the composition that way we could
actually declutter the UI and basically
help them focus on content over this
dance of cultivations competing with
each other there is so much we could do
with materials the world in the real
world materials communicate a lot
they’re extremely important and
essential to how we interact with the
real world yet when you sit down in your
computer it feels like you’re
compromising it feels that you live in a
cheap version of what that could be
the more we believe we can leverage
materials inauthentic digital way the
better we’ll serve our users and
especially as we transition to fully
immersive through the environment right
so the last thing that I’d like to talk
about is sounds in the context of
notifications sound is an extremely
important part of any design system yet
it’s normally overlooked after the
aesthetics and the behavior of it we
need sound if we want to create a
multi-sensory experience in the real
world
sound is key to locate ourselves he
provides feedback anticipation and it
gives us a spatial hierarchy it is
essential to how we navigate the world
just trying going out there with your
headphones let’s say noise cancelling
headphones you suddenly missing a
gigantic part of what it takes to what
it makes you basically replacing in this
world sound is also essential to how we
communicate with others and how we
communicate with again other parts of
the world beyond the specific sound of
words every spoken language has musical
patterns and that melody and rhythm has
an emotional meaning that has the power
to create an emotional connection so the
sound team actually worked on this idea
of analyzing the speech contour of a
different set of languages to try to
synthesize what could be a universal
pattern
that could be meaningful to everybody
beyond the culture or country of origin
or their language the hypothesis was
that if we can define the foundation of
our sound system based on the emotional
connection we have with others and with
the world we’re going to be able to
bring this gigantic piece of emotional
connection we have with the real world
in our digital interfaces and that is
really powerful we’ll be able to talk to
a global audience as well so what I want
to do now I want to share a few examples
of that this is for instance the
reminder sound on Windows already and
this is basically the sentence they
selected to come up with that sound and
we’re gonna listen to that sentence in
different languages so this is how it
sounds in English ready to go Japanese
yo edict our mantra Mahalo MA
in Spanish it’s does lista
so hopefully you were able to already
figured out the melody identify the
melody but just in case let’s play all
of them at the same time I’ll be crazy
alright so from there hopefully doesn’t
happen but from there this is the
abstract contour the team came up with
ready to go and this is how it sounds
your desktop machine
another example is new message if you
get a new message in Windows this is
basically this is how they got there so
new message in English new message
French Nouvelle massage and Russian Nova
/ Sonya and this is the abstract contour
now how it sounds in your desktop
alright so all the other examples are
for instance getting a new mail and the
center was message for you background
alerts everything is good and for
Granollers place it now so this really
all these sounds they already exist in
Windows today I encourage you to go and
experience there for yourself
so finally hold on a second what a
second I still have one minute this is
just really everything that I just
talked about those are just really a few
examples of what we’re aiming for right
we believe that by tapping into the
human innate engagement that we have
with the real world but in a digitally
authentic way we’re gonna be able to
create experiences that feel lighter
more natural and actually that leverage
our knowledge of the world again
remember since then from the moment we
were born we are hardwired to understand
everything around us we have we’re
equipped to actually explore this world
and understand what we have to do right
so I guess I’ll take away from me here
for all the creators in in the room and
that might be watching this talk later
is that it’s important that we as
creators look beyond the screen and get
back in touch with those what shapes our
perception of the world which is
ultimately what makes us human
thank you so much
you
Please follow and like us: