truth is reality this idea goes through
the whole history of philosophy already
are it’s totally said to say of what is
then it is is true and he again most
certainly adapt to the idea from Plato
as all do for millennia
scientists have been breaking their
heads about how to establish unity
between the thing in itself how we
perceive it and how we then speak about
it a quest for the most efficient and
effective representation of the external
world a representation that helps to
forage for information support
decision-making with analytics generate
feedback automatically create
personalized learning environments and
many many other applications but what if
we ask the wrong questions what if
instead of trying to represent reality
we could actually make reality what are
the rules of the spiel when we treat
reality as a medium the key to unlock
reality as a medium is certainly found
in perception so let’s untangle that a
little bit perception is an active
process a reconstruction process the
mind reconstructs reality through
perception this is evident in for
example the way the retina works
pre-processing of light beams happens
already on the retina some light beams
are executed some are inhibited the
primary visual cortex here at the back
of the brain processes shapes detail
color blobs angles and then it goes up
the what and where streams cross the
whole brain almost this is most visible
on optical illusions that is an active
process I don’t know if you can see it
already this is where the active comes
in
the process works in two directions it
works from the outside in but also from
the inside out
thinking influences perception think for
example of hallucinations the thing I
like most about this optical illusion is
that once you’ve seen it you cannot
unsee it our visual memory is just too
powerful if you haven’t seen it by now
you might see it now this is actually a
cigar sticking out of the wall
with at the left of the box a tip and at
the right of the box the end sticking in
the wall perception reconstructs reality
that’s why the arrow goes to the right
perception also helps to build
experience we have an episodic memory we
have a sensory semantic memory we’re a
muscle memory we have many different
types of memories if we want to hack
into this we have to hack into
perception and we can do so with
augmented reality and wearable
technologies we can put gateway devices
in between the real world and what we
perceive to help us build them different
types of experience there are many
different gateway devices available for
augmented reality they’re ready no
displays head-mounted displays their
handheld devices some of you may have
already played pokemon go on a mobile
phone there are spatial see-through
displays their projection based displays
many different types head mounted
displays also known as smart glasses
make possible an example like this
the little inset video shows the view
from the glasses a nice Halloween prank
er I’ve been told if you very strongly
believe in it and you walk through the
skeleton you feel a shiver running down
your spine if you believe if you believe
the other technology wearable technology
allows us to hack into additional senses
use additional sensors we can for
example use electromyography in an EMG
such as the tannic lab smile as you see
here on the picture the armband to
measure orientation acceleration and
muscle movement on the 8 pads to control
interaction in the smart glasses here’s
another fun example that makes use of
this the lightsaber is enabled with
voice commands on and it is controlled
using the sensor fusion between the
optical tracking of the glasses with the
sender tracking of the armband of quite
fun as you can imagine yeah we have a
lot of fun in in my lab I should have
mentioned that what you’ve just seen in
this video is a flat 2d recording of
what is a 3d experience the light beams
from the object passed unfiltered
through the display unfiltered through
the glass of the smart glasses and
additional light beams are projected in
from the display if you tried this
lightsaber experience the only thing
that you would see that is not from the
real world is a lightsaber itself kind
of a floating shimmering hologram in
your field of view I guess that’s my dry
explanation for if you try it out for
real it is much cooler
clearly very entertaining but what about
if we use these superpowers for good
let’s take this to a workplace context
we have taken this technology to look
transport in Norway halfway up the
Arctic Circle to see if we can train
maintenance engineers of ambulance
airplanes with it we have first recorded
what an expert would do we have asked
them to Don the glasses and some
additional hardware record what they
would do and use a kind of think aloud
protocol to speak out what they are
actually doing this deliver the
explanation required we can clean up
this recording and make it available to
trainees to have them have the same
experience this is now what the trainee
sees following in the footsteps we have
also taken this to Italy to Genoa to
test a training procedure of a new
ultrasound feature with radiologists the
procedure was about checking the main
karate’d to see if there are any signs
of clogging using the ultrasound and
using some some Doppler measurements
that I don’t fully understand and I’m
not going to explain but with the help
of this example I will tell you a little
bit more about the authoring that we
have created kind of a reality editor
that allows us to use in instructional
design tools beyond the ghost tracking
and to think aloud that you’ve already
seen to create these experiences in situ
in the place you can see here this
radial menu that allows the author to
select different types of annotations
for example
textual annotation for example audio
recordings or also the ability to place
3d content or to place kind of a visual
language of what people are allowed to
touch and where they should look and
what they’re not allowed to do this
makes authoring of real-world
experiences using augmented reality
fairly easy we have also tested this in
terrine in Italy with astronaut trainers
in a module of the International Space
Station on the ground to test the guided
assembly of a storage rack this is what
it looks like now from the perspective
of the trainee the trainee uses voice
interaction to bring us to a task list
and then go step by step through this
linear process receiving the instruction
where required being guided to the right
locations getting videos images pointers
icons that guide through the successful
assembly of such a storage rack it could
be any other part that is up there on
the International Space Station
hopefully successfully but hopefully
also providing the training that is
needed to survive in extreme
environments in this case you can see
how our actor is demonstrating the
procedure that we have tested let me
come to the conclusion I say we can
close the dissociative gap between
knowledge and its practical application
by putting representations into the real
world yes we can make reality treat
reality as a knowledge medium at least
if done well we can commit to accepting
this as a form of reality
maybe not the real real but the new real
and if we can make reality why don’t we
just stop there
maybe we do not have to produce
everything we want maybe we don’t have
to print a handbook not manufacture an
object that we want to try reuse a
physical button for multiple functions
maybe we can save the scarce resources
of this planet by sometimes having
something super real rather than real
making the world a little bit more
sustainable thank you
[Applause]
the whole history of philosophy already
are it’s totally said to say of what is
then it is is true and he again most
certainly adapt to the idea from Plato
as all do for millennia
scientists have been breaking their
heads about how to establish unity
between the thing in itself how we
perceive it and how we then speak about
it a quest for the most efficient and
effective representation of the external
world a representation that helps to
forage for information support
decision-making with analytics generate
feedback automatically create
personalized learning environments and
many many other applications but what if
we ask the wrong questions what if
instead of trying to represent reality
we could actually make reality what are
the rules of the spiel when we treat
reality as a medium the key to unlock
reality as a medium is certainly found
in perception so let’s untangle that a
little bit perception is an active
process a reconstruction process the
mind reconstructs reality through
perception this is evident in for
example the way the retina works
pre-processing of light beams happens
already on the retina some light beams
are executed some are inhibited the
primary visual cortex here at the back
of the brain processes shapes detail
color blobs angles and then it goes up
the what and where streams cross the
whole brain almost this is most visible
on optical illusions that is an active
process I don’t know if you can see it
already this is where the active comes
in
the process works in two directions it
works from the outside in but also from
the inside out
thinking influences perception think for
example of hallucinations the thing I
like most about this optical illusion is
that once you’ve seen it you cannot
unsee it our visual memory is just too
powerful if you haven’t seen it by now
you might see it now this is actually a
cigar sticking out of the wall
with at the left of the box a tip and at
the right of the box the end sticking in
the wall perception reconstructs reality
that’s why the arrow goes to the right
perception also helps to build
experience we have an episodic memory we
have a sensory semantic memory we’re a
muscle memory we have many different
types of memories if we want to hack
into this we have to hack into
perception and we can do so with
augmented reality and wearable
technologies we can put gateway devices
in between the real world and what we
perceive to help us build them different
types of experience there are many
different gateway devices available for
augmented reality they’re ready no
displays head-mounted displays their
handheld devices some of you may have
already played pokemon go on a mobile
phone there are spatial see-through
displays their projection based displays
many different types head mounted
displays also known as smart glasses
make possible an example like this
the little inset video shows the view
from the glasses a nice Halloween prank
er I’ve been told if you very strongly
believe in it and you walk through the
skeleton you feel a shiver running down
your spine if you believe if you believe
the other technology wearable technology
allows us to hack into additional senses
use additional sensors we can for
example use electromyography in an EMG
such as the tannic lab smile as you see
here on the picture the armband to
measure orientation acceleration and
muscle movement on the 8 pads to control
interaction in the smart glasses here’s
another fun example that makes use of
this the lightsaber is enabled with
voice commands on and it is controlled
using the sensor fusion between the
optical tracking of the glasses with the
sender tracking of the armband of quite
fun as you can imagine yeah we have a
lot of fun in in my lab I should have
mentioned that what you’ve just seen in
this video is a flat 2d recording of
what is a 3d experience the light beams
from the object passed unfiltered
through the display unfiltered through
the glass of the smart glasses and
additional light beams are projected in
from the display if you tried this
lightsaber experience the only thing
that you would see that is not from the
real world is a lightsaber itself kind
of a floating shimmering hologram in
your field of view I guess that’s my dry
explanation for if you try it out for
real it is much cooler
clearly very entertaining but what about
if we use these superpowers for good
let’s take this to a workplace context
we have taken this technology to look
transport in Norway halfway up the
Arctic Circle to see if we can train
maintenance engineers of ambulance
airplanes with it we have first recorded
what an expert would do we have asked
them to Don the glasses and some
additional hardware record what they
would do and use a kind of think aloud
protocol to speak out what they are
actually doing this deliver the
explanation required we can clean up
this recording and make it available to
trainees to have them have the same
experience this is now what the trainee
sees following in the footsteps we have
also taken this to Italy to Genoa to
test a training procedure of a new
ultrasound feature with radiologists the
procedure was about checking the main
karate’d to see if there are any signs
of clogging using the ultrasound and
using some some Doppler measurements
that I don’t fully understand and I’m
not going to explain but with the help
of this example I will tell you a little
bit more about the authoring that we
have created kind of a reality editor
that allows us to use in instructional
design tools beyond the ghost tracking
and to think aloud that you’ve already
seen to create these experiences in situ
in the place you can see here this
radial menu that allows the author to
select different types of annotations
for example
textual annotation for example audio
recordings or also the ability to place
3d content or to place kind of a visual
language of what people are allowed to
touch and where they should look and
what they’re not allowed to do this
makes authoring of real-world
experiences using augmented reality
fairly easy we have also tested this in
terrine in Italy with astronaut trainers
in a module of the International Space
Station on the ground to test the guided
assembly of a storage rack this is what
it looks like now from the perspective
of the trainee the trainee uses voice
interaction to bring us to a task list
and then go step by step through this
linear process receiving the instruction
where required being guided to the right
locations getting videos images pointers
icons that guide through the successful
assembly of such a storage rack it could
be any other part that is up there on
the International Space Station
hopefully successfully but hopefully
also providing the training that is
needed to survive in extreme
environments in this case you can see
how our actor is demonstrating the
procedure that we have tested let me
come to the conclusion I say we can
close the dissociative gap between
knowledge and its practical application
by putting representations into the real
world yes we can make reality treat
reality as a knowledge medium at least
if done well we can commit to accepting
this as a form of reality
maybe not the real real but the new real
and if we can make reality why don’t we
just stop there
maybe we do not have to produce
everything we want maybe we don’t have
to print a handbook not manufacture an
object that we want to try reuse a
physical button for multiple functions
maybe we can save the scarce resources
of this planet by sometimes having
something super real rather than real
making the world a little bit more
sustainable thank you
[Applause]
Please follow and like us: