on a collaborative project together with
the ETH Zurich game Technology Center
and Disney Research about augmented
reality
so what’s augmented reality you’ve
probably heard this term together with
virtual reality these usually go
hand-in-hand
virtual reality is the idea where you
put on a virtual reality headset and
you’re transformed into the virtual
world augmented reality is kind of the
flipside of this idea augmented reality
takes virtual objects and brings them
into your world these are very powerful
concepts and what I’ve come to learn
over the past years with our experiments
is that the real winning proposition
here the real transformative aspect of
this technology is the ability it has to
enhance creativity so why is that that’s
really my core message and if you think
about it over the past thousands of
years you know humans we’ve developed
amazing tools to interact and create in
the real world and in more recent
history since the advent of computers we
have built interfaces within the
computers digital tools that allow us to
do things that seem like magic compared
to what was possible only 50 years ago
and the real promise of augmented
reality is to merge these two worlds to
merge the virtual and the physical and
it allows us to take the magic of what’s
possible in a digital tool and bring it
into the world around us and when you
combine that power of digital processing
with the intuitiveness of working in the
real world then there’s this
exponential increase and what’s possible
to create and that’s a really exciting
concept so in this bigger spectrum I
want to talk about one aspect that I’m
especially interested in which is the
idea of creative play this is the
concept of children interacting with
their physical environment in a playful
and creative way this is an important
part of early childhood and childhood
development but in fact because we have
such amazing content ready for
consumption there’s a risk and a fear
that children in the end become passive
consumers and they lose that direct
physical interaction with the
environment around us and I really
believe that augmented reality holds a
unique potential to impact a situation
by providing magical digital overlays on
top of traditional real world creative
activities and this is the concept we’re
calling augmented creativity where we
use augmented reality to enhance
creative play so to illustrate this idea
I want to show you several live demos on
stage using my iPad before I get started
I want to give a big personal thanks to
the team who helped put this together so
with that we are switching to my iPad
screen so what you see behind me is
actually exactly what I see on my screen
and I want to start with coloring so
this is one of childhoods most early
creative endeavors I have a coloring
book page here and when I view that page
our software recognizes the line art
that’s printed on it and displays this
digital character that matches the line
art that’s pretty cool already however
at the moment she’s blank because we
haven’t colored her so I’m going to do
some live coloring on her dress right
now just to add a little stripe there
and now
I tap the screen this software we’ve
written dynamically captures that
texture that I just colored and applies
it to the dress so in fact I have a
fully colored version that I made just
this morning
so let me capture this one
[Music]
and what’s colored on the front what you
see on the front is copied directly
however we also have an algorithm that
invents what I might have colored on the
back based on what I colored on the
front now coloring books also often are
accompanied with puzzles and I have here
maze and you see our system actually
validates that the maze was solved
correctly and displays this digital
character who follows the path that I’ve
drawn on the page if I had made a
mistake he would have in fact stop and
wait we also have a circle the word
puzzle so it checks that I’ve circled
the proper words and delivers this
reward for completing it correctly so
we’re not restricted to you know things
right in front of me on this table
there’s a painting beside me which is
something you might see at a museum
museums have a lot of cultural
importance but for children they can be
super boring they’re not the most
exciting experience Augmented creativity
can help with that so when I view this
painting with my app while we see it but
because we’re using the system we can do
some magical things so if I tap on the
screen we throw balls of paint and are
able to recolor this painting to create
something more exciting and I can even
go farther than that if I swipe we can
explore some new hairstyles
for this character let’s find one that’s
pretty good I can even move the nose
around a little bit oops let’s try to
make the eyes a little bigger and I
would say you know with these changes
he’s a pretty happy guy so this allows a
child to become their own Picasso when
they visit a museum we’ve also explored
bringing gameplay into this suite of
demos and for that I have some rubber
stamps so these are actually ones we
created in our office with our laser
cutter and I’m gonna show you how we can
meld games into the world around us
using these stamps and for that I’m
gonna just stamp right here on the paper
and when I create that stamp the system
actually recognizes what I’ve just done
and creates this corresponding character
that matches the stamp it even knew that
the ink I used was blue and colored the
character blue gonna make a green one
right now
and once this is done we actually can
implement a simple game where these two
and the full interaction really comes
from this physical kind of tangible
movement in the real world now these are
two different concepts while we have
visuals we saw we can create new
interaction what about other sensing
modalities so how about listening or
hearing and for that I have a number of
a number of cards on the table and these
cards have different instruments and
musical styles printed on them so the
first one is the vocals and when I view
this card with my iPad we display a
caring for my baby
how they visit around who’s now tree
forming for us he’s singing solo right
now so let’s give him some instruments
to accompany so just like that we have a
rock version of this song but we can
also bring in some other some other
instruments so now I’ve actually created
a reggae track pretty interesting what
I’m gonna add this add this string
quartet so we have a more classic feel
which is interesting but I think for
this group we should rather go full-on
techno
now the only thing missing is some
lighting effects really limitless
possibilities we can do with this
technology I can even for example bring
in the character that I colored earlier
to join the performance in a duet or
the theme today is people first but we
had a whole lot of fun
developing this technology over the past
years and putting together these demos
and I think it shows that you know
sometimes the most satisfying endeavor
is when you really think about putting
the children first thank you very much
[Applause] [Music]