we’re in an era now of technologies
advancing faster than they ever have
before raising important questions about
labor market disruptions but I’m going
to talk to you instead about how we can
leverage these technologies to learn
skills and ways that were never before
possible and unlock our full potential
and in particular I’m going to talk
about how we can digitize the master
apprentice relationship so it’s hard to
avoid today’s stories about AI and
machine learning advancing in their
capabilities and really reaching
unprecedented levels at the same time
robotics and automation systems have the
potential to streamline physical tasks
as you see here in this amazing video
from amazon fulfillment warehouse now in
some cases these change the nature of
the work that humans do but in other
cases they completely remove the humans
from the equation so with autonomous
vehicles alone it’s projected that this
is going to cost approximately 300,000
jobs to be lost per year once they reach
their peak yeah that’s a scary number
and it’s natural to be frightened about
this but we’re still quite a ways away
from AI and machine learning replacing
everything we do and robots taking over
our jobs so we’re not quite off the hook
yet instead and what we’ve learned in
the past is that these disruptive
technologies don’t take away our jobs
they change the nature of the work that
we do and in essence they create these
hybrid systems where humans and
technologies work together so in the
context of architecture which is where I
work at Autodesk research architects now
learn to use computer-aided design
software they no longer need to become
masters of physical drafting skills so
now is a time more than ever when we
need to have adaptable expertise often
learning new skills on the job and I’m
going to talk to you about how we can
digitize the master apprentice
relationship to do this now technology
guided learning is not
new and in fact even before computers
existed psychologists were developing
what they call teaching machines that
were thought to lead to an industrial
revolution of teaching it was to the
point where is even question whether the
learning experience would become fully
automated and our teachers would be
replaced by robots of course it’s a
vision that was never realized although
I always questioned about my own grade
for science teacher but the capabilities
of today’s technologies do allow us to
learn in ways that were never before
possible like I said I work at Autodesk
which makes some of the world’s most
advanced technology and software for
doing 3d design from towering
skyscrapers to special effects and
Hollywood movies it also happens to be
some of the most complex software to
learn in the world having hundreds and
often thousands of commands and
workflows that you need to become a
master of now with my colleagues at
Autodesk research in the user-interface
research group I’ve spent the last 10
years of my research trying to
understand how technology can aid in the
learning process in these complex
environments for example we can
instrument the software to track
everything you’re doing and by analyzing
what the experts are doing and what the
rest of the community is doing we can
recommend new commands and workflows for
you to use so with this project which we
called community commands the learning
is occurring in real time as you’re
working with the system personalized
recommendations are coming to you just
as if you were working right next to an
expert but I ask myself can we extend
this paradigm of what we call
over-the-shoulder learning beyond just
computer-based tasks can we use
technology to achieve mastery in the
physical realm and unlock our potential
in essence what I was asking is can we
digitize the master apprentice
relationship let’s look at a couple
examples to motivate this here’s an
athlete using a training apparatus to
improve their baseball swing now surely
this type of repetitive practice is
gonna have some benefits but he might be
making some errors and a device like
this is completely passive it’s
providing no feedback on the errors he’s
making and no guidance on specific
things that he could improve upon what I
want to do is to provide ways for
systems like this to provide a model of
mentorship so people can learn in the
midst of an activity let’s consider a
best-in-class example of this so what
this person needs is a mr. Miyagi this
is of course from the the movie The
Karate Kid and if you go back and watch
the scene or mr. Miyagi is teaching
Danielle all these all these different
movements it’s actually a really great
example of the rich interactions that
occur between a master and an apprentice
mr. Miyagi doesn’t just show the
movements he’s constantly providing
feedback and guidance to Daniel he’s
also creating this close personal
connection so we can truly understand
what it is that Daniel needs to learn
best now to add a personal touch to this
story I had my own mr. Miyagi perhaps a
unique hobby for a computer scientist I
trained in Muay Thai so this is the Thai
equivalent to kickboxing and there with
me is Simon Marcus who is my master or
anti crew now I was incredibly fortunate
to work with Simon he’s actually a
two-time world champion in kickboxing
and still an undefeated professional my
Thai fighter so Simon provided me all
the necessary things I needed to know to
learn the physical skills surrounding
MoIT I was also very clear from working
with Simon who seems very good at
understanding each individual student
and what their strengths and weaknesses
were and to teach accordingly and that’s
a fundamental aspect of the master
apprentice relationship and something
that doesn’t always occur and the
one-size-fits-all approach to education
and I’m sure every one of you here can
think about a mentor that you’ve had in
your life whether it was as a child or
even as an adult that’s had a
significant impact on who you are today
for me I was fortunate to work with
Simon now not everyone has their mr.
Miyagi or their Simon Marcus
they need them the most so the question
remains can we digitize and by doing so
democratize this master apprentice
relationship and I would argue that with
today’s emerging technologies we can do
just that so here’s some examples this
is a project we did call you move where
we used an augmented reality mirror to
teach physical skills such as dance and
athletics and get the feedback that you
typically typically get from a trainer
the system would capture your movements
with a 3d depth camera and then compare
them to the recorded movements of an
expert personalized feedback was then
projected back on the mirror and you’d
see it overlaid on your own physical
reflection applying this to the trades
this is a project we called the smart
makerspace which emerged digitized
learning content within a physical
makerspace environment as you worked on
a physical task the system would guide
you on how to use the required tools and
materials it would track your progress
through the tasks and update the
instructions in real time but can we
take this even further what about
large-scale construction like this can
we take the traditional mentorship model
and digitize it within this context with
our crowdsource fabrication project we
did exactly that so with this project we
had over a hundred volunteers
spontaneously come together and build
this 12-foot tall Pavilion structure
what was interesting about this project
is that the volunteers had no prior
knowledge of the of the project the
structure or the tools and materials
that were needed to build it instead we
use the combination of wearable devices
and sensors integrated in the
environment to provide real-time
learning here’s a brief overview of the
experience so when volunteers came to
the environment they were given a
SmartWatch and the SmartWatch provided
just-in-time
and location-aware instructions they
were first directed to gather three
pieces of bamboo and then take those
bamboo pieces to a robotic device where
they’d load them onto
special tool they would tie a string to
the bamboo and then the robot would
begin a special fabrication method to
create individual modules there were two
hundred and twenty four unique modules
in the construction of this pavilion now
this is showing a powerful concept where
again humans and technology are working
in a collaborative way together to do
something that would be difficult for
either of them to do independently once
a module was completed the human worker
would remove it from the robot and have
to figure out where it goes because each
module was unique so what we did was we
embedded wireless LEDs within the actual
materials that would illuminate and show
the worker exactly where their module
had to go this entire process was
coordinated by a back-end intelligent
agent which we called a digital Foreman
engine and in the end we had the world’s
first example of what we called
crowdsource fabrication where a group of
unskilled volunteers came came together
to build an architectural scale
structure using a combination of user
interface technologies now here’s where
this gets really exciting there’s
volunteer efforts across the world such
as Habitat for Humanity where volunteers
come together to construct housing for
low-income families imagine the
implications of each worker came to the
site and received the learning and
training resources they needed in real
time while they contributed to the build
another example that’s very relevant is
the construction and assembly of
shelters for refugees and asylum seekers
with new fabrication techniques today
there’s companies like better shelter
that are creating shelters that are more
robust and can provide families with
more security but the assembly is still
a major bottleneck and let’s just say
the instructions for assembling these
are suboptimal they’re literally like
following IKEA instructions to try to
build a house so the technologies I’ve
been showing you could potentially lead
to builds like this occurring in
fractions of the time and also giving
the workers lifelong skills that they
can carry forward the master apprentice
relationship is something that has been
around with us for centuries but with
these new technology
we have opportunities to learn in ways
that were never before possible we can
digitize the master apprentice
relationship and unlock our full
potential now my challenge to you going
forward for developers designers and
computer scientists is to think about
ways we can incorporate the personal
aspects of these master apprentice
relationships into our broader
interactions with technology thank you
[Applause]
you
advancing faster than they ever have
before raising important questions about
labor market disruptions but I’m going
to talk to you instead about how we can
leverage these technologies to learn
skills and ways that were never before
possible and unlock our full potential
and in particular I’m going to talk
about how we can digitize the master
apprentice relationship so it’s hard to
avoid today’s stories about AI and
machine learning advancing in their
capabilities and really reaching
unprecedented levels at the same time
robotics and automation systems have the
potential to streamline physical tasks
as you see here in this amazing video
from amazon fulfillment warehouse now in
some cases these change the nature of
the work that humans do but in other
cases they completely remove the humans
from the equation so with autonomous
vehicles alone it’s projected that this
is going to cost approximately 300,000
jobs to be lost per year once they reach
their peak yeah that’s a scary number
and it’s natural to be frightened about
this but we’re still quite a ways away
from AI and machine learning replacing
everything we do and robots taking over
our jobs so we’re not quite off the hook
yet instead and what we’ve learned in
the past is that these disruptive
technologies don’t take away our jobs
they change the nature of the work that
we do and in essence they create these
hybrid systems where humans and
technologies work together so in the
context of architecture which is where I
work at Autodesk research architects now
learn to use computer-aided design
software they no longer need to become
masters of physical drafting skills so
now is a time more than ever when we
need to have adaptable expertise often
learning new skills on the job and I’m
going to talk to you about how we can
digitize the master apprentice
relationship to do this now technology
guided learning is not
new and in fact even before computers
existed psychologists were developing
what they call teaching machines that
were thought to lead to an industrial
revolution of teaching it was to the
point where is even question whether the
learning experience would become fully
automated and our teachers would be
replaced by robots of course it’s a
vision that was never realized although
I always questioned about my own grade
for science teacher but the capabilities
of today’s technologies do allow us to
learn in ways that were never before
possible like I said I work at Autodesk
which makes some of the world’s most
advanced technology and software for
doing 3d design from towering
skyscrapers to special effects and
Hollywood movies it also happens to be
some of the most complex software to
learn in the world having hundreds and
often thousands of commands and
workflows that you need to become a
master of now with my colleagues at
Autodesk research in the user-interface
research group I’ve spent the last 10
years of my research trying to
understand how technology can aid in the
learning process in these complex
environments for example we can
instrument the software to track
everything you’re doing and by analyzing
what the experts are doing and what the
rest of the community is doing we can
recommend new commands and workflows for
you to use so with this project which we
called community commands the learning
is occurring in real time as you’re
working with the system personalized
recommendations are coming to you just
as if you were working right next to an
expert but I ask myself can we extend
this paradigm of what we call
over-the-shoulder learning beyond just
computer-based tasks can we use
technology to achieve mastery in the
physical realm and unlock our potential
in essence what I was asking is can we
digitize the master apprentice
relationship let’s look at a couple
examples to motivate this here’s an
athlete using a training apparatus to
improve their baseball swing now surely
this type of repetitive practice is
gonna have some benefits but he might be
making some errors and a device like
this is completely passive it’s
providing no feedback on the errors he’s
making and no guidance on specific
things that he could improve upon what I
want to do is to provide ways for
systems like this to provide a model of
mentorship so people can learn in the
midst of an activity let’s consider a
best-in-class example of this so what
this person needs is a mr. Miyagi this
is of course from the the movie The
Karate Kid and if you go back and watch
the scene or mr. Miyagi is teaching
Danielle all these all these different
movements it’s actually a really great
example of the rich interactions that
occur between a master and an apprentice
mr. Miyagi doesn’t just show the
movements he’s constantly providing
feedback and guidance to Daniel he’s
also creating this close personal
connection so we can truly understand
what it is that Daniel needs to learn
best now to add a personal touch to this
story I had my own mr. Miyagi perhaps a
unique hobby for a computer scientist I
trained in Muay Thai so this is the Thai
equivalent to kickboxing and there with
me is Simon Marcus who is my master or
anti crew now I was incredibly fortunate
to work with Simon he’s actually a
two-time world champion in kickboxing
and still an undefeated professional my
Thai fighter so Simon provided me all
the necessary things I needed to know to
learn the physical skills surrounding
MoIT I was also very clear from working
with Simon who seems very good at
understanding each individual student
and what their strengths and weaknesses
were and to teach accordingly and that’s
a fundamental aspect of the master
apprentice relationship and something
that doesn’t always occur and the
one-size-fits-all approach to education
and I’m sure every one of you here can
think about a mentor that you’ve had in
your life whether it was as a child or
even as an adult that’s had a
significant impact on who you are today
for me I was fortunate to work with
Simon now not everyone has their mr.
Miyagi or their Simon Marcus
they need them the most so the question
remains can we digitize and by doing so
democratize this master apprentice
relationship and I would argue that with
today’s emerging technologies we can do
just that so here’s some examples this
is a project we did call you move where
we used an augmented reality mirror to
teach physical skills such as dance and
athletics and get the feedback that you
typically typically get from a trainer
the system would capture your movements
with a 3d depth camera and then compare
them to the recorded movements of an
expert personalized feedback was then
projected back on the mirror and you’d
see it overlaid on your own physical
reflection applying this to the trades
this is a project we called the smart
makerspace which emerged digitized
learning content within a physical
makerspace environment as you worked on
a physical task the system would guide
you on how to use the required tools and
materials it would track your progress
through the tasks and update the
instructions in real time but can we
take this even further what about
large-scale construction like this can
we take the traditional mentorship model
and digitize it within this context with
our crowdsource fabrication project we
did exactly that so with this project we
had over a hundred volunteers
spontaneously come together and build
this 12-foot tall Pavilion structure
what was interesting about this project
is that the volunteers had no prior
knowledge of the of the project the
structure or the tools and materials
that were needed to build it instead we
use the combination of wearable devices
and sensors integrated in the
environment to provide real-time
learning here’s a brief overview of the
experience so when volunteers came to
the environment they were given a
SmartWatch and the SmartWatch provided
just-in-time
and location-aware instructions they
were first directed to gather three
pieces of bamboo and then take those
bamboo pieces to a robotic device where
they’d load them onto
special tool they would tie a string to
the bamboo and then the robot would
begin a special fabrication method to
create individual modules there were two
hundred and twenty four unique modules
in the construction of this pavilion now
this is showing a powerful concept where
again humans and technology are working
in a collaborative way together to do
something that would be difficult for
either of them to do independently once
a module was completed the human worker
would remove it from the robot and have
to figure out where it goes because each
module was unique so what we did was we
embedded wireless LEDs within the actual
materials that would illuminate and show
the worker exactly where their module
had to go this entire process was
coordinated by a back-end intelligent
agent which we called a digital Foreman
engine and in the end we had the world’s
first example of what we called
crowdsource fabrication where a group of
unskilled volunteers came came together
to build an architectural scale
structure using a combination of user
interface technologies now here’s where
this gets really exciting there’s
volunteer efforts across the world such
as Habitat for Humanity where volunteers
come together to construct housing for
low-income families imagine the
implications of each worker came to the
site and received the learning and
training resources they needed in real
time while they contributed to the build
another example that’s very relevant is
the construction and assembly of
shelters for refugees and asylum seekers
with new fabrication techniques today
there’s companies like better shelter
that are creating shelters that are more
robust and can provide families with
more security but the assembly is still
a major bottleneck and let’s just say
the instructions for assembling these
are suboptimal they’re literally like
following IKEA instructions to try to
build a house so the technologies I’ve
been showing you could potentially lead
to builds like this occurring in
fractions of the time and also giving
the workers lifelong skills that they
can carry forward the master apprentice
relationship is something that has been
around with us for centuries but with
these new technology
we have opportunities to learn in ways
that were never before possible we can
digitize the master apprentice
relationship and unlock our full
potential now my challenge to you going
forward for developers designers and
computer scientists is to think about
ways we can incorporate the personal
aspects of these master apprentice
relationships into our broader
interactions with technology thank you
[Applause]
you
Please follow and like us: