when I was studying at UMass Amherst in
2000 I was walking between classes when
with a friend when we saw some graduate
students walking around and the head his
large backpacks on and they had laptops
in the backpacks and that he’s like
ridiculously large head mounted display
units it was like a big bulky piece of
hardware sitting on this guy’s head and
he had these large glasses that looked
to be projecting something onto his face
it looked really cool and my my friend
sarcastically said to them are you from
the future and it was really
embarrassing but it turns out that they
were from the future and the future
being roughly 2017 to 2018 when
augmented reality started going
mainstream and that’s what I want to
talk about today
augmented reality and how it’s being
applied today and how it will be
practically applied to our everyday
lives in the near future so it’s an
exciting time for me to be in the mobile
development space from new hardware
being released often and different more
streamlined form factors faster
processors and better cameras the
ability to create very realistic
augmented and virtual reality
experiences is becoming well a reality
finally we can live those experiences
that we’ve seen for decades and read
about in science fiction but now in a
form factor that regular consumers
already have so before we dive too much
into augmented reality I just want to
give a quick background on what the
different realities are so virtual
versus augmented for starters the most
common ended in getting mainstream
attention in his virtual reality video
games have been the main driver for
adoption in this technology and
companies like HTC Sony and Facebook are
the biggest players in the space
currently a virtual reality experience
is a fully immersive experience where
your entire point of view has changed VR
uses goggles that cover your eyes and
soundproof headphones are used to create
a fully immersive experience by
essentially taking control of two or
three of your five senses you can travel
to new worlds you can experience things
that you never have before and you can
even feel deep emotions then there is
augmented reality which in my opinion
and we’ll have a lot more utility in our
day-to-day lives but maybe not as
dramatic so this video is not
necessarily a good example of the
utility of augmented reality since this
is just a game
and so that that’s my son he’s real he’s
not part of the AR experience but you
know it shows what an AR experience is
so essentially in augmented reality you
can see the real world through a head
mounted display or through your phone
and computer-generated images are
superimposed over the camera view the
field of computer vision and image
recognition has advanced significantly
since augmented reality start first
started being touted as a consumer ready
tech in this video example the phone I’m
holding is quickly scanning the ground
for horizontal services calculating the
distance from my phone’s camera to those
surfaces using the various servers using
the various movements of the phone using
internal hardware to track my movements
and then determining where 3d objects
can be rendered in the scene so the
amount of computations happening to
create this effect so smoothly has not
been possible on computers let alone
even on the smart phones we have today
so in addition to be able to track
surfaces and superimpose objects in the
scene these devices are also able to
like you know scan and recognize objects
and faces and various products very
quickly
over this past summer both Google and
Apple released major software updates
there their smartphone operating systems
to enable developers to create these
augmented reality experiences that are
already working tens of millions of
smartphones and that’s what makes this
year very exciting for augmented reality
after decades of speculation we now see
a path for this technology to go
mainstream now the final component to
augment reality that is poised to really
make this technology valuable in my
opinion is the ability to store detailed
3d information about a space in the
cloud or a remote server by having
detailed 3d models of interior spaces
stored in the cloud any of the mobile
devices anybody any of us with mobile
devices can retrieve that data about a
location that has been scanned now you
know this this may be like detailed
geometry about the interior of a concert
venue or an airport or an apartment
complex or even a store and
just store 3d information can then be
coupled with augmented reality to
display to display or guide us to our
seat at that concert venue or help us
find our baggage you know at the airport
or you know help first responders
navigate that apartment complex quickly
or even just help a shopper find an item
easily in the store so the challenge of
creating this AR cloud is this being
called is one of the one that’s being
worked on or rumored to be being worked
on by many big companies including
Google Microsoft and Apple so let’s talk
a little bit more about the utility of
augmented reality in our day to day
lives and where it’s being used now and
where it will be used the near future
there are already companies that have
employed augmented reality for things
like training engineering and
architectural visualization but you know
how will consumers actually use this and
expect to use augmented reality so from
my perspective I see immediate
application of this technology in retail
and tourism so brick-and-mortar retail
as we all know is struggling in the age
of Amazon and retailers are aggressively
looking for ways to improve that
experience whether it’s you know that
getting consumers to find their item
quicker and easier or keeping them
captivated and in the store longer and
one of the reasons that brick-and-mortar
retail struggles compared to online
shopping obviously is the difficulty of
searching and finding a particular item
you need you know online you can simply
just type in a search and you can add it
to your cart and you know wait a day or
two and I’ll show up in the mail however
consumers still want the convenience of
having particular products right away
and have the option to really look at an
item in person before making their
purchase as a first example of how a
major retailer is using augmented
reality let’s take a look at the Lowe’s
vision app so this app was created with
the Google tango team for a while and it
does an excellent job of showing what
can be done with augmented reality of
retail you can place items in your house
see how they’ll look design full room
layouts and then even search for their
products in the store using augmented
reality and then it’ll guide you to the
exact location on the exact shelf and
where on that shelf that product is that
you need it’s an amazing app and it’s
currently but it’s currently
– just a handful of devices that have
very specific google hardware in it
called project tango this hardware is
essentially a phone that has an infrared
camera an extra camera and death sensors
on it to scan a space in real time it’s
very expensive hardware with
unfortunately low adoption
however just recently with the release
of a our kit by Apple along with a slew
of new phones IKEA released its place
app for the iPhone this app uses
augmented reality more so to drive
purchase intent versus finding a
particular item in the IKEA store in
this app you get to browse the IKEA
catalog measure and place items in your
store and see how they look in AR you
can save and share pictures of the
furniture and in theory this will
increase your our desire for that piece
of furniture to order through the IKEA
IKEA store online or you know brave IKEA
in person which is a thing in itself so
let’s look at another hypothetical
example where AR can be used in retail
so I personally don’t know anything
about fixing cars or tools for cars and
I hate going to car parts store and you
know my dad’s in the audience which were
super embarrassed but like when I walk
in I know roughly what I want but I
don’t know what what size I want I will
particular tool is called and definitely
don’t know where it is in the store so
it’s said there’s already technology
available to make this process less
embarrassing for me but someone needs to
build it we can currently scan cars the
technology we have to determine what
make it is we can scan bolts to
determine what size they are and not
only can you walk into a car parts store
knowing you know exactly what you need
but because the interior of the store
can be scanned and stored in 3d your
phone can tell you exactly where their
product is and guide you to it with a
simple AR experience you actually look
like you know what you’re doing
so now you know now let me know you can
find certain products and complete tasks
within the store using AR and image
recognition is to scan improve our
shopping experience but let’s let’s look
a way we can use AR to design measure
price and shop for a landscaping project
at your home
so who here has ever used a ruler
hopefully everybody okay so with
Ottomans reality that could be a relic
of the past so by using augmented
reality you’re able to or soon will be
able to measure large areas using your
phone to a fraction of an inch in
addition you’ll be able to place items
in 3d within a real environment so for
example let’s say you want to build a
patio out your back door you can you
know use your imagination place posts in
the corners of your patio a measure the
area you want choose a brick pattern and
painful eerie measure and probably mess
it up or or I’d mess it up so or
alternatively you can use an AR app that
will allow you to essentially a place a
patio in AR space see what it look like
by your house select different brick
patterns to see what it will look like
and instantly get you know the
measurements the brick counts that you
need the cost is going to make to do
that and that way you can make your
purchase decision without having to go
to the store to price things look things
up measure and re measure and again
screw it up so in addition you can then
take this design save it to your phone
to the cloud visit a store and actually
pick up the materials you will need
including the bricks and the required
stone dust and as a bonus when you get
to the app to the store the output self
will actually guide you in AR to the
location of those bricks you wanted for
your patio so all of this requires a lot
less thinking which is kind of nice
phones have been graded absorbing our
focus on giving us 2d information about
some other place that is not necessarily
relevant to where we are this current
time with the advent of augmented
reality coupled with location-based data
from our phone and the intricate details
of interior space of any location that
has been scanned and stored in a cloud
apps will now not only be able to guide
us or a venue or other building but also
show us exactly where every location and
anything that has been tagged with the
interior of that location is so we
talked about retail let’s talk a little
bit about tourism so this is another
fantastic application of augmented
reality
and AR is already allowing us to visit
museums ancient ruins go on city tours
and engage with locations in new ways
and there are a few examples of this
working really well and improving our
overall visit experience so one of the
cooler examples I’ve seen in the museum
space is an app called skin and bone
that was created for the Smithsonian’s
Natural History Museum essentially this
app app uses image recognition you know
on your iPad or your iPhone to identify
skeletons at museums and through
augmented reality show what that
creature looked like when it was live
you can scan fish in various vertebrates
and see them come to life in full color
it’s really amazing I think you’ll see a
lot of other museums adopt this
technology in the future another amazing
implementation of augmented reality
that’s being tested is allowing us to
see ancient ruins for what they used to
look like
there are dozens of examples of apps
that are currently currently scan and
identify an ancient ruin by combining
our GPS location and the unique shape of
an ancient ruin then the app can
superimpose on top of it what that ruin
ruin looked like in all its glory so for
example the app may scan and recognize a
Parthenon
or Rome’s Colosseum and allow you to see
what it looked like 2500 or 2,000 years
ago so all of these examples both in
retail and tourism are apps that have
been available for some time or just
being released but you know where is
this technology going and why should you
why should you care so with the
introduction of mobile augmented reality
platforms this past summer we’re
starting to see a path to mass adoption
of this hardware and software that is
the computational power at a very small
scale that can handle these very
complicated applications so video games
like Pokemon go and a slew of others
that are just released on the Apple
Store now will continue to push the
adoption and understanding of this
technology for to more users then at the
same time this tech this the software is
improving and the adoption of technology
is increasing head-mounted display units
are being designed and developed that
are smaller and streamlined and dare I
say almost practical keep in mind 17
years ago when I was at UMass there are
people attempting to do
you just this and is taking this long to
get to the hololens which you know is
still not really that practical of a
device trust me
kind of crushes your head it’s cool but
yeah so however what I do see happening
in the next two to four years is we’ll
see a transition from smartphone based
augmented reality like the examples I’ve
shared to a streamlined head mounted
display unit that people might actually
want to wear until then I’d encourage
you to take a look so from the augmented
reality apps that are on the market
today see how they work and get familiar
with the technology it’s now mainstream
and is only going to become more
prevalent in our day to day lives
2000 I was walking between classes when
with a friend when we saw some graduate
students walking around and the head his
large backpacks on and they had laptops
in the backpacks and that he’s like
ridiculously large head mounted display
units it was like a big bulky piece of
hardware sitting on this guy’s head and
he had these large glasses that looked
to be projecting something onto his face
it looked really cool and my my friend
sarcastically said to them are you from
the future and it was really
embarrassing but it turns out that they
were from the future and the future
being roughly 2017 to 2018 when
augmented reality started going
mainstream and that’s what I want to
talk about today
augmented reality and how it’s being
applied today and how it will be
practically applied to our everyday
lives in the near future so it’s an
exciting time for me to be in the mobile
development space from new hardware
being released often and different more
streamlined form factors faster
processors and better cameras the
ability to create very realistic
augmented and virtual reality
experiences is becoming well a reality
finally we can live those experiences
that we’ve seen for decades and read
about in science fiction but now in a
form factor that regular consumers
already have so before we dive too much
into augmented reality I just want to
give a quick background on what the
different realities are so virtual
versus augmented for starters the most
common ended in getting mainstream
attention in his virtual reality video
games have been the main driver for
adoption in this technology and
companies like HTC Sony and Facebook are
the biggest players in the space
currently a virtual reality experience
is a fully immersive experience where
your entire point of view has changed VR
uses goggles that cover your eyes and
soundproof headphones are used to create
a fully immersive experience by
essentially taking control of two or
three of your five senses you can travel
to new worlds you can experience things
that you never have before and you can
even feel deep emotions then there is
augmented reality which in my opinion
and we’ll have a lot more utility in our
day-to-day lives but maybe not as
dramatic so this video is not
necessarily a good example of the
utility of augmented reality since this
is just a game
and so that that’s my son he’s real he’s
not part of the AR experience but you
know it shows what an AR experience is
so essentially in augmented reality you
can see the real world through a head
mounted display or through your phone
and computer-generated images are
superimposed over the camera view the
field of computer vision and image
recognition has advanced significantly
since augmented reality start first
started being touted as a consumer ready
tech in this video example the phone I’m
holding is quickly scanning the ground
for horizontal services calculating the
distance from my phone’s camera to those
surfaces using the various servers using
the various movements of the phone using
internal hardware to track my movements
and then determining where 3d objects
can be rendered in the scene so the
amount of computations happening to
create this effect so smoothly has not
been possible on computers let alone
even on the smart phones we have today
so in addition to be able to track
surfaces and superimpose objects in the
scene these devices are also able to
like you know scan and recognize objects
and faces and various products very
quickly
over this past summer both Google and
Apple released major software updates
there their smartphone operating systems
to enable developers to create these
augmented reality experiences that are
already working tens of millions of
smartphones and that’s what makes this
year very exciting for augmented reality
after decades of speculation we now see
a path for this technology to go
mainstream now the final component to
augment reality that is poised to really
make this technology valuable in my
opinion is the ability to store detailed
3d information about a space in the
cloud or a remote server by having
detailed 3d models of interior spaces
stored in the cloud any of the mobile
devices anybody any of us with mobile
devices can retrieve that data about a
location that has been scanned now you
know this this may be like detailed
geometry about the interior of a concert
venue or an airport or an apartment
complex or even a store and
just store 3d information can then be
coupled with augmented reality to
display to display or guide us to our
seat at that concert venue or help us
find our baggage you know at the airport
or you know help first responders
navigate that apartment complex quickly
or even just help a shopper find an item
easily in the store so the challenge of
creating this AR cloud is this being
called is one of the one that’s being
worked on or rumored to be being worked
on by many big companies including
Google Microsoft and Apple so let’s talk
a little bit more about the utility of
augmented reality in our day to day
lives and where it’s being used now and
where it will be used the near future
there are already companies that have
employed augmented reality for things
like training engineering and
architectural visualization but you know
how will consumers actually use this and
expect to use augmented reality so from
my perspective I see immediate
application of this technology in retail
and tourism so brick-and-mortar retail
as we all know is struggling in the age
of Amazon and retailers are aggressively
looking for ways to improve that
experience whether it’s you know that
getting consumers to find their item
quicker and easier or keeping them
captivated and in the store longer and
one of the reasons that brick-and-mortar
retail struggles compared to online
shopping obviously is the difficulty of
searching and finding a particular item
you need you know online you can simply
just type in a search and you can add it
to your cart and you know wait a day or
two and I’ll show up in the mail however
consumers still want the convenience of
having particular products right away
and have the option to really look at an
item in person before making their
purchase as a first example of how a
major retailer is using augmented
reality let’s take a look at the Lowe’s
vision app so this app was created with
the Google tango team for a while and it
does an excellent job of showing what
can be done with augmented reality of
retail you can place items in your house
see how they’ll look design full room
layouts and then even search for their
products in the store using augmented
reality and then it’ll guide you to the
exact location on the exact shelf and
where on that shelf that product is that
you need it’s an amazing app and it’s
currently but it’s currently
– just a handful of devices that have
very specific google hardware in it
called project tango this hardware is
essentially a phone that has an infrared
camera an extra camera and death sensors
on it to scan a space in real time it’s
very expensive hardware with
unfortunately low adoption
however just recently with the release
of a our kit by Apple along with a slew
of new phones IKEA released its place
app for the iPhone this app uses
augmented reality more so to drive
purchase intent versus finding a
particular item in the IKEA store in
this app you get to browse the IKEA
catalog measure and place items in your
store and see how they look in AR you
can save and share pictures of the
furniture and in theory this will
increase your our desire for that piece
of furniture to order through the IKEA
IKEA store online or you know brave IKEA
in person which is a thing in itself so
let’s look at another hypothetical
example where AR can be used in retail
so I personally don’t know anything
about fixing cars or tools for cars and
I hate going to car parts store and you
know my dad’s in the audience which were
super embarrassed but like when I walk
in I know roughly what I want but I
don’t know what what size I want I will
particular tool is called and definitely
don’t know where it is in the store so
it’s said there’s already technology
available to make this process less
embarrassing for me but someone needs to
build it we can currently scan cars the
technology we have to determine what
make it is we can scan bolts to
determine what size they are and not
only can you walk into a car parts store
knowing you know exactly what you need
but because the interior of the store
can be scanned and stored in 3d your
phone can tell you exactly where their
product is and guide you to it with a
simple AR experience you actually look
like you know what you’re doing
so now you know now let me know you can
find certain products and complete tasks
within the store using AR and image
recognition is to scan improve our
shopping experience but let’s let’s look
a way we can use AR to design measure
price and shop for a landscaping project
at your home
so who here has ever used a ruler
hopefully everybody okay so with
Ottomans reality that could be a relic
of the past so by using augmented
reality you’re able to or soon will be
able to measure large areas using your
phone to a fraction of an inch in
addition you’ll be able to place items
in 3d within a real environment so for
example let’s say you want to build a
patio out your back door you can you
know use your imagination place posts in
the corners of your patio a measure the
area you want choose a brick pattern and
painful eerie measure and probably mess
it up or or I’d mess it up so or
alternatively you can use an AR app that
will allow you to essentially a place a
patio in AR space see what it look like
by your house select different brick
patterns to see what it will look like
and instantly get you know the
measurements the brick counts that you
need the cost is going to make to do
that and that way you can make your
purchase decision without having to go
to the store to price things look things
up measure and re measure and again
screw it up so in addition you can then
take this design save it to your phone
to the cloud visit a store and actually
pick up the materials you will need
including the bricks and the required
stone dust and as a bonus when you get
to the app to the store the output self
will actually guide you in AR to the
location of those bricks you wanted for
your patio so all of this requires a lot
less thinking which is kind of nice
phones have been graded absorbing our
focus on giving us 2d information about
some other place that is not necessarily
relevant to where we are this current
time with the advent of augmented
reality coupled with location-based data
from our phone and the intricate details
of interior space of any location that
has been scanned and stored in a cloud
apps will now not only be able to guide
us or a venue or other building but also
show us exactly where every location and
anything that has been tagged with the
interior of that location is so we
talked about retail let’s talk a little
bit about tourism so this is another
fantastic application of augmented
reality
and AR is already allowing us to visit
museums ancient ruins go on city tours
and engage with locations in new ways
and there are a few examples of this
working really well and improving our
overall visit experience so one of the
cooler examples I’ve seen in the museum
space is an app called skin and bone
that was created for the Smithsonian’s
Natural History Museum essentially this
app app uses image recognition you know
on your iPad or your iPhone to identify
skeletons at museums and through
augmented reality show what that
creature looked like when it was live
you can scan fish in various vertebrates
and see them come to life in full color
it’s really amazing I think you’ll see a
lot of other museums adopt this
technology in the future another amazing
implementation of augmented reality
that’s being tested is allowing us to
see ancient ruins for what they used to
look like
there are dozens of examples of apps
that are currently currently scan and
identify an ancient ruin by combining
our GPS location and the unique shape of
an ancient ruin then the app can
superimpose on top of it what that ruin
ruin looked like in all its glory so for
example the app may scan and recognize a
Parthenon
or Rome’s Colosseum and allow you to see
what it looked like 2500 or 2,000 years
ago so all of these examples both in
retail and tourism are apps that have
been available for some time or just
being released but you know where is
this technology going and why should you
why should you care so with the
introduction of mobile augmented reality
platforms this past summer we’re
starting to see a path to mass adoption
of this hardware and software that is
the computational power at a very small
scale that can handle these very
complicated applications so video games
like Pokemon go and a slew of others
that are just released on the Apple
Store now will continue to push the
adoption and understanding of this
technology for to more users then at the
same time this tech this the software is
improving and the adoption of technology
is increasing head-mounted display units
are being designed and developed that
are smaller and streamlined and dare I
say almost practical keep in mind 17
years ago when I was at UMass there are
people attempting to do
you just this and is taking this long to
get to the hololens which you know is
still not really that practical of a
device trust me
kind of crushes your head it’s cool but
yeah so however what I do see happening
in the next two to four years is we’ll
see a transition from smartphone based
augmented reality like the examples I’ve
shared to a streamlined head mounted
display unit that people might actually
want to wear until then I’d encourage
you to take a look so from the augmented
reality apps that are on the market
today see how they work and get familiar
with the technology it’s now mainstream
and is only going to become more
prevalent in our day to day lives
Please follow and like us: