relax okay everyone good afternoon came
true as they say around here yes okay so
I am gonna be talking about building
your own eyes while we get the
technology sorted how many of you have
working eyes well I’m here to tell you
that whether you do or you don’t it
doesn’t matter you can build your own
eye it’s particularly useful if you
don’t have working eyes but even if you
do you can add to your senses and we’re
going to go through that shortly so what
I’m wearing
are it’s a smart it’s a pair of smart
glasses and I’m okay right now I’m
wearing multiple headphones because I
have to hear the presentation but I’m
wearing bone conduction headphones and I
am getting visual input as I speak so as
I pan this room okay we are gonna get
started with the formal presentation
shortly controller all right yeah we go
everyone slide one okay where’s the
laptop let’s do it this way okay okay
the idea is to see the world the way you
want well why bother with vision okay
especially if you don’t have working
eyes we adapt why bother
okay so
for one thing if you use your hands to
feel you get only 10 degrees field of
view okay the second problem with touch
okay or any of our other senses is you
cannot sense things at a distance so if
you are crossing the street is hard for
you to figure out if somewhere come is
coming towards you okay
and how many of us have collided with a
half-open door when we are in a hurry
yeah that happens – you need to be able
to sense an opening as you walk in okay
so how do we get vision okay so how do
we use light if you can’t see light
sounds strange I know but there are a
few ways to do it okay one way is
sensory substitution which I’ll talk
more about the other way to do it is you
use chips and you stimulate the retina
because the retinas were partially so
you can do that or you bypass the eye
and stick wire in your brain in the
visual centers use electrodes and
stimulate those now the problem with
stimulating the retina the chips and you
know putting wires directly for one
thing it is major surgery which means
you have a lot of post-operative care
which is required the second thing is
the technology is still developing it’s
very much at the research stage so if
you read the literature you have
okay
so how do we get around these what we
use the of an approach called sensory
substitution what that means is we can
word input from one sense to another so
in my case I am converting vision to
sound has anyone heard of cochlear
implants for the deaf they also work on
sensory substitution what they do is
they are converting sound once again to
electrical impulses which would allow
we have other advantages with sensory
substitutions so that you can use
off-the-shelf hardware so nothing
special is required and it’s flexible
this presentation is about flexibility
and I’ll be talking more about that in
subsequent slides okay now all right the
plastic brain don’t worry your brains
are not made of plastic that’s what I’m
saying you see the brain is a task-based
machine it really doesn’t care from
where it gets input because for the
brain whether you touch the chairs
you’re on whether you smell whether you
hear what I’m saying you tap your feet
on the ground you look at the screen you
look at something else you look at your
phone everything is converted to
electrical impulses and those impulses
are what form your senses now what
happens in the brain is you have neurons
in people who are congenitally blind
like I am these neurons are used for
other functions but when the brain gets
visual input it says hey wait a second
what’s coming here I have special cells
to be able to deal with this so let me
use those cells so the ability of the
brain to reira croute its cells to
perform functions is called a neural
plasticity
yeah it’s flexible plastic so that is
the principle used in sensory
substitution okay so what’s the system I
use it’s called the voice oh it’s
spelled the way it’s spelled a small V
capital o capital i capital c’ small e
oh i see you know as an exclamation it’s
an algorithm made by a Dutch physicist
dr. Peter BL Myer and what it allows you
to do is to convert images to sound and
it’s a generic solution so it’s not that
you have one image and you have a
different set of sound you have a
separate image you have a different
algorithm okay so now we have how does
how does it all work okay there are
three rules used for decoding vision and
I’ll get all of you to hear what I’m
seeing shortly d how many of us have
been to a movie and he want me into a
theater here yes you know you hear the
sound in the screen in a car-chase when
an aircraft takes off it goes from left
to right yeah that’s called panning now
I’m wearing headphones so the panning
tells me where an object is on the
horizontal plane okay the pitch of the
sound which is the frequency maps to
height so the higher the pitch the
higher the object the lower the pitch
the lower the object okay
and the amount of light or brightness
maps to volume which means that the
louder the sound the brighter the object
now these rules are what I have anybody
using this technology keeps in mind as
we decode soundscapes okay the input is
called a soundscape one iteration and
this is what you keep in mind to start
okay now well it’s up anywhere here in
software development anybody done
programming yeah maybe some people at
some point so this is the timeline which
shows you how the software and the I
have evolved okay so as you can see we
started at 90 98 we added the speaking
of color yeah I don’t sense color but we
can certainly get to know what color is
we added the ability to read so we are
moving towards enhanced vision and you
can see you know we are talking about
off-the-shelf hardware so as you can see
some of us used mobile phones to see so
the mobile phones in your pocket
everyone take them out all these mobile
phones that all of you have can be used
to restore sight okay specially those
okay well why build your own eye
you got to working eyes right why bother
when of course it’s enhancement so if
you want to add sonar to see in the dark
you can do that
okay if you can’t see you also have cost
and scale considerations because you’re
using different hardware so it’s up to
you what to do okay different ways of
seeing what I mean by this is I gave you
these the three rules of sound right
panning pitch and volume well if you
want to change these around you can do
things that way
okay it also depends on how much work
you want to put into this kind of thing
you know how much work you want to put
into building your own eye what are your
resources available and if you already
okay minimum requirements you basically
need a camera computer for the voice and
a pair of headphones okay now I’m going
to take you to some of the eyes I have
used I’ll be on eleven yeah all right
so combining vision with sonar that’s a
single board machine with a Raspberry Pi
that’s a single board computer that
means your computer is on one credit
the you can navigate in the dark as I
told you you judge distance with greater
axes because you’re using sound not a
single camera okay and this is called
the after side model one okay something
a little bit different it’s called the
immersive vision kit again we are using
a different piece of hardware called a
no droid we have this time it works on
Android so more mobile phone
compatibility you get automatic updates
okay and it’s a little bit easier to do
because you don’t do soldiering it’s
okay well here is the kangaroo PC once
again now this is a computer-based thing
for the eye okay now it’s all very well
to say that I want to see but what if
you want to do something more let’s say
you want to view spreadsheets when using
your I of course one way to do it is
look at it but if you want to use the
same machine we have something called
the kangaroo PC okay
similar thing the only advantage with
this is you’re not dealing with external
batteries so it’s far more compact but
it still has a lot of wire well whatever
we don’t want to do all that we have a
shortcut which the vision 800 glasses
which is what I am wearing compact you
still need an external battery but it’s
a one-button start okay fa Q’s here for
those of you are still wondering why we
are still doing this the main thing you
have to remember with the vision is that
it’s an exploratory process what are the
use cases are entirely up to you yeah
right now I can’t drive with a thing nor
can I fly an aeroplane but I can see
fountains I can see waterfalls I can
take pictures and do a range of other
things you can even see graphs on
blackboards okay so in an education
setting you’re talking about a class of
60 people or like yesterday when I went
into this palace I could see all the
swords behind glass cases there’s no way
to touch any of that okay and of course
the other thing is like detecting
sources for more information very
quickly seeing would sound calm and my
own blog at Tech esoterica calm but I’m
chronicling my journey
using synthetic vision ladies and
gentlemen thank you very much for
listening [Applause]
true as they say around here yes okay so
I am gonna be talking about building
your own eyes while we get the
technology sorted how many of you have
working eyes well I’m here to tell you
that whether you do or you don’t it
doesn’t matter you can build your own
eye it’s particularly useful if you
don’t have working eyes but even if you
do you can add to your senses and we’re
going to go through that shortly so what
I’m wearing
are it’s a smart it’s a pair of smart
glasses and I’m okay right now I’m
wearing multiple headphones because I
have to hear the presentation but I’m
wearing bone conduction headphones and I
am getting visual input as I speak so as
I pan this room okay we are gonna get
started with the formal presentation
shortly controller all right yeah we go
everyone slide one okay where’s the
laptop let’s do it this way okay okay
the idea is to see the world the way you
want well why bother with vision okay
especially if you don’t have working
eyes we adapt why bother
okay so
for one thing if you use your hands to
feel you get only 10 degrees field of
view okay the second problem with touch
okay or any of our other senses is you
cannot sense things at a distance so if
you are crossing the street is hard for
you to figure out if somewhere come is
coming towards you okay
and how many of us have collided with a
half-open door when we are in a hurry
yeah that happens – you need to be able
to sense an opening as you walk in okay
so how do we get vision okay so how do
we use light if you can’t see light
sounds strange I know but there are a
few ways to do it okay one way is
sensory substitution which I’ll talk
more about the other way to do it is you
use chips and you stimulate the retina
because the retinas were partially so
you can do that or you bypass the eye
and stick wire in your brain in the
visual centers use electrodes and
stimulate those now the problem with
stimulating the retina the chips and you
know putting wires directly for one
thing it is major surgery which means
you have a lot of post-operative care
which is required the second thing is
the technology is still developing it’s
very much at the research stage so if
you read the literature you have
okay
so how do we get around these what we
use the of an approach called sensory
substitution what that means is we can
word input from one sense to another so
in my case I am converting vision to
sound has anyone heard of cochlear
implants for the deaf they also work on
sensory substitution what they do is
they are converting sound once again to
electrical impulses which would allow
we have other advantages with sensory
substitutions so that you can use
off-the-shelf hardware so nothing
special is required and it’s flexible
this presentation is about flexibility
and I’ll be talking more about that in
subsequent slides okay now all right the
plastic brain don’t worry your brains
are not made of plastic that’s what I’m
saying you see the brain is a task-based
machine it really doesn’t care from
where it gets input because for the
brain whether you touch the chairs
you’re on whether you smell whether you
hear what I’m saying you tap your feet
on the ground you look at the screen you
look at something else you look at your
phone everything is converted to
electrical impulses and those impulses
are what form your senses now what
happens in the brain is you have neurons
in people who are congenitally blind
like I am these neurons are used for
other functions but when the brain gets
visual input it says hey wait a second
what’s coming here I have special cells
to be able to deal with this so let me
use those cells so the ability of the
brain to reira croute its cells to
perform functions is called a neural
plasticity
yeah it’s flexible plastic so that is
the principle used in sensory
substitution okay so what’s the system I
use it’s called the voice oh it’s
spelled the way it’s spelled a small V
capital o capital i capital c’ small e
oh i see you know as an exclamation it’s
an algorithm made by a Dutch physicist
dr. Peter BL Myer and what it allows you
to do is to convert images to sound and
it’s a generic solution so it’s not that
you have one image and you have a
different set of sound you have a
separate image you have a different
algorithm okay so now we have how does
how does it all work okay there are
three rules used for decoding vision and
I’ll get all of you to hear what I’m
seeing shortly d how many of us have
been to a movie and he want me into a
theater here yes you know you hear the
sound in the screen in a car-chase when
an aircraft takes off it goes from left
to right yeah that’s called panning now
I’m wearing headphones so the panning
tells me where an object is on the
horizontal plane okay the pitch of the
sound which is the frequency maps to
height so the higher the pitch the
higher the object the lower the pitch
the lower the object okay
and the amount of light or brightness
maps to volume which means that the
louder the sound the brighter the object
now these rules are what I have anybody
using this technology keeps in mind as
we decode soundscapes okay the input is
called a soundscape one iteration and
this is what you keep in mind to start
okay now well it’s up anywhere here in
software development anybody done
programming yeah maybe some people at
some point so this is the timeline which
shows you how the software and the I
have evolved okay so as you can see we
started at 90 98 we added the speaking
of color yeah I don’t sense color but we
can certainly get to know what color is
we added the ability to read so we are
moving towards enhanced vision and you
can see you know we are talking about
off-the-shelf hardware so as you can see
some of us used mobile phones to see so
the mobile phones in your pocket
everyone take them out all these mobile
phones that all of you have can be used
to restore sight okay specially those
okay well why build your own eye
you got to working eyes right why bother
when of course it’s enhancement so if
you want to add sonar to see in the dark
you can do that
okay if you can’t see you also have cost
and scale considerations because you’re
using different hardware so it’s up to
you what to do okay different ways of
seeing what I mean by this is I gave you
these the three rules of sound right
panning pitch and volume well if you
want to change these around you can do
things that way
okay it also depends on how much work
you want to put into this kind of thing
you know how much work you want to put
into building your own eye what are your
resources available and if you already
okay minimum requirements you basically
need a camera computer for the voice and
a pair of headphones okay now I’m going
to take you to some of the eyes I have
used I’ll be on eleven yeah all right
so combining vision with sonar that’s a
single board machine with a Raspberry Pi
that’s a single board computer that
means your computer is on one credit
the you can navigate in the dark as I
told you you judge distance with greater
axes because you’re using sound not a
single camera okay and this is called
the after side model one okay something
a little bit different it’s called the
immersive vision kit again we are using
a different piece of hardware called a
no droid we have this time it works on
Android so more mobile phone
compatibility you get automatic updates
okay and it’s a little bit easier to do
because you don’t do soldiering it’s
okay well here is the kangaroo PC once
again now this is a computer-based thing
for the eye okay now it’s all very well
to say that I want to see but what if
you want to do something more let’s say
you want to view spreadsheets when using
your I of course one way to do it is
look at it but if you want to use the
same machine we have something called
the kangaroo PC okay
similar thing the only advantage with
this is you’re not dealing with external
batteries so it’s far more compact but
it still has a lot of wire well whatever
we don’t want to do all that we have a
shortcut which the vision 800 glasses
which is what I am wearing compact you
still need an external battery but it’s
a one-button start okay fa Q’s here for
those of you are still wondering why we
are still doing this the main thing you
have to remember with the vision is that
it’s an exploratory process what are the
use cases are entirely up to you yeah
right now I can’t drive with a thing nor
can I fly an aeroplane but I can see
fountains I can see waterfalls I can
take pictures and do a range of other
things you can even see graphs on
blackboards okay so in an education
setting you’re talking about a class of
60 people or like yesterday when I went
into this palace I could see all the
swords behind glass cases there’s no way
to touch any of that okay and of course
the other thing is like detecting
sources for more information very
quickly seeing would sound calm and my
own blog at Tech esoterica calm but I’m
chronicling my journey
using synthetic vision ladies and
gentlemen thank you very much for
listening [Applause]
Please follow and like us: