[Music]
good morning everyone if I ask you what
see in this picture what me
you answer you see a dog there is a
as born somewhere they actually
there are no vintage spots
black and white if I ask if you see
a white triangle where everyone is
you will see there is not really your
foresight the most recent part of the
our brain that fuses the flow of
information that comes from the eyes and
with what I told you you must
search in the image and then complete
the perception in this visual case is
a thin layer of 2 3 mm around the
brain that with only one algorithm
it deals with the same way and creates
the visual tactile perceptions leads us to
reasoning leads us to interact and is
structured in a hierarchical way then the
let’s see with this example
following the flow of light that
through the optic nerve comes to the
occipital area of our brain
meets the first certitude 1 and there are i
neurons that process information
simpler their output goes up in q2
and then from there to v4 where they come from
process the information more
complex such as those of the movement
so the perceptions is actually the
result of two information flows
1 from bottom to top and one from above
downwards then the object of the
my research and the brain
only those who discovered it late when I got it
started the doctorate school
sant’anna di pisa and the interest was ed
it’s still for human interaction
machine interested robots like i
robots interact with humans
their medical implications above all
and how they necessarily interact with
the environment in which they move taking
as a source of inspiration like man
betrays the senses with the environment
the main ones for me were therefore the
sight and touch in both directions
explorers par excellence, however
it is a complex animal too complex
to be taken but brought in
robotics
so I need more models then
simple epa we see 21 for how much
it concerns the visual system and one for
regarding the tactile system they have
something in common the praying mantis
and this mole is called with lura
cristata or mole with a starry face and the
let’s find out with another experiment
now I ask you to close your eyes and
to put your right hand on it
on our range without moving it
you will have a tactile perception some
they will understand what he did
other fabric no
everything will be easier if
this hand started to move it e
so your tactile perception yes
enriches both these animals for
have a perception complete with
where is the prey in the case of the mantis
with eyes to the case of the mole
with the big nose they move the mantia of moves
right and left and the nose instead
of the mole with a starry snout moves
made the 22 used sensor appendices say
that are moved by the animal
so my approach to interaction
man machine was and is that of the
intelligent discretion
which goes through systems for
transform experiences into memories
memories that can be interrogated
used to plan the movement
therefore necessary ex in machine learning
that I discovered a step too
time
this is an architecture that goes a lot
in fashion today in deep architecture
organized hierarchically and are
nodes each of which has one becomes one
model in the part of the world to which it is
exposed and passes this model to the nodes
superiors we see simple way how
it works we think of the cimas nodes more in
low those who are in contact with i
sensors for example in this case with
microphone and then recognize the
letters sequences of letters from the origin
to words therefore at the best level
we have nodes that learn sequences
of letters then words the sequences of
words are songs
so while I’m inside of
a song all the network that is
essentially a distributed memory
it will be in a consistent state the
node high above dimension in the
song hicks the knots at the bottom will know
what words are hearing why
they are in their memory and therefore can
Also tell us if we ask her which ones
these are the next words we must
expect us then this network is in
able to learn to memorize and therefore
also predicting what is going to
happen when a
understanding of the situation
now we start in the low level of the
more knots in contact with the physical world
this is the subject of my doctorate
and it is a was a robotic system that
he had to allow navigation
endoscopic inside
the spinal subarachnoid space then
it is a liquid environment among the most loved
of the dura mater in contact with the marrow
populated with nerve roots and vessels
blood as seen in image a
left that must be before
all the bellati is therefore said
recognized by the system and then avoided
you have to stop the surgeon from hitting them
with the endoscope head because
they could be damaged
in this case there is nothing that in
I have been various in techniques of
digital image processing to say
to teach the system this nerve yes
this means this is a lot of it
it involves what and I could do it because
because the environment is enough
simple
from the anatomical point of view the problem
here it was more than shared control and
also important ethical questions up to
what point does the robot have to watch
to what extent it can be exempted
from intervening in the event that the
surgeon is doing something
potentially dangerous I perceived
as dangerous and if the perception of the
robot is wrong
this is one of the reasons why the
our society is full of robots
industrialists who do wonderful things
in the company but there are still robots
humanoids around the streets or in the
our homes
one of the reasons is that they have one
perception of the world still limited for
example, nobody has a tactile skin
worthy of this name therefore covering
completely the robot that in genres
real-time information that will allow
their way of knowing if he’s hitting one
person in fact you’re hurting a
someone
the feel is anything but a
problem if this is the challenge
launched by toyota to my blog always
santanna and the challenge was precisely that
to be able to grasp with your hand
robotics without any programming from
from above or a piece of tofu this
Japanese slimy cheese and then
the first thing from
from bioengineer this fact I am
studied how it works and physiology
from and anatomy of human touch therefore I have
discovered that we have four types
mercando receptors in our skin ne
we have so many behaves in the world
different you are sensitive and aspects
different from the tactile experience
however, the raid is a model too
complex then the game comes back into play
Our mole Star-nosed be
see his nose and an appendix
covered with a large number of organs of a reimer
they are called are receptors
natural that I have replaced one more
sensor that I had available in
Lab silicon manufactured Friend
it’s not just an open array of a little
free has become a skin and has
made so that it generated of the gods
images of these tactile images
here then the problem of
process these tactile images in a
so that it was compatible, not like 30
40 sensors that I had available to
potentially thousands of sensors
that you buy is a tactile skin of a
humanoid robot so I was looking for a
soft computing approach that maybe
parallel that just could not
work the whole image of a shot
alone and that he was able to generate
tactile images of the maps
tactiles of elaborate images in which
each of which made sense to
what the sensors felt but there was
a sensitive one that indicated areas a
stable contact
another that indicated areas instead
they were subject to slipping another
which indicated which areas were going to
be subject to slipping and then
I had managed to climb a level
of my pyramid then to have
of the semantic labels then
safe contact contact with secure then
shake hands and the result is there
this was it
a film of the robotic hand that
manages to raise a piece of tofu
safely and then demonstrate it by touching it
which is really a piece of tofu
in a piece of plastic
now you should see the system that
hurry here it is the movement well
however of this this way a lot
simple was only open and close
raise and lower and others when moving
it starts to be more complex
example a movement of a human being
and I want to analyze it I need to
climb another step into the pyramid e
then develop some nodes that succeed
to learn time sequences
much longer in this case was
asked ten people in ours
laboratories to carry out the sequence
yoga of greeting in the sun wearing some
wireless accelerometers
two wrists and 2 ankles an hour and the out
of these sensors has been used
to teach an algorithm that has
I learned everything by myself what was the
sequence of this greeting to the sun the
fact that this trajectory that
represents the memory of this node
is closed and due to the fact that
the exercise is a periodic exercise
children the position wants a position
12 are in the same
so when then the
the system is operated in
real time with a person never seen
first it is able to navigate in the
memory using its input e
identify the neuron that vault for
time is the most active and when
Europe while in some provisions
of memory means that in one
position of the greeting and here we see the
currently recognized position is the
position that the system expects to
see later if the person is doing
exercise correctly
the image we see here is not
used in any way for the
classification but only for
see this was brought in a
processor of a smartphone and
it worked in real time
what’s the use of all this we’ve seen
that for the human machine interaction there
are three basic ingredients a
artificial brain of the sensors the
ast movement make us a moment we have
need an architecture that is in
degree at the same time to memorize e
to compute say of sensors and a
they are a body but around the environment
welcoming however three challenges and they are three
challenges that make the difference 1 being
capable of handling a large quantity
of data and like the one that comes from
eyes a beautiful tactile distributed by or
accelerometers work in real
time and do it without needing a
data center
to work for example using
a smartphone with the platform then
machine learning alone intelligence
artificial can not solve
this problem requires an approach
holistic
I believe all three aspects
what can you do for example in the
moment when the ability to move
it comes less than one person
we have scenarios that today are like
this at the top we see a session of
rehabilitation of a person for
example a post stroke in a hall
immersive virtual reality there were
it would appear that there are systems
that acquire movement, however, see
which is located in a furnished room
just for this then there is a hand
prosthetic and a hand that recently
has been controlled with the brain by
a pinned one is however now needs
of a cortical implant then and
think about how much better it would be if
this hand could alone manage the
all low-level control
so do everything you need for
ensure a secure grip of the object
or many other activities that normally
let’s do as our hand leaving
to the person with others now his
brain only control
of the goal of what I want to do
in my hand I thought just like
it would be nice if all you need
to acquire the movement analyze it e
give feedback to the patient could
be hosted on a smartphone that
what we are doing here we see
a person that some accelerometers
wireless around him is doing a
rehabilitation exercise of this
exercise is analyzed stored
first on the therapist and then on the person
and lemon abstract in temporary of the
olive trees of the indices and that come
returned to the patient so
allow him to have an experience of
better rehabilitation shorten i
times and improve results though
another important thing
the limits of current rehabilitation
they are the activities of daily life
when a person is finished maybe later
a stroke after rehabilitation
of the hospital daprà rehabilitation a
home enters everyday life
does what and spices often so
pejorative is not maintained
quality of rehabilitation
so a system that alone manages to
understand if I’m washing the dishes yes I’m
while walking , I’m settling down
beating the computer and telling me that it goes from
if you tell me no careful remembrance to roll out
on one arm to stretch on the leg a
draw a great help
or if the same exercise or another
I do it with a helmet in
head and register the brain activity
before and during the bike I can
satisfy the dream and we are
making to unveil what the alphabet
instead on the cerebral level of the commands
motor actions engines so
allow a person who can not
more make it to command a robot one
prosthesis something that does it in
functional replacement sees the challenges
are there are so many are making this
artificial intelligence and well
running on small platforms
allow it to adapt automatically
to individual people because everyone of
we move each one differently
we think differently and effort
to train these of these
prosthesis must go down must allow to
make these prostheses work in
continuous way all day long to understand
alone when it is time to intervene and
when no, no fascinating challenges
we are far from the
reach the goal this is the
small group of researchers with
the honor of coordinating the names of
others who give us their hands on themes
specific because as I said before
the approach can only be holistic
there are sanctis task there are i
mathematicians there are the physicists are the
material scientists is so and
all this must be handled with care
[Applause]
good morning everyone if I ask you what
see in this picture what me
you answer you see a dog there is a
as born somewhere they actually
there are no vintage spots
black and white if I ask if you see
a white triangle where everyone is
you will see there is not really your
foresight the most recent part of the
our brain that fuses the flow of
information that comes from the eyes and
with what I told you you must
search in the image and then complete
the perception in this visual case is
a thin layer of 2 3 mm around the
brain that with only one algorithm
it deals with the same way and creates
the visual tactile perceptions leads us to
reasoning leads us to interact and is
structured in a hierarchical way then the
let’s see with this example
following the flow of light that
through the optic nerve comes to the
occipital area of our brain
meets the first certitude 1 and there are i
neurons that process information
simpler their output goes up in q2
and then from there to v4 where they come from
process the information more
complex such as those of the movement
so the perceptions is actually the
result of two information flows
1 from bottom to top and one from above
downwards then the object of the
my research and the brain
only those who discovered it late when I got it
started the doctorate school
sant’anna di pisa and the interest was ed
it’s still for human interaction
machine interested robots like i
robots interact with humans
their medical implications above all
and how they necessarily interact with
the environment in which they move taking
as a source of inspiration like man
betrays the senses with the environment
the main ones for me were therefore the
sight and touch in both directions
explorers par excellence, however
it is a complex animal too complex
to be taken but brought in
robotics
so I need more models then
simple epa we see 21 for how much
it concerns the visual system and one for
regarding the tactile system they have
something in common the praying mantis
and this mole is called with lura
cristata or mole with a starry face and the
let’s find out with another experiment
now I ask you to close your eyes and
to put your right hand on it
on our range without moving it
you will have a tactile perception some
they will understand what he did
other fabric no
everything will be easier if
this hand started to move it e
so your tactile perception yes
enriches both these animals for
have a perception complete with
where is the prey in the case of the mantis
with eyes to the case of the mole
with the big nose they move the mantia of moves
right and left and the nose instead
of the mole with a starry snout moves
made the 22 used sensor appendices say
that are moved by the animal
so my approach to interaction
man machine was and is that of the
intelligent discretion
which goes through systems for
transform experiences into memories
memories that can be interrogated
used to plan the movement
therefore necessary ex in machine learning
that I discovered a step too
time
this is an architecture that goes a lot
in fashion today in deep architecture
organized hierarchically and are
nodes each of which has one becomes one
model in the part of the world to which it is
exposed and passes this model to the nodes
superiors we see simple way how
it works we think of the cimas nodes more in
low those who are in contact with i
sensors for example in this case with
microphone and then recognize the
letters sequences of letters from the origin
to words therefore at the best level
we have nodes that learn sequences
of letters then words the sequences of
words are songs
so while I’m inside of
a song all the network that is
essentially a distributed memory
it will be in a consistent state the
node high above dimension in the
song hicks the knots at the bottom will know
what words are hearing why
they are in their memory and therefore can
Also tell us if we ask her which ones
these are the next words we must
expect us then this network is in
able to learn to memorize and therefore
also predicting what is going to
happen when a
understanding of the situation
now we start in the low level of the
more knots in contact with the physical world
this is the subject of my doctorate
and it is a was a robotic system that
he had to allow navigation
endoscopic inside
the spinal subarachnoid space then
it is a liquid environment among the most loved
of the dura mater in contact with the marrow
populated with nerve roots and vessels
blood as seen in image a
left that must be before
all the bellati is therefore said
recognized by the system and then avoided
you have to stop the surgeon from hitting them
with the endoscope head because
they could be damaged
in this case there is nothing that in
I have been various in techniques of
digital image processing to say
to teach the system this nerve yes
this means this is a lot of it
it involves what and I could do it because
because the environment is enough
simple
from the anatomical point of view the problem
here it was more than shared control and
also important ethical questions up to
what point does the robot have to watch
to what extent it can be exempted
from intervening in the event that the
surgeon is doing something
potentially dangerous I perceived
as dangerous and if the perception of the
robot is wrong
this is one of the reasons why the
our society is full of robots
industrialists who do wonderful things
in the company but there are still robots
humanoids around the streets or in the
our homes
one of the reasons is that they have one
perception of the world still limited for
example, nobody has a tactile skin
worthy of this name therefore covering
completely the robot that in genres
real-time information that will allow
their way of knowing if he’s hitting one
person in fact you’re hurting a
someone
the feel is anything but a
problem if this is the challenge
launched by toyota to my blog always
santanna and the challenge was precisely that
to be able to grasp with your hand
robotics without any programming from
from above or a piece of tofu this
Japanese slimy cheese and then
the first thing from
from bioengineer this fact I am
studied how it works and physiology
from and anatomy of human touch therefore I have
discovered that we have four types
mercando receptors in our skin ne
we have so many behaves in the world
different you are sensitive and aspects
different from the tactile experience
however, the raid is a model too
complex then the game comes back into play
Our mole Star-nosed be
see his nose and an appendix
covered with a large number of organs of a reimer
they are called are receptors
natural that I have replaced one more
sensor that I had available in
Lab silicon manufactured Friend
it’s not just an open array of a little
free has become a skin and has
made so that it generated of the gods
images of these tactile images
here then the problem of
process these tactile images in a
so that it was compatible, not like 30
40 sensors that I had available to
potentially thousands of sensors
that you buy is a tactile skin of a
humanoid robot so I was looking for a
soft computing approach that maybe
parallel that just could not
work the whole image of a shot
alone and that he was able to generate
tactile images of the maps
tactiles of elaborate images in which
each of which made sense to
what the sensors felt but there was
a sensitive one that indicated areas a
stable contact
another that indicated areas instead
they were subject to slipping another
which indicated which areas were going to
be subject to slipping and then
I had managed to climb a level
of my pyramid then to have
of the semantic labels then
safe contact contact with secure then
shake hands and the result is there
this was it
a film of the robotic hand that
manages to raise a piece of tofu
safely and then demonstrate it by touching it
which is really a piece of tofu
in a piece of plastic
now you should see the system that
hurry here it is the movement well
however of this this way a lot
simple was only open and close
raise and lower and others when moving
it starts to be more complex
example a movement of a human being
and I want to analyze it I need to
climb another step into the pyramid e
then develop some nodes that succeed
to learn time sequences
much longer in this case was
asked ten people in ours
laboratories to carry out the sequence
yoga of greeting in the sun wearing some
wireless accelerometers
two wrists and 2 ankles an hour and the out
of these sensors has been used
to teach an algorithm that has
I learned everything by myself what was the
sequence of this greeting to the sun the
fact that this trajectory that
represents the memory of this node
is closed and due to the fact that
the exercise is a periodic exercise
children the position wants a position
12 are in the same
so when then the
the system is operated in
real time with a person never seen
first it is able to navigate in the
memory using its input e
identify the neuron that vault for
time is the most active and when
Europe while in some provisions
of memory means that in one
position of the greeting and here we see the
currently recognized position is the
position that the system expects to
see later if the person is doing
exercise correctly
the image we see here is not
used in any way for the
classification but only for
see this was brought in a
processor of a smartphone and
it worked in real time
what’s the use of all this we’ve seen
that for the human machine interaction there
are three basic ingredients a
artificial brain of the sensors the
ast movement make us a moment we have
need an architecture that is in
degree at the same time to memorize e
to compute say of sensors and a
they are a body but around the environment
welcoming however three challenges and they are three
challenges that make the difference 1 being
capable of handling a large quantity
of data and like the one that comes from
eyes a beautiful tactile distributed by or
accelerometers work in real
time and do it without needing a
data center
to work for example using
a smartphone with the platform then
machine learning alone intelligence
artificial can not solve
this problem requires an approach
holistic
I believe all three aspects
what can you do for example in the
moment when the ability to move
it comes less than one person
we have scenarios that today are like
this at the top we see a session of
rehabilitation of a person for
example a post stroke in a hall
immersive virtual reality there were
it would appear that there are systems
that acquire movement, however, see
which is located in a furnished room
just for this then there is a hand
prosthetic and a hand that recently
has been controlled with the brain by
a pinned one is however now needs
of a cortical implant then and
think about how much better it would be if
this hand could alone manage the
all low-level control
so do everything you need for
ensure a secure grip of the object
or many other activities that normally
let’s do as our hand leaving
to the person with others now his
brain only control
of the goal of what I want to do
in my hand I thought just like
it would be nice if all you need
to acquire the movement analyze it e
give feedback to the patient could
be hosted on a smartphone that
what we are doing here we see
a person that some accelerometers
wireless around him is doing a
rehabilitation exercise of this
exercise is analyzed stored
first on the therapist and then on the person
and lemon abstract in temporary of the
olive trees of the indices and that come
returned to the patient so
allow him to have an experience of
better rehabilitation shorten i
times and improve results though
another important thing
the limits of current rehabilitation
they are the activities of daily life
when a person is finished maybe later
a stroke after rehabilitation
of the hospital daprà rehabilitation a
home enters everyday life
does what and spices often so
pejorative is not maintained
quality of rehabilitation
so a system that alone manages to
understand if I’m washing the dishes yes I’m
while walking , I’m settling down
beating the computer and telling me that it goes from
if you tell me no careful remembrance to roll out
on one arm to stretch on the leg a
draw a great help
or if the same exercise or another
I do it with a helmet in
head and register the brain activity
before and during the bike I can
satisfy the dream and we are
making to unveil what the alphabet
instead on the cerebral level of the commands
motor actions engines so
allow a person who can not
more make it to command a robot one
prosthesis something that does it in
functional replacement sees the challenges
are there are so many are making this
artificial intelligence and well
running on small platforms
allow it to adapt automatically
to individual people because everyone of
we move each one differently
we think differently and effort
to train these of these
prosthesis must go down must allow to
make these prostheses work in
continuous way all day long to understand
alone when it is time to intervene and
when no, no fascinating challenges
we are far from the
reach the goal this is the
small group of researchers with
the honor of coordinating the names of
others who give us their hands on themes
specific because as I said before
the approach can only be holistic
there are sanctis task there are i
mathematicians there are the physicists are the
material scientists is so and
all this must be handled with care
[Applause]
Please follow and like us: