
everything I want to show you today was
impossible
five years ago I’m gonna talk to you
about a new technology that lets us look
inside your brain and look at the
circuit diagram a technology that
michelle obama referred to as high
definition imaging this is a technology
that perhaps in the next decade may
benefit a member of your family but let
me start with a strange phone call I had
so the FBI once urgently talked to you
about your work now I don’t know about
you but what went through my mind wasn’t
necessarily good so I call it back the
number Oh dr. Walter Schneider what
would you like to know
[Music]
I’m from the FBI victims unit dealing
with the Boston Marathon bombing and we
had some 250 people there and the Army
says you’ve got the best technology to
look inside their brains can we send
them yes
and they came I’m going to talk to you
about this technology this is a
technology that deals with one of the
major healthcare costs that we have in
our nation the capability somebody this
is running on auto so if you could
switch it so it is not Auto forwarding
on whoever’s running the slides that
would be helpful the what we deal with
is tumors North almond disorders such as
autism no degenerative disorders such as
Alzheimer’s that impacts ten point four
million people a year and put that in
perspective that’s 1,200 people in this
hour the student body of the University
of Pittsburgh today that’s the number of
people implanted we want to build a
technology to look at the circuit
diagram of the brain to understand it
we this is a big reach the technology
doesn’t yet exist we have technology for
x-rays in terms of what bone is broken
we have something called high-definition
fiber tracking that wants to look at the
pattern of the cables of the brain so
after trauma something breaks in the
case of a car crash maybe you lose your
lights the radio something broke in the
circuit diagram after trauma for a loved
one you made me lose language effect
control motor function these are all
things we want back now if your family
needs the technology you want to be able
to say what’s broken what does it mean
and what do we do about the loss
functions so the technology we use is an
MRI based technology so we put a person
in a magnet we then do something called
tag labeling of the water molecules so
we use the machine to label a given part
of the brain and we watch how the water
spreads and it spreads along the cables
and then we build a circuit diagram of
an individual within that system so that
our goal here now is we start with the
structural image the colors you see are
the connections of that system we’re
still fighting the sequence here and
what we then do is we can take this is
the language cable of the brain and you
can see the projection of the cable so
each of these places where it ends is
what’s picking up information we can
then take an individual and take the
cables of their brain and dissect this
is where you hear things this is where
you say things this is where you connect
what you see to what you say and this is
the inter link of the processes that
you’re working with so we can then build
a circuit diagram in about the time of
this talk that is in ten minutes we can
scan
and build this capability now we’re
creating a personalized circuit diagram
for locations within each of these
cables of the brain to understand the
bandwidth of those components to look at
the different cables just like bones in
your skeleton to identify the processes
I’m going to talk about three
applications the first is clinical when
a nurse surgeon has to go in they’re
going to cut something out better it not
be language we want to know where the
cables are with an individual so I’m
going to review a few cases here is an
example of a push that is the tumor
pushed it out on the right there and
then they’re gonna be very assertive
take it all out okay
here is an example of guidance in
neurosurgery so you could go down you go
to the side whichever way you go you’re
gonna take something out and Sylla
straited in the middle and then on the
right is what the result was so we can
predict what’s gonna be taken out
because again we want to take out the
least damaging piece of the brain here
is a woman who had a tumor around the
system that controls all her motor
function had they taken it out she would
have been paralyzed what we did is we
instrumented in the o.r that green line
is the probe of the surgeon so we can
see where it is and they very gently
aspirated the tumor think of it as it’s
like trying to vacuum cleaner curtain
but never move it because if you move it
she loses control of her lips so the
idea is that we by seeing the cables and
knowing where we are we can be more
precise here’s a woman who went to was
progressively being paralyzed on one
part of her body went to several places
they said we can’t do anything because
there’s been too much damage so she came
to us and we looked closely at the
tracks and we found that there was an
existing hole about where the problem
that she had was within that Robert
Freelander then our search
said I can do the repair through that
hole without doing more damage and here
you see after the surgery this is his
cut the surgical knife where he went in
and did the repair now what happens with
this High Definition fiber tracking
Robert says that Ashley could not move
half her body we needed to operate on
her we understood where all the normal
fibers are then we can avoid those –
look at this and let me let Ashley
describe the result I am right now to
actually get out on my own so to go from
progressive paralysis to cooking driving
on a full-time job we have that
capability because we could see what to
miss how to get in to repair the damage
my second topic is autism where one of
the things that happens is in
development sometimes the cables grow in
differently if you make really
interesting images all of a sudden 60
minutes in Discovery magazine show up on
campus which is an interesting
experience and I had the privilege of
working with Temple Grandin a noted
person in autism and we scanned her
brain and we looked at the different
cables that we had in that brain and
what we found was that part of her brain
was down 99% cable was 1% of what it is
in your brain and another part was 300
percent higher than what it would be in
the normal brain
she is the example of a neuro a typical
different somethings better somethings
worse what we want to do when somebody
presents with autism is we want to have
the ability to identify for that
individual well can we map the person’s
language system because it’s really
important in the first 5 years to give
this child the ability to say I want to
go to the bathroom or
want to be fed what if I got to work
with identify the approaches that work
target rehabilitation and measure the
growth finding out what strategies work
better
my major effort is the traumatic brain
injury with my colleague David okonkwo
if you have a major traumatic brain
injury in western Pennsylvania he’s the
guy that’s kind of see you and will do
the surgery if it’s needed we want to
know what tracks are broken whether it
is from Falls Auto injuries war wounds
what are the cables and what are their
current States and I’m gonna give you
two rules to help you understand how we
can see traumatic brain injury
the first is symmetry if you see an
individual and they look like this the
brain and the and the body are enormous
ly symmetric and if you see somebody
like this you can guess that they had
trauma sometime in their life in terms
of the hand elements well we look at
that and we look at their systems so
that you can see these colors are these
cables and there’s a hole it doesn’t
have the symmetry so rule number one is
loss of symmetry rule number two is
trauma reduces branching so in the upper
left you see a Central Park after
Hurricane sandy went through it blew
away the small branches we look at the
branching and in the lower right you see
a special-forces person who had 400
blasts blow blow and it takes out the
small fibers so we can look at these
elements now what I’m going to do is
show you some images and let’s see now
that I’ve described what you can look
for can you see it
so here we go we have three individuals
look in the middle of the upper set and
note the red and the tan a B or C whose
missing part of the red okay that took
all of ten seconds on the bottom look at
the middle who’s missing those five
projections going
okay you in one minute of instruction
were able to do something that nobody in
the planet could do three years ago
because we didn’t have the technology we
can now see asymmetry we can look at
this change in arborisation this makes
the invisible wound of war visible this
is very important because if you take a
marine and you tell him that there’s a
problem in his brain they’re not gonna
say much if you if he’s worried about
malingering he may commit suicide but if
you show him the damage then he their
family and the clinicians all work as a
team if a loved one has a brain injury
you want to know what’s broken what does
it mean and how do we cover functions
the standard MRI scan gives you
something like this this is of uncertain
clinical significance not because they
don’t want to help they don’t yet have
the technology what we do is we give you
a 20 page report and for each cable
where is it how what is the integrity of
that cable we can show the patient their
brain the yellow here in the image are
the the images of the fibers you can see
on one side you’re missing a big chunk
on the other side it’s fully there so
they could understand so they can move
on in the process we want to understand
the functional implications so we had an
individual we had an all-terrain vehicle
accident flipped over in a coma for 14
days came out first thing I want to know
because he’s paralyzed I have his body
am I ever gonna walk again he’s no
question while we say if I plot your
flavors I can identify how much you have
and in the leg you’ve lost 60% and you
have a little bit of function in the arm
67% in the hand
97% by this is very important because if
we work on the hand and it’s not going
anywhere
we’re gonna blow this investment of 300
hours to try and bring back function so
we worked first on the lake it came back
then the arm came back
he’s never opened his hand by knowing
what we have we can give them
expectations and we can target the
therapy now I’m engaged in a big reach I
want to develop with my team members by
colleagues something that humanity has
never had a map of the anatomical
connectivity of any individual who it is
useful for us to identify that we need
to drive innovation the technology does
not yet exist it’s gonna be a five to
fifteen year effort it’s a reach to
figure out how we do that now I’m gonna
give you an example that you’re aware of
for reaching which is the experience of
technology to make possible remote
driving vehicles in town right I want to
take you back ten years ago to the first
vehicle challenge where DARPA put up a
bunch of Hill bails they ran off over
the hill bails they ran into the sides
of the barriers they flipped over okay
desert 15 robots zero the brilliance of
the dark DARPA campaign was to expect
make a big reach but expect failure it’s
going to be a bumpy road well they also
provided incremental tests of capability
and what happened is that was an open
competition it brought world talent
together they accomplished steps in that
process well in Pittsburgh we’re doing
what I call the brain connectivity
challenge we want to create a new
technology to make a map of each
individual person’s brain this has to be
done at a level of detail called the
fasciculus sets of bundles of axons
typically several
of those and the individual cables
because we want to map for each
individual what those are here’s an
illustration of the first challenge
which is we want to map from it 500
positions in each eye all the way
through the first stage in the middle of
the brain called the lateral geniculate
of getting exactly the spatial topology
of every one of those particular right
if we do this we can then move on to
other parts of the brain now the problem
is remember of DARPA when it did this
challenge it could put up hay bales to
make a sort of simulated little bitty
road that’s a little bit easy to work
with we needed our version of that and
we created what I call the textile brain
wiring machine where we have the ability
of routing through a complex pattern at
the ten micron accuracy the full path of
one of these fibers so I can see if the
available technology can get the answer
right because now I have ground truth
within that set so up the hill we built
[Music]
what we’re going to do is then map out
64,000 640,000 patterns of tubes within
that set for an early visual system then
we will work on that then we will see
can we image that where we know the
right answer is of every one of those
tubes this is just an illustration of
this for each of these little lines is
sort of the example of an axon that now
is manufactured to look at that
performance we then do an a we’ll be
doing an annual incremental challenge
test can you find the track routes of
the crossings that you’re working with
how did the fascicular come together
travel and pull apart can we get that
accurately and then we will start with
vision but then go on the language and
then go on to attention control just
like DARPA the first trick was could you
not run over the bail hails we will
start with can you get the eyeball to
lgn right once we get that we’ll move on
so what I’ve described for you is a big
reach it’s gonna take harnessing world
talent including perhaps some of the
people in this room we want to put this
together so we can deliver a technology
that hasn’t existed before this reaches
a big team effort it takes lots of
people from different disciplines
including coders by the way is a big
group of that element but researchers
that deal with both the neuroscience the
clinical issues the imaging issues
within that set so I’m gonna leave you
with my favorite quote from arthur c
clarke any sufficiently advanced
technology is indistinguishable from
magic
five years ago everything that i showed
you was impossible ten years ago from
now it may be routine it’s our job here
to reach
make it happen thank you
[Applause]
impossible
five years ago I’m gonna talk to you
about a new technology that lets us look
inside your brain and look at the
circuit diagram a technology that
michelle obama referred to as high
definition imaging this is a technology
that perhaps in the next decade may
benefit a member of your family but let
me start with a strange phone call I had
so the FBI once urgently talked to you
about your work now I don’t know about
you but what went through my mind wasn’t
necessarily good so I call it back the
number Oh dr. Walter Schneider what
would you like to know
[Music]
I’m from the FBI victims unit dealing
with the Boston Marathon bombing and we
had some 250 people there and the Army
says you’ve got the best technology to
look inside their brains can we send
them yes
and they came I’m going to talk to you
about this technology this is a
technology that deals with one of the
major healthcare costs that we have in
our nation the capability somebody this
is running on auto so if you could
switch it so it is not Auto forwarding
on whoever’s running the slides that
would be helpful the what we deal with
is tumors North almond disorders such as
autism no degenerative disorders such as
Alzheimer’s that impacts ten point four
million people a year and put that in
perspective that’s 1,200 people in this
hour the student body of the University
of Pittsburgh today that’s the number of
people implanted we want to build a
technology to look at the circuit
diagram of the brain to understand it
we this is a big reach the technology
doesn’t yet exist we have technology for
x-rays in terms of what bone is broken
we have something called high-definition
fiber tracking that wants to look at the
pattern of the cables of the brain so
after trauma something breaks in the
case of a car crash maybe you lose your
lights the radio something broke in the
circuit diagram after trauma for a loved
one you made me lose language effect
control motor function these are all
things we want back now if your family
needs the technology you want to be able
to say what’s broken what does it mean
and what do we do about the loss
functions so the technology we use is an
MRI based technology so we put a person
in a magnet we then do something called
tag labeling of the water molecules so
we use the machine to label a given part
of the brain and we watch how the water
spreads and it spreads along the cables
and then we build a circuit diagram of
an individual within that system so that
our goal here now is we start with the
structural image the colors you see are
the connections of that system we’re
still fighting the sequence here and
what we then do is we can take this is
the language cable of the brain and you
can see the projection of the cable so
each of these places where it ends is
what’s picking up information we can
then take an individual and take the
cables of their brain and dissect this
is where you hear things this is where
you say things this is where you connect
what you see to what you say and this is
the inter link of the processes that
you’re working with so we can then build
a circuit diagram in about the time of
this talk that is in ten minutes we can
scan
and build this capability now we’re
creating a personalized circuit diagram
for locations within each of these
cables of the brain to understand the
bandwidth of those components to look at
the different cables just like bones in
your skeleton to identify the processes
I’m going to talk about three
applications the first is clinical when
a nurse surgeon has to go in they’re
going to cut something out better it not
be language we want to know where the
cables are with an individual so I’m
going to review a few cases here is an
example of a push that is the tumor
pushed it out on the right there and
then they’re gonna be very assertive
take it all out okay
here is an example of guidance in
neurosurgery so you could go down you go
to the side whichever way you go you’re
gonna take something out and Sylla
straited in the middle and then on the
right is what the result was so we can
predict what’s gonna be taken out
because again we want to take out the
least damaging piece of the brain here
is a woman who had a tumor around the
system that controls all her motor
function had they taken it out she would
have been paralyzed what we did is we
instrumented in the o.r that green line
is the probe of the surgeon so we can
see where it is and they very gently
aspirated the tumor think of it as it’s
like trying to vacuum cleaner curtain
but never move it because if you move it
she loses control of her lips so the
idea is that we by seeing the cables and
knowing where we are we can be more
precise here’s a woman who went to was
progressively being paralyzed on one
part of her body went to several places
they said we can’t do anything because
there’s been too much damage so she came
to us and we looked closely at the
tracks and we found that there was an
existing hole about where the problem
that she had was within that Robert
Freelander then our search
said I can do the repair through that
hole without doing more damage and here
you see after the surgery this is his
cut the surgical knife where he went in
and did the repair now what happens with
this High Definition fiber tracking
Robert says that Ashley could not move
half her body we needed to operate on
her we understood where all the normal
fibers are then we can avoid those –
look at this and let me let Ashley
describe the result I am right now to
actually get out on my own so to go from
progressive paralysis to cooking driving
on a full-time job we have that
capability because we could see what to
miss how to get in to repair the damage
my second topic is autism where one of
the things that happens is in
development sometimes the cables grow in
differently if you make really
interesting images all of a sudden 60
minutes in Discovery magazine show up on
campus which is an interesting
experience and I had the privilege of
working with Temple Grandin a noted
person in autism and we scanned her
brain and we looked at the different
cables that we had in that brain and
what we found was that part of her brain
was down 99% cable was 1% of what it is
in your brain and another part was 300
percent higher than what it would be in
the normal brain
she is the example of a neuro a typical
different somethings better somethings
worse what we want to do when somebody
presents with autism is we want to have
the ability to identify for that
individual well can we map the person’s
language system because it’s really
important in the first 5 years to give
this child the ability to say I want to
go to the bathroom or
want to be fed what if I got to work
with identify the approaches that work
target rehabilitation and measure the
growth finding out what strategies work
better
my major effort is the traumatic brain
injury with my colleague David okonkwo
if you have a major traumatic brain
injury in western Pennsylvania he’s the
guy that’s kind of see you and will do
the surgery if it’s needed we want to
know what tracks are broken whether it
is from Falls Auto injuries war wounds
what are the cables and what are their
current States and I’m gonna give you
two rules to help you understand how we
can see traumatic brain injury
the first is symmetry if you see an
individual and they look like this the
brain and the and the body are enormous
ly symmetric and if you see somebody
like this you can guess that they had
trauma sometime in their life in terms
of the hand elements well we look at
that and we look at their systems so
that you can see these colors are these
cables and there’s a hole it doesn’t
have the symmetry so rule number one is
loss of symmetry rule number two is
trauma reduces branching so in the upper
left you see a Central Park after
Hurricane sandy went through it blew
away the small branches we look at the
branching and in the lower right you see
a special-forces person who had 400
blasts blow blow and it takes out the
small fibers so we can look at these
elements now what I’m going to do is
show you some images and let’s see now
that I’ve described what you can look
for can you see it
so here we go we have three individuals
look in the middle of the upper set and
note the red and the tan a B or C whose
missing part of the red okay that took
all of ten seconds on the bottom look at
the middle who’s missing those five
projections going
okay you in one minute of instruction
were able to do something that nobody in
the planet could do three years ago
because we didn’t have the technology we
can now see asymmetry we can look at
this change in arborisation this makes
the invisible wound of war visible this
is very important because if you take a
marine and you tell him that there’s a
problem in his brain they’re not gonna
say much if you if he’s worried about
malingering he may commit suicide but if
you show him the damage then he their
family and the clinicians all work as a
team if a loved one has a brain injury
you want to know what’s broken what does
it mean and how do we cover functions
the standard MRI scan gives you
something like this this is of uncertain
clinical significance not because they
don’t want to help they don’t yet have
the technology what we do is we give you
a 20 page report and for each cable
where is it how what is the integrity of
that cable we can show the patient their
brain the yellow here in the image are
the the images of the fibers you can see
on one side you’re missing a big chunk
on the other side it’s fully there so
they could understand so they can move
on in the process we want to understand
the functional implications so we had an
individual we had an all-terrain vehicle
accident flipped over in a coma for 14
days came out first thing I want to know
because he’s paralyzed I have his body
am I ever gonna walk again he’s no
question while we say if I plot your
flavors I can identify how much you have
and in the leg you’ve lost 60% and you
have a little bit of function in the arm
67% in the hand
97% by this is very important because if
we work on the hand and it’s not going
anywhere
we’re gonna blow this investment of 300
hours to try and bring back function so
we worked first on the lake it came back
then the arm came back
he’s never opened his hand by knowing
what we have we can give them
expectations and we can target the
therapy now I’m engaged in a big reach I
want to develop with my team members by
colleagues something that humanity has
never had a map of the anatomical
connectivity of any individual who it is
useful for us to identify that we need
to drive innovation the technology does
not yet exist it’s gonna be a five to
fifteen year effort it’s a reach to
figure out how we do that now I’m gonna
give you an example that you’re aware of
for reaching which is the experience of
technology to make possible remote
driving vehicles in town right I want to
take you back ten years ago to the first
vehicle challenge where DARPA put up a
bunch of Hill bails they ran off over
the hill bails they ran into the sides
of the barriers they flipped over okay
desert 15 robots zero the brilliance of
the dark DARPA campaign was to expect
make a big reach but expect failure it’s
going to be a bumpy road well they also
provided incremental tests of capability
and what happened is that was an open
competition it brought world talent
together they accomplished steps in that
process well in Pittsburgh we’re doing
what I call the brain connectivity
challenge we want to create a new
technology to make a map of each
individual person’s brain this has to be
done at a level of detail called the
fasciculus sets of bundles of axons
typically several
of those and the individual cables
because we want to map for each
individual what those are here’s an
illustration of the first challenge
which is we want to map from it 500
positions in each eye all the way
through the first stage in the middle of
the brain called the lateral geniculate
of getting exactly the spatial topology
of every one of those particular right
if we do this we can then move on to
other parts of the brain now the problem
is remember of DARPA when it did this
challenge it could put up hay bales to
make a sort of simulated little bitty
road that’s a little bit easy to work
with we needed our version of that and
we created what I call the textile brain
wiring machine where we have the ability
of routing through a complex pattern at
the ten micron accuracy the full path of
one of these fibers so I can see if the
available technology can get the answer
right because now I have ground truth
within that set so up the hill we built
[Music]
what we’re going to do is then map out
64,000 640,000 patterns of tubes within
that set for an early visual system then
we will work on that then we will see
can we image that where we know the
right answer is of every one of those
tubes this is just an illustration of
this for each of these little lines is
sort of the example of an axon that now
is manufactured to look at that
performance we then do an a we’ll be
doing an annual incremental challenge
test can you find the track routes of
the crossings that you’re working with
how did the fascicular come together
travel and pull apart can we get that
accurately and then we will start with
vision but then go on the language and
then go on to attention control just
like DARPA the first trick was could you
not run over the bail hails we will
start with can you get the eyeball to
lgn right once we get that we’ll move on
so what I’ve described for you is a big
reach it’s gonna take harnessing world
talent including perhaps some of the
people in this room we want to put this
together so we can deliver a technology
that hasn’t existed before this reaches
a big team effort it takes lots of
people from different disciplines
including coders by the way is a big
group of that element but researchers
that deal with both the neuroscience the
clinical issues the imaging issues
within that set so I’m gonna leave you
with my favorite quote from arthur c
clarke any sufficiently advanced
technology is indistinguishable from
magic
five years ago everything that i showed
you was impossible ten years ago from
now it may be routine it’s our job here
to reach
make it happen thank you
[Applause]
Please follow and like us: