thank you every generation or so
emerging technologies converge and
something revolutionary happens such is
the case with artificial intelligence or
we know it as AI it’s about to bring
upon another Industrial Revolution you
see back in the day in first Industrial
Revolution it was based upon the fact
that we had discovered the source of
artificial power prior to that in
Agricultural Revolution everything we
did or make was through our muscular
power or animal power the innovation of
Industrial Revolution was based on the
fact that we had harnessed steam feel
fossil power to discover things and make
things the way we want it later down the
road we found electricity and that
helped us electrify everything now with
the advent of computers internet and the
massive amounts of data that we’re
gathering we’re at the forefront years
of we call artificial intelligence yet
another source of artificial power to
enhance what we do and it’s all around
us you see that in Netflix recommending
movies to you you see that in Amazon
recommending products to you you see
that in your snapchats creating those
funny filters ai is all around us yet
it’s not in places that it needs to be
we have better ways of choosing products
on Amazon than we do in recommending
optimal treatment plans for cancer
patients and that is an area that AI
will revolutionize that’s health care
this is an image of a chest CT scan at a
Harvard University study 24 highly
trained radiologists were asked to look
for signs of lung cancer in a series of
images like these astonishingly 20 out
of 24 radiologists failed to discover
that there is a hairy gorilla embedded
in this image
and this is not an indictment of
radiologists who are otherwise highly
trained they’re brilliant and skilled
practitioner of their field however it
is an indication that we have inherent
in abilities and limitations that that
do in hinder on what we look at and that
causes miss diagnosis and ultimately
incorrect treatment that can be gentle
mental to patients so I am a
computational cancer scientist what I do
is we develop AI algorithm that can
automatically go in and read this
medical imaging particularly in my field
digital pathology or pathology tissue
slides and able to not only provide
diagnosis but skip a step and predict
what the outcome of the cancer patient
will be and that helps us in figuring
out an optimal a curated a personalized
treatment plan for cancer patients which
is truly lacking at the moment and will
I’ll talk a little bit more detail about
how the treatment plan at the moment is
so let’s go through the current clinical
workflow of a cancer patient if a
patient is suspected to have a tumor or
anomalies a biopsy is performed a small
tissue sample is taken out for further
investigation we have significant
advancements now to digitize that sample
so no longer need to look at a
microscope you can look at this in our
computer a pathologist
then further investigates and looks at
these tissue slides to figure out if
there’s a cancer if there is then what
is the extent of this cancer what they
look at is the molecular changes that
manifest themselves on cells the cells
change size and shapes and texture and
in some cases orientation and this
process is called grading because a
pathologist will then assign a grade
the grade better the prognosis higher
the grade the more aggressive the
diseases so for example on a protein
prostate cancer uh there’s a rubric
that’s called a Gleason Gleason grading
scheme a grade three cancer might look
like this there’s certain shapes and
size of the cells and the glands they’re
observed by a pathologist similarly a
grade four may look like that in grade
five and that’s based on a pathologists
managed reading these slides and
determining the stage once the stage is
figured out a treatment plan is
suggested physicians get together and
they look at previous clinical trials or
outcome from patients who had similar
great and they look at the average
treatment response of those patients and
a treatment plan is adjusted for that
cancer patient few problems with this
first of all we all know that we all are
different cancer biology is complex
there are thousands of factors that
causes cancer and that drive the cancer
treating to the mean as they call it in
medical terms it’s not always an optimal
plan or it never wrong but it doesn’t
always work out for the patient secondly
the reading of these pathology slides
are highly subjective there are times
where pathologists do not have agreement
amongst themselves particularly on the
borderline cases they say the shape and
the sizes of the cells may differ in
appearance to one pathologist versus
other depending on the number of years
of training they have the institution
they came out of thirdly there are
cities in the world where the entire
city has only two pathologists so what
we do at Case Western University under
dr. malucci’s lab we keep these goals in
mind and come up with an AI that can
automatically read to digital imaging
here which is the tissue pathology and
develop this algorithm that can assess
it
and not only just assess it and predict
at that stage if this patient’s going to
become aggressive or not there are two
benefits of that firstly if the patient
is predicted to become an aggressive
disease we can treat it accordingly
if the tumor is not going to become
aggressive then we may just opt for a
minimal treatment for example in breast
cancer there are the reports suggest
that almost 50% of women do not need
chemotherapy and they can just go they
could just survive with hormonal therapy
but we don’t have an essay or a
predictor that can accurately tell us
that
similarly in prostate cancer if we can
predict if the biochemical recurrence or
the cancer is going to come back or in
other words it’s going to be an
aggressive disease then we can treat it
accordingly and may and for lower
aggressive disease we do not have to
provide chemotherapy or active
surveillance they call it just watch and
wait so how do we go about training in
AI like this well that can read a
pathology slide so I want to give an
example from what you guys may be
familiar with is the facial recognition
the way facial recognition algorithm
works is that it looks for mathematical
relationships at the pixel level that
define a face once it’s done that it
constructs a mathematical relationship
that is unique to just that face so for
example Will Ferrell’s face generates
this unique signature that is only
unique to his face and these are complex
mathematical relationships although
they’re just numbers so next time in a
series of images or videos his face
comes up and that’s all this is getting
tagged or recognized you see the
application of this in facebook tagging
you see this and snapchat now you see
this many other places in fact the new
iPhone has a similar technology but it
looks at three-dimensional stuff rather
than two-dimensional taking this further
the first step in having us read this
pathology slides is to figure out where
the cells are and that’s called an
object recognition or image segmentation
in AI so I want to give it a general
example of how AI does that or how we
built this there’s an image here and
there’s a symbol R which is my alma
mater Rutgers University symbol an AI
starts with an arbitrary shape obviously
it has no idea what it’s looking for but
when we infuse information and knowledge
and mathematical scheme it’s able to go
in and start with this arbitrary shape
but actually conforms to R and the way
it’s doing that is using physics
thermodynamics energy conservation it
what we do is we take this image and
turn it into a field of energies and so
we use the known principles of
mathematics and we train our AI to look
for these sources of energies in an
image and as we know from physics
there’s an equilibrium process and it
reaches an equilibrium when it’s found
object is looking for and as you saw
that it evolved and stretched into that
object
similarly in pathology we deal with a
lot of elliptical things in nature
however they are at times either
connected overlapped or just occluded to
train that what we did here is we have
an ellipse that we can see but there’s a
bar that is running through an ellipse
and that confuses a machine that doesn’t
have a knowledge of how ellipse looks
like so we came up with an AI that
learns the shapes and sizes of ellipse
and it is able to generalize so if you
see here it’s able to evolve and ignores
the bar and gives the ellipse back so
now we’re ready to let the AI go in and
look at the full scale of a digital
digital tissue pathology image and here
it is this is a micro array tissue image
of a prostate cancer the color coded
objects that you see are the cells it’s
gone in and accurately identified all
the cell’s even though they’re
overlapping touching and stuff so I’m
gonna zoom in a bit on the left side you
see
in green outline AI is able to go in and
give us the cells back on this is a very
important step here now that we actually
know cell where the cells are and and
what the boundaries are we’re able to go
and then do measurements of shape size
their orientation theirs their texture
just like a pathologist would do but now
we actually have a quantitative way of
doing this similarly with the face
algorithm now we can generate this
unique signature of this tumor that is
unique to that patient then we run it
through something called machine
learning algorithm it learns the
signature and creates a model once we
have this model now we can feed in
thousands of previous cases of patients
who either survived the cancer or who
did not survive the cancer who were a
high-risk cancer at the beginning were
low risk cancer and how their follow-up
was we take all of that information and
AI then learns on because it’s able to
see this cells and be able to
quantifiably measure them we went beyond
that we actually start to look at what
happens to a cell structure when when a
cancer texts the tissue or the molecular
level it actually changed their
orientation and that’s something very
difficult to see with the human eye you
can see on the right side the AI is
actually created little arrows that
indicates that what orientation these
cells are and so that’s another feature
that is typically very difficult to
appreciate with the human eye but we can
do this at a quantifiable manner this is
actually a facebook graph and the reason
why I’ve brought this up here is that
when we are on Facebook where one entity
let’s imagine that as a tissue or world
as a tissue image if you will a large
tissue image and we’re all cells at
times our connections are close to each
other at times we are farther from each
other so what Facebook does it is it
creates this mathematical graph or a
network
just like this and that helps them
predict who your next friend should be
and that’s how it it’s able to tell you
well you may know this person that’s how
Netflix tells you that well this is a
movie you liked well and the network
graph of things and grand scheme of
things you may like this so we took a
page out of this and we so if you go
back to the image here well these are
all cells but human eye we cannot tell
the relationships here but can we do
something with this algorithm that I
just talked about mathematically created
relationships between them and that’s
exactly what we did so this is another
tissue image and there we created an
algorithm any eye that actually goes in
and looks for these networks of
relationships mathematically so from the
zoom in a little bit here you can notice
here in different colors there are
smaller networks that appear the
interesting about this is that there are
these recurring patterns that can
predict if this cancer is going to
become aggressive or not if this cancer
is going to come back or not and and we
can begin the benefit of doing such a
thing
is that now it gives a wide array of
options to physicians to not just rely
on that average patient previously but
now we can take in these factors and we
have a way of predicting if this
patient’s going to become aggressive and
treat them that way and if the patient’s
not going to become aggressive let’s not
give them chemotherapy so what we did is
we ran experiments like any scientists
we ran on 251 breast cancer images these
were women who had lymph node negative
er positive breast cancers on the left
as a tissue image of an early recurrence
early recurrence means cancer came back
pretty quickly and aggressively and on
the right we have death distant
recurrence that means the the prognosis
was good the cancer didn’t come back for
for a long time
ai was accurately able to identify them
and
the images here are tiny to appreciate
but these are all graph based the
networks that we created and those
networks were a we are able to create
because of the cells that we were able
to identify and those measurements we
were able to do and the relationships
that we’ve built these hidden factors
and patterns that we never could see
with the human eyes now any eye can see
and figure out the predictions we were
able to do 76% but with the testing that
we did and that number may not be as
impressive to other computer scientists
but this is significant if you keep in
mind that the current state of at
laboratory tests that actually does
something similar can only do up to 66
percent and it takes about two weeks
cost $4,000 and it’s only limited to
California so tissue slides have to be
shipped over there this can be done in
two hours over a cloud network worldwide
we ran another set of tests or
experiments on head and neck and tumor
imagism oral pharyngeal cancer we were
able to distinguish between progresses
and non progresses aggressive versus non
progresses but 90 percent and again so
this this was just one other element we
tested we also did a prostate cancer
we’ve also done it for lung cancer so in
the end I want to say that we’re just at
the beginning of the beginning we’re
just the first hour of this revolution
of AI and we’re not gonna sell we’re not
gonna solve cancer by classical clinical
trials but we have this new arsenal new
tool that will help us curate and
personalize a treatment plan for every
cancer patient because every cancer
patient truly deserves to have their own
equation thank you
[Applause]
emerging technologies converge and
something revolutionary happens such is
the case with artificial intelligence or
we know it as AI it’s about to bring
upon another Industrial Revolution you
see back in the day in first Industrial
Revolution it was based upon the fact
that we had discovered the source of
artificial power prior to that in
Agricultural Revolution everything we
did or make was through our muscular
power or animal power the innovation of
Industrial Revolution was based on the
fact that we had harnessed steam feel
fossil power to discover things and make
things the way we want it later down the
road we found electricity and that
helped us electrify everything now with
the advent of computers internet and the
massive amounts of data that we’re
gathering we’re at the forefront years
of we call artificial intelligence yet
another source of artificial power to
enhance what we do and it’s all around
us you see that in Netflix recommending
movies to you you see that in Amazon
recommending products to you you see
that in your snapchats creating those
funny filters ai is all around us yet
it’s not in places that it needs to be
we have better ways of choosing products
on Amazon than we do in recommending
optimal treatment plans for cancer
patients and that is an area that AI
will revolutionize that’s health care
this is an image of a chest CT scan at a
Harvard University study 24 highly
trained radiologists were asked to look
for signs of lung cancer in a series of
images like these astonishingly 20 out
of 24 radiologists failed to discover
that there is a hairy gorilla embedded
in this image
and this is not an indictment of
radiologists who are otherwise highly
trained they’re brilliant and skilled
practitioner of their field however it
is an indication that we have inherent
in abilities and limitations that that
do in hinder on what we look at and that
causes miss diagnosis and ultimately
incorrect treatment that can be gentle
mental to patients so I am a
computational cancer scientist what I do
is we develop AI algorithm that can
automatically go in and read this
medical imaging particularly in my field
digital pathology or pathology tissue
slides and able to not only provide
diagnosis but skip a step and predict
what the outcome of the cancer patient
will be and that helps us in figuring
out an optimal a curated a personalized
treatment plan for cancer patients which
is truly lacking at the moment and will
I’ll talk a little bit more detail about
how the treatment plan at the moment is
so let’s go through the current clinical
workflow of a cancer patient if a
patient is suspected to have a tumor or
anomalies a biopsy is performed a small
tissue sample is taken out for further
investigation we have significant
advancements now to digitize that sample
so no longer need to look at a
microscope you can look at this in our
computer a pathologist
then further investigates and looks at
these tissue slides to figure out if
there’s a cancer if there is then what
is the extent of this cancer what they
look at is the molecular changes that
manifest themselves on cells the cells
change size and shapes and texture and
in some cases orientation and this
process is called grading because a
pathologist will then assign a grade
the grade better the prognosis higher
the grade the more aggressive the
diseases so for example on a protein
prostate cancer uh there’s a rubric
that’s called a Gleason Gleason grading
scheme a grade three cancer might look
like this there’s certain shapes and
size of the cells and the glands they’re
observed by a pathologist similarly a
grade four may look like that in grade
five and that’s based on a pathologists
managed reading these slides and
determining the stage once the stage is
figured out a treatment plan is
suggested physicians get together and
they look at previous clinical trials or
outcome from patients who had similar
great and they look at the average
treatment response of those patients and
a treatment plan is adjusted for that
cancer patient few problems with this
first of all we all know that we all are
different cancer biology is complex
there are thousands of factors that
causes cancer and that drive the cancer
treating to the mean as they call it in
medical terms it’s not always an optimal
plan or it never wrong but it doesn’t
always work out for the patient secondly
the reading of these pathology slides
are highly subjective there are times
where pathologists do not have agreement
amongst themselves particularly on the
borderline cases they say the shape and
the sizes of the cells may differ in
appearance to one pathologist versus
other depending on the number of years
of training they have the institution
they came out of thirdly there are
cities in the world where the entire
city has only two pathologists so what
we do at Case Western University under
dr. malucci’s lab we keep these goals in
mind and come up with an AI that can
automatically read to digital imaging
here which is the tissue pathology and
develop this algorithm that can assess
it
and not only just assess it and predict
at that stage if this patient’s going to
become aggressive or not there are two
benefits of that firstly if the patient
is predicted to become an aggressive
disease we can treat it accordingly
if the tumor is not going to become
aggressive then we may just opt for a
minimal treatment for example in breast
cancer there are the reports suggest
that almost 50% of women do not need
chemotherapy and they can just go they
could just survive with hormonal therapy
but we don’t have an essay or a
predictor that can accurately tell us
that
similarly in prostate cancer if we can
predict if the biochemical recurrence or
the cancer is going to come back or in
other words it’s going to be an
aggressive disease then we can treat it
accordingly and may and for lower
aggressive disease we do not have to
provide chemotherapy or active
surveillance they call it just watch and
wait so how do we go about training in
AI like this well that can read a
pathology slide so I want to give an
example from what you guys may be
familiar with is the facial recognition
the way facial recognition algorithm
works is that it looks for mathematical
relationships at the pixel level that
define a face once it’s done that it
constructs a mathematical relationship
that is unique to just that face so for
example Will Ferrell’s face generates
this unique signature that is only
unique to his face and these are complex
mathematical relationships although
they’re just numbers so next time in a
series of images or videos his face
comes up and that’s all this is getting
tagged or recognized you see the
application of this in facebook tagging
you see this and snapchat now you see
this many other places in fact the new
iPhone has a similar technology but it
looks at three-dimensional stuff rather
than two-dimensional taking this further
the first step in having us read this
pathology slides is to figure out where
the cells are and that’s called an
object recognition or image segmentation
in AI so I want to give it a general
example of how AI does that or how we
built this there’s an image here and
there’s a symbol R which is my alma
mater Rutgers University symbol an AI
starts with an arbitrary shape obviously
it has no idea what it’s looking for but
when we infuse information and knowledge
and mathematical scheme it’s able to go
in and start with this arbitrary shape
but actually conforms to R and the way
it’s doing that is using physics
thermodynamics energy conservation it
what we do is we take this image and
turn it into a field of energies and so
we use the known principles of
mathematics and we train our AI to look
for these sources of energies in an
image and as we know from physics
there’s an equilibrium process and it
reaches an equilibrium when it’s found
object is looking for and as you saw
that it evolved and stretched into that
object
similarly in pathology we deal with a
lot of elliptical things in nature
however they are at times either
connected overlapped or just occluded to
train that what we did here is we have
an ellipse that we can see but there’s a
bar that is running through an ellipse
and that confuses a machine that doesn’t
have a knowledge of how ellipse looks
like so we came up with an AI that
learns the shapes and sizes of ellipse
and it is able to generalize so if you
see here it’s able to evolve and ignores
the bar and gives the ellipse back so
now we’re ready to let the AI go in and
look at the full scale of a digital
digital tissue pathology image and here
it is this is a micro array tissue image
of a prostate cancer the color coded
objects that you see are the cells it’s
gone in and accurately identified all
the cell’s even though they’re
overlapping touching and stuff so I’m
gonna zoom in a bit on the left side you
see
in green outline AI is able to go in and
give us the cells back on this is a very
important step here now that we actually
know cell where the cells are and and
what the boundaries are we’re able to go
and then do measurements of shape size
their orientation theirs their texture
just like a pathologist would do but now
we actually have a quantitative way of
doing this similarly with the face
algorithm now we can generate this
unique signature of this tumor that is
unique to that patient then we run it
through something called machine
learning algorithm it learns the
signature and creates a model once we
have this model now we can feed in
thousands of previous cases of patients
who either survived the cancer or who
did not survive the cancer who were a
high-risk cancer at the beginning were
low risk cancer and how their follow-up
was we take all of that information and
AI then learns on because it’s able to
see this cells and be able to
quantifiably measure them we went beyond
that we actually start to look at what
happens to a cell structure when when a
cancer texts the tissue or the molecular
level it actually changed their
orientation and that’s something very
difficult to see with the human eye you
can see on the right side the AI is
actually created little arrows that
indicates that what orientation these
cells are and so that’s another feature
that is typically very difficult to
appreciate with the human eye but we can
do this at a quantifiable manner this is
actually a facebook graph and the reason
why I’ve brought this up here is that
when we are on Facebook where one entity
let’s imagine that as a tissue or world
as a tissue image if you will a large
tissue image and we’re all cells at
times our connections are close to each
other at times we are farther from each
other so what Facebook does it is it
creates this mathematical graph or a
network
just like this and that helps them
predict who your next friend should be
and that’s how it it’s able to tell you
well you may know this person that’s how
Netflix tells you that well this is a
movie you liked well and the network
graph of things and grand scheme of
things you may like this so we took a
page out of this and we so if you go
back to the image here well these are
all cells but human eye we cannot tell
the relationships here but can we do
something with this algorithm that I
just talked about mathematically created
relationships between them and that’s
exactly what we did so this is another
tissue image and there we created an
algorithm any eye that actually goes in
and looks for these networks of
relationships mathematically so from the
zoom in a little bit here you can notice
here in different colors there are
smaller networks that appear the
interesting about this is that there are
these recurring patterns that can
predict if this cancer is going to
become aggressive or not if this cancer
is going to come back or not and and we
can begin the benefit of doing such a
thing
is that now it gives a wide array of
options to physicians to not just rely
on that average patient previously but
now we can take in these factors and we
have a way of predicting if this
patient’s going to become aggressive and
treat them that way and if the patient’s
not going to become aggressive let’s not
give them chemotherapy so what we did is
we ran experiments like any scientists
we ran on 251 breast cancer images these
were women who had lymph node negative
er positive breast cancers on the left
as a tissue image of an early recurrence
early recurrence means cancer came back
pretty quickly and aggressively and on
the right we have death distant
recurrence that means the the prognosis
was good the cancer didn’t come back for
for a long time
ai was accurately able to identify them
and
the images here are tiny to appreciate
but these are all graph based the
networks that we created and those
networks were a we are able to create
because of the cells that we were able
to identify and those measurements we
were able to do and the relationships
that we’ve built these hidden factors
and patterns that we never could see
with the human eyes now any eye can see
and figure out the predictions we were
able to do 76% but with the testing that
we did and that number may not be as
impressive to other computer scientists
but this is significant if you keep in
mind that the current state of at
laboratory tests that actually does
something similar can only do up to 66
percent and it takes about two weeks
cost $4,000 and it’s only limited to
California so tissue slides have to be
shipped over there this can be done in
two hours over a cloud network worldwide
we ran another set of tests or
experiments on head and neck and tumor
imagism oral pharyngeal cancer we were
able to distinguish between progresses
and non progresses aggressive versus non
progresses but 90 percent and again so
this this was just one other element we
tested we also did a prostate cancer
we’ve also done it for lung cancer so in
the end I want to say that we’re just at
the beginning of the beginning we’re
just the first hour of this revolution
of AI and we’re not gonna sell we’re not
gonna solve cancer by classical clinical
trials but we have this new arsenal new
tool that will help us curate and
personalize a treatment plan for every
cancer patient because every cancer
patient truly deserves to have their own
equation thank you
[Applause]
Please follow and like us: