I love driving I not only love driving I
also love technology and that is why I
was amazed when I saw in 2007 the DARPA
Grand Challenge to take place in the
Mojave Desert where 150 miles across
this desert car is completely
autonomously drove and reached their
destination I was much more amazed when
I saw in 2013 that mercedes-benz took a
route Bertha Benz took in August 1888
with the first Benz from Mannheim to
Heidelberg and when Mercedes completely
autonomously drove all their over this
bridge and actually this bridge crossing
Heidelberg is the bridge I use for my
morning jog and I always thought well
why don’t we have this in medicine and
just to give you a little bit of
information about Medicine I can tell
you one thing surgery is complex much
more complex than you think we always
think about surgery only about the
single operation but it’s not it we have
a preoperative phase and we have a
post-operative phase and if you don’t
care about that you’re not a good
surgeon and you’re not a good physician
why because you can do a perfect
operation but on the wrong patient if
your preoperative assessment is not the
right one
not so good same thing you do perfect
operation perfect preoperative
preparation and then you’re after the
after the operation I did it everything
is fine that’s not going to work out you
have to take care of the patient and you
have to see what is going on with him
after the operation as well so it’s a
whole process and just to give you an
image how difficult it is and how much a
data we already have for one single
patient I show you this diagram this is
a patient who received a liver
transplantation in our hospital and what
you see here is all his laboratory
parameters and on the x-axis you can see
the days and on the YX in axis you can
see the relative faith of the parameters
so who in this room can tell
me whether this patient is well or not
well
impossible I can’t neither usually it’s
not represented like that it’s much more
in a tabular forum but I’ll show you one
thing what you have to look for is
complication patterns and this
complication pattern for example is the
operation here in the first days
elevates liver enzymes inside of the
blood which is okay which is normal but
then they fall down which is also normal
but then there is this double peak
formation on day nine it’s building up
and then it’s fully there if you take
care of this patient on day nine he will
be fine that is what happened in this
case but if you wait too long time to
response this patient might die he did
not he is now well off but just to show
you this is the patterns that we trained
and that we have to provide all patients
because there are more patients inside
of our Hospital than only one and they
come in different phases and there is
also a different effort and also
different diseases and not only that
inside of the hospital we also have
different specialities so what do we do
how do we train all this and how do we
go there well one thing is we
specialized specialization for
everything but we also have to look on
medical literature and you have to train
your whole life how do you do that well
you go on MEDLINE MEDLINE is our
worldwide Medical Library and there just
to give you an impression how much it is
last year where 185 thousand
publications released tagged surgery I
read them all okay so reality is we have
to take care of our patients we have to
take care and taking into account all
his individual data may be imaging may
be laboratory parameters or whatever and
also our experience and also experience
from other people in concerning studies
or guidelines etc all this in a very
complex
setting well this may lead to error and
what you can see here is the Swiss
cheese model I don’t know if you are
familiar with it but it’s very
fascinating to err is human you see here
a lot of layers and each layer
represents a human so a nurse or a
doctor and on the far left corner you
can see an error and this error passes
each layer so it’s not seen by the nurse
it’s not seen by the doctor happens
every day but what comes out if we have
a lot of layers and nobody sees the
error maybe it becomes a fatal error and
what do we need there is so many
possibilities there are so many
different diseases and also so much data
that we need an assistant
we need and cognitive as assistant we
need a human-like machine that perceives
information uses a knowledge base
interprets the data based on the
perception and the knowledge base
knowledge base to perform an intelligent
context-aware action and i will show you
some examples that we performed in this
area and i want to say that none of this
is only my work it’s rather a joint
effort of a multidisciplinary team and
mostly together with other medical
colleagues but also with the council we
Institute of Technology and the German
Cancer Research Center and together we
made all this possible so just an
example perioperative
imaging maybe you are familiar with cat
scans so this is an example of a cat
scan of a patient with a colorectal
metastases so whoever now sees it
already is very good we are very good in
pattern recognition I will help you
the liver is on the left side of the
picture and there in the middle you can
see this small well we say hi to dens
lesion small lesion so that’s what
usually the radiologist sees and
yesterday radiology worked like this
he took his dictaphone he saw this
lesion and he dictated a prose text so
his individual analog human
interpretation
this lesion and then the next
radiologist came and next radiologist
next radiologist and you had the picture
on one side and yet on the other side
the texts but they were not together in
one database so what is possible today
you not only look at that but you just
really measure each individual lesion
and then you can see a diameter but not
only the diameter there are lots of
parameters that you can generate out of
this and also follow it up through the
course of the disease well what can you
do with this then is to implement
clinical guidelines that are respected
internationally in this case resist one
point one guideline and together with
those profiles and those categories and
attributes you can really perform a
context adaptive report so you will have
one for example for prostate cancer you
will have one for lung cancer you will
have one for chemotherapy follow-up but
what is also much more important is that
you just combined the image and the
information in one report and what you
really can do then is that all of your
images on all your data is then a
searchable database in clinical routine
use imagine what is possible with a
technology like that well I’m a surgeon
so I have to talk about surgery and I’m
also inside of the operating room I
looked at all the pictures
preoperatively well I can show you one
of our modern hybrid ORS and what you
can see there is a very complex setting
it got more and more and more and there
are a lot of medical devices in fact in
our Hospital alone there are 40,000
medical devices today just to improve or
just to take care of the patient in the
best way possible 40,000 and all of
these medical devices are effectors and
that’s what they but they are well meant
to be they do something for the patient
or we do something with them for the
patient but all I want to say to you is
that they are not only effectors they
are in in reality also
census just to give you an example we
use to open the abdomen that is
necessary for the operation an electric
scalpel and this electric scalpel
we use and then operation begins usually
when you’re not seeing an electric
scuttle as a sensor the nurse and the
anesthesiologist will ask after the
operation homage wow it took a long time
what do you think when did we start the
operation and the nurse will say I think
it was 2:30 and then the still
anesthesiologist will say I think it was
250 ok let’s say 240 well that’s maybe
not how it should be done it should not
be done by a human it’s also a surplus
effort of documentation and you have the
Electric scalpel who knew when it was if
it were not that dump so you have to
integrate those information as sensor
data well in these images are not for
the faint of heart but I think you will
have a nice impression of how a 3d
representation of the abdomen can be
made on the upper left corner and how
clinical tasks can be assessed and can
be seen and extracted from the imaging
which you can see on the upper images
and on the lower right you can see that
the different parameters change
throughout the operation so for each
phase of the operation you will have a
different activation of set of medical
devices while you take this and this is
how an abdomen looks like when you do
laparoscopic surgery so you take this
and give this a computer and then he can
interpret what phase of the operation
are we had and that scan that you can
see on the right hand corner and then
you can show perioperative
imaging preoperative imaging intra
operative Lee in a context-aware manner
so you can see the relevant information
for this phase or you can say well
probably the operation is in phase 3 it
will take another 30 minutes to finish
it finish it so I will inform the nurse
on the world to bring the next patient
and she can organize
herself in a better way it’s not only
that process optimization it’s also
automation of surgical tasks so what we
did is we trained a robot to guide the
laparoscopic camera and this
laparoscopic camera was trained on 20
operations and these 20 operations as
we’re then calculated like good image
bad image medium image and from that it
was able to anticipate the next moves up
of the surgeon and was truly able to
assist the surgeon and not only follow
the surgical instruments because
sometimes they are out of you and they
should be out of you because the more
interesting part is in the lower abdomen
and focused on the other instrument but
it’s not enough to talk about the
operation room it’s also about clinical
processes and it’s also about the
hospital here you can see 18 thousand
patients following a clinical process
inside of our hospital and we depict
this and what you can see here are these
cohorts and what is very astounding
about these cohorts is that there are
similarities between cohorts that were
not obvious before because it were
different specialties one was miss Royal
surgery we won Ross urology and they did
not realize that they have patients that
go together very well well I talked
about building we will have a new
building a new surgical building in
Heidelberg which I can take off and it
will be open in 2019 and will be what we
do there is you have to give all of this
home and we will have a very nice home
and these rooms will be staffed with
parts of this technology to really
enable a digital clinic and what you can
see here is the digital sibling that we
already established which is now our
intermediate care unit and what you can
see here is the clinical processes
evolve inside of the hospital you can
see the nurse caring for the patient you
can see a visitor and you can follow all
of those processes what
can you do with it you can do education
you can do validation you can do
training on it and you can also do
process optimization and also there are
doctors walking over there in the LA and
you can watch them go on the morning
round and after the morning route go to
the or are all inside of this virtual
model speaking of the doctor now one of
the most fascinating things are these
cognitive agents that are already out
there so maybe you know the IBM Watson
system and we call the system
I am Watson MD and what we did we had
access together with our friends and at
the cows were Institute of Technology to
the IBM Watson Explorer program and
Watson Explorer program lets you well
work on a certain project and we said
hey why not pass the USMLE USMLE is the
United States medical licensing exam and
we tried to pass this exam with that
system well what you can see here is an
example question and you can see that
they are rather complex so there’s a
long story and again there is relevant
information there is much irrelevant
information and you have nine possible
answers guessing rate is about 15% if
you are pretty knowledgeable or with
human with good human sense you would
make it probably 225 but not much more
we trained the system and this system
learned by Wikipedia all medical
articles there which were around 40,000
and also some open source medical
textbooks and then also trained the
system on 800 USMLE questions and guess
what how much we did we got well we did
not pass the exam
otherwise I would have changed my
profession maybe long ago no it we are
close we were close it was 50% which is
much more than an average surgeon would
be able to do when he when he is five or
six years in in in the clinic 65% are
there to pass and you won’t want a
system that passes with a 65% that is
clear and take care of this but just
imagine what it means United States
medical licensing exam when you pass it
you are allowed to diagnose you are
allowed to administer therapies and let
me say this I think by 2020 this system
will pass that test and will change our
future and what I really love is driving
and I would love to drive a Tesla this
is not possible now that what they do is
right it’s amazing what they did they
issued an update and the Model S could
drive autonomously just like that
isn’t it amazing and then another issue
and the car drove faster a issue of an
update and the car drove faster so I
think this is really a paradigm that we
will have to adapt adopt this is the
thing that updates will be driving the
healthcare revolution why because we
started eminence based eminence based is
the one very good search and one very
good physician who has 100 cases in his
mind but the next step will be to
implement evidence-based medicine inside
of them you can remember a few studies
but not all of them but combine them
inside of a system a cognitive system
with real-world data and then you will
be truly there that software will be a
drug software is a drug that will
update our healthcare system and
perceive information based on a large
base interpret it and give an
appropriate context away action this is
true context of an intelligent assistant
assistants will come and they will
provide everybody the patient the
surgeon the nurse even the manager with
the right action the relevant interval
information at the right time at the
right place and this will happen very
soon thank you
[Applause]
also love technology and that is why I
was amazed when I saw in 2007 the DARPA
Grand Challenge to take place in the
Mojave Desert where 150 miles across
this desert car is completely
autonomously drove and reached their
destination I was much more amazed when
I saw in 2013 that mercedes-benz took a
route Bertha Benz took in August 1888
with the first Benz from Mannheim to
Heidelberg and when Mercedes completely
autonomously drove all their over this
bridge and actually this bridge crossing
Heidelberg is the bridge I use for my
morning jog and I always thought well
why don’t we have this in medicine and
just to give you a little bit of
information about Medicine I can tell
you one thing surgery is complex much
more complex than you think we always
think about surgery only about the
single operation but it’s not it we have
a preoperative phase and we have a
post-operative phase and if you don’t
care about that you’re not a good
surgeon and you’re not a good physician
why because you can do a perfect
operation but on the wrong patient if
your preoperative assessment is not the
right one
not so good same thing you do perfect
operation perfect preoperative
preparation and then you’re after the
after the operation I did it everything
is fine that’s not going to work out you
have to take care of the patient and you
have to see what is going on with him
after the operation as well so it’s a
whole process and just to give you an
image how difficult it is and how much a
data we already have for one single
patient I show you this diagram this is
a patient who received a liver
transplantation in our hospital and what
you see here is all his laboratory
parameters and on the x-axis you can see
the days and on the YX in axis you can
see the relative faith of the parameters
so who in this room can tell
me whether this patient is well or not
well
impossible I can’t neither usually it’s
not represented like that it’s much more
in a tabular forum but I’ll show you one
thing what you have to look for is
complication patterns and this
complication pattern for example is the
operation here in the first days
elevates liver enzymes inside of the
blood which is okay which is normal but
then they fall down which is also normal
but then there is this double peak
formation on day nine it’s building up
and then it’s fully there if you take
care of this patient on day nine he will
be fine that is what happened in this
case but if you wait too long time to
response this patient might die he did
not he is now well off but just to show
you this is the patterns that we trained
and that we have to provide all patients
because there are more patients inside
of our Hospital than only one and they
come in different phases and there is
also a different effort and also
different diseases and not only that
inside of the hospital we also have
different specialities so what do we do
how do we train all this and how do we
go there well one thing is we
specialized specialization for
everything but we also have to look on
medical literature and you have to train
your whole life how do you do that well
you go on MEDLINE MEDLINE is our
worldwide Medical Library and there just
to give you an impression how much it is
last year where 185 thousand
publications released tagged surgery I
read them all okay so reality is we have
to take care of our patients we have to
take care and taking into account all
his individual data may be imaging may
be laboratory parameters or whatever and
also our experience and also experience
from other people in concerning studies
or guidelines etc all this in a very
complex
setting well this may lead to error and
what you can see here is the Swiss
cheese model I don’t know if you are
familiar with it but it’s very
fascinating to err is human you see here
a lot of layers and each layer
represents a human so a nurse or a
doctor and on the far left corner you
can see an error and this error passes
each layer so it’s not seen by the nurse
it’s not seen by the doctor happens
every day but what comes out if we have
a lot of layers and nobody sees the
error maybe it becomes a fatal error and
what do we need there is so many
possibilities there are so many
different diseases and also so much data
that we need an assistant
we need and cognitive as assistant we
need a human-like machine that perceives
information uses a knowledge base
interprets the data based on the
perception and the knowledge base
knowledge base to perform an intelligent
context-aware action and i will show you
some examples that we performed in this
area and i want to say that none of this
is only my work it’s rather a joint
effort of a multidisciplinary team and
mostly together with other medical
colleagues but also with the council we
Institute of Technology and the German
Cancer Research Center and together we
made all this possible so just an
example perioperative
imaging maybe you are familiar with cat
scans so this is an example of a cat
scan of a patient with a colorectal
metastases so whoever now sees it
already is very good we are very good in
pattern recognition I will help you
the liver is on the left side of the
picture and there in the middle you can
see this small well we say hi to dens
lesion small lesion so that’s what
usually the radiologist sees and
yesterday radiology worked like this
he took his dictaphone he saw this
lesion and he dictated a prose text so
his individual analog human
interpretation
this lesion and then the next
radiologist came and next radiologist
next radiologist and you had the picture
on one side and yet on the other side
the texts but they were not together in
one database so what is possible today
you not only look at that but you just
really measure each individual lesion
and then you can see a diameter but not
only the diameter there are lots of
parameters that you can generate out of
this and also follow it up through the
course of the disease well what can you
do with this then is to implement
clinical guidelines that are respected
internationally in this case resist one
point one guideline and together with
those profiles and those categories and
attributes you can really perform a
context adaptive report so you will have
one for example for prostate cancer you
will have one for lung cancer you will
have one for chemotherapy follow-up but
what is also much more important is that
you just combined the image and the
information in one report and what you
really can do then is that all of your
images on all your data is then a
searchable database in clinical routine
use imagine what is possible with a
technology like that well I’m a surgeon
so I have to talk about surgery and I’m
also inside of the operating room I
looked at all the pictures
preoperatively well I can show you one
of our modern hybrid ORS and what you
can see there is a very complex setting
it got more and more and more and there
are a lot of medical devices in fact in
our Hospital alone there are 40,000
medical devices today just to improve or
just to take care of the patient in the
best way possible 40,000 and all of
these medical devices are effectors and
that’s what they but they are well meant
to be they do something for the patient
or we do something with them for the
patient but all I want to say to you is
that they are not only effectors they
are in in reality also
census just to give you an example we
use to open the abdomen that is
necessary for the operation an electric
scalpel and this electric scalpel
we use and then operation begins usually
when you’re not seeing an electric
scuttle as a sensor the nurse and the
anesthesiologist will ask after the
operation homage wow it took a long time
what do you think when did we start the
operation and the nurse will say I think
it was 2:30 and then the still
anesthesiologist will say I think it was
250 ok let’s say 240 well that’s maybe
not how it should be done it should not
be done by a human it’s also a surplus
effort of documentation and you have the
Electric scalpel who knew when it was if
it were not that dump so you have to
integrate those information as sensor
data well in these images are not for
the faint of heart but I think you will
have a nice impression of how a 3d
representation of the abdomen can be
made on the upper left corner and how
clinical tasks can be assessed and can
be seen and extracted from the imaging
which you can see on the upper images
and on the lower right you can see that
the different parameters change
throughout the operation so for each
phase of the operation you will have a
different activation of set of medical
devices while you take this and this is
how an abdomen looks like when you do
laparoscopic surgery so you take this
and give this a computer and then he can
interpret what phase of the operation
are we had and that scan that you can
see on the right hand corner and then
you can show perioperative
imaging preoperative imaging intra
operative Lee in a context-aware manner
so you can see the relevant information
for this phase or you can say well
probably the operation is in phase 3 it
will take another 30 minutes to finish
it finish it so I will inform the nurse
on the world to bring the next patient
and she can organize
herself in a better way it’s not only
that process optimization it’s also
automation of surgical tasks so what we
did is we trained a robot to guide the
laparoscopic camera and this
laparoscopic camera was trained on 20
operations and these 20 operations as
we’re then calculated like good image
bad image medium image and from that it
was able to anticipate the next moves up
of the surgeon and was truly able to
assist the surgeon and not only follow
the surgical instruments because
sometimes they are out of you and they
should be out of you because the more
interesting part is in the lower abdomen
and focused on the other instrument but
it’s not enough to talk about the
operation room it’s also about clinical
processes and it’s also about the
hospital here you can see 18 thousand
patients following a clinical process
inside of our hospital and we depict
this and what you can see here are these
cohorts and what is very astounding
about these cohorts is that there are
similarities between cohorts that were
not obvious before because it were
different specialties one was miss Royal
surgery we won Ross urology and they did
not realize that they have patients that
go together very well well I talked
about building we will have a new
building a new surgical building in
Heidelberg which I can take off and it
will be open in 2019 and will be what we
do there is you have to give all of this
home and we will have a very nice home
and these rooms will be staffed with
parts of this technology to really
enable a digital clinic and what you can
see here is the digital sibling that we
already established which is now our
intermediate care unit and what you can
see here is the clinical processes
evolve inside of the hospital you can
see the nurse caring for the patient you
can see a visitor and you can follow all
of those processes what
can you do with it you can do education
you can do validation you can do
training on it and you can also do
process optimization and also there are
doctors walking over there in the LA and
you can watch them go on the morning
round and after the morning route go to
the or are all inside of this virtual
model speaking of the doctor now one of
the most fascinating things are these
cognitive agents that are already out
there so maybe you know the IBM Watson
system and we call the system
I am Watson MD and what we did we had
access together with our friends and at
the cows were Institute of Technology to
the IBM Watson Explorer program and
Watson Explorer program lets you well
work on a certain project and we said
hey why not pass the USMLE USMLE is the
United States medical licensing exam and
we tried to pass this exam with that
system well what you can see here is an
example question and you can see that
they are rather complex so there’s a
long story and again there is relevant
information there is much irrelevant
information and you have nine possible
answers guessing rate is about 15% if
you are pretty knowledgeable or with
human with good human sense you would
make it probably 225 but not much more
we trained the system and this system
learned by Wikipedia all medical
articles there which were around 40,000
and also some open source medical
textbooks and then also trained the
system on 800 USMLE questions and guess
what how much we did we got well we did
not pass the exam
otherwise I would have changed my
profession maybe long ago no it we are
close we were close it was 50% which is
much more than an average surgeon would
be able to do when he when he is five or
six years in in in the clinic 65% are
there to pass and you won’t want a
system that passes with a 65% that is
clear and take care of this but just
imagine what it means United States
medical licensing exam when you pass it
you are allowed to diagnose you are
allowed to administer therapies and let
me say this I think by 2020 this system
will pass that test and will change our
future and what I really love is driving
and I would love to drive a Tesla this
is not possible now that what they do is
right it’s amazing what they did they
issued an update and the Model S could
drive autonomously just like that
isn’t it amazing and then another issue
and the car drove faster a issue of an
update and the car drove faster so I
think this is really a paradigm that we
will have to adapt adopt this is the
thing that updates will be driving the
healthcare revolution why because we
started eminence based eminence based is
the one very good search and one very
good physician who has 100 cases in his
mind but the next step will be to
implement evidence-based medicine inside
of them you can remember a few studies
but not all of them but combine them
inside of a system a cognitive system
with real-world data and then you will
be truly there that software will be a
drug software is a drug that will
update our healthcare system and
perceive information based on a large
base interpret it and give an
appropriate context away action this is
true context of an intelligent assistant
assistants will come and they will
provide everybody the patient the
surgeon the nurse even the manager with
the right action the relevant interval
information at the right time at the
right place and this will happen very
soon thank you
[Applause]
Please follow and like us: