[Music]
so I’m gonna start with a quote the
intuitive mind is a sacred gift the
irrational mind is a faithful servant
we’ve created a society that has honest
servant and has forgotten the gift I
found this quote from when I was looking
around for other people’s perspective on
intuition because it’s really
fascinating how our brain comes up with
these gut feelings to tell you stuff and
I found this one attributed to Albert
Einstein and it really resonated with me
particular this slightly ominous part
that I think was never as timely as
today in a world where we are deeply
enamored with AI machine learning and
all things data I remember I’m the first
one to plead guilty I fell in love with
data as a 21 year old socially awkward
exchange student in Boulder Colorado
during high school I had no idea what to
do with my life I was good at math but
have no true convictions otherwise and
so I talked to my dad and he said they
will use computers everywhere to study
computer science and you can do anything
and that’s how I found myself enrolled
in the course in artificial intelligence
and artificial neural networks now what
was it about data well for the introvert
itself was a revelation you had this
virtual window on the world through
which you could observe and understand
the world and the people without having
to talk to them
and the words of my dad should prove
truly prophetic a good 17 years later
because not only are computers
everywhere today data science is
everywhere and I had the great luxury
and pleasure to work with incredibly
smart people from all kinds of
industries medicine finance and
advertising but as I progressed in my
career and I started teaching more and
mentoring there was this nagging thing
that couldn’t be taught something else
than algorithms and math it was this gut
feeling thing and that brings us back to
the quote that I started with now I have
a confession to make it’s actually not
by Albert Einstein
sorry guys which is a lesson in
intuition itself because he was known to
have a deep appreciation for intuition
but he was not known to be a very
religious man and so that word of the
sacred gift should have raised some red
flag but then my intuition for data not
language sorry I quote I can actually
attribute this by my friend and
colleague Hilary Mason and she talks
about this fallacy that there’s some
supposed to be some tension between data
and intuition and she claims the data as
a tool to enhance intuition and I’m
taking a slightly different perspective
on this and saying intuition is an
absolutely crucial tool for data science
and the reason it’s so important is
because data is extremely compelling
many of you know this code by Deming
that In God We Trust and all other bring
data but that implies there are some
divine truth to data but it’s there
really may be the facts are true but
often our interpretation and
understanding
it’s wrong the rational mind is really
easily fooled by data an intuition is
the only safeguard
standing in between since then I have
moved on I work today in advertising now
you may wonder why well if you love data
working in advertising is like living in
Disneyland it’s it’s like this huge
playground where you can do almost
anything you want and you can really
push these technologies and see how far
they work what they can do based on this
incredible richness of data and for the
better other words one of the things
that marketers care about is whether you
click on ads because that means that
you’re interested in the product and may
actually want to buy it now technically
that’s a simple question was there an
event that was recorded by the software
of your browser has happened clicking on
an ad but does it actually mean that the
person clicked on an ad now this might
be a bit of a straumann but I do have a
cat that when she gets bored actually
walks over my keyboard it is really fun
if you’re in a video conference but we
have a lot of bots and fraudulence
player that have written software or
programs that pretend to click on ads
and that way make huge amounts of money
selling those ads and also benefiting a
lot from things like fake news that we
see a lot these days but ok fine let’s
assume that it actually is a person
anybody remembers last time you clicked
on an ad all right let me rephrase do
you remember last time something popped
up out of nowhere and you realize that
instead of closing it you must have
accidentally clicked on something there
you go so one of the things where my
expectation and iteration got concerned
is the models that are we
be good at predicting who clicks on ads
always addressed that people who are
using the flashlight app are really into
credit cards and pizzas and hotels we’re
really only the more or less found is
there fumbling in the dark in chances of
accidentally clicking up pretty high so
there may be some notion of technical
truth to that event but the reality is
the interpretation of this is certainly
wrong so there is this paradox that were
having now that on the one hand side as
a society we have a possibly misplaced
reverence for data and then by the same
token the one topic that data scientists
love to lament about is all the tedious
work they have to do cleaning up data
but here’s the thing if you don’t like
cleaning data you shouldn’t be data
scientist that is your job and there’s
nothing trivial about it in fact there’s
nothing dirty about data data often just
wants you to understand where it came
from and it’s our job as a data
scientist to listen one of the parts of
cleaning often means replacing missing
values and yes it’s tedious if you don’t
know the age of the patient but imagine
you don’t have the measurement for the
white blood cell count is that missing
do you need to replace that or is it not
there for very good reason the doctor
actually thinks everything’s fine and
didn’t even bother to measure so the
absence of evidence is in fact evidence
that everything is all right so in this
tension in medicine you also have a deep
understanding that intuition brings to
the place it’s not that they
has the truth but you need to think as a
data scientist about what it means so
data really isn’t dirty
it’s just misunderstood intuition also
helps you to find order of randomness
should be now that sounds weird okay let
me give you an example I have a
thirteen-year-old
and if I come home and his room looks
like this I’m worried I mean that’s the
thing about thirteen year olds they’re
not good with order and this usually
means that either he has something
really bad to confess so he’s trying to
beg for that new video game
either way I’m a bit concerned here’s
another story um he has a very good
friend who lives around the block and
they often hang out and they afternoon
do homework and then one week Monday
Tuesday he came home at exactly 810 two
days maybe that’s a coincidence comes
Thursday he still comes home and when I
take him outside he actually confesses
that well they weren’t exactly doing
homework the whole time they actually
went to a carnival that closed at
exactly eight o’clock thus the very
predictable pattern of him throwing up
at home because usually teenagers are
not terribly predictable so be concerned
when things are not random enough
talking of prediction the human kind has
for the longest time being absolutely
obsessed with predicting the futures
Oracle’s prophecies and there is now a
field in AI a machine learning that
focuses erect Lee on this being able to
teach machines based on historical data
to predict the future or to diagnose
things
now what do people who like to do this
do for fun
anybody here preparing for the Boston
Marathon so we have kind of marathons –
not quite
prize money is about the same about 20
years ago some geeks in machine learning
because back then it wasn’t data science
came up with the idea of data mining
competitions where they take data set
and then they hand it out to the
participants and they try to build the
best possible model on the second half
of the data set they keep the truth of
what actually happened and then they
will declare the winner as whoever has
the most predictive model so you’ve
heard of these maybe as the Netflix
prize we could make a million bucks so
there’s some money in this now but most
of us are in it for the glory one of
these competitions was organized by
Siemens medical and was around detecting
breast cancer they shared with us the
fMRI data of 2000 patients and the task
was to identify either a group of people
who were fou-fou healthy or a good risk
score so doing the most likely people
who had a malignant tumor to the least
and we won this competition not because
we knew anything about medicine not
because we actually detected some secret
signal from this data in fact it had
nothing to do with the algorithm what we
found was something wrong with the data
so let me show you what it looked like
so this is the data set where you have a
patient identifier which is a random
number and this random ID was assigned
to protect the identity of the patient
now anything strikes me as interesting
well the first thing is that the sting
is indeed sorted which is fine but if it
is sorted then what exactly is going on
here I mean what happened to all these
people between half a million and four
million the plot thickens when you look
at the other column which is whether or
not that patient had cancer so remember
what I said about missingness trying to
tell you about the story how the data
was created won’t turns out you really
have two separate groups of patients
here there were the lucky ones was the
low numbers and the unlucky ones but of
course it had nothing to do with luck
one group came from a treatment facility
in the other one from a screening center
now of course if you observe that
winning the competition wasn’t that hard
and again it had nothing to do with the
algorithm but there’s a deeper important
meaning here this data is completely
useless to build any model that detects
cancer because all the algorithm does it
finds out what the average greyscale is
and therefore which of the two centers
it was it doesn’t learn anything about
cancer and that is an important skill
for data scientists to have to tell you
that this is not working the interesting
thing here is that the model actually
looked too good to be true
so one form of intuition is to warn you
if things are actually looking too good
the leading cause of death in intensive
care units is one
Kamille pramanya that means while the
person is in hospital they contract form
of ramonja
that sometimes it’s even resistant
strain and it would be incredibly
important and really save lives if we
could understand how these infections
work and identify people at risk so we
build the model to detect this using all
of the patient information and diagnosis
and medication and the model was
incredibly good it founds this pocket of
people that it was certain would get
this infection a smoking gun
and then you looked into the model and
here’s what it said people who have
nothing else wrong with them will get
this infection now why are they in
hospital in the first place well they
aren’t Hospital in the first place
because they showed up on a Friday
evening in the emergency with an acute
pneumonia and because of all the chaos
it didn’t get recorded until Monday
so the timestamp of the diagnosis wasn’t
the time some of the I diagnosis but
when a person actually got around to
type into the system and now it looked
as if they actually contracted it while
they were in hospital so the lesson here
is if your model is too good to be true
maybe you want to go back and check the
data but it also means in the bigger
picture that the models we built and all
of these AIS are really only as good as
the data intuition of their creators and
they are only to human when you look at
for instance modern culture themes like
person of interest there’s a machine
here that can tell with certainty who in
the next 24 hours will die of a violent
death and this is science fiction today
and will always be
these machines will never get that good
and not just because we aren’t that good
but in fact I refuse to believe that the
world will ever be predictable because I
do wish to believe in the power of free
will and if I can decide what to do with
my next 24 hours there will always be
limits to what we can predict but that
poses a different question if they
aren’t perfect when shall we trust them
with our lives now even if they’re not
perfect they’re pretty damn good they
have beaten us and tress trêpa d poker
turns out poker players think that they
are not predictable in fact they are but
if you know what to look for or if a
machine knows what to look for and today
even our current versions of
self-driving cars are a lot better than
the average teenage driver so 3 years
from now I will sleep very well when my
son comes home in a self-driving car
even if on occasion accidentally it
breaks for traffic because I thought it
was a cat thank you
[Applause]
so I’m gonna start with a quote the
intuitive mind is a sacred gift the
irrational mind is a faithful servant
we’ve created a society that has honest
servant and has forgotten the gift I
found this quote from when I was looking
around for other people’s perspective on
intuition because it’s really
fascinating how our brain comes up with
these gut feelings to tell you stuff and
I found this one attributed to Albert
Einstein and it really resonated with me
particular this slightly ominous part
that I think was never as timely as
today in a world where we are deeply
enamored with AI machine learning and
all things data I remember I’m the first
one to plead guilty I fell in love with
data as a 21 year old socially awkward
exchange student in Boulder Colorado
during high school I had no idea what to
do with my life I was good at math but
have no true convictions otherwise and
so I talked to my dad and he said they
will use computers everywhere to study
computer science and you can do anything
and that’s how I found myself enrolled
in the course in artificial intelligence
and artificial neural networks now what
was it about data well for the introvert
itself was a revelation you had this
virtual window on the world through
which you could observe and understand
the world and the people without having
to talk to them
and the words of my dad should prove
truly prophetic a good 17 years later
because not only are computers
everywhere today data science is
everywhere and I had the great luxury
and pleasure to work with incredibly
smart people from all kinds of
industries medicine finance and
advertising but as I progressed in my
career and I started teaching more and
mentoring there was this nagging thing
that couldn’t be taught something else
than algorithms and math it was this gut
feeling thing and that brings us back to
the quote that I started with now I have
a confession to make it’s actually not
by Albert Einstein
sorry guys which is a lesson in
intuition itself because he was known to
have a deep appreciation for intuition
but he was not known to be a very
religious man and so that word of the
sacred gift should have raised some red
flag but then my intuition for data not
language sorry I quote I can actually
attribute this by my friend and
colleague Hilary Mason and she talks
about this fallacy that there’s some
supposed to be some tension between data
and intuition and she claims the data as
a tool to enhance intuition and I’m
taking a slightly different perspective
on this and saying intuition is an
absolutely crucial tool for data science
and the reason it’s so important is
because data is extremely compelling
many of you know this code by Deming
that In God We Trust and all other bring
data but that implies there are some
divine truth to data but it’s there
really may be the facts are true but
often our interpretation and
understanding
it’s wrong the rational mind is really
easily fooled by data an intuition is
the only safeguard
standing in between since then I have
moved on I work today in advertising now
you may wonder why well if you love data
working in advertising is like living in
Disneyland it’s it’s like this huge
playground where you can do almost
anything you want and you can really
push these technologies and see how far
they work what they can do based on this
incredible richness of data and for the
better other words one of the things
that marketers care about is whether you
click on ads because that means that
you’re interested in the product and may
actually want to buy it now technically
that’s a simple question was there an
event that was recorded by the software
of your browser has happened clicking on
an ad but does it actually mean that the
person clicked on an ad now this might
be a bit of a straumann but I do have a
cat that when she gets bored actually
walks over my keyboard it is really fun
if you’re in a video conference but we
have a lot of bots and fraudulence
player that have written software or
programs that pretend to click on ads
and that way make huge amounts of money
selling those ads and also benefiting a
lot from things like fake news that we
see a lot these days but ok fine let’s
assume that it actually is a person
anybody remembers last time you clicked
on an ad all right let me rephrase do
you remember last time something popped
up out of nowhere and you realize that
instead of closing it you must have
accidentally clicked on something there
you go so one of the things where my
expectation and iteration got concerned
is the models that are we
be good at predicting who clicks on ads
always addressed that people who are
using the flashlight app are really into
credit cards and pizzas and hotels we’re
really only the more or less found is
there fumbling in the dark in chances of
accidentally clicking up pretty high so
there may be some notion of technical
truth to that event but the reality is
the interpretation of this is certainly
wrong so there is this paradox that were
having now that on the one hand side as
a society we have a possibly misplaced
reverence for data and then by the same
token the one topic that data scientists
love to lament about is all the tedious
work they have to do cleaning up data
but here’s the thing if you don’t like
cleaning data you shouldn’t be data
scientist that is your job and there’s
nothing trivial about it in fact there’s
nothing dirty about data data often just
wants you to understand where it came
from and it’s our job as a data
scientist to listen one of the parts of
cleaning often means replacing missing
values and yes it’s tedious if you don’t
know the age of the patient but imagine
you don’t have the measurement for the
white blood cell count is that missing
do you need to replace that or is it not
there for very good reason the doctor
actually thinks everything’s fine and
didn’t even bother to measure so the
absence of evidence is in fact evidence
that everything is all right so in this
tension in medicine you also have a deep
understanding that intuition brings to
the place it’s not that they
has the truth but you need to think as a
data scientist about what it means so
data really isn’t dirty
it’s just misunderstood intuition also
helps you to find order of randomness
should be now that sounds weird okay let
me give you an example I have a
thirteen-year-old
and if I come home and his room looks
like this I’m worried I mean that’s the
thing about thirteen year olds they’re
not good with order and this usually
means that either he has something
really bad to confess so he’s trying to
beg for that new video game
either way I’m a bit concerned here’s
another story um he has a very good
friend who lives around the block and
they often hang out and they afternoon
do homework and then one week Monday
Tuesday he came home at exactly 810 two
days maybe that’s a coincidence comes
Thursday he still comes home and when I
take him outside he actually confesses
that well they weren’t exactly doing
homework the whole time they actually
went to a carnival that closed at
exactly eight o’clock thus the very
predictable pattern of him throwing up
at home because usually teenagers are
not terribly predictable so be concerned
when things are not random enough
talking of prediction the human kind has
for the longest time being absolutely
obsessed with predicting the futures
Oracle’s prophecies and there is now a
field in AI a machine learning that
focuses erect Lee on this being able to
teach machines based on historical data
to predict the future or to diagnose
things
now what do people who like to do this
do for fun
anybody here preparing for the Boston
Marathon so we have kind of marathons –
not quite
prize money is about the same about 20
years ago some geeks in machine learning
because back then it wasn’t data science
came up with the idea of data mining
competitions where they take data set
and then they hand it out to the
participants and they try to build the
best possible model on the second half
of the data set they keep the truth of
what actually happened and then they
will declare the winner as whoever has
the most predictive model so you’ve
heard of these maybe as the Netflix
prize we could make a million bucks so
there’s some money in this now but most
of us are in it for the glory one of
these competitions was organized by
Siemens medical and was around detecting
breast cancer they shared with us the
fMRI data of 2000 patients and the task
was to identify either a group of people
who were fou-fou healthy or a good risk
score so doing the most likely people
who had a malignant tumor to the least
and we won this competition not because
we knew anything about medicine not
because we actually detected some secret
signal from this data in fact it had
nothing to do with the algorithm what we
found was something wrong with the data
so let me show you what it looked like
so this is the data set where you have a
patient identifier which is a random
number and this random ID was assigned
to protect the identity of the patient
now anything strikes me as interesting
well the first thing is that the sting
is indeed sorted which is fine but if it
is sorted then what exactly is going on
here I mean what happened to all these
people between half a million and four
million the plot thickens when you look
at the other column which is whether or
not that patient had cancer so remember
what I said about missingness trying to
tell you about the story how the data
was created won’t turns out you really
have two separate groups of patients
here there were the lucky ones was the
low numbers and the unlucky ones but of
course it had nothing to do with luck
one group came from a treatment facility
in the other one from a screening center
now of course if you observe that
winning the competition wasn’t that hard
and again it had nothing to do with the
algorithm but there’s a deeper important
meaning here this data is completely
useless to build any model that detects
cancer because all the algorithm does it
finds out what the average greyscale is
and therefore which of the two centers
it was it doesn’t learn anything about
cancer and that is an important skill
for data scientists to have to tell you
that this is not working the interesting
thing here is that the model actually
looked too good to be true
so one form of intuition is to warn you
if things are actually looking too good
the leading cause of death in intensive
care units is one
Kamille pramanya that means while the
person is in hospital they contract form
of ramonja
that sometimes it’s even resistant
strain and it would be incredibly
important and really save lives if we
could understand how these infections
work and identify people at risk so we
build the model to detect this using all
of the patient information and diagnosis
and medication and the model was
incredibly good it founds this pocket of
people that it was certain would get
this infection a smoking gun
and then you looked into the model and
here’s what it said people who have
nothing else wrong with them will get
this infection now why are they in
hospital in the first place well they
aren’t Hospital in the first place
because they showed up on a Friday
evening in the emergency with an acute
pneumonia and because of all the chaos
it didn’t get recorded until Monday
so the timestamp of the diagnosis wasn’t
the time some of the I diagnosis but
when a person actually got around to
type into the system and now it looked
as if they actually contracted it while
they were in hospital so the lesson here
is if your model is too good to be true
maybe you want to go back and check the
data but it also means in the bigger
picture that the models we built and all
of these AIS are really only as good as
the data intuition of their creators and
they are only to human when you look at
for instance modern culture themes like
person of interest there’s a machine
here that can tell with certainty who in
the next 24 hours will die of a violent
death and this is science fiction today
and will always be
these machines will never get that good
and not just because we aren’t that good
but in fact I refuse to believe that the
world will ever be predictable because I
do wish to believe in the power of free
will and if I can decide what to do with
my next 24 hours there will always be
limits to what we can predict but that
poses a different question if they
aren’t perfect when shall we trust them
with our lives now even if they’re not
perfect they’re pretty damn good they
have beaten us and tress trêpa d poker
turns out poker players think that they
are not predictable in fact they are but
if you know what to look for or if a
machine knows what to look for and today
even our current versions of
self-driving cars are a lot better than
the average teenage driver so 3 years
from now I will sleep very well when my
son comes home in a self-driving car
even if on occasion accidentally it
breaks for traffic because I thought it
was a cat thank you
[Applause]
Please follow and like us: