[Music]
[Applause]
[Music]
Bell Gibson was a happy young Australian
she lived in Perth and she loves
skateboarding but in 2009 Beldon that
she had brain cancer and four months to
live
two months of chemo and radiotherapy had
no effect but Bell was determined she’d
been a fighter her whole life from age
six she had a cook for her brother who
had autism and her mother had multiple
sclerosis her father was out of the
picture so Bell fought with exercise
with meditation and by ditching meat for
fruit and vegetables and she made a
complete recovery Bell story went viral
it was tweeted blogged about shared and
reach millions of people it showed the
benefits of shunning traditional
medicine for diet and exercise in August
2013 Bell launched a healthy eating app
the whole pantry downloaded two hundred
thousand times in the first month but
Bell’s story was a lie Bell never had
cancer people shared a story without
ever checking if it was true this is a
classic example of confirmation bias we
accept a story on critically if it
confirms what we’d like to be true and
we reject any story that contradicts it
how often do we see this in the stories
that we share and we ignore in politics
in business and health advice the oxford
dictionaries word of 2016 was post truth
and the recognition that we now live in
a post truth world has led to a much
needed emphasis on checking the facts
but the punchline of my talk is that
just checking the facts is not enough
even if Bell’s story were true it would
be just as irrelevant why well let’s
look at one of the most fundamental
techniques in statistics
it’s called Bayesian inference and the
very simple version is this
we care about does the data support the
theory does the data increase our belief
that the theory is true but instead we
end up asking is the data consistent
with the theory but being consistent
with the theory does not mean that the
data supports the theory why because of
a crucial but forgotten third term the
data could also be consistent with rival
theories but due to confirmation bias we
never consider the rival theories
because we’re so protective of our own
pet theory now let’s look at this
football story well we care about does
Bell story support the theory that dark
you–is cancer but instead we end up
asking is Bell story consistent with
diet curing cancer and the answer is yes
if diet did cure cancer we’d see stories
like balance but even if diet did not
cure cancer we’d still see stories like
Bell’s a single story in which a patient
apparently self cured just due to being
misdiagnosed in the first place just
like even if smoking was bad for your
health
you’d still see one smoker who lived
into 100 just like even if education was
good for your income you’d still see one
multi-millionaire who didn’t go to
university
so the biggest problem with Belle’s
story is not that it was false it’s that
it’s only one story there might be
thousands of other stories where diet
alone failed but we never hear about
them we share the outlier cases because
they are new and therefore they are news
we never share the ordinary cases
they’re too ordinary that they’re what
normally happens and that’s the true 99%
that we ignore just like in society you
can’t just listen to the 1% the outliers
and ignore the 99% the ordinary because
that’s the second example of
confirmation bias we accept a fact as
data the biggest problem is not that we
live in a post truth world it’s that we
live in a post data world we prefer a
single story to tons of data now stories
are powerful their vivid they bring it
to life they tell you to start every
talk with the story I did but a single
story is meaningless
and misleading unless it’s backed up by
large-scale data but even if we had
large-scale data that might still not be
enough because it could still be
consistent with rival theories let me
explain a classic study by psychologist
Peter wastin gives you a set of three
numbers and asked you to think of the
rule that generated them so if you were
given two four six what’s the rule well
most people would think it’s successive
even numbers how would you test it well
you’d propose other sets of successive
even numbers 4 6 8 or 12 14 16 and Peter
would say these sets also work but
knowing that these sets also work know
that perhaps hundreds of sets of
successive even numbers also work tells
you nothing because this is still
consistent with rival theories
perhaps the rule is any three even
numbers or any three increasing numbers
and that’s the third example of
confirmation bias accepting data as
evidence even if it’s consistent with
rival theories data is just a collection
of facts evidence is data that supports
one theory and rules out others so the
best way to support your theory is
actually to try to disprove it to play
devil’s advocate so test something like
four twelve twenty six well if you got a
yes to that that would disprove your
theory of successive even numbers yet
this test is powerful because if you got
a no it would rule out any three even
numbers and any three increasing numbers
it would rule out the rival theories but
not rule out yours but most people are
too afraid of testing the four twelve
twenty six because they don’t want to
get a yes and prove their pet theory to
be wrong
like confirmation bias is not only about
failing to search for a new data but
it’s also about misinterpreting data
once you receive this and this applies
outside the lab to important real world
problems
indeed Thomas Edison famously said I
have not failed I have found 10,000 ways
that won’t work finding out the out that
you’re gone is the only way to find out
what’s right say your University
Admissions Director and your theories
the only students with good grades from
rich families do well so you only let in
such students and they do well but
that’s also consistent with a rival
Theory perhaps all students with good
grades do well rich or poor but you
never test that theory because you never
let in poor students because you don’t
want to be proven wrong
so what have we learned a story is not
fact
because it may not be true a fact is not
data it may not be representative if
it’s only one data point and data is not
evidence it may not be supportive if
it’s consistent with rival theories so
what do you do when you’re at the
inflection points of life deciding on a
strategy for your business a parenting
technique for your child or a regimen
for your health how do you ensure that
you don’t have a story but you have
evidence let me give you three tips the
first is to actively seek other
viewpoints read and listen to people you
flagrantly disagree with 90% of what
they say may be wrong in your view but
what if 10% is right as our Stovall said
the mark of an educated man is the
ability to entertain a thought without
necessarily accepting it surround
yourself with people who challenge you
and create a culture that actively
encourages dissent some banks suffered
from groupthink where staff were too
afraid to challenge management’s lending
decisions contributing to the financial
crisis in a meeting appoint someone to
be devil’s advocate against your pet
idea and don’t just hear another
viewpoint listen to it as well as
psychologist Stephen Covey said listen
with the intent to understand not the
intent to reply a dissenting viewpoint
is something to learn from not to argue
against which takes us to the other
forgotten terms in Bayesian influence
because data allows you to learn but
learning is only relative to a starting
point if you started with complete
certainty that your pet theory must be
true then your view won’t change
regardless of what data that you see
only if you are truly open to the
possibility of being wrong
can you ever learn as Leo Tolstoy wrote
the most difficult subjects can be
explained to the most slow-witted man if
he has not formed any idea of them
already but the simplest thing cannot be
made clear to the most intelligent man
if he is firmly persuaded that he knows
already tip number two is listen to
experts now that has the most unpopular
advice that I could give you British
politician Michael Gove famously said
that people in this country have had
enough of experts a recent poll showed
that more people would trust their
hairdresser or the man on the street
then they would leaders of businesses
the health service and even charities so
we respect a teeth whitening formula
discovered by a mum or we listen to an
actresses view on vaccinations we like
people who tell it like it is you go
where they got and we call them
authentic but God feel can only get you
so far God feel would tell you never to
give water to a baby with diarrhea
because it would just flow out the other
end
expertise tells you otherwise you’d
never trust your surgery to the man on
the street you’d want an expert who
spent years doing surgery and knows the
best techniques but that should apply to
every major decision politics business
health advice require expertise just
like surgery so then why are experts so
mistrusted well one reason is they’re
seen as out of touch a Millionaire CEO
couldn’t possibly speak for the man on
the street but true expertise is found
on evidence and evidence stands up for
the man on the street and against the
elites because evidence forces you to
prove it evidence prevents the elites
from imposing their own view with
out proof a second reason why experts
are not trusted is that different
experts say different things for every
expert who claimed that leaving the EU
would be bad for Britain
another expert claimed it would be good
right half of these so-called experts
will be wrong and I have to admit that
most papers written by experts are wrong
or at best make claims that the evidence
doesn’t actually support so we can’t
just take an experts word for it in
November 2016 a study on executive pay
hit national headlines even though none
of the newspapers who covered the study
had even seen the study it wasn’t even
out yet they just took the author’s word
for it
just like with bail nor does it mean
that we can just handpick any study that
happens to support our viewpoint that
would again be confirmation by us nor
does it mean that if seven studies show
a and three show B that a must be true
what matters is the quality and not the
quantity of expertise so we should do
two things first we should critically
examine for credentials of the authors
just like you’ve critically examined the
credentials of a potential surgeon are
they truly experts in the matter or do
they have a vested interest second we
should pay particular attention to
papers published in the top academic
journals now academics are often accused
of being detached from the real world
but this detachment gives you years to
spend on a study to really nail down a
result to rule out those rival theories
and a distinguished correlation from
causation and academic journals involve
peer review where a paper is rigorously
scrutinized by the world’s leading minds
the better the journal the higher the
standard the most elite journals reject
95% of papers now academic evidence
is not everything real-world experience
is critical also and and peer review is
not perfect mistakes are made but it’s
better to go with something checked than
something unchecked if we latch on to a
study because we like the findings
without considering who it’s by or
whether it’s even been vetted there is a
massive chance that that study is
misleading and those of us who claim to
be experts should recognize the
limitations of our analysis very rarely
is it possible to prove or predict
something with certainty yet it’s so
tempting to make a sweeping unqualified
statement
it’s easier to turn into a headline or
to be tweeted in 140 characters but even
evidence may not be proof it may not be
Universal it may not apply in every
setting so don’t say Redwine causes
longer life when the evidence is only
that red wine is correlated with longer
life and only then in people who
exercise as well tip number three is
pause before sharing anything the
Hippocratic oath says first do no harm
what we share is potentially contagious
so be very careful about what we spread
our goal should not be to get likes or
retweets otherwise we only share the
consensus we don’t challenge anyone’s
thinking otherwise we only share what
sounds good
regardless of whether it’s evidence
instead we should ask the following if
it’s a story is it true if it’s true is
it backed up by large-scale evidence if
it is who’s that by what are their
credentials is it published how rigorous
is the journal and ask yourself the
million-dollar question if the same
study was written by the same authors
with the same credentials but found the
opposite result would you still be
willing to believe it and to share it
treating any problem a nation’s economic
problem or an individual’s health
problem is difficult so you must ensure
that we have the very best evidence to
guide us only if it’s true can it be
fact only of its representative can it
be data only if it’s supportive can it
be evidence and only with evidence
can we move from a post truth world to a
pro truth world thank you very much
[Applause]
[Applause]
[Music]
Bell Gibson was a happy young Australian
she lived in Perth and she loves
skateboarding but in 2009 Beldon that
she had brain cancer and four months to
live
two months of chemo and radiotherapy had
no effect but Bell was determined she’d
been a fighter her whole life from age
six she had a cook for her brother who
had autism and her mother had multiple
sclerosis her father was out of the
picture so Bell fought with exercise
with meditation and by ditching meat for
fruit and vegetables and she made a
complete recovery Bell story went viral
it was tweeted blogged about shared and
reach millions of people it showed the
benefits of shunning traditional
medicine for diet and exercise in August
2013 Bell launched a healthy eating app
the whole pantry downloaded two hundred
thousand times in the first month but
Bell’s story was a lie Bell never had
cancer people shared a story without
ever checking if it was true this is a
classic example of confirmation bias we
accept a story on critically if it
confirms what we’d like to be true and
we reject any story that contradicts it
how often do we see this in the stories
that we share and we ignore in politics
in business and health advice the oxford
dictionaries word of 2016 was post truth
and the recognition that we now live in
a post truth world has led to a much
needed emphasis on checking the facts
but the punchline of my talk is that
just checking the facts is not enough
even if Bell’s story were true it would
be just as irrelevant why well let’s
look at one of the most fundamental
techniques in statistics
it’s called Bayesian inference and the
very simple version is this
we care about does the data support the
theory does the data increase our belief
that the theory is true but instead we
end up asking is the data consistent
with the theory but being consistent
with the theory does not mean that the
data supports the theory why because of
a crucial but forgotten third term the
data could also be consistent with rival
theories but due to confirmation bias we
never consider the rival theories
because we’re so protective of our own
pet theory now let’s look at this
football story well we care about does
Bell story support the theory that dark
you–is cancer but instead we end up
asking is Bell story consistent with
diet curing cancer and the answer is yes
if diet did cure cancer we’d see stories
like balance but even if diet did not
cure cancer we’d still see stories like
Bell’s a single story in which a patient
apparently self cured just due to being
misdiagnosed in the first place just
like even if smoking was bad for your
health
you’d still see one smoker who lived
into 100 just like even if education was
good for your income you’d still see one
multi-millionaire who didn’t go to
university
so the biggest problem with Belle’s
story is not that it was false it’s that
it’s only one story there might be
thousands of other stories where diet
alone failed but we never hear about
them we share the outlier cases because
they are new and therefore they are news
we never share the ordinary cases
they’re too ordinary that they’re what
normally happens and that’s the true 99%
that we ignore just like in society you
can’t just listen to the 1% the outliers
and ignore the 99% the ordinary because
that’s the second example of
confirmation bias we accept a fact as
data the biggest problem is not that we
live in a post truth world it’s that we
live in a post data world we prefer a
single story to tons of data now stories
are powerful their vivid they bring it
to life they tell you to start every
talk with the story I did but a single
story is meaningless
and misleading unless it’s backed up by
large-scale data but even if we had
large-scale data that might still not be
enough because it could still be
consistent with rival theories let me
explain a classic study by psychologist
Peter wastin gives you a set of three
numbers and asked you to think of the
rule that generated them so if you were
given two four six what’s the rule well
most people would think it’s successive
even numbers how would you test it well
you’d propose other sets of successive
even numbers 4 6 8 or 12 14 16 and Peter
would say these sets also work but
knowing that these sets also work know
that perhaps hundreds of sets of
successive even numbers also work tells
you nothing because this is still
consistent with rival theories
perhaps the rule is any three even
numbers or any three increasing numbers
and that’s the third example of
confirmation bias accepting data as
evidence even if it’s consistent with
rival theories data is just a collection
of facts evidence is data that supports
one theory and rules out others so the
best way to support your theory is
actually to try to disprove it to play
devil’s advocate so test something like
four twelve twenty six well if you got a
yes to that that would disprove your
theory of successive even numbers yet
this test is powerful because if you got
a no it would rule out any three even
numbers and any three increasing numbers
it would rule out the rival theories but
not rule out yours but most people are
too afraid of testing the four twelve
twenty six because they don’t want to
get a yes and prove their pet theory to
be wrong
like confirmation bias is not only about
failing to search for a new data but
it’s also about misinterpreting data
once you receive this and this applies
outside the lab to important real world
problems
indeed Thomas Edison famously said I
have not failed I have found 10,000 ways
that won’t work finding out the out that
you’re gone is the only way to find out
what’s right say your University
Admissions Director and your theories
the only students with good grades from
rich families do well so you only let in
such students and they do well but
that’s also consistent with a rival
Theory perhaps all students with good
grades do well rich or poor but you
never test that theory because you never
let in poor students because you don’t
want to be proven wrong
so what have we learned a story is not
fact
because it may not be true a fact is not
data it may not be representative if
it’s only one data point and data is not
evidence it may not be supportive if
it’s consistent with rival theories so
what do you do when you’re at the
inflection points of life deciding on a
strategy for your business a parenting
technique for your child or a regimen
for your health how do you ensure that
you don’t have a story but you have
evidence let me give you three tips the
first is to actively seek other
viewpoints read and listen to people you
flagrantly disagree with 90% of what
they say may be wrong in your view but
what if 10% is right as our Stovall said
the mark of an educated man is the
ability to entertain a thought without
necessarily accepting it surround
yourself with people who challenge you
and create a culture that actively
encourages dissent some banks suffered
from groupthink where staff were too
afraid to challenge management’s lending
decisions contributing to the financial
crisis in a meeting appoint someone to
be devil’s advocate against your pet
idea and don’t just hear another
viewpoint listen to it as well as
psychologist Stephen Covey said listen
with the intent to understand not the
intent to reply a dissenting viewpoint
is something to learn from not to argue
against which takes us to the other
forgotten terms in Bayesian influence
because data allows you to learn but
learning is only relative to a starting
point if you started with complete
certainty that your pet theory must be
true then your view won’t change
regardless of what data that you see
only if you are truly open to the
possibility of being wrong
can you ever learn as Leo Tolstoy wrote
the most difficult subjects can be
explained to the most slow-witted man if
he has not formed any idea of them
already but the simplest thing cannot be
made clear to the most intelligent man
if he is firmly persuaded that he knows
already tip number two is listen to
experts now that has the most unpopular
advice that I could give you British
politician Michael Gove famously said
that people in this country have had
enough of experts a recent poll showed
that more people would trust their
hairdresser or the man on the street
then they would leaders of businesses
the health service and even charities so
we respect a teeth whitening formula
discovered by a mum or we listen to an
actresses view on vaccinations we like
people who tell it like it is you go
where they got and we call them
authentic but God feel can only get you
so far God feel would tell you never to
give water to a baby with diarrhea
because it would just flow out the other
end
expertise tells you otherwise you’d
never trust your surgery to the man on
the street you’d want an expert who
spent years doing surgery and knows the
best techniques but that should apply to
every major decision politics business
health advice require expertise just
like surgery so then why are experts so
mistrusted well one reason is they’re
seen as out of touch a Millionaire CEO
couldn’t possibly speak for the man on
the street but true expertise is found
on evidence and evidence stands up for
the man on the street and against the
elites because evidence forces you to
prove it evidence prevents the elites
from imposing their own view with
out proof a second reason why experts
are not trusted is that different
experts say different things for every
expert who claimed that leaving the EU
would be bad for Britain
another expert claimed it would be good
right half of these so-called experts
will be wrong and I have to admit that
most papers written by experts are wrong
or at best make claims that the evidence
doesn’t actually support so we can’t
just take an experts word for it in
November 2016 a study on executive pay
hit national headlines even though none
of the newspapers who covered the study
had even seen the study it wasn’t even
out yet they just took the author’s word
for it
just like with bail nor does it mean
that we can just handpick any study that
happens to support our viewpoint that
would again be confirmation by us nor
does it mean that if seven studies show
a and three show B that a must be true
what matters is the quality and not the
quantity of expertise so we should do
two things first we should critically
examine for credentials of the authors
just like you’ve critically examined the
credentials of a potential surgeon are
they truly experts in the matter or do
they have a vested interest second we
should pay particular attention to
papers published in the top academic
journals now academics are often accused
of being detached from the real world
but this detachment gives you years to
spend on a study to really nail down a
result to rule out those rival theories
and a distinguished correlation from
causation and academic journals involve
peer review where a paper is rigorously
scrutinized by the world’s leading minds
the better the journal the higher the
standard the most elite journals reject
95% of papers now academic evidence
is not everything real-world experience
is critical also and and peer review is
not perfect mistakes are made but it’s
better to go with something checked than
something unchecked if we latch on to a
study because we like the findings
without considering who it’s by or
whether it’s even been vetted there is a
massive chance that that study is
misleading and those of us who claim to
be experts should recognize the
limitations of our analysis very rarely
is it possible to prove or predict
something with certainty yet it’s so
tempting to make a sweeping unqualified
statement
it’s easier to turn into a headline or
to be tweeted in 140 characters but even
evidence may not be proof it may not be
Universal it may not apply in every
setting so don’t say Redwine causes
longer life when the evidence is only
that red wine is correlated with longer
life and only then in people who
exercise as well tip number three is
pause before sharing anything the
Hippocratic oath says first do no harm
what we share is potentially contagious
so be very careful about what we spread
our goal should not be to get likes or
retweets otherwise we only share the
consensus we don’t challenge anyone’s
thinking otherwise we only share what
sounds good
regardless of whether it’s evidence
instead we should ask the following if
it’s a story is it true if it’s true is
it backed up by large-scale evidence if
it is who’s that by what are their
credentials is it published how rigorous
is the journal and ask yourself the
million-dollar question if the same
study was written by the same authors
with the same credentials but found the
opposite result would you still be
willing to believe it and to share it
treating any problem a nation’s economic
problem or an individual’s health
problem is difficult so you must ensure
that we have the very best evidence to
guide us only if it’s true can it be
fact only of its representative can it
be data only if it’s supportive can it
be evidence and only with evidence
can we move from a post truth world to a
pro truth world thank you very much
[Applause]
Please follow and like us: