there you place a huge bunch of grapes
on the scale and get a nasty surprise
because the numbers shown on the scale
is not what you expect but you you need
the grapes but you would hate to pay too
much and so you think to yourself never
mind but then you notice the greengrocer
look in your direction and he’s looking
too intensely at you you think and to
make make matters worse he’s holding a
large wad of cash in his hand which by
now you’re thinking he’s probably
appropriated from the poor fools who
stood there before you but little does a
green grocer know that you are no fool
you know something about collective
intelligence and now you plan to conjure
up the wisdom of the crowds right there
in the green growth grocery store
because you’ve watched all against one
the block posted Danish gameshow on
based on wisdom of the crowds and you’ve
seen just how easily the average opinion
of ordinary viewers competed with the
particularly bright participants in the
studio so now you long to approach the
greedy green green greedy green grocer
with accurate information and tell me
about the error of his ways and a scale
so you begin to ask your fellow shoppers
what they think about this bunch of
graves so that you can calculate the
average opinion and after many awkward
moments you have your answer
875 grams 25 grams short of the 900
shown on the scale now admittedly this
25 gram deviation is a lot less than you
thought but cheating is cheating you
think and triumphantly you make your way
to the counter the green grocer has been
expecting you which obviously bugs you
bought what really troubles you is what
you instantly see behind his cancer it’s
a large pile of literature on collective
intelligence and it even includes
similar works on the wisdom of crowds
and then the Green Grove spree
I see what you’ve been doing he says but
there’s something that you should know
while your strategy is remarkably
powerful under normal circumstances it
and not my scale is wrong here because
you’re dealing with an item far from our
typical experience and then it begins to
lecture now we have long known that the
average opinion of many people provides
excellent information about the truth
but there’s no reason to believe that
the average opinion errors for reasons
of psychology when the magnitude being
presented is extreme in some way now
what you have to do he almost whispers
is look at the entire distribution of
judgments and not just the central
tendencies in particular you have to
look at the way judgments skew or slant
because that’s information that you can
use to infer when the average opinion is
erroring you could say concludes the
knowledgeable greengrocer that skew is a
cue for our collective error and with
that you can do even better perhaps even
as well as my mechanical scale now the
greengrocer is correct and that’s
important for all the reasons why M good
decisions and good judgments matter in
our lives but let me explain why it’s
correct and turn to the question of
significance as I conclude my talk and
along the way I’ll invoke three things
an old probability machine and two
psychological theories now psychologists
have long established that when we face
the same magnitude many times our
judgments is subject to variation it’s
kind of like when you go home from work
and it did 30 minutes it actually takes
sometimes feels longer and sometimes
feels shorter and the probability
machine I’m invoking does an excellent
job in capturing this phenomenon it was
invented and built by the half cousin of
Charles Darwin Sir Francis Galton
in 1873 their golden coal this machine
the Queen comes but we’ll just call it
the golden board it consists of displays
rows of pins at the top and equally
spaced their compartments at the bottom
that serve the purpose of capturing
balls have dropped into and dropped
through the machine now you may think
about the individual path of
such a ball as a line of thought in the
cognitive process and you may think
about the particular compartment that a
ball ultimately settles into as an
estimate of magnitude in our case an
estimate of weight now along the way
this thought process is disrupted by
random events as captured by bolts at
head pins in the otherwise straight fall
and where exactly this ball will
ultimately end up and we cannot actually
predict for certain but we can say
something about the pattern that emerges
when we drop many balls into the system
one after the other and that pattern you
might know it’s a famous bell curve but
the bell curve is symmetric and skewed
judgment distributions occur much more
frequently than predicted by the golden
board and so we must change it in some
way in order to learn why that is and
the first change that we make to the
golden board is to combine that with
what’s called adaptation level theory
according to which we become used to the
typical magnitudes of our experience and
presume these magnitudes as we then
confront new items in our case we become
used to the typical weight of a bunch of
grapes from our shopping experience in
the setting of the the Queen conquers
are the golden board this implies that
the ball begins its journey and starting
at a magnitude that is typical in our
case let’s assume that’s 500 grams but
of course it’s often important
differences between the particular and
the typical and the second change that
we make to the golden board is to assume
that people will try to account for
these differences and do so according to
what’s called sequential sampling theory
now the basic idea here is that what we
judge has features and these features
contain information and this information
is consistent with one of two hypotheses
that relate to the typical value in our
case heavier than typical or lighter
than typical for example a bunch of
grapes can have particularly plump
fruits which would tends to suggest that
we’re dealing with an item that’s
heavier than typical but that same bunch
of grapes can have particularly sparse
fruits in places which would suggest
suggests
opposite now the these features are now
captured by the rows of pins in the
golden board with the probability of a
ball moving in the same direction as the
evidence provided by the particular
feature capturing the expertise or the
adeptness of the judge at the task for
example a wine producer will be much
more adept at extracting useful
information from a bunch of grapes then
will your typical shoppers so it’s by
combining the gulten board with
adaptation level theory and sequential
sampling theory that we finally reach M
the Augmented Queen Kong’s or the aq for
short and the aq can tell us why skew is
a cue for our collective error the aq
predicts then when the percent in
magnitude is typical than for the people
in the crowd then these people form
their average opinion of these people
will be perfectly aligned with the truth
irrespective of how they use information
moreover the judgment distribution say
together form will be symmetric under
these circumstances the wisdom of the
crowds is almost magic but when the
percent of magnitude is in some way
extreme then how we use information
matters for both collective error and
for skewness in actual fact it’s when we
have some level of expertise that skewed
judgment distributions are predicted to
occur in the first place it’s almost
like a swarm of birds where the majority
head towards one clear direction but a
few laggards trailed behind leaving the
impression of a long tail in the
distribution so for example when the
presented magnitude is smaller than
typical then these swarm not of birds
but of judgments head towards small
values whereas it heads towards large
values when the percent of magnitude is
is larger as in the case of the
greengrocers store and you might think
that well that’s all well and good but
in both cases the average is dragged
down by the tail end of this
distribution in an unfortunate way and
because skewness well and that brings me
to the final most important point of my
talk namely that the deviation between
truth and the average opinion
systematically varies with this
observable skewness and that’s because
it’s observable that we can simply say
that skew is a cue for our collective
error now you might think measuring the
weight of a bunch of grapes is using the
collective minds of shoppers is a simple
and perhaps impractical example but
looking more broadly using modern
information technology we can now hope
to harness the wisdom of crowds much
more efficiently than we have previously
done now that we know that skew is a cue
for error
indeed doctors managers and even
politicians who care about the facts who
do look do well to look at the entire
distribution judgments because they
might just save their patients and them
a higher profit or increase the welfare
of citizens by bringing these important
decision-makers closer to the truth so
it doesn’t matter if you are buying a
bunch of grave so if you’re running a
country if you want to know the truth
then skew is your cue
[Applause] [Music]