slide looked like this and I thought
well that’s the good news right away so
let’s just get that out of the way well
that didn’t exactly happen as we know
and I was shocked I mean I was shocked
about the course American politics is
taking by what is also shocked because I
am doing election forecasting and
together with three American colleagues
we are running this website called the
Polly vote where we are predicting the
outcome of this election and we expected
that Hillary Clinton would win the
popular vote which she ended up winning
and we were within the margin of error
but who cares right we predicted her to
also win the electoral college and
that’s how Americans decide who the
president is and there was actually a
very well humbling experience and it was
a big mistake but we weren’t the only
ones pretty much everyone who was doing
electoral college predictions got it
wrong now to be fair somewhere a little
bit more conservative from Clinton’s
chances than others but the consensus
really was that she is going to win this
and even people in the Trump campaign
didn’t have a clue that they really have
a chance of this so the question really
is what went wrong with the predictions
now many people now blame the polls but
actually if you look at the national
polls for example they were pretty good
the error was actually lower than it was
in 2012 but nobody paid attention
because it didn’t matter the problem
this year was that we had large error in
some states that ended up being decisive
for the outcome of this election
particular Pennsylvania Wisconsin and
Michigan now to be honest it’s still too
early to say what exactly went wrong we
know that polls doing polls is difficult
it’s harder and harder to reach people
it’s harder and harder to get people to
respond to polls in the first place and
you really have a problem if certain
demographic groups that tend maybe
towards a certain candidate don’t
respond to polls at all now it looks
like this is what kind of happened it’s
when you don’t have a representative
it’s really hard to model the electorate
and what we think what happened is that
the pollsters underestimated the turnout
for Trump from white voters with other
college education and on the other hand
over a stood Clinton support among
different demographic groups and there
was also big uncertainty about the share
of undecided voters who appear to have
broken towards Donald Trump very late in
the campaign but it wasn’t just the
polls and I think we can learn a lot by
looking at other forecasts and how they
failed to learn about what we didn’t
understand about this election so I’ve
been involved in election forecasting
for almost eight years now I was a
visiting scholar at the University of
Pennsylvania where Weber Scott Armstrong
who is one of the leading force
researchers in in forecasting and he got
me involved in the Pali Road project and
the idea of this project was to use
findings from forecasting research and
applied them to election forecasting and
demonstrate that they work because what
we do you can use it for other problems
as well so one major finding in
forecasting research over the last 50
years is actually to not put too much
faith in any single forecasts instead
what you should do is you should combine
forecasts from many different sources
different methods that use different
information and that is the principle
that we rely upon in the Pali vote now
how can we predict elections probably
most of you now think well election
forecasting that’sthat’s polling right
and by the way we are not pollsters
there’s many more methods some of which
are older than polling so before the
days of polling people would rely on
other people’s expectations and that’s
what we do as well so we reach out to
experts and ask them it’s political
scientists across the country and we ask
them to provide us the forecasts of the
election outcome and you would think
that experts have knowledge about how to
read polls they know that polls have
errors but they should be able to kind
of tell you in which direction the arrow
is and move the forecasts into the right
direction
now this hasn’t really happened in 2016
the chart that you see up there shows
you the error of the experts in
historical elections on the left and in
this election on the right and it shows
you that in this election the era was
much higher and what happened is is that
the expert actually thought that the
polls are under estimating Clinton now
this might sound crazy now but there was
lots of talk about you know good early
voting numbers for Clinton that Trump
didn’t really have a turnout game or a
good ground game so you could find all
these pieces of evidence but there was
also evidence to the contrary such as
large share of undecided voters lots of
uncertainty it looks like there might
have been some wishful thinking going on
by the experts who tend to be more
liberal and maybe they just wanted their
forecasts to be true and therefore
focused on information that they
supported it another method that we use
are so-called prediction markets on
these markets people bet on the outcome
of the elections that is real money at
stake this has a very long history goes
back to the 19th century in the United
States it has a very good track record
in this election the prediction markets
were really far off as you can see in
the chart the error was about five times
higher than it used to be in previous
elections now the question is why is
that we still have to analyze but it
looks a lot that it depends on who
participates in these markets we know
from prior research that that’s people
who tend to be highly educated they tend
to be wealthier tend to be politically
engaged and and interested so it’s a
very special group of the demographic
group as the more elite group of society
and it to me this kind of shows that
these people they just couldn’t imagine
that other so many people would vote for
Donald Trump’s or the dramatically
overestimated Clinton’s vulture it kind
of shows us also a little bit about the
deep polarization in the country the
next method is my favorite method
because it’s so simple instead of asking
people who you’re going to vote for ask
them who is going to win this question
has existed since the 1930s but it’s
often overlooked
this is the most accurate method that we
have to predict election outcomes
unfortunately it’s not asked that often
impulse and we only have data for the
national level it would have been good
to see this for some really critical
state of swing states what the voters
would have think thought in states such
as Pennsylvania and then of course you
know 1930s we saw the rise of scientific
polling and polls have era we knew that
polls had big errors like this one in
1948 the 2016 error will be big in some
states but there has also been some
excellent work on how to improve upon
the accuracy of polls which is by again
combining combining forecasts so we can
reduce the error of polls and these
methods to work really well
another group of methods that we use are
so-called political economy models these
models are based on the idea that voting
is retrospective so voters look back
they look at well how well has the
president done his job in particular in
hanging the economy so they use economic
indicators and they also lose use
factors such as the electoral cycle
because we know from history that
Americans after a while they won’t
change in who is leading the government
and if you use these factors you can
make a prediction about future elections
and ironically this method provided the
most accurate predictions in this
election and I’m saying ironically
because first of all it doesn’t have a
very great track record and second of
all many of these models assume that the
candidates play no role for the outcome
of the election which is completely
counterintuitive when you look at how
candidate focused this election was and
then the final group of methods that we
have in contrast look specifically about
what impact do campaigns have on the
election outcome so they try to measure
for example how voters perceive the
candidates on handling the issues facing
the country these models again failed
badly and they overestimated Clinton
which she was favored in many issues
which might also tell us something about
maybe a
of media coverage about which issues are
really important and who might be a
better candidate to solve them so what
we know from prior research in
forecasting from elections but also in
other fields is that the accuracy of
different methods varies a lot over time
and we see it again in 2016 methods that
have worked well in the past don’t
necessarily work well in the future we
saw that with prediction markets this
time and we seed with political economy
models as well who did not do well in
previous elections but really have done
well now well what does this mean the
point is that it’s really difficult to
predict which method will be the most
accurate for predicting a particular
event if you would have asked me before
the election which method is most likely
wrong I would have probably picked
political economy models simply because
not a good track record and assume that
candidates don’t really play a role so
it’s not really the conditions under
which you would assume this method to
work I would have been proved really
wrong and that’s of course the reason
why the principle of combining works
because if you use all this information
that’s out there and you put it into one
forecast you usually have the effect
that you use more information and second
of all usually the error of the
different methods cancel out because
usually methods have errors but in
different directions
now the problem this year was that most
of these methods had the same error in
the same direction most of the methods
overestimated Clinton’s share and many
of the methods overestimated it by a lot
and this is of course the situation when
combining cannot work as well it still
does as well as the typical forecasts
and it prevents you from making really
large errors by picking a poor
prediction but it cannot be as good as
the best forecasts but that’s not what
combining forecast claims the claim is
that if you do it over and over again
not just for one election over and over
again you will have the effect because
all the other methods vary in the
accuracy that you will get a really
accurate prediction and again it
protects you from large errors
and I think another big benefit is
because we aggregate all this
information we really have it in front
of us and I think that’s the biggest
puzzle about this election that the
experts and the betting markets which
are also self selected experts they bet
money on their information they really
missed what’s going on and some of the
information was there so I think it’s
not really a lack of polling accuracy
this time it was really a failure of
pundits and maybe also the media because
the media there were models that say
like look the electoral cycle is
important look the economy’s an
important factor
look there is lots of undecided voters
that could have been an important factor
to point out and it hadn’t happen so
niels bohr once famously said prediction
is very difficult especially if it’s
about the future forecasting is not an
exact science they will get better and I
think the 2016 failure will ignite a lot
of new research we’re already thinking
about what we can do better of course
we’re thinking about how to maybe
combine our forecast in a different way
or how to challenge experts that they
also can kind of consider contradictory
information and I know there’s a lot of
work now in the polling industry to
really figure out what happened and how
to improve but there will always be a
random component of error and maybe the
FBI letter about Clinton’s emails was
some sort of random October Surprise
that really had an effect on the outcome
of the election we don’t know but the
point is don’t take the forecast too
seriously don’t don’t take them for
granted don’t take them as an insurance
not to go to vote or as an excuse not to
go to vote don’t rely on us to predict
the future but rather shape the future by going out and vote thank you