[Music]
[Applause]
thank you
I’m collided to be speaking here today
I’m going to talk about election polling
in the 2016 election and my first
thought about this is well what extent
you you all want to hear about the 2016
election again an opinion polling and
all the missed predictions and so forth
on the other hand Donald Trump seems to
like to talk about this continually even
though the election has has long past so
what happened in 2016 2016 arguably was
the biggest debacle of predictions of
elections of all time in the United
States it harkens back to the 1948
election where there was this famous
photo which you’ll see further in a
second of the Chicago Tribune reporting
that Dewey defeats Truman that is
Republican Thomas Dewey defeated Harry
Truman in the 1948 election based on
pre-election polling and predictions and
it’s useful to think back historically
on this in particular I think the best
analysts of this is the legendary
baseball player and language master Yogi
Berra I think he had it right and the
terms that his phrases that apply here
or in 2016 compared to 1948 was deja vu
all over again but the similarities are
different and of course the old
all-encompassing fallback is hard to
predict especially about the future now
to put this in broader historical
context I want to go back to 1936 which
before 1948 was a polling debacle of a
particular sort that’s worth reflecting
further on from 1916 to 1936 the
literary digest did public did what they
called straw polls on presidential
elections these were polls based on
mailing out cards to Ted to basically 10
million or more people the people were
selected by their subscriber list
by automobile registration telephone
directories and off and also voter lists
now these constituted by current
standards bias samples in the sense that
they just sent out cards to as many
people as possible and let people
self-select whether to send in their
cards and despite that self-selected
kind of sample which arguably was more
of an upscale scale sample that is
people had telephones people has cars
and so forth you would think that would
introduce certain kinds of biases and
biases of the sort that comport to the
stereotype of Republican voters who were
more at least in those days more of
middle middle class and upscale so there
were apparent potential biases there but
from 1916 to 1932 the literary digest
correctly predicted the results of
national elections so in 1933 when
Roosevelt took off Franklin Roosevelt
took office he’s on the front page of
the literary digest they got it right
come 1936 they’re doing their poll once
more but but what happens there they
predict that Alfred Landon would win the
election
however the upstart George Gallup began
doing what he called scientific polling
based on sampling of geographic
locations and sending interviewers out
to those locations to fill certain kinds
of demographic quotas based on age and
gender and other characteristics and
Gallup argued that his method was more
scientific and that he would his for
prediction of the election would be
right compared to the literary digest
and he predicted based on his
pre-election polling that Roosevelt
would win the election and not not
landed and he consistently had Roosevelt
the head head in the polls as she as you
see as you see in the chart and at the
same time other legendary pollsters
Archibald Crosley and Elmo Roper were
entering the polling game using methods
similar to Gallup and we’re making the
same kinds of predictions what happened
as a result of the election George
Gallup became famous he got the election
right the literary digest got it wrong
what happened afterwards with was that
the literary digest went under in 1938
scientific polling with with the word
scientific in quotes established Gallup
as a major figure in public affairs and
in polling and so forth but there was
that but the question is well why the
deliberate was that was literary digest
wrong in 1936 and not earlier and what
happened was there was a major political
realignment such that the kinds of
biases that existed in the polls were
correlated with how people voted and so
what happened was literary digest
overestimated the Republican vote what
also happened that that that people
don’t fully appreciate is that Gallup
underestimated Roosevelt’s support by
fully five points in the graph that you
saw he was actually five point five
point short so galip’s poll had those
actually similar kinds of biases but he
got the winner right and he got the
winner right in 1940 and 1944 but again
underestimated Roosevelt’s support so
there were problems in the polls that
then came home to roost in 1948 in 1948
prior to prior to the election the
gallop gallop polls had predicted that
Dewey would win by somewhere around five
points in the election and other
pollsters were getting the same kinds of
results in including academic posters
who were analyzing the election as well
and what happened there was there are
obviously problems in the Gallup poll
the uproar was so great there were Wars
in government there were up growers all
throughout Society and it prompted the
Social Science Research Council to put
together a committee to investigate the
pre-election polls of nineteen forty
eight headed by a famous statistician
named Frederick Mosteller
so the big question is what did the what
are the one of the SSRC report find well
they found that the following things
the biggest thing problem they found was
what they argue were biases in the polls
that led to Gallup and others
interviewing upscale voters more than
downscale voters introducing the same
kinds of biases that we we had seen all
along through nineteen nineteen forty
four and the bias came in not because of
the selection of geographic locations
but because the interviewers had
discretion with regard to what people
they would interview and talk to and the
more interview friendly looking people
tended to be more middle-class and
upscale and the interviewers would
gravitate toward those people so that
was the first problem the other problem
and this
thanks to what I call the people like
the lack of humility or to be less
polite the arrogance of the pollsters
and predictors of because of the
confidence that they had in their
polling methods and the fact that you’ve
gotten prior elections right and the
arrogance was such that they stopped
polling two weeks before the election
and it turned out there was a lot going
on in those two weeks in terms of the
campaign Truman was actively campaigning
there was a sense that the tide was
turning may sound a little familiar to
what happened in 2016 but we’ll get
there in a second and so the fact that
they stopped polling earlier early that
was considered a problem as well further
there’s always the problem and how to
allocate undecided voters that was a
possible major problem and we’ll come
back to that today and in addition to
that it’s always difficult and this
speaks to Leo G barrows comment about
its being difficult to predict
especially about the future it’s
difficult to predict what the electorate
on election day is going to look like in
based on pre-election polls so there was
at this time this exaggerated impression
of the accuracy of polls and that led to
the reporting and projecting of the
polls and the election to turnout in a
certain way and it didn’t so there was a
major problem there Gallup thereafter
corrected his methods and in subsequent
years was able to better predict the
polls in in the 1958 election he was
within the Gallup polls were within one
point of Dwight Eisenhower’s victory
over at least Stevenson following that
there were advanced further advances in
Survey Research leading to the
proliferation of polling the shifting
from in-person interviews to telephone
surveys the use of random digit dialing
computer assisted interviewing and so
forth and up until a point there were
sufficiently high response rates so that
the quality of polls was good but what
happened afterwards was that because of
the proliferation of polling through the
proliferation of solicitation calls and
other kinds of calls that people get
there’s been a decline over the years
and response rates to surveys especially
in the modern day with the proliferation
of cell phones cell phones in the United
States today there are more than 50% of
the households in the United States that
are that are cell phone only so that the
pollsters had to start calling with
people on cell phones
people were less likely to talk to
pollsters on cellphones there other
issues involved in cell phones as well
so that the response rates began to
decline and there were worries about
increasing errors in surveys and these
were compounded in the 2016 election by
other factors including especially the
ability of the pollsters to identify
that the nature the composition of the
electorate in the 2016 election the
other complication in the 2016 election
was the for lack of a better term again
arrogance associated with projections
about how the election would occur
because of what was referred to as the
blue wall that the Democrats had now
Obama was able to win substantial
electoral college majorities in 2008 and
2012 he was able to win the state the
key states of Ohio and Florida and in
addition he won North Carolina and
Indiana in the 2008 election and so that
in the electoral map that you see
there’s a lot of blue there and the blue
there is such that it basically gave
Obama the Democrats many different ways
of winning the electoral college if you
go back to the two thousand election
everybody there’s a big uproar about
what happened in Florida and have had al
gore won Florida he would have won the
election but what people forget is that
Gore could have won the election without
Florida if he had taken his home state
of Tennessee or if he had taken the
state of New Hampshire
so they fast forward to the 2016
election there were great many ways that
Hillary Clinton could win the election
in the blue that you see in in the in
the map in front of you you see a lot of
blue there but if we if we eliminate the
states of Florida Ohio North Carolina
and Iowa Hillary Clinton could have
could have could have still won the one
the election the key things being the
key difference between to that the 2000
election is that the states of Colorado
and Virginia have become democratic
states it was always assumed because of
decades of election results that
Michigan Wisconsin and Pennsylvania
could never go Republican that what that
was assumed and so the worst-case
scenario would be if she if she lost
North Carolina floor
and Ohio but as long as she took the
states and Iowa as long as she took the
states of New Mexico Nevada and New
Hampshire she could win she could win
the election and if you look at what
happened in the 2016 election she in
fact lost North Carolina Florida Ohio
she won Nevada and New Hampshire and
would have won the election except for
one small detail
she lost Pennsylvania Wisconsin in
Michigan and that’s that that’s what no
one saw coming
so what happened in the 2016 election
there were all these wild predictions of
the probability of Hillary Clinton
winning ranging from 71 percent to 99
percent based on simulations of
electoral college results based on the
pre-election polling the problem was
that the polling in key states was error
ridden you remember the uproar that was
that the that the election caused that
led of course to a report that was just
completed by the American Association
for public opinion research a poor had
actually put together a task force to do
a report on pre on the 2016 election
polling not expecting any you know
surprising results but they were
confronted with a major surprise so it
basically shifted the focus of the
report that was done by a blue-ribbon
committee of the American Association
for public opinion research as you see
as you see listed here and the big
question is what did they find
well what they found first and foremost
was that the national polls were
generally generally accurate by
historical standards that is Hillary
Clinton won the popular vote by two
percentage points and the polls had her
winning by three or four points and that
two point difference was similar to
historical averages and actually the
polls in 2016 were better than the polls
in 2012 and that and that’s something
that that got lost in the shuffle the
problem was is that the states had a bad
year and in particular there was a bad
year in the key states of Michigan
Wisconsin and in Pennsylvania now the
problems with the polls
particular slide basically shows that
the poles had her winning by an average
of two points based on polls by
different by different polling
organizations the the error that was
found in the polls with the magnitude of
the error was basically about 2.2
percentage points with the bias in favor
of Hillary Hillary Clinton in the graph
that you see that the orange line
represents the magnitude of the of the
error and the blue bars represent the
bias the direction so was there was an
overestimate of the Hillary Clinton vote
if you go back and look at 1936 by the
way there was a huge underestimate
underestimate of the Democratic vote
even though in 1936 1940 in 1944 the
Gallup poll got the winner right but got
the winner right but by high margins
that were that were substantially
substantially off in the states the
result the results were different the
magnitudes were on the order of five
percentage points which is a lot and
that’s an average in some states it was
bigger in some states it was smaller and
the bias was in the direction of of the
Democrats that is they underestimated
Donald Trump spoke and the problem that
occurred has to do with the quality of
the polls which for me
and this isn’t cited in the report is
indicated by the fact that in the last
two days before the election there were
about a there were 11 high quality
public opinion polls that were done
nationally in contrast in the states in
the last week before the election in
Wisconsin Michigan in Pennsylvania there
were just a handful of polls and the
quality of those polls the a poor report
called in called into question what what
had happened in the election well there
were a lot there were a lot of things at
the a poor report sites one is voters
broke in the last day of week of the
election more toward Donald Trump than
Hillary Clinton based on the exit polls
people who decided late in Wisconsin
Michigan and Pennsylvania
disproportionately voted for Donald
Trump in addition in addition there was
a question of what happened with regard
to voter turnout in the state and the
representativeness of the samples what
they did find is that one problem in the
polls is that the polls in the states in
many cases did not weight the weight and
or adjust their data statistically for
education that is white voters without a
college degree voted disproportionately
for Trump and
and the there their non-response the
polls was not compensated for by by any
waiting in addition and this isn’t in
the report but based on the latest
census data we know that voter turnout
among african-american voters was less
and we also know from the exit polls is
that African American voters supported
Hillary Clinton at slightly lower rates
and they had supported Barack Obama and
the same team same thing for Latino
voters and in addition with the aiport
cited in some states there were called
ballot order effects that is that is
that isn’t there were states in which
Trump was disproportionately listed
first on the ballot and that may have
led to some slight biases and then last
but not least the report talks about
this but I think under states it the big
problem was not so much in the polls
themselves but in the reporting and in
the exaggerating were the perceived
exaggerating of the likelihood that
Hillary Clinton would win the election
and the fact that the that people did
not fully appreciate sampling
variability and probabilities in the
1948 report the Mosteller report they
talked also about the lack of
appreciation of variability and survey
error particularly among the reporters
and predictors of the poll results 71%
it’s not a prediction that Hillary
Clinton when it would win is not a slam
dunk but if you repeat 71 percent enough
online and on network news and on cable
news people get the impression that she
got the impression that she was a
shoo-in and now and that that perception
was a big part I would say the most
significant part of the debacle that
occurred and the final conclusion of the
a poor report is that election polls are
not all polls what the pollsters all
worried about is that when they when
they failed to predict elections right
people lose faith in polling in this in
this in this in this election the
national polls were reasonably accurate
but more importantly election polls are
different from all polls in the sense
they you’re trying to make predictions
and wait or adjust your data to an
electorate that you don’t know where as
normal public opinion polls that are
done between elections and don’t focus
on elections are samples of the adult
population for which we have census data
at the benchmark what
the composition of the public is so
there should be much less error in
polling overall compared to election
polls what what are predictions about
the future quality of survey is giving
the even declining response rates and
other issues in polling that’s a topic
for another occasion if you want to read
about this further you should see the
latest Pew Research Center report
entitled what low response rates mean
for telephone surveys the basic finding
of that study is that for the most part
good high-quality Poland could be done
but there’s certain kinds of problems
that we need to be cognizant about thank
[Applause]
[Applause]
thank you
I’m collided to be speaking here today
I’m going to talk about election polling
in the 2016 election and my first
thought about this is well what extent
you you all want to hear about the 2016
election again an opinion polling and
all the missed predictions and so forth
on the other hand Donald Trump seems to
like to talk about this continually even
though the election has has long past so
what happened in 2016 2016 arguably was
the biggest debacle of predictions of
elections of all time in the United
States it harkens back to the 1948
election where there was this famous
photo which you’ll see further in a
second of the Chicago Tribune reporting
that Dewey defeats Truman that is
Republican Thomas Dewey defeated Harry
Truman in the 1948 election based on
pre-election polling and predictions and
it’s useful to think back historically
on this in particular I think the best
analysts of this is the legendary
baseball player and language master Yogi
Berra I think he had it right and the
terms that his phrases that apply here
or in 2016 compared to 1948 was deja vu
all over again but the similarities are
different and of course the old
all-encompassing fallback is hard to
predict especially about the future now
to put this in broader historical
context I want to go back to 1936 which
before 1948 was a polling debacle of a
particular sort that’s worth reflecting
further on from 1916 to 1936 the
literary digest did public did what they
called straw polls on presidential
elections these were polls based on
mailing out cards to Ted to basically 10
million or more people the people were
selected by their subscriber list
by automobile registration telephone
directories and off and also voter lists
now these constituted by current
standards bias samples in the sense that
they just sent out cards to as many
people as possible and let people
self-select whether to send in their
cards and despite that self-selected
kind of sample which arguably was more
of an upscale scale sample that is
people had telephones people has cars
and so forth you would think that would
introduce certain kinds of biases and
biases of the sort that comport to the
stereotype of Republican voters who were
more at least in those days more of
middle middle class and upscale so there
were apparent potential biases there but
from 1916 to 1932 the literary digest
correctly predicted the results of
national elections so in 1933 when
Roosevelt took off Franklin Roosevelt
took office he’s on the front page of
the literary digest they got it right
come 1936 they’re doing their poll once
more but but what happens there they
predict that Alfred Landon would win the
election
however the upstart George Gallup began
doing what he called scientific polling
based on sampling of geographic
locations and sending interviewers out
to those locations to fill certain kinds
of demographic quotas based on age and
gender and other characteristics and
Gallup argued that his method was more
scientific and that he would his for
prediction of the election would be
right compared to the literary digest
and he predicted based on his
pre-election polling that Roosevelt
would win the election and not not
landed and he consistently had Roosevelt
the head head in the polls as she as you
see as you see in the chart and at the
same time other legendary pollsters
Archibald Crosley and Elmo Roper were
entering the polling game using methods
similar to Gallup and we’re making the
same kinds of predictions what happened
as a result of the election George
Gallup became famous he got the election
right the literary digest got it wrong
what happened afterwards with was that
the literary digest went under in 1938
scientific polling with with the word
scientific in quotes established Gallup
as a major figure in public affairs and
in polling and so forth but there was
that but the question is well why the
deliberate was that was literary digest
wrong in 1936 and not earlier and what
happened was there was a major political
realignment such that the kinds of
biases that existed in the polls were
correlated with how people voted and so
what happened was literary digest
overestimated the Republican vote what
also happened that that that people
don’t fully appreciate is that Gallup
underestimated Roosevelt’s support by
fully five points in the graph that you
saw he was actually five point five
point short so galip’s poll had those
actually similar kinds of biases but he
got the winner right and he got the
winner right in 1940 and 1944 but again
underestimated Roosevelt’s support so
there were problems in the polls that
then came home to roost in 1948 in 1948
prior to prior to the election the
gallop gallop polls had predicted that
Dewey would win by somewhere around five
points in the election and other
pollsters were getting the same kinds of
results in including academic posters
who were analyzing the election as well
and what happened there was there are
obviously problems in the Gallup poll
the uproar was so great there were Wars
in government there were up growers all
throughout Society and it prompted the
Social Science Research Council to put
together a committee to investigate the
pre-election polls of nineteen forty
eight headed by a famous statistician
named Frederick Mosteller
so the big question is what did the what
are the one of the SSRC report find well
they found that the following things
the biggest thing problem they found was
what they argue were biases in the polls
that led to Gallup and others
interviewing upscale voters more than
downscale voters introducing the same
kinds of biases that we we had seen all
along through nineteen nineteen forty
four and the bias came in not because of
the selection of geographic locations
but because the interviewers had
discretion with regard to what people
they would interview and talk to and the
more interview friendly looking people
tended to be more middle-class and
upscale and the interviewers would
gravitate toward those people so that
was the first problem the other problem
and this
thanks to what I call the people like
the lack of humility or to be less
polite the arrogance of the pollsters
and predictors of because of the
confidence that they had in their
polling methods and the fact that you’ve
gotten prior elections right and the
arrogance was such that they stopped
polling two weeks before the election
and it turned out there was a lot going
on in those two weeks in terms of the
campaign Truman was actively campaigning
there was a sense that the tide was
turning may sound a little familiar to
what happened in 2016 but we’ll get
there in a second and so the fact that
they stopped polling earlier early that
was considered a problem as well further
there’s always the problem and how to
allocate undecided voters that was a
possible major problem and we’ll come
back to that today and in addition to
that it’s always difficult and this
speaks to Leo G barrows comment about
its being difficult to predict
especially about the future it’s
difficult to predict what the electorate
on election day is going to look like in
based on pre-election polls so there was
at this time this exaggerated impression
of the accuracy of polls and that led to
the reporting and projecting of the
polls and the election to turnout in a
certain way and it didn’t so there was a
major problem there Gallup thereafter
corrected his methods and in subsequent
years was able to better predict the
polls in in the 1958 election he was
within the Gallup polls were within one
point of Dwight Eisenhower’s victory
over at least Stevenson following that
there were advanced further advances in
Survey Research leading to the
proliferation of polling the shifting
from in-person interviews to telephone
surveys the use of random digit dialing
computer assisted interviewing and so
forth and up until a point there were
sufficiently high response rates so that
the quality of polls was good but what
happened afterwards was that because of
the proliferation of polling through the
proliferation of solicitation calls and
other kinds of calls that people get
there’s been a decline over the years
and response rates to surveys especially
in the modern day with the proliferation
of cell phones cell phones in the United
States today there are more than 50% of
the households in the United States that
are that are cell phone only so that the
pollsters had to start calling with
people on cell phones
people were less likely to talk to
pollsters on cellphones there other
issues involved in cell phones as well
so that the response rates began to
decline and there were worries about
increasing errors in surveys and these
were compounded in the 2016 election by
other factors including especially the
ability of the pollsters to identify
that the nature the composition of the
electorate in the 2016 election the
other complication in the 2016 election
was the for lack of a better term again
arrogance associated with projections
about how the election would occur
because of what was referred to as the
blue wall that the Democrats had now
Obama was able to win substantial
electoral college majorities in 2008 and
2012 he was able to win the state the
key states of Ohio and Florida and in
addition he won North Carolina and
Indiana in the 2008 election and so that
in the electoral map that you see
there’s a lot of blue there and the blue
there is such that it basically gave
Obama the Democrats many different ways
of winning the electoral college if you
go back to the two thousand election
everybody there’s a big uproar about
what happened in Florida and have had al
gore won Florida he would have won the
election but what people forget is that
Gore could have won the election without
Florida if he had taken his home state
of Tennessee or if he had taken the
state of New Hampshire
so they fast forward to the 2016
election there were great many ways that
Hillary Clinton could win the election
in the blue that you see in in the in
the map in front of you you see a lot of
blue there but if we if we eliminate the
states of Florida Ohio North Carolina
and Iowa Hillary Clinton could have
could have could have still won the one
the election the key things being the
key difference between to that the 2000
election is that the states of Colorado
and Virginia have become democratic
states it was always assumed because of
decades of election results that
Michigan Wisconsin and Pennsylvania
could never go Republican that what that
was assumed and so the worst-case
scenario would be if she if she lost
North Carolina floor
and Ohio but as long as she took the
states and Iowa as long as she took the
states of New Mexico Nevada and New
Hampshire she could win she could win
the election and if you look at what
happened in the 2016 election she in
fact lost North Carolina Florida Ohio
she won Nevada and New Hampshire and
would have won the election except for
one small detail
she lost Pennsylvania Wisconsin in
Michigan and that’s that that’s what no
one saw coming
so what happened in the 2016 election
there were all these wild predictions of
the probability of Hillary Clinton
winning ranging from 71 percent to 99
percent based on simulations of
electoral college results based on the
pre-election polling the problem was
that the polling in key states was error
ridden you remember the uproar that was
that the that the election caused that
led of course to a report that was just
completed by the American Association
for public opinion research a poor had
actually put together a task force to do
a report on pre on the 2016 election
polling not expecting any you know
surprising results but they were
confronted with a major surprise so it
basically shifted the focus of the
report that was done by a blue-ribbon
committee of the American Association
for public opinion research as you see
as you see listed here and the big
question is what did they find
well what they found first and foremost
was that the national polls were
generally generally accurate by
historical standards that is Hillary
Clinton won the popular vote by two
percentage points and the polls had her
winning by three or four points and that
two point difference was similar to
historical averages and actually the
polls in 2016 were better than the polls
in 2012 and that and that’s something
that that got lost in the shuffle the
problem was is that the states had a bad
year and in particular there was a bad
year in the key states of Michigan
Wisconsin and in Pennsylvania now the
problems with the polls
particular slide basically shows that
the poles had her winning by an average
of two points based on polls by
different by different polling
organizations the the error that was
found in the polls with the magnitude of
the error was basically about 2.2
percentage points with the bias in favor
of Hillary Hillary Clinton in the graph
that you see that the orange line
represents the magnitude of the of the
error and the blue bars represent the
bias the direction so was there was an
overestimate of the Hillary Clinton vote
if you go back and look at 1936 by the
way there was a huge underestimate
underestimate of the Democratic vote
even though in 1936 1940 in 1944 the
Gallup poll got the winner right but got
the winner right but by high margins
that were that were substantially
substantially off in the states the
result the results were different the
magnitudes were on the order of five
percentage points which is a lot and
that’s an average in some states it was
bigger in some states it was smaller and
the bias was in the direction of of the
Democrats that is they underestimated
Donald Trump spoke and the problem that
occurred has to do with the quality of
the polls which for me
and this isn’t cited in the report is
indicated by the fact that in the last
two days before the election there were
about a there were 11 high quality
public opinion polls that were done
nationally in contrast in the states in
the last week before the election in
Wisconsin Michigan in Pennsylvania there
were just a handful of polls and the
quality of those polls the a poor report
called in called into question what what
had happened in the election well there
were a lot there were a lot of things at
the a poor report sites one is voters
broke in the last day of week of the
election more toward Donald Trump than
Hillary Clinton based on the exit polls
people who decided late in Wisconsin
Michigan and Pennsylvania
disproportionately voted for Donald
Trump in addition in addition there was
a question of what happened with regard
to voter turnout in the state and the
representativeness of the samples what
they did find is that one problem in the
polls is that the polls in the states in
many cases did not weight the weight and
or adjust their data statistically for
education that is white voters without a
college degree voted disproportionately
for Trump and
and the there their non-response the
polls was not compensated for by by any
waiting in addition and this isn’t in
the report but based on the latest
census data we know that voter turnout
among african-american voters was less
and we also know from the exit polls is
that African American voters supported
Hillary Clinton at slightly lower rates
and they had supported Barack Obama and
the same team same thing for Latino
voters and in addition with the aiport
cited in some states there were called
ballot order effects that is that is
that isn’t there were states in which
Trump was disproportionately listed
first on the ballot and that may have
led to some slight biases and then last
but not least the report talks about
this but I think under states it the big
problem was not so much in the polls
themselves but in the reporting and in
the exaggerating were the perceived
exaggerating of the likelihood that
Hillary Clinton would win the election
and the fact that the that people did
not fully appreciate sampling
variability and probabilities in the
1948 report the Mosteller report they
talked also about the lack of
appreciation of variability and survey
error particularly among the reporters
and predictors of the poll results 71%
it’s not a prediction that Hillary
Clinton when it would win is not a slam
dunk but if you repeat 71 percent enough
online and on network news and on cable
news people get the impression that she
got the impression that she was a
shoo-in and now and that that perception
was a big part I would say the most
significant part of the debacle that
occurred and the final conclusion of the
a poor report is that election polls are
not all polls what the pollsters all
worried about is that when they when
they failed to predict elections right
people lose faith in polling in this in
this in this in this election the
national polls were reasonably accurate
but more importantly election polls are
different from all polls in the sense
they you’re trying to make predictions
and wait or adjust your data to an
electorate that you don’t know where as
normal public opinion polls that are
done between elections and don’t focus
on elections are samples of the adult
population for which we have census data
at the benchmark what
the composition of the public is so
there should be much less error in
polling overall compared to election
polls what what are predictions about
the future quality of survey is giving
the even declining response rates and
other issues in polling that’s a topic
for another occasion if you want to read
about this further you should see the
latest Pew Research Center report
entitled what low response rates mean
for telephone surveys the basic finding
of that study is that for the most part
good high-quality Poland could be done
but there’s certain kinds of problems
that we need to be cognizant about thank
[Applause]
Please follow and like us: