Let’s take two fairly obvious data-driven conclusions from the 2016 election and see if there’s any link between them.
The first conclusion: Education was almost everything in explaining the results of the race. Donald Trump substantially improved on Mitt Romney’s performance among voters without college degrees — especially white voters without college degrees. Hillary Clinton somewhat improved on President Obama’s performance with college-educated voters. The link between education levels and the shift in the vote is robust, even when controlling for other factors, such as income levels.
The second conclusion: The polling was bad. Actually, let me amend that: The polling wasn’t that bad. (Reporters and analysts should have been more prepared for the possibility that Trump might win. We’re going to keep being really annoying about this.) With Clinton’s lead in the popular vote still expanding, the national polls are going to wind up having been pretty good (they showed her winning by 3 to 4 percentage points, and she’ll eventually win by about 2 points). The state polls? Not so hot. What matters, though, is not only the magnitude of the error in the state polls but the direction of it. The errors were correlated from state to state, and Clinton underperformed in a trio of states in the Rust Belt — Michigan, Wisconsin and Pennsylvania — that were supposed to have been part of her firewall; that underperformance cost her the Electoral College.
So, are these two things connected? Did Trump beat his polls in states with large numbers of white voters without college degrees?
The short answer is “yes.” Below, you’ll find a table that sorts the states by the share of their 2016 electorate that consists of white voters without college degrees, as according to our Swing-O-Matic interactive.1 The table also lists Trump’s actual margin of victory or defeat as of this writing compared to FiveThirtyEight’s adjusted polling average in each state.
|TRUMP MARGIN OF VICTORY|
|STATE||WHITE NON-COLLEGE SHARE||ADJ. POLLING AVERAGE||ACTUAL||ACTUAL VS. POLLS|
Adjusted polling average is from the FiveThirtyEight polls-only forecast as of Nov. 8, 2016. Actual results are accurate as of Nov. 30.
Sources: David Wasserman, U.S. Census Bureau
In the 10 states with the largest share of white voters without college degrees, Trump beat his polling average by an average of 8 percentage points — a major polling miss. But in the 10 states with the lowest share of white voters without college degrees, Clinton beat her polls by an average of 3 points (or 4 points if you count the District of Columbia as a state). Overall, the correlation between the share of white non-college voters in a state and the amount by which Trump overperformed (or underperformed) his polls is quite high.2
But there are a few complications. For one thing, most of the states with large numbers of white voters without college degrees were pretty red to begin with. So it could just be that Trump outperformed his polls in red states, regardless of voters’ education levels. Or it could be that Trump beat his polls in states with lots of white voters, whether or not they had college degrees. So we should check for those possibilities. And, finally, it may be that the high correlation is driven by outlier-y states such as West Virginia and Hawaii, which didn’t get polled very much. Does the conclusion hold among well-polled states?
There’s still a pretty strong relationship between education levels and polling errors, even if you account for all of these factors. Gory details ahead: I ran a regression that accounted for Romney’s margin against Obama in 2012 (as a measure of each state’s overall redness or blueness) and for the share of white voters with college degrees in addition to the share of white voters without college degrees. And I weighted the regression by FiveThirtyEight’s aggregate polling weight in each state, so that Ohio and Florida (for example) are much more influential than West Virginia or Hawaii. The share of non-college white voters was still a highly statistically significant predictor of the polling error, although Romney’s performance in 2012 was too. (Trump beat his polls more in red states, as I mentioned earlier.) The share of white voters with college degrees in a state didn’t matter much one way or the other, however. (See this footnote for the regression output.3)
So it looks to me as though the polls may not have reached enough non-college voters. It’s a bit less clear whether this is a longstanding problem or something particular to the 2016 campaign. As Nate Cohn has found at The Upshot, the exit polls almost certainly have too many college-educated voters in their samples, which is why I haven’t cited exit polling data here. Pew Research reported in 2015 that less-well-educated voters are less likely to respond to pre-election surveys, although a pollster I spoke with earlier this week told me that he’d also seen the opposite problem.
This is not the only potential type of non-response bias; white voters are also more likely to respond to telephone surveys than black or Hispanic ones, older people are more likely to do so than younger ones, and women are more likely to do so than men. But most pollsters apply demographic weighting by race, age and gender to try to compensate for this problem. It’s less common (although by no means unheard of) to weight by education, however. As education levels increasingly cleave voters from one another, more pollsters may need to consider weighting their samples accordingly.