“We could rule some things out, but it’s hard to prove beyond a certainty what happened,” said Josh Clinton, a professor at Vanderbilt University and the chair of the association’s 2020 election task force. “Based on what we know about polling, what we know about politics, we have some good prime suspects as to what may be going on.”
Those “prime suspects” will hardly be comforting to pollsters and those who depend on them, from political campaigns to the news media. The most likely — if far from certain — culprit for off-kilter polling results is that key groups of people don’t answer polls in the first place.
Decreasing response rates have been a major source of concern for pollsters for more than a decade. But the politicization of polling during the Trump era — including the feedback loop from the former president, who has falsely decried poll results he doesn’t like as “fake” or deliberately aimed at suppressing enthusiasm for answering polls among GOP voters — appears to be skewing the results, with some segment of Republicans refusing to participate in surveys.
But pollsters say they can’t be sure that’s the main reason, because you never know exactly whom you’re not talking to.
That makes the problems with polling a lot harder to fix than the diagnosis four years ago, which mostly focused on adjusting surveys to account for Trump’s popularity with voters who haven’t earned college degrees and his corresponding weakness with college degree-holders.
“It seems plausible to the task force that, perhaps, the Republicans who are participating in our polls are different from those who are supporting Republican candidates who aren’t participating in our polls,” Clinton said. “But how do you prove that?”
The task force’s first job was to evaluate the performance of the 2020 public election polls. On that measure, polling earned a failing grade. While the national polls were the worst in four decades, the state-level polls of the presidential, Senate and gubernatorial races were as bad as they’ve been as far back as there are records (20 years).
According to the report, national polls of the presidential race conducted in the final two weeks of the election were off by an average of 4.5 percentage points, while the state polls were off by just over 5 points. Most of the error was in one direction: Looking at the vote margin, the national polls were too favorable to now-President Joe Biden by 3.9 points, and the state polls were 4.3 points too favorable for Biden.
Most of the error came from underestimating Trump’s support, as opposed to overestimating Biden’s. Comparing the final election results to the poll numbers for each candidate, Trump’s support was understated by a whopping 3.3 points on average, while Biden’s was overstated by a point — turning what looked like a solid Biden lead into a closer, if still decisive, race.
It wasn’t just a Trump effect, either. The polls of Senate and governor’s races were off by an even greater margin: 6 points on average.
“Within the same state, polling error was often larger in senatorial contests than the presidential contest,” the AAPOR report reads. “Whether the candidates were running for president, senator, or governor, poll margins overall suggested that Democratic candidates would do better and Republican candidates would do worse relative to the final certified vote.”
No one methodology performed head-and-shoulders above the others. According to the report, there were only “minor differences” whether polls were conducted on the phone, over the internet or using a mixed methodology, including texts and smartphone apps — or whether they contacted voters randomly versus off a list of registered…