How will we know if the polls are wrong again?

Look at the rank ordering of states, not margins

It’s generally a bad look to be a polling denier, dismissing the current polls and mentally adjusting them to favor a candidate (usually, the one you support). A better approach is to treat the polls as presumptively accurate but with the understanding that could miss equally in either direction. That’s the “high accuracy, low precision” view that Nate Silver has, and it was the argument I made back in December, when Trump was leading the national polls. Over the long haul, polling error is a fact of life. It’s not necessarily getting better or worse over time, and you can’t predict which direction it will go in any future election.

In the postwar period (that is when polls weren’t conducted of Republican-leaning magazine subscribers) polls have overstated Democratic support in two thirds of elections and Republican support in one third. You might assign a slightly pro-Democratic “tilt” to the polls on that basis, but that’s hardly a smoking gun that the polls will be wrong to Trump’s eventual benefit this year.

But now that Trump is behind in the national polls, all the theories of why polls were wrong in 2016 or 2020 — shy Trump voters or nonresponse bias — are back with a vengeance. Rather than speculating too much on these theories for the umpteenth time, I’m interested looking systematically at where the polls missed and equipping readers with a framework to use this information to predict if they’re likely to miss again in 2024.

While polling error is random across cycles and candidates, it was not necessarily random in its treatment of the same candidate, Donald Trump, in the two elections in which he ran. In 2016 and 2020, polling missed in pretty much the same ways across the same states, showing that at least in the Trump context, polling error was not purely random. The overall presence or magnitude of error might be impossible to predict, but the data shows clear signs of trouble in certain states.

Here’s what you want to look out for to know whether the 2024 polls are likely to be wrong again: