2022 Midterm Predictions

Always start with the base rate

Note: I’ll be giving a talk on the forthcoming election and what’s going on with polling at the Salem Center at the University of Texas, Austin, on November 7 at 4PM. You can join us live or through Zoom. Details here.

I’ve already written about the discrepancy between what polls are saying compared to prediction markets and forecasting websites. Matt Yglesias adds another point about why you should start with an assumption that Republicans are going to do well.

According to 538, Republicans have a 72 percent chance of taking the House, while Democrats have a 64 percent chance of holding the Senate. Those forecasts seem D-skewed to me. I will be genuinely shocked if Democrats hold the House. The only precedents for that happening in remotely recent history are the 9/11 election and the 1962 midterms held right after the resolution of the Cuban Missile Crisis. I’m not saying it’s impossible — we do have those two examples — but it’s difficult to understand why that would happen this fall. By contrast, Republicans picking up a net of one Senate seat would be a completely banal outcome. The fact that polling is giving it only a 1-in-3 chance of happening is, I believe, a consequence of the polling being skewed.

This strikes me as correct. Here’s a chart showing how the president’s party has done in every midterm since 1946.

In 17 out of 19 cases, the party out of power has picked up House seats. Right now, Democrats have a majority of only 8, meaning any notable shift towards the Republicans should mean they lose the House. And the Senate races that will determine control of that body are in states that are perennial toss-ups.

With the election a week away, I’d like to invite anyone who hasn’t had a chance to sign up for the Salem Center/CSPI forecasting tournament to do so. In designing the questions, I wanted to not only get forecasts for the midterms, but ask questions testing whether people think that the polling is likely to be biased. First, we asked if there would be at least a 3-point polling miss towards the Democrats in the Senate races in Ohio, Pennsylvania, and Wisconsin, defined relative to the popular vote deluxe forecasts at FiveThirtyEight. As I write this around 7AM Eastern Time on October 31, that number is at 55%, which is extremely high given that ideally there should only be a 50% chance of a pro-Democrat bias of any size. The second thing we did was ask whether the FiveThirtyEight deluxe forecast would favor Republicans on election day. That number is at 59%, which is indistinguishable from their odds of actually winning the Senate (60%). Until very recently, there had been a consistent gap between those markets, with the latter being higher. I think the convergence between the two numbers is likely temporary, as it seems somewhat difficult to square with the belief of the market in the likelihood of a major polling error in favor of the Democrats.

Between Salem/CSPI, Metaculus, PredictIt, and Polymarket, we have four different kinds of forecasting platforms with various kinds of incentive structures. We can compare them all to FiveThirtyEight, which, as alluded to above, has a “polls only” model, and a “deluxe” model, which takes into account things like fundraising, expert opinion, and how states lean, but still puts a lot of stock in the polls. Here’s what we get when we compare the odds. See my previous post for numbers from September in specific House, Senate, and governor’s races.

Chances of Republicans taking the House and Senate, as of October 31, 2022, 7AM ET. There are no notable differences between mean and median estimates at Metaculus.

It appears that Metaculus is much closer to the polls than the other platforms. I wonder if this might have to do with the politics of its users, as with the other websites. Salem/CSPI participants might lean right because of the two organizations sponsoring the tournament. Polymarket is a betting website that uses crypto, so you can imagine what kind of views its users hold. I’m less sure of PredictIt, but the fans of the website that I know are all conservative, though who knows how representative they are. Betting markets in general seem like something right-leaning people are interested in, hence their biggest public boosters being figures like Bryan Caplan and Robin Hanson. Meanwhile, other than occasional tournament prizes, Metaculus pays out in prestige. Maybe this brings in proportionally more liberals relative to prediction markets? This would be similar to the way in which prestige-heavy but low-pay professions like journalism and academia attract people on the left, while conservatives have more of a tendency to go into areas with the opposite profile, becoming small business owners or working in oil and gas production.

While I’ve previously implied that I thought Salem/CSPI and PredictIt were just giving Republicans better odds than polls because they were closer to the truth, seeing Metaculus believe more in the polls makes me suspect that the divergence might be based mostly on the politics of who is doing the forecasting in each case. Nonetheless, nothing has changed with regards to the case I recently made about why, given recent experience, we should expect a polling bias towards the Democrats. Seeing Republicans starting to do a bit better in the polls and reflecting on the historical factors Yglesias points to, I am more confident than I had previously been that they should win both the House and Senate. I’d put the House at 90%, and the Senate at around 65%. Given that I suspect politics is influencing how people forecast, I should be honest and say I’m hoping Republicans win. While I leaned that way before, liberal positions on masking and other pandemic restrictions have increased the emotional intensity of my opposition to the Democrats and my desire to see them out of power.

Finally, I usually try not to go by “gut feelings” on these things, but I’ll allow myself the indulgence in one race. I bought 2,000 shares of Oz on PredictIt for an average of 34 cents earlier this year. Even though he’s now at 65, I’m still not selling. Voters can be mysterious, and people often confuse their own aesthetic reactions to candidates with those of the broader public. This is why pundits, despite polling to the contrary, found it so hard to accept that Trump was the favorite throughout the 2016 Republican primaries and Biden was the Democratic favorite in 2020. With that danger in mind, I still can’t believe Pennsylvania will choose Fetterman over Oz. This race has for a long time shown particularly large discrepancies between polls and prediction markets, and I think for good reason. It’s no accident that Oz was successful on TV, while no talk show hosts look or talk like his opponent. Football analysts often do a “pick of the week.” I’ll do the same here, and call Pennsylvania for Oz.

This article was cross-posted at the Metaculus Journal.