Wednesday, November 5, 2008

Wednesday Math, Vol. 46: Margin of error and confidence of victory revisited

Things were a little busy this morning, and I wanted to wait to see how the states ended up before doing this week's Wednesday math, a post-election wrap-up of the state by state presidential contests using both the conventional margin of error method, which is actually the 95% confidence interval, and the confidence of victory method that I've been working on.

Using the most recent polls from every state and Washington, D.C., things changed slightly from the final Sunday numbers. Two states became flat footed ties, and so in the confidence of victory method, Obama and McCain both had a 50% chance to win in Missouri and Indiana. Both of these contests were too close to call for most of the evening last night, with Indiana being called for Obama late last evening and Missouri called for McCain today.

On most websites as of Wednesday evening, only North Carolina has yet to be put in either column.

Using the Confidence of Victory method, there was a favorite in 49 of the 51 contests, and the CoV method using the latest poll or the median of the three latest polls in the case more than three polls in the last week picked the correct winner in 48 of 49 contests. The only error is if Obama wins North Carolina, as McCain was a slim 60% to 40% favorite in that state. Missing one 60%-40% contest is nothing to be ashamed of.

In other words, the CoV method was extremely accurate in 2008. In 2004, it missed two states, Ohio and Florida.

As for the standard margin of error, since it is supposed to be 95% accurate and we have 102 results, we should expect the 96.9 numbers to be within the margin and 5.1 number to be wrong on average, which rounds to 97 right and 5 wrong. In reality in 2008, there were a total of four mistakes in presidential vote predictions.

Washington DC, two errors: McCain's result was far lower than expected and Obama's far higher than expected. It was predicted to be about 86% to 14%, it was actually 93% to 7%. Massively increased black voter turnout is the reason for the error.

Nevada, one error: The prediction was 52.2% for Obama and 47.8% for McCain, with a 4.1% margin of error. Obama got 55%, within the margin of error, but McCain's 43% fell outside the the margin of error to the low side. Reason for the error is assumed to be greater turnout than expected, with Hispanic turnout given large credit.

Rhode Island, one error : Prediction was 59.8% to 40.2% for Obama, with a margin of error of 4.4%. Obama's 64% was inside the margin, but McCain's 35% was lower than anticipated. Again, voter turnout is the most likely culprit as to why the numbers were off.

The Obama effect is that the polls got it awfully damn right. Obama did somewhat better than expected 18 times, and McCain did marginally better than expected 16 times. The number of times they underperformed outweighs the number of times they overperformed in both cases because the numbers are stretched to give them 100% of the vote, and in many states other candidates pulled in 1% or 2% of the total vote.

The Matty Boy effect is that the Confidence of Victory method works SCARY well.

I gots to figure out how to GIT PAID behind this thing.

Matt said...

Getting paid is easy. Figure out how to transfer this to football. I know you have a site for that but once you can get 95% you'll be beyond rich. Like King of Saudia Arabia rich.

I'm just sayin.

Matt

Jess Wundrun said...

Sounds to me like you should work for fivethirtyeight.com after you find a way for their site not to crash on election night!

dguzman said...

Ditto what Jess said.