This blog is still alive, just in semi-hibernation.
When I want to write something longer than a tweet about something other than math or sci-fi, here is where I'll write it.

Sunday, November 18, 2012

Watching the watchmen:
Final grades for all polling companies with 3 or more predictions in final week


At least part of my pay is for grading papers, so it seems only fair that I grade the numbers from the polls released in the final week of the election.  I would have done this earlier but I gave three midterms so I have been busy... grading papers.

A+ companies

Company: PPP (d)
Total races polled in final week: 33
Record: 33-0
Worst miss: Predicted McCaskill by 4.0% MO Senate, won by 15.5%
Average distance from correct percentages: 1.9% toward GOP
Median distance from correct percentages: 2.0% toward GOP

It's really hard to argue with 33-0 being at the top of the list, and PPP deservedly earns the Busy Bee Medal, but notice how much PPP skewed towards the Republicans in their sampling. You will see how big a trend this is in all the companies, with damned few exceptions.

===============

Company: SurveyUSA
Total races polled in final week: 11
Record: 11-0
Worst miss: Predicted Heller by 6.0% in NV Senate, won by 1.2%
Average distance from correct percentages: 0.9% toward GOP
Median distance from correct percentages: 2.2% toward GOP

Yet another company with a spotless record, several tracking websites like Real Clear Politics are slow to publish the stuff SurveyUSA puts out. If someone has put them in the penalty box for earlier infractions, it's time to let them back in the game.

===============

Company: Marist
Total races polled in final week: 10
Record: 10-0
Worst miss: Had Obama by 6.0% in OH, won by 1.9%
Average distance from correct percentages: 1.5% toward GOP
Median distance from correct percentages: 2.0% toward GOP

Median is farther to the right than the average because of this big miss towards Obama.
 ===============

Company: Grove Insight [d]
Total races polled in final week: 7
Record: 7-0
Worst miss: Predicted Obama by 6.7% in MI, won by 9.5%
Average distance from correct percentages: 0.1% toward GOP
Median distance from correct percentages: 0.1% toward Dem


I'm giving this partisan pollster the Fair and Balanced Medal for this election cycle. Only seven races called, but their median and average away from true are incredibly close, and even more pretty, one is on the GOP side and the other on the Dem. Nobody else did that.

===============

Company: Gravis
Total races polled in final week: 6
Record: 6-0
Worst miss: Predicted Nelson by 3.0% in FL, won by 13.0%
Average distance from correct percentages: 2.7% toward GOP
Median distance from correct percentages: 1.6% toward GOP

Lots of companies were off by a bunch on the margin of victory in the senate races in Florida and Missouri, and Gravis was no exception.

===============

Company: Zogby
Total races polled in final week: 6
Record: 6-0
Worst miss: Predicted Nelson by 20% in FL, won by 13.0%
Average distance from correct percentages: 3.4% toward Dems
Median distance from correct percentages: 3.5% toward Dems

Zogby gets the In the Tank for the Dems medal. Not even the partisan Democratic polling companies skewed as far left as Zogby. Over-predicting Nelson's margin of victory was a big, big outlier.

===============

Company: Angus Reid
Total races polled in final week: 4
Record: 4-0
Worst miss: Predicted Baldwin by 2.0% in WI Senate, won by 5.6%
Average distance from correct percentages: 1.6% toward GOP
Median distance from correct percentages: 1.6% toward GOP

Another perfect score that oversampled GOP voters and never got pulled off course.

===============

Company: Mellman [d]
Total races polled in final week: 4
Record: 4-0
Worst miss: Had Obama by 2.0% in both FL and IA, won by 0.9% in both
Average distance from correct percentages: 0.4% toward Dem
Median distance from correct percentages: 0.5% toward Dem

The partisan company Mellman oversampled Democrats slightly, but only by about a half a point instead of the bigger oversampling of Republicans shown by many companies who got perfect records.

The reason this is possible is because this election really wasn't that close.
 
===============

A companies

Company: WeAskAmerica
Total races polled in final week: 12
Record: 11-1
Missed with Romney by 0.9% in FL where Obama won by 0.9%Worst miss: Predicted McCaskill by 3.4% MO Senate, won by 15.5%
Average distance from correct percentages: 1.3% toward GOP
Median distance from correct percentages: 1.6% toward GOP

A miss is a miss, but this was so close I might only take a half point off if I were grading a student who turned this in.

===============

Company: Reuters/Ipsos
Total races polled in final week: 6
Record: 5-1
Missed with Romney by 1.0% in FL where Obama won by 0.9%
Worst miss: Predicted Brown by 9.0% in OH Senate, won by 5.2%
Average distance from correct percentages: 0.1% toward GOP
Median distance from correct percentages: 1.0% toward GOP

Yet again, a compnay misses a perfect record by missing the toughest election to call by a single percentage point. 5 of 6 would count as 83%, which is a middle B, but 5.5 of 6 is 91.7%, a low A.

===============

F companies

Company: Rasmussen/Pulse Opinion Research (subsidiary)
Total races polled in final week: 16
Record: 9-5-2
Missed with Romney by 2% in FL, Obama wins by 0.9%
Missed with tie in OH, Obama wins by 2.0%
Missed with Romney by 1% in IA, Obama wins by 5.6%
Missed with Romney by 2% in VA, Obama wins by 3.0%
Missed with Romney by 3% in CO, Obama wins by 4.7%
Missed with tie in VA senate, Kaine wins by 5.0%
Missed with Thompson by 1% WI senate, Baldwin wins by 6.7%
Worst miss: Predicted Nelson by 4.0% in FL Senate, won by 13.0%
Average distance from correct percentages: 4.5% toward GOP
Median distance from correct percentages: 4.4% toward GOP

According to their websites, Rasmussen contracts out the actual polling to Pulse Opinion Research, and Pulse Opinion says they use Scott Rasmussen's model.  According to a Twitter conversation I had with Josh Marshall of Talking Points Memo, he has done research that says Pulse Opinion is a subsidiary of Rasmussen, though he did not provide a link.  In either case, these two are joined at the hip. There were a few races they polled more than once in the last week, and in that case I take the most recent, or if reported on the same day, the one with the larger sample size.

Okay 16 points possible, 9 definitely right.  This reminds me of the grade lawyers I see in class after midterms begging to have their grades changed.  I'll give a half point each for the close presidential races in FL and OH, but no way I give a half point for calling a tie in a race decided by 5%. That means 10 right, 6% wrong, 62.5% on the test.

When I am being really generous, that might be a D or D-, but looking at the massive difference between Rasmussen/Pulse and the next company better than they are, I'm calling it an F.

===============

Company: Wenzel [r]
Total races polled in final week: 3
Record: 1-2
Worst miss: Predicted Mandel by 5.0% in OH Senate, Brown won by 5.2%
Also pickey Romney by 3% in OH, Obama won by 0.9%
Average distance from correct percentages: 6.6% toward GOP
Median distance from correct percentages: 4.9% toward GOP

Most Republican partisans only did one or two polls in the last week, but Wentzel did three, so they get to do the Walk of Shame in this week's grading.

But the real shame belongs to a non-partisan poll that stunk the place up just a little bit worse.

===============

Company: Mason-Dixon
Total races polled in final week: 3
Record: 1-2
Worst miss: Predicted Rehberg by 4.0% in MT Senate, lost by 3.9%
Also called Romney by 6% in OH, where Obama won by 0.9%
Average distance from correct percentages: 7.3% toward GOP
Median distance from correct percentages: 7.0% toward GOP

Yet again, it's a non-partisan company that looks the most partisan as Mason-Dixon gets the In the Tank for the GOP medal. 
===============

Do you believe in duality, professor?

Hypothetical question asker, it's nice to have you back. And no I don't. Duality is for wimps.

Like Gaul, I am split in three parts.

When it comes to politics, I am definitely a left leaning partisan. When it comes to gathering poll data, I always play it down the middle, preferring the median to the average, which can drift badly when pulled by outliers. But when it comes to the polling industry, I have to hope the free market prevails. A ridiculous amount of money was spent on this election, including the hiring of more polling companies than I have seen in any election cycle, way more that 2004 or 2008. Some people know they didn't get their money's worth, most obviously when their pollster said they were ahead or tied when they actually were far behind.  The companies that did well should get gold stars by their names and the companies that did badly should have their bad grades posted like a weak report from the Board of Health.

The polling industry could use some new blood to bring them into the 21st Century, a company with a lot of smart people who are good with numbers and data and ideas of how to collect it in the modern age. Nate Silver has an interesting story that Google Consumer Surveys was one of the most accurate predictors of the presidential popular vote tally. Nate, may he be blessed by Odin, Vishnu and the little baby Jebus, works at The New York Times and has lots of access. I live in Oakland, far from Olympus but surprisingly close to Silicon Valley. I have a very well-placed source that tells me Google Consumer Surveys is about to take a big step in that direction and pick up the crown dropped by the people now dragging the late George Gallup's name through the mud.

Remember, you heard it here first.

Sunday, November 11, 2012

Watching the watchmen:
Final 2012 grades for the smaller partisan polls

Public Policy Polling is officially a Democratic partisan poll. Rasmussen Polls is officially not a Republican partisan poll, much in the same way that Fox News is not officially the propaganda arm of the Republican Party.

Which is why Karl Rove is part of their election night news team.

In any case, in the final week of polling, PPP went 33-0 picking both Democrats and Republican winners, while Rasmussen became a shrinking violet, going 5-6, all six losses occurring because they said a Republican would win or it was a tie when the Democrat actually prevailed.

So let's look at the polling companies that have either an [r] or a [d] after their names that don't produce as many polls as the big boys.

Garin Hart Yang [d]: 1-0. This company is a significant reason why my system picked Montana, a race that helped me beat Nate Silver this year.  Thanks, guys.

Keating [d]: 1-0. Keating said Obama by 4 in Colorado, when the actual lead was 4.7% That's nice close work.

Global Strategy [d]: 1-0. They predicted a 7% lead in the Indiana senate race, which finished at 6.6%.Given that the company rounded to the nearest percent and the final answer rounded to the nearest tenth of a percent, this is as close to golden as they could get.

Lake Research Partners [d]: 1-0. Arguably the worst of the Democratic partisans because they were so far from the correct percentage. predicting Obama by 1% when it was actually 4.7%. Notice that this company actually gave Romney more of an advantage than he actually deserved, given the numbers.

Mellman [d]: 4-0. They got it right on the presidential side in Florida, Iowa, Colorado and Virginia. very strong work.

Grove Insight [d]: 6-0. They got it right on the presidential side in Florida, Iowa, Colorado, Michigan, Ohio and Wisconsin.

Let it here be noted, that none of these companies ever picked a Republican to win. PPP correctly chose six GOP winners on their way to going 33-0.

And now the Republicans.

NMB Research [r]: 0-1. They said it was Romney by 1% in Minnesota. It was Obama by 7.7%. Maybe this is the poll George Will saw when he predicted Mitt would win the Land of 10,000 Lakes and One Big Mistake. 
 
Glengariff Group [r]: 2-0. This is the best Republican partisan polling result by far, but they picked two easy races, the Michigan senate race and Romney vs. Obama in that same state. Thing is, these were two huge wins for the Dems, roughly 10% for Obama and 21% for Debbie Stabenow, and Glengoniff missed the actual victory margin by about 7% favoring the Republicans each time.

Did I call them Glengoniff? Oops, my bad.  It's easy to look good stinking the place up in races like this.

Kimball [r]: 1-1. Like Glengoniff, Kimball only polled one state, Massachusetts. They were even worse though, missing the actual results each time about about 10%. That means they gave a win to the loser Scott Brown and gave Obama only a 13% margin instead of the 23.2% the people of Massachusetts actually gave him.

Susquehanna [r]: 1-0-1. Another one state poll, Pennsylvania. They said Casey by 1% in the Senate, he won by 8.9%. They said it was a tie between Mitt and Barry, Barry won by 5.2%

Public Opinion Strategies [r]: 1-0-1.  Though they have the unfortunate acronym of P.O.S., for a Republican polling company they almost come up smelling like roses. They polled New Mexico, and missed the Senate by calling it a tie when Heinrich won by 5.6%, they actually did some creditable polling in the presidential race, giving Obama and 8% lead when he actually won by 9.9%.

Wenzel [r]: 1-2. Wenzel polled in the last week in both Ohio races and in Wisconsin presidential. It's odd that they didn't poll Wisconsin Senate as well, but I won't speculate as to why not. The one they got right was Obama winning Wisconsin, though they guessed 2% when it was actually 6.7%. They completely sucked eggs in Ohio, predicting a 5% win in the Senate race when it was a 5.2% loss and picking Romney by 3% when it was Obama by 1.9%

I have already said I will NOT use fudge factors in my system ever, even though I have seen how dishonest the partisan polls can be, especially the Republican side. Though I am a state employee - and so in the view of Pat Sajak a super moocher who does not deserve citizenship - I'm going to go with the free market in this case. People running campaigns in either party need reliable data, and the Republican partisan polls look like blatant puffery. The people who paid for it this time are going to think twice about laying down actual cash for this crap. I also expect that some of these companies will fold in the night and sprout up again, the same crooks using a different name, but then again, that's the free market, too, isn't it? Besides clowns paying for this garbage, there's the question if any news outlet will publish it when the track record this year is made public.

The Republicans have some serious structural problems going forward. They are not producing grumpy old white guys fast enough and a lot of their infrastructure looks to be run by people who think Jack Abramoff just chose unwisely when he selected the people he would fleece. That said, in midterms the turnout goes down, and less people voting is often an advantage for the GOP.

It's now a while off, but stay tuned.




Watching the watchmen:
Final grades 2012, PPP vs. Rasmussen


Officially, Public Policy Polling (PPP) is a Democratic funded polling company. Unofficially, Rasmussen Polls is independent.

The numbers from the final week of polling tell a simply astonishing tale.

Some Bush administration official called the mainstream press, which they considered hopelessly biased, "the reality based community". The underlying point was the press would report what happened, but the new GOP would create what happened, and the press would simply fall behind.

The GOP lost a lot of power in the past few years and reality wants some payback.


PPP worked a LOT harder than Rasmussen in the last week at the state level for both presidential and senatorial races. Remarkably, they did not call a single tie.  When it was tied, they let guys like me see the secret stuff, the exact counts of polls. By my way of measuring things as a person with nearly two decades of grading experience, they made some very wise choices by showing all work.

Presidential races.

PPP polls show Romney ahead in AZ, MO, MT and NC: Romney wins all those states.

PPP polls show Obama ahead in CO, CT, FL, IA, ME, MI, MN, NH, NV, OH, OR, PA, VA, WA and WI: Obama wins all those states.

Senatorial races

PPP shows Republicans ahead in AZ and NV: Republicans win both those races.

PPP shows Democrats ahead in CT, FL, MA, MI, MN, MO, MT, OH, PA, VA, WA and WI: Democrats win all those races.

That's 33-0, pretty stunning. You might think that is something like going 32-0 in the first round of March Madness, but this is actually much easier.  Some of those contests really weren't contests at all and every poll agreed for months.

If any right winger stumbles around the part of the Internet they don't control and see this, you might have read somewhere in your comfortable nest that this election was stolen.

Let me remind you of a quote from Joe Kennedy, the arch conservative father of the Kennedy clan. "I'll buy you an election, Jack, but I'll be damned if I'll buy you a landslide."

And given all the money on the other side, buying a landslide now would be prohibitively expensive. All evidence points to this being the will of the people, sick of being insulted in droves.




And now to Rasmussen. One remarkable thing here is how little polling they did in the last seven days. I'm guessing Scott Rasmussen is in a quiet dark room, fumbling with prayer beads and asking the Almighty to let him keep his syndicated TV show long enough to make up for all the polling contracts he's going to lose.

Presidential races.

Rasmussen polls show Romney ahead in CO, IA, IN and VA: Romney wins Indiana and loses the rest. The worst numbers are in Colorado, where Rasmussen misses the margin of victory by 7.7%.

Rasmussen reports tie contests in OH and WI: Obama wins Ohio by 1.9% and Wisconsin by 6.7%.

Rasmussen polls show Obama ahead in NH: Rasmussen gets one right, though under-reporting the margin by 3.8%.

Senatorial races

Rasmussen shows Democrats ahead in IN, MT and OH: Three successes in terms of win-loss, but still under-reporting the margins by, 3.6%, 2,9% and 3.2% respectively.

Rasmussen shows a tie in WI:The Democrat wins by 5.6%.

Okay, if we throw out widespread voter fraud, what other options are there?

The Republicans really are this stupid and incompetent.  This is possible, as you will see in the companies I call the off-brand partisan polls, The Democrats do very well and the Republicans make mistake after mistake. I have a hard time believing that no Republican company can hire a single honest competent person. Just because it's hard to believe does not prove that this isn't the case.

The Republican infrastructure played a long con this year:  Citizens United opened a floodgate and even with the economy a mess, demographics are not in the Republicans' favor and will never be again unless the party changes dramatically on every level, local, state and federal. I do not believe the Republicans are all inept, but I do believe more than a few of them are dishonest. If Jack Abramoff salivated at the idea of naive owners of Indian casinos paying for access, think of how Karl Rove felt when he landed a solitary super-whale like Sheldon Adelson. Adelson backed Newt Gingrich in the primary, a strong sign he is not the sharpest crayon in the box, and even decided to attack Romney with anti-Bain ads. The word was he hated Santorum and even when Newt was clearly road kill, he made sure Santorum would not be the final and successful Not Romney.

Looking at the relative lengths of my explanations for each case, you might well think I have my own pet theory.

You might very well think that, I couldn't possibly comment.

The second part dealing with the lesser partisan polls will be published this evening.

Friday, November 9, 2012

How hard is it to beat Nate Silver?
Apparently, not that hard at all.

I've been doing my election snapshot system which I call Confidence of Victory for three presidential elections now, 2004, 2008 and 2012. I have made small adjustments to my system over that time, but always in the spirit of mathematics, trying to correct problems in the simplest, most elegant way possible. Many other people do this, too, but they use the statistical method, which usually involves adding "fudge factors" to calculations. Even Sam Wang at Princeton, whose method most resembles mine, does this. Statisticians do this all the time and mathematicians, especially someone like me trained as a pure mathematician, avoid that kind of fiddling with numbers like it's the plague.

To make a comparison, it would be like journalists changing quotes. It's dishonest and unethical. Even worse, the best mathematical system can avoid this nonsense and make better predictions, so it's also completely unnecessary.

A fellow named Dean Chambers did a lot more fudge factor work and came up Unskewedpolls.com.  People paid attention to him. His method was praised by that great mathematician and physicist, Governor Rick Perry of Texas.  That should have been an early clue this guy had his head up his ass, but conservatives really did buy this, as I found out in a short back and forth with a conservative Facebook friend of my good buddy Padre Mickey, now Doctor Padre Mickey. El Doctor Padre is not conservative. As he used to say, he is only a registered Democrat because Sandinistas don't get to vote in the primaries.

I started my system in 2004. I didn't have a blog then, but I posted online to a website no longer available. My last prediction online was recorded by the Wall Street Journal and I thought Kerry had a 74% chance on Sunday because both Ohio and Florida were barely in his camp. Monday things changed a little toward Bush as Florida moved into his column but I didn't post it and it was past the Journal's deadline.

I did remember it, though. I made sure my last prediction would always include the last polls.

While my system has been refined, a great advantage is how much more data I have to work with now. The worst thing that can happen is to rely on a single poll in a difficult to determine contest. In battleground states now, there is no way they will be "underpolled".

In 2008, I had a blog. I only posted numbers on my blog on Sunday because by October there was no excitement in this race, unlike this year when the press in general and the conservative press in particular thought this thing was neck and neck, but I did update with a final post.I had it at 353-174, with Indiana a flat footed tie in my system.

In 2008, Nate Silver was "just a blogger", but he was actually connected to Daily Kos. As you can see, his last guess had Indiana a light red, and his final numbers were 353-185.

The real result was 365-173.  Everything I put a number on was right, and Nate messed up in Indiana. We both missed District 2 in Omaha for that last electoral vote, but I didn't do the work on it and he did and got it wrong.

My final result. 50-0 with one too close to call.

Nate's final result. 54-2, all that I had right, right in the Maine districts and two Nebraska districts but screwing up in Omaha and Indiana. Where we both said something, he went 50-1-0 and I went 50-0-1, where the order of the numbers is wins, losses and abstentions.

I didn't do the Senate that year.

So here's the two election totals. I made predictions in 135 races and went 133-1-1. My system said Florida was too close to call this year, but I went all stupid and tried to be a pundit. I guessed Romney and now it's Saturday morning in Oakland and I was wrong. My system said no comment, but I made a comment and and my word is my bond. Nate Silver's system said slight advantage Obama, so he beat me there.

In those 135 races, Silver never abstained and went 132-3. Two more mistakes and one less correct pick, he has yet to abstain.


Nate's system works really well, but mine works better. We almost never disagree, but when we do this is what happens. If I abstain, it's because the poll in the middle says the race is a flat footed tie. If I make a statement one way or the other, it's because a majority of the recent polls I counted favored either Obama or Romney.

When Silver's system disagrees with mine, there is some poll in the minority that is an outlier, screaming at the top of its lungs a result almost no one else agrees with, at least in degree.  Because I use the median, the screaming outlier just gets a single vote like everyone else.  Nate's system gets pulled away from the opinion of the majority and into the minority.

That might be to his advantage sometime, but that's not the way to bet.  In the long run, the Law of Large Numbers is on my side.  A casino doesn't get lucky when a roulette player finally goes broke. The way a casino gets lucky is when the roulette player takes out the wallet and decides to play. In the long run, the house wins, and my system is the house and Nate's isn't.

Math loves beauty as well as truth. Stats tinkers and fidgets and gets fooled by randomness way too often. So it was in the beginning and unless statisticians wake up, so it shall be at the end.

Here endeth the lesson.


Wednesday, November 7, 2012

Mouth and money time, part 2:
Hubbard 83, Silver 82
Game over, dude.


I made prognostications in 84 races on Monday, the 33 Senate races and the 51 electoral contests, all the states and the District of Columbia. Nate Silver made predictions in all these and many more.

Let me point out that this is just a hobby for me and he's getting paid and has a staff.

Florida goes to Obama.That's a win for Silver and a loss for me. Technically it is a loss for Matthew Hubbard and not for my system, since my system thought it was a toss-up.

No matter. Silvers gets a win and I get a loss.

The first two disagreements to be called were the Senate races in North Dakota and Montana. My system had the Democrats ahead and Nate's said they were both going Republican.

DOWN GOES SILVER! DOWN GOES SILVER!

Sorry, I couldn't help myself. I need to stay in practice with my online Howard Cosell impression.

After 84 contests, I lead 83-82. In races where both of us made predictions in 2008 when I only followed the presidential contests, I had 50 correct with one abstention, Indiana. Silver predicted McCain in Indiana, which means I won that year as well by an ever narrower margin, 50-0-1 to 50-1-0.

Look, people have called me a genius many, many times in my life. If I compare myself to the big dogs, people like Newton and Gauss and Euler and Von Neumann, I know how ordinary I am. But if Nate Silver's a genius in this world, I'm a super-genius.

No brag, just fact. My simple system is better than his arcane one. It's the difference between the elegance of math and the constant tinkering of statistics.

Mathematicians really do outrank statisticians. It's all part of the "physics envy" nature of the ranking of the sciences. I didn't create this bias, but I do essentially agree with it and if it plays to my favor, why should I argue?

No disrespect to guys like Persi Diaconis, who works in both the math and stats department at Stanford and uses one of my algorithms in his class.

There are so many people on the right despising Nate Silver right now, I want to distinguish myself from them. He's not bad at what he does. He's very good. Getting 82 of 84 contests right is excellent, even though some of them are very obvious and everyone will get them right.

I went 83 of 84 on the same predictions. That is clearly better.

I do not want Nate Silver driven from the national stage. I want my chance to be part of the conversation as well. I have important things to say and ways to make polling data better, if anyone thought what I had to say was worth noticing.

With all my heart, I know it is. There are many important things I know that no one on the national stage is saying. 

This is two presidential elections in a row for me beating him. I look forward to 2014. Who knows, maybe he'll know who I am by then and offer me a $1,000 bet like he did Joe Scarborough.

Back when I gambled a lot more, I made it a rule never to borrow money to place a bet. If need be, in this case, I'll make an exception.


Monday, November 5, 2012

It's mouth and money time:
The predictions for the Senate and electoral college

Enough of the calculations, it's time for prognostication. No more probabilities, it's time to put money on the table.

The Senate.

Republican wins: MS NE* TN TX UT WY AZ NV


Nebraska's Ben Nelson is retiring and the race looks good for the Republicans there, so a net Republican gain of 1.


Democratic wins: CT** FL IN* MA* MO MT ND* NM OH PA VA WI CA DE HI MD MI MN NJ NY RI VT WA WV

The three marked with asterisks will be changes from Republican to Democratic. The double asterisk is Connecticut switching from the independence of Joe Lieberman to a real Democrat.  Technically, this would be a gain of 3, so the Democrats get an overall gain of 2.


Independent wins: ME*

This is a switch from Republican to an Independent who sounds like he'll caucus with the Dems, which makes for a net gain of 3 for the Democrats.

As for differences with Nate Silver, he has the Republicans winning in Montana and North Dakota.

Here's my prediction for the electoral college as a picture.



My system says the most likely result is Obama 303, Romney 235.  Plenty of other people have come to this conclusion as well. I haven't seen him put his numbers in black and white, but it looks like the only disagreement I have with  Nate Silver is Florida, which he thinks Obama will win for a final total of 332 to 206.

Nate Silver isn't a crazy person. His system has just a glimmer of an advantage in Florida. After some moving around today, Florida is a flat footed tie in my system, but my computer program also spits out the probability for every possible combination and slightly favors 303 to 332, 3.65% to 2.86%. That's what the numbers of my system say and that's where I'll make my stand.

As for the popular vote, I've only done some back of the envelope calculations, less than an hour fiddling with an idea in Excel. I'm not as convinced as I am with the winners and losers I've picked, but I'm going with 49.6% Obama, 48.4% Romney and 2.0% for the other candidates. Silver has it as 50.9% for Obama, 48.3% Romney and  0.8% Other.  That means against Silver, I'm bullish on Mitt and Gary Johnson while he's bullish on Obama.

Now all we have to do is wait for the votes to be tallied.  I'll be live tweeting tomorrow night under the name @ConfidenceOfVic.

Thanks to everyone for all your kind words. Make sure you get out and vote. If you have the time and inclination, do what you can to get out the vote as well.

Soon enough, we will get the proof of the pudding.  This is an exciting time for me. After months collecting and analyzing data, I can no longer hide behind the previously valid excuse of "if the polls are taken on the day the election is held". With early and absentee ballots, the election is being held now and has been going on for weeks.

I will be back here doing post mortems for most of the week.

Sunday numbers, 11/4, the final countdown:
The Senate Races


Here is an almost final snapshot of the Senate races. An Asterisk next to a race means a seat changing hands

Solidly Republican.
MS NE TN TX UT WY

===
Strongly Republican.
AZ    0.9526
NV    0.8425

==
Leaning Democratic.
MT    0.5898
ND *    0.5957
PA    0.6299
WI    0.6311
MA *    0.6789
VA    0.8309

==
Strongly Democratic
MO    0.8870
NM    0.9844
CT    0.9929
FL    0.9867
IN*    0.9616
OH    0.9893

 ==
Solidly Democratic.
CA DE HI MD MI MN NJ NY RI VT WA WV


The unmentioned state in this list is Maine, where independent Angus King looks solid to take a seat away from the Republicans with Olympia Snowe's retirement. If these races go according to form, the Democratic caucus could pick up three seats and possibly four if King decides to join them, which is likely but not yet set in stone.

I'll come back to this on Thursday to see how the Confidence of Victory numbers did in terms of prediction.

Sunday numbers, 11/4, the final countdown:
The Electoral College

It feels like I've been doing this for years, but this is the next to last update.  There will be a few more polls reported on Monday and then comes election day.  The picture is getting very clear. Let's first look at the states in play

As before, the colors red and blue mean Republican and Democratic leaning respectively, purple means a flat footed tie under my system and the state with the gold background is the state both side need to win in the simplest path, which I call The Golden Spike.

As states are declared for one candidate or another on Election Night, the Golden Spike can move.  I'll be live tweeting the updates under my Twitter name ConfidenceOfVic.

North Carolina
============

 Most recent poll: October 31
Number of polls within one week of most recent: 7
Obama leads: 2
Tie: 1
Romney leads: 4
Confidence of victory: 58.3% Romney


Florida
======
 

Most recent poll: November 4
Number of polls within one week of most recent: 11
Obama leads: 5
Tie: 1
Romney leads: 5
Confidence of victory: 50.0% for both Obama and Romney
Improvement for Romney


Virginia
============

 Most recent poll: November 4
Number of polls within one week of most recent: 7
Obama leads: 6
Tie: 0
Romney leads: 1
Confidence of victory: 63.7%
Improvement for Romney

Colorado
=======

 Most recent poll: November 3
Number of polls within one week of most recent: 9
Obama leads: 7
Tie: 0
Romney leads: 2
Confidence of victory: 64.0% Obama

Iowa
====

 Most recent poll: November 2
Number of polls within one week of most recent: 9
Obama leads: 8
Tie: 0
Romney leads: 1
Confidence of victory: 75.9%
Improvement for Romney

New Hampshire
=============
 

Most recent poll: Nov. 4
Number of polls within one week of most recent: 6
Obama leads: 5
Tie: 1
Romney leads: 0
Confidence of victory: 76.6%
Slight improvement for Obama

Ohio
====

 Most recent poll: Nov 3
Number of polls within one week of most recent:14
Obama leads: 12
Tie: 1
Romney leads: 1
Confidence of victory: 79.0% Obama
Note: This is currently the state with The Golden Spike. If Obama wins here and every state that has a better Confidence of Victory for him, he wins. The same is true for Romney.
Slight improvement for Romney

 

Pennsylvania
======

 Most recent poll: November 3
Number of polls within one week of most recent: 4
Obama leads: 3
Tie: 1
Romney leads: 0
Confidence of victory: 83.1% Obama
Improvement for Romney

Wisconsin
============

 Most recent poll: November 3
Number of polls within one week of most recent: 9
Obama leads: 8
Tie: 1
Romney leads: 0
Confidence of victory: 85.7% for Obama
Improvement for Romney

Nevada
======

 Most recent poll: October 31
Number of polls within one week of most recent: 7
Obama leads: 7
Tie: 0
Romney leads: 0
Confidence of victory: 89.1% Obama

Michigan
============
 

Most recent poll: November 3
Number of polls within one week of most recent: 5
Obama leads: 4
Tie: 0
Romney leads: 1
Confidence of victory: 93.0% for Obama 



The numbers improved slightly for Romney and the median result is now 303 electoral votes for Obama and 235 for Romney.  The median result is now the most likely result as well, which will happen if Romney wins all the states in which he is favored and the toss-up state Florida as well.

The numbers can't change all that much and that will very likely be the final answer I go with as my prediction when the last polls come in this Monday evening.



The Confidence of Victory number is now at 93.5% for an Obama win and 6.5% for a win for Romney. In comparison, Nate Silver has the number at 86% for Obama.

As someone who played a lot of backgammon back in the day, I know that 93.5% is not a lock. I played plenty of games where the final roll of a game came down to double sixes or double fives to win and the person who needed to get lucky did so.  The favorite in that position had 94.4% Confidence of Victory.

One last electoral college update Monday night.

Stay tuned. Go out and vote if you haven't done so already.

Sunday, November 4, 2012

Saturday Numbers, 11/3, the final countdown:
The electoral college

It's just a couple more days, I promise.

Then again, about a week from now after all the post-mortem is done, I'm going to need to find another hobby.

The state in red is favored to be won by Romney.
The states in blue are favored to be won by Obama.
The state with the gold background is the Golden Spike. If either candidate wins that state and all the states that the system says are easier for the candidate to win, that candidate becomes president.


North Carolina
============

 Most recent poll: October 31
Number of polls within one week of most recent: 7
Obama leads: 2
Tie: 1
Romney leads: 4
Confidence of victory: 58.3% Romney
No change from yesterday


Florida
======
 

Most recent poll: November 3
Number of polls within one week of most recent: 12
Obama leads: 6
Tie: 2
Romney leads: 4
Confidence of victory: 52.1% for Obama
Slight improvement for Romney from 52.3%

Colorado
=======

 Most recent poll: November 3
Number of polls within one week of most recent: 9
Obama leads: 7
Tie: 0
Romney leads: 2
Confidence of victory: 64.0% Obama
Slight improvement for Obama

New Hampshire
=============

 Most recent poll: Nov. 1
Number of polls within one week of most recent: 6
Obama leads: 5
Tie: 1
Romney leads: 0
Confidence of victory: 73.5%
Slight improvement for Romney

Virginia
============
 

Most recent poll: November 2
Number of polls within one week of most recent: 8
Obama leads: 5
Tie: 1
Romney leads: 2
Confidence of victory: 74.8% for Obama

Iowa
====
 

Most recent poll: November 2
Number of polls within one week of most recent: 10
Obama leads: 9
Tie: 0
Romney leads: 1
Confidence of victory: 79.2%
4% move for Obama

Ohio
====
 

Most recent poll: Nov 2
Number of polls within one week of most recent:13
Obama leads: 11
Tie: 1
Romney leads: 1
Confidence of victory: 80.5% Obama
Big gain for Obama
Note: This is currently the state with The Golden Spike. If Obama wins here and every state that has a better Confidence of Victory for him, he wins. The same is true for Romney.




Nevada
======

 Most recent poll: October 31
Number of polls within one week of most recent: 7
Obama leads: 7
Tie: 0
Romney leads: 0
Confidence of victory: 89.1% Obama

 

Pennsylvania
======

 Most recent poll: November 3
Number of polls within one week of most recent: 2
Obama leads: 2
Tie: 0
Romney leads: 0
Confidence of victory: 91.4% Obama

Michigan
============

 Most recent poll: November 3
Number of polls within one week of most recent: 4
Obama leads: 4
Tie: 0
Romney leads: 0
Confidence of victory: 93.7% for Obama
 


Before any votes are cast, both candidates need Ohio.  As the east coast polls close and states are declared for the candidates, the Golden Spike can move. Romney can win without some of the states on the list before Ohio because he has a 16 vote cushion if he takes them all. But if he loses more than 16 votes in the Eastern time zone, and the most likely way to do that right now is Obama winning New Hampshire and Virginia, then the Golden Spike moves into territory even more solid for Obama than Ohio currently is.

Here are the earliest things that can happen on Tuesday night that can make Obama supporters breathe easier.

Least likely: Obama wins North Carolina.
Next least likely: Obama wins Florida. He has a very slim lead by my system's reckoning and it is not hyperbole to say Florida has a history of very shaky election results.
Most likely: Obama wins both New Hampshire and Virginia.  The odds of winning both together are just over 50%, very close to the odds of Obama winning Florida.  If I had money to put on any of these things, I'd bet the NH-VA daily double before I'd bet Florida for Obama.


In the spirit of being fair and balanced, here is the earliest thing that can happen on Tuesday night that can make Romney supporters breathe easier.

Romney wins Pennsylvania. It's not very likely and it doesn't guarantee victory in the electoral college, but it is within the realm of possibility, so as a mathematician I feel compelled to bring it up.

Now here are the pictures.  If you support the Democrats, you might well call them The Pretty Pictures.

You might well call them that. I couldn't possibly comment.

Gratuitous Francis Urquhart reference aside, my comment is not to be complacent. Vote and get out the vote. It's not over yet.

 The Confidence of Victory number edges up 95% for the first time since October 7.

I want to be clear as a person who used to gamble as a hobby that I have lost poker games and backgammon games where I was much better than a 95% favorite.  Then again, if someone gave the opportunity to be a 95% favorite or a 5% underdog, I would take the former position in a heartbeat, saying please and thank you like my momma taught me.


The median result climbs to 306 electoral votes. Here are the most likely current results.

303-235 Obama: 3.7% chance of this.  Easiest way for this to happen is Romney winning Florida and all the states where he is currently favored and Obama picking up all the rest.

332-208 Obama: 3.2% chance. Obama wins in Florida and all the other states where he is favored. 

My next update late Sunday night will include the Senate numbers.  I get e-mails from the Democrats saying they could easily lose the Senate. My system currently and emphatically says that's not likely. As for the House, I have no numbers I consider reliable enough on which to base a prediction. From what I've seen, the Democrats will likely pick up a few seats but taking back control would take a miracle.

Stay tuned.








Saturday, November 3, 2012

Friday numbers, 11/2, the final countdown:
The electoral college

My usual caveats about not celebrating early still apply, but with all the early voting in states across the country, it's not all that early anymore.  Here are the states that are most closely contested according to recent polls.

Red means leaning toward Romney. 
Blue means leaning toward Obama.
Gold background indicates the state that both sides need to cross the 270 vote threshold.

North Carolina
============

Most recent poll: October 31
Number of polls within one week of most recent: 7
Obama leads: 2
Tie: 1
Romney leads: 4
Confidence of victory: 58.3% Romney
No change from yesterday

 

Florida
======
 

Most recent poll: November 1
Number of polls within one week of most recent: 12
Obama leads: 6
Tie: 1
Romney leads: 5
Confidence of victory: 52.3% for Obama
Slight improvement for Romney

 

Colorado
=======

Most recent poll: November 1
Number of polls within one week of most recent: 10
Obama leads: 8
Tie: 0
Romney leads: 2
Confidence of victory: 62.6% Obama
No change from yesterday


New Hampshire
=============

Most recent poll: November 1
Number of polls within one week of most recent: 5
Obama leads: 4
Tie: 0
Romney leads: 0
Confidence of victory: 74.2%
No change from yesterday


Virginia
============

Most recent poll: November 2
Number of polls within one week of most recent: 9
Obama leads: 5
Tie: 1
Romney leads: 1
Confidence of victory: 74.8% for Obama
Big improvement for Obama

 

Iowa
====

Most recent poll: November 1
Number of polls within one week of most recent: 8
Obama leads: 7
Tie: 0
Romney leads: 1
Confidence of victory: 75.9%
An improvement of 6% for Romney


Ohio
====

Most recent poll: November 1
Number of polls within one week of most recent:15
Obama leads: 13
Tie: 1
Romney leads: 1
Confidence of victory: 76.1% Obama
Slight improvement from 74.9 yesterday
Note: This is currently the state with The Golden Spike. If Obama wins here and every state that has a better Confidence of Victory for him, he wins. The same is true for Romney.
 


The big change today is Iowa moving from the approximately 80%-20% range to the 75%-25% range and into the states easier for Romney than Ohio. What this means is if Romney sweeps all seven races, he will have 285 electoral votes. This is a lot of leeway, so he doesn't have to sweep all of these to win. Given that the House is likely to stay in Republican hands, a 269-269 tie will go Republican. 


The median result ticks down slightly to 304-234 in favor of Obama. The most likely result is at about 303-235.

As the polls close across the country, the first battleground states appear to be Florida, North Carolina, Virginia and New Hampshire. For Romney, losing Florida would be a devastating blow.  The other trouble combinations would be losing any two of the other three.  Should any of those combos take place, Ohio alone won't be enough.

I'll be live tweeting the results Tuesday night, which means it will show up on my Facebook page as well.


As can happen, the median result ticked down but the Confidence of Victory ticked up, from 93.7% to 94.2% for Obama. It is absolutely better to be ahead than behind at this late date, but it is not a mortal lock. By this time in 2008, Obama's CoV was over 99.9% and STILL I told people not to celebrate early. I've played enough Texas Hold 'Em to have lost on the last turn on the card multiple times, even when I had 43 good cards to my opponent's 1.  

It's not over and you can still make a difference. Make sure you vote and if you have the time, do what you can to get out the vote. If you have a phone, you can help get out the vote in the battleground states even if you live in a solid red or blue state.

The next update on Saturday evening. Stay tuned.


Thursday, November 1, 2012

Thursday numbers, 11/1, the final countdown:
The electoral college

Another day goes by and I am surprised at the level of change today. 23 state polls concerning the electoral college are published, which is a high number but not unprecendented. The big change happens because of older polls timing out and most of the change is to Mitt Romney's disadvantage.  Here is the story of the six states currently in the middle. Mitt needs to win all of North Carolina, Florida, Virginia and Ohio, and at least one of New Hampshire and Colorado. If he fails this, there are some states currently very blue (Confidence of Victory numbers over 80% Democratic but under 90%) that could make up a deficit, but those are serious long shots.

Let's look at the six.  Red means the state leans Republican, blue means it leans Democratic and a gold background means this is the state with the Golden Spike.  If a candidate wins the Golden Spike state and every state that my system considers to be easier to win for that candidate, he will become president.

As of today, there are no toss-up states of consequence.  The single electoral vote in Omaha is a coin flip, but that is largely due to how little data I have been able to find concerning it.

North Carolina
============

Most recent poll: October 31
Number of polls within one week of most recent: 7
Obama leads: 2
Tie: 1
Romney leads: 4
Confidence of victory: 58.3% Romney
 

North Carolina gets much worse for Romney than it was yesterday. In the last two elections, all the states that went the opposite way from expected were in the 60%-40% range or closer.



Florida
======

Most recent poll: November 1
Number of polls within one week of most recent: 11
Obama leads: 6
Tie: 1
Romney leads: 4
Confidence of victory: 54.6% for Obama
 

Polls added and good polls for Romney timed out. Vital for Romney and moving the wrong direction. His winning this state by double digits - a big mouth claim of some Romney spokesman yesterday - would mean a result completely out of line with every poll taken since April. That's more than 70 polls.


Colorado
=======
Most recent poll: October 31
Number of polls within one week of most recent: 8
Obama leads: 7
Tie: 0
Romney leads: 1
Confidence of victory: 62.6% Obama


The big move in Romney's favor. It didn't make a huge dent in terms of odds of victory because Romney could lose this and still win if he runs the table of NC, FL, VA, NH and OH.

Virginia

 ============Most recent poll: November 1
Number of polls within one week of most recent: 8
Obama leads: 4
Tie: 2
Romney leads: 2
Confidence of victory: 64.6% for Obama


Old polls favoring Romney time out and Virginia trends big for Obama. Rasmussen will definitely have a say before Election Day, but who else will speak well for the Mittster?

New Hampshire


=============
Most recent poll: October 29
Number of polls within one week of most recent: 5
Obama leads: 4
Tie: 0
Romney leads: 1
Confidence of victory: 74.2%
 

Note: A new poll added and old poll goes away.  Not completely crucial for Romney, like Colorado, but losing both is probably fatal.

Ohio
====

Most recent poll: October 29
Number of polls within one week of most recent:14
Obama leads: 12
Tie: 0
Romney leads: 2
Confidence of victory: 74.9% Obama

Note: This is currently the state with The Golden Spike. If Obama wins here and every state that has a better Confidence of Victory for him, he wins. The same is true for Romney.



And now the aggregate numbers. The median electoral college result now stands at 305-233 for Obama. The most likely scenarios come is pairs because of the toss-up nature of Omaha right now, with a spike of 4.7% chance of 303 or 304 and another spike of 4.4% at 319 or 320.

The 305 median ties the highest value enjoyed by Obama since October 7 when I switched from weekly reporting to day by day updates.

 The Confidence of Victory number has risen to 93.7%, slightly lower than the numbers Obama was at on October 7 and 8 when his numbers first began to deteriorate.

As always, I insist on saying that these numbers would be valid only if the polling was done on the day the election was held, but in many ways, this close to Tuesday the election IS being held.  I turned in my mail-in ballot directly to the Registrar of Voters this week. Many people have taken advantage of early voting all over the country. Time is definitely running out.



There are counter-indicative data, notably some daily tracking polls that still give Romney a lead, but in no way are stat scrunchers like myself and Nate Silver are just voices in the wilderness as some of the numerically challenged members of the pundit class would have people believe. The gambling site Intrade has never considered Romney a favorite, and after a positive run up to being about a 45% underdog as of Nov. 24, the gambling community has given Mitt a strong vote of no confidence and his odds have sunk to about 33% chance to win.

One thing everyone can agree on is that it won't be over until Tuesday night at the very earliest.

More numbers tomorrow. Stay tuned.