Wednesday, October 15, 2008

Projection post mortem

Over the past few weeks, I engaged in one of my favourite pastimes - predictions. To aid me, I developed a bit of an aid in the form a projection model. Here I try to evaluate how it did and what its strengths and weaknesses were.

First, its biggest flaw is that it is almost wholly dependant on outside factors. It is a poll-driven machine and, if the polls are not accurate, then it is not accurate.

Here is how the polling worked out vs. the actual result (both contrasted against the 2006 results):

 CPCLibBQNDPGrn
Poll AvgActualPoll AvgActualPoll AvgActualPoll AvgActualPoll AvgActual
Atl-9.2%-4.9%-2.0%-4.5%--+3.8%+3.4%+7.2%+3.7%
QC-4.4%-2.9%-0.1%+3.0%-0.8%-4.0%+3.9%+4.7%+1.8%-0.5%
ON-0.7%+4.1%-5.1%-6.1%--+1.3%-1.2%+4.9%+3.3%
Pra-0.6%+5.4%-6.4%-7.2%--+3.4%+0.1%+5.0%+2.7%
AB-7.1%-0.4%+3.5%-3.9%--+2.9%+1.0%+1.8%+2.3%
BC+1.5%+7.1%-5.6%-8.3%---2.7%-2.5%+7.5%+4.1%
Natl*-1.9%+1.3%-3.6%-4.0%--+1.6%+0.7%+4.7%+2.3%
*National numbers excluding the BQ were used to project the 3 northern seats

As you can see, the pollsters were a bit off (or at least my average of their polls was). No pollster got it exactly right, though Angus-Reid came very close in both national and regional numbers.

Using the actual regional results, my model gives the following:
CPC 145
Lib 77
BQ 48
NDP 35
Ind 3

Using only Angus-Reid's poll, my model gives the following:
CPC 141
Lib 76
BQ 49
NDP 39
Oth 3

So, I think I am relatively pleased with the model, even with the slightly off polling results, I seem to have done better than most prognosticators for a change.

Here is a list of the gross error for all the predictions listed on Calgary Grit's round-up and myself:
  1. Ekos Predictions (18)

  2. Calgary Grit's model (23.6)
  3. Barry Kay Seat Projections (24)
  4. Andrew Coyne (26)
  5. nbpolitico (28)
  6. UBC Stock Market (30)
  7. David Akin (30)

  8. Kady O'Malley (35)
  9. Democratic Space (36)
  10. Election Prediction Project (38)
  11. Scott Reid (40)
  12. Andrew Steele (44)
If I had had a crystal ball or a better gut, using the real results would have projected a gross error of 8, while using the best poll (Angus-Reid) would have given an error of 6 - obviously showing that the model needs a bit of work!!

But of course I am forgetting my blogging colleague NBT. Who needs fancy projection models when you've got him? His gross error: 22, good enough for second place! As the prominently displayed link on my other site says, a better prognosticator than I!

Projection history: Final :: Eighth :: Seventh :: Sixth :: Fifth :: Fourth :: Third :: Second :: First :: Methodology

7 comments:

nuna d. above said...

As you love the numbers, how many votes was Harper away from a majority? Vancouver South, Edmonton Strathcona and other close ridings could have put him over the top.

nbpolitico said...

That sounds like a fun challenge...

Censurer General said...

"If I had had a crystal ball or a better gut, using the real results would have projected a gross error"

Hey - even better would be having access to the hidden programming code used in the e-voting machines (1).

Then you could really give us some accurate 'polls' for next time.

nbpolitico said...

3628 votes taken from the right parties in the right ridings would have given the CPC 154 seats, enough for a majority with an opposition speaker. Another 746 votes (for a total of 4374) would have given them 155.

Here are the magic 12:

Esquimalt--Juan de Fuca
lost to the Liberals by 35

Brampton West
lost to the Liberals by 62

Edmonton--Strathcona
lost to the NDP by 222

Welland
lost to the NDP by 246

Western Arctic
lost to the NDP by 262

Brampton--Springdale
lost to the Liberals by 360

Vancouver South
lost to the Liberals by 390

Burnaby--Douglas
lost to the NDP by 400

Malpeque
lost to the Liberals by 463

Sault Ste. Marie
lost to the NDP by 571

Moncton--Riverview--Dieppe
lost to the Liberals by 617

New Westminster--Coquitlam
lost to the NDP by 746

nbpolitico said...

Sorry, I misspoke there. Those seats were lost by double - 1 of those totals.

For instance, the Liberals won Esquimalt--Juan de Fuca by 68 votes. If the CPC took 35 votes from the Liberals and moved them to their column, they would have won by 2 votes.

nuna d. above said...

I wonder if Esquimalt would have gone NDP or Con if Martin had stepped down.

Wilson said...

Thank you for your kind words and sharing your thoughts.

oil painting on canvas