Example: Assessing Bias From the Margins of an American Pre-Election Survey with surveybiasi

 
. surveybiasi , popvalues(47.6 48.8 3.6) samplevalues(46 48 5) n(563)
------------------------------------------------------------------------------
      catvar |      Coef.   Std. Err.      z    P>|z|     [95% Conf. Interval]
-------------+----------------------------------------------------------------
A'           |
      1 |  -.0455767   .0845014    -0.54   0.590    -.2111965    .1200431
      2 |  -.0126154   .0843287    -0.15   0.881    -.1778965    .1526658
      3 |   .3537155   .1924563     1.84   0.066    -.0234919    .7309229
-------------+----------------------------------------------------------------
B            |
      B |   .1373025          .        .       .            .           .
    B_w |   .0405846          .        .       .            .           .
------------------------------------------------------------------------------

    Ho: no bias
    Degrees of freedom: 2
    Chi-square (Pearson) = 3.4542892
    Pr (Pearson) = .17779136
    Chi-square (LR) = 3.0856308
    Pr (LR) = .21377838

A week before the 2012 election for the US House of Representatives, 563 likely voters were polled for CBS/The New York Times. 46 per cent said they would vote for the Republican candidate in their district, 48 per cent said they would vote for the Democratic candidate. Three per cent said it would depend, and another two per cent said they were unsure, or refused to answer the question. In the example these five per cent are treated as ‘other’. Due to rounding error, the numbers do not exactly add up to 100, but surveybiasi takes care of the necessary rescaling.

In the actual election, the Republicans won 47.6 and the Democrats 48.8 per cent of the popular vote, with the rest going to third-party candidates. Given the small sample size and the close match between survey and electoral counts, it is not surprising that there is no evidence for statistically or substantively significant bias in this poll.

Agree? Disagree? Leave a reply (also works with Facebook, G+, Twitter...)