http://www.kai-arzheimer.com/blog/2009/08/31/weighting-survey-data-not-necessarily-a-brilliant-idea/ A political science blog Wed, 19 May 2010 20:31:00 +0000 hourly 1 http://wordpress.org/?v=3.0 http://www.kai-arzheimer.com/blog/2009/08/31/weighting-survey-data-not-necessarily-a-brilliant-idea/comment-page-1/#comment-146 kai Wed, 02 Sep 2009 07:32:10 +0000 http://www.kai-arzheimer.com/blog/?p=307#comment-146 You are obviously right about the interaction though I feel that if there is an interaction in reality, it should be reflected in the model. The trouble is of course that you have to know it is there in the first place. Equivalently, you have to know which variables you would like to use in your weighting procedure. In an ideal world, we would do all our research with people who are absolutely randomly sampled and then randomly assigned to perfectly valid experiments. In lieu of that, we have to do with kludge of different sorts. :-( You are obviously right about the interaction though I feel that if there is an interaction in reality, it should be reflected in the model. The trouble is of course that you have to know it is there in the first place. Equivalently, you have to know which variables you would like to use in your weighting procedure. In an ideal world, we would do all our research with people who are absolutely randomly sampled and then randomly assigned to perfectly valid experiments. In lieu of that, we have to do with kludge of different sorts.

]]> http://www.kai-arzheimer.com/blog/2009/08/31/weighting-survey-data-not-necessarily-a-brilliant-idea/comment-page-1/#comment-145 gabriel Mon, 31 Aug 2009 17:18:06 +0000 http://www.kai-arzheimer.com/blog/?p=307#comment-145 My instincts are also not to weight, and I agree with your argument <i>if all effects are additive</i>, but what about if you're worried about omitted variable bias not for main effects, but interaction effects? For instance, for the sake of argument, let's assume that blacks get different income returns to education than whites (ie, there's a race*edu interaction) and that your data have an oversample of blacks such that they are half the sample. If you control but do not weight for race you're only controlling for the possibly different intercepts by race but not the interaction with education. You'll thus have an estimate of the grand slope that is the mean of the black slope and white slope, when in reality it should be more similar to the white slope because in the population whites are more numerous. On the other hand weighting should produce the correct grand slope. Maybe you should just specify the interaction effect, but a) interactions are a huge pain to interpret and b) it may not occur to you that a specific interaction has appreciable effects. My instincts are also not to weight, and I agree with your argument if all effects are additive, but what about if you're worried about omitted variable bias not for main effects, but interaction effects?
For instance, for the sake of argument, let's assume that blacks get different income returns to education than whites (ie, there's a race*edu interaction) and that your data have an oversample of blacks such that they are half the sample. If you control but do not weight for race you're only controlling for the possibly different intercepts by race but not the interaction with education. You'll thus have an estimate of the grand slope that is the mean of the black slope and white slope, when in reality it should be more similar to the white slope because in the population whites are more numerous. On the other hand weighting should produce the correct grand slope. Maybe you should just specify the interaction effect, but a) interactions are a huge pain to interpret and b) it may not occur to you that a specific interaction has appreciable effects.

]]>