Using Polls

Matt doesn’t like polling:

I think the reason is that referring to polling data, even bad data, is a good CYA mechanism when you need to make difficult decisions. A consultant who says “we don’t have any valid data on this question, but I think you should do X” is going to get blamed if X doesn’t turn out right. But if he can point to some data, and say that he’s not making the recommendation, he’s just pointing to the numbers then if things go south it isn’t really his fault.

This is a pretty common organizational flaw. The natural tendency is to try to maximize whatever it is that you have a good measurement of, even if the measured quantity is only questionably related to what you’re trying to do. Politicians know how to get an issue poll in the field, and there aren’t great metrics for getting the information you would really want. So campaigns often go to war with the data they have, even while knowing that the data’s no good.

He’s remarking on the thesis behind The Opinion Makers, to wit, that our polling data isn’t very good because pollsters don’t differentiate between rock solid opinion and people who don’t pay much attention and/or particularly care about the topic. The problem here, at least with Matt’s analysis, is that it confuses the polls you see at RCP or on CNN with the polls that campaigns use. Why exactly do you think campaigns commit gobs of resources to doing their own polling? It’s precisely because they don’t necessarily trust the media driven data, and because they want to control for as many variable as they can in the data. So, for what it’s worth, the real lesson here is to take media polling with a grain of salt, unless you can digest the totality of the data yourself.