Tracking Poll Mechanisms

Kevin Drum tamps down teacking poll enthusiasm, and gives me a great springboard from which to tear them to bits:

As for the world tour, it’s gotten him a ton of coverage during a month that’s usually light on people paying attention to politics, and the coverage has mostly been very positive. That’s a good thing. It may not have given him an instantaneous bump, but honestly, we should all calm down over stuff like this. Not everything shows up in tracking polls within 24 hours.

Actually, nothing shows up in tracking polls within 24 hours, because tracking polls don’t really track day to day numbers. You’ve got to get into the fine print to catch it, but if you notice, somewhere in Gallup’s tracking poll there’s always the line that “data is based on 3 day rolling averages,” which is sort of confusing to layman and generally gets ignored in favor of the, seemingly, more straightforward “daily tracking poll” header. The problem is that “daily tracking polls” don’t do what we would logically think they would do, and this structural problem, and general misunderstanding, is what contributes mostly to the widespread confusion around why they look differently than other polls.

The way tracking polls work is fairly straight forward. The “daily numbers” are really 3 days worth of data. So if it’s Thursday, we’re looking at the data from Monday, Tuesday, and Wednesday. On Friday, the data from Thursday is added, and the data from Monday is dropped, with Tuesday and Wednesday being held as a control (because taking totally different samples every day would lead to really big sampling error). So when you look at the Friday “daily” numbers that cable news thinks is the difference between Thursday and Friday, you’re really looking at the difference between Monday and Thursday. But because a full 2/3 of the data is a control, the very design of the polling mechanism lends itself to creating very little movement in the data, and to creating tighter numbers than bi-weekly or monthly snapshots will.

In other words, tracking polls differ from, say, the monthly Q polls because their methodology is fundamentally difference, and tracking polls have a systematic bias towards small margins and little movement. Which isn’t necessarily their fault, it’s simply an extension of the fact that it’s really impossible to create daily polling. But the big polling organizations want to get their name out there through reporting, and cable news needs a horserace to fill time. So pollsters put out the tracking number because that’s what the media wants to report, even though I suspect they recognize that it’s a fraudulent number.

I’ve never encountered a campaign who gave a mind to tracking polls, and certainly have never seen one that did tracking polls for their internals. The only people who care about them are talking heads who need material.