Crowley Political Report

Crowley Political Report takes a closer look at this weekend's Times/Herald Republican presidential campaign poll for Columbia Journalism Review. The polls stories were written by Tampa Bay Times political editor Adam Smith and Miami Herald political writer Marc Caputo. Some excerpts from the CJR report:

Both reporters went to considerable length to describe the complexities of the campaign and the reasons for Romney’s apparent resurgence and Gingrich’s apparent slide. Where they failed, however, is in what they did not report.

Newspaper polling (and the coverage of such polling) has long been a pet peeve of mine. Smith, Caputo and others with whom I worked as a political reporter have been subjected to my railing about newspaper polls countless times.

In Caputo’s and Smith’s stories, there is a clear example of what is wrong with newspaper polls and the coverage thereof. Both reporters wrote that their poll showed that Romney has a 24-point lead over Gingrich among Hispanic voters—52 to 28 percent. But readers were not told important details about these numbers. I interviewed Brad Coker of Mason-Dixon Polling and Research who conducted the poll for the newspapers. He said the survey of 500 registered, likely voting, Republicans included just 75 Hispanics. Of those 75 Hispanics surveyed, the number consisted “heavily” of Cuban Americans who live in Miami-Dade county.

Now, the argument can be made that Miami-Dade Cuban Americans will be the overwhelming majority of Hispanic voters in the Florida primary, as Caputo explains in his story. But neither Caputo nor Smith tells readers that their Hispanic survey is, in fact, not a comprehensive look at Florida Hispanic voters. Coker told me that if he were doing a detailed survey of Hispanics he would have surveyed 400 not 75.

Once you realize that the survey sample is 75, it begs the next question: What is the margin of error? The Times and Herald reported that the margin of error for the 500 Republicans surveyed was 4.5 percent. True. But what they did not tell readers was the margin of error for the 75 Hispanic voters surveyed, which Coker told me was plus or minus 12 percent. That means that the percentage of Hispanics who support Romney ranges from 40 percent to 64 percent and the percentage supporting Gingrich ranges from 16 percent to 40 percent.

What does it really mean? That there is little statistical value in the number of Hispanics surveyed. This is a common problem when news organizations report subgroups in their polls. Often the numbers surveyed are too small to reveal any meaningful information.

. . . 

Another problem is that neither the Times nor the Herald released its complete polling results with questions and crosstabs (the Times posted poll questions online). While print news holes may be too small, there is no reason not to post all of this information on the newspaper websites. The timing of the release is also problematic. The poll was conducted during the period of January 24-26. The results were not released on the newspaper websites until very late on Saturday, January 28. The results then appeared in the Sunday paper, three days after the survey was completed. Pollsters will tell you there is simply too much volatility that close to an election to hold on to poll results for three days. Even voters surveyed on the 24th could have changed their opinion by the 25th.

. . .

News organizations are getting increasingly sloppy with reporting on—and addressing the shortcomings of—their own polls, not to mention asking tough questions of all the other polls that seem to pop up every day.

The National Council on Public Polls offers 20 questions journalists should ask about poll results. Let’s ask them.

You can read the entire Columbia Journalism Review report here.

Follow on Twitter @crowleyreport