Ziegler vs Silver

John Ziegler, a conservative talk show host, has started a website called HowObamaGotElected.com, a project which includes a documentary film and now two national polls that have gotten a lot of media attention.

Backstory: Ziegler believes that favorable treatment by the media was directly responsible for Obama's electoral success. He argues that Obama voters based their choice on inadequate coverage of Obama's weaknesses, and he has set out to publicize his opinions in the national media.

His first venture came in the form of a YouTube video that showed Obama voters incorrectly answering political questions, such as "Which party currently controls Congress?" Ziegler then commissioned Zogby International to conduct a national poll of Obama voters, asking them the twelve questions from the video.

Nate Silver of FiveThirtyEight.com (as well as BaseballProspectus.com) sharply critisized the Zogby poll, calling it a push poll. Ziegler naturally responded angrily, and Silver then interviewed Ziegler. You can read most of the transcript here.

Since then, Zogby defended the poll, but then stated publicly that he would not have approved the poll in the form it took, and that he was on a book tour when Zogby International conducted the poll.

Recent News: Ziegler then asked Zogby for another poll that would include McCain voters, but was refused. Ziegler then commissioned the poll from Wilson Research Strategies. The Wilson poll asks the same questions as the first poll, with a few wording changes (I'll get into that in bit).

As a follow-up to this controversy, Silver and Zogby recently went head to head on the B Cast, which I guess is some sort of online show. I couldn't figure out if it had political leanings.

My Take: Whatever the dictionary definition or commonly understood meaning of the phrase "push poll", this poll is hideously biased AND Ziegler's conclusions are not even supported by the data it does provide. For the purposes of talking about the questions, I'm mostly going to talk about the Wilson poll, because I have an actual copy of the survey.

1. Twelve total questions, all negative statements about McCain, Palin, Obama or Biden. Seven answers are Obama/Biden, while five are McCain/Palin. (Sidenote: One of the Palin questions isn't actually something she did!) Of those five, only two are about McCain, which is the same as the number of questions about Biden. Buried on Ziegler's website's presentation of the poll numbers is the following quote: "...in general, the voters did universally worse on questions where the negative information was about their candidate."

2. The statements about Palin and McCain well-publicized facts, while the statements about Obama are frequently misleading or disputed. Palin definitely has a pregnant daughter, no doubt about it. But did Obama actually say BOTH that his policies "would bankrupt any company that built a coal burning power plant" AND that his policies "would make electricity prices skyrocket"? Because I have not found any quote that supports the second statement. When Ziegler asks a well-publicized and straightforward question about Obama, such as "Which candidate said the government should redistribute wealth?", no problem, everyone gets it right. Are we really shocked that most Obama voters don't remember that Obama once accidently said he'd visited 57 states instead of 47?

3. In the B Cast interview, Silver and Ziegler argue about changing the wording of the questions about Bill Ayers. The original poll asked which candidate started his political career at the home of two former members of the Weather Underground, while the second poll asked, "Which candidate reportedly held their first political fundraiser at the home of Bill Ayers and his wife, two leaders of the 1960’s terrorist organization the Weather Underground?" More Obama voters answered the question from the second poll correctly the questions from the first. Silver argues that the second question is more accurate, resulting in more correct answers. Ziegler makes the following statement on his website: "We added Ayers name... and it significantly increased the rate of correct response, indicating a very superficial grasp of the overall story." There is absolutely no way for anyone to draw statistical conclusions about WHY respondents did better in the second poll, since there are multiple variables changed.

4. How do we know this is even tied to media coverage? All this poll shows is that McCain voters watch Fox News and answer questions one way, while Obama voters watch CNN and answer questions differently. There is very little attempt to tie the answers to the questions directly to the news media.

Last Thoughts: This poll completely fails any test of academic rigor. Is this a joke?