Strange things afoot at the PDK

For the last 38 years, Phi Delta Kappa and The Gallup Organization have conducted a national survey on the public’s attitudes toward public schools. (Go here for complete results of this year’s survey.) It’s an influential survey, put out by two credible entities, and one would be inclined to take its results at face value. But it seems as if there are some shenanigans happening behind the scenes that would severely compromise the integrity of this work.

Given the inroads made in recent years by voucher initiatives (programs in Milwaukee, Cleveland, and DC for example), it would seem as if the idea of school choice is gaining political and public acceptance. Yet, according to PDK/Gallup:

Since 1991, the PDK/Gallup polls have approached this issue with a question that measures approval of the voucher concept — “allowing parents and students to choose a private school to attend at public expense” — without using the politically charged word “vouchers.” Table 5 provides this year’s results. In considering the results, it is useful to keep in mind that choice, independent of a specific program, is popular with the public.

Findings. The percentage favoring vouchers dropped from 38% a year ago to 36% this year, while opposition grew from 57% to 60%. Support for vouchers started at 24% in 1993, fluctuated up and down for years, and peaked at 46% in 2002. It is now at the mid-Nineties level.

Sounds bad for the voucher folks, right? It would – if this were the whole story.

PDK/Gallup is technically correct when they say that they’ve been asking about the voucher concept without using the word “vouchers” since 1991. What they don’t say is that they’ve been asking about vouchers since 1970; the difference is that they actually used the word “vouchers” in previous years.

According to Andrew Coulson of The Cato Institute, on the Cato blog:

PDK actually started asking the American public about vouchers back in 1970, with a rather more informative question:

In some nations, the government allots a certain amount of money for each child for his or her education. The parents can then send the child to any public, parochial, or private school they choose. This is called the “voucher system.” Would you like to see such an idea adopted in this country?

Response to this question was initially somewhat unfavorable, but those answering favorably began outnumbering those opposed in 1981, and that pattern was never reversed. The last time PDK ever asked this question, in 1991, 50 percent of respondents were in favor while only 39 percent were opposed.

What Cato did not mention, but also can be found in the 1991 PDK report, is that “The voucher plan finds its strongest support among non-whites and blacks (57% in both groups), inner-city dwellers (57%), people with children under 18 (58%), and nonpublic school parents (66%).”

If you want to politicize this, you can argue that The Cato Institute has an agenda of its own – but they’re simply presenting facts here, as evidenced by past editions of PDK/Gallup reports (archived here).

It gets even stickier. Sensing that the new question (“Do you favor or oppose allowing students and parents to choose a private school to attend at public expense?”) promoted a bias against vouchers, The Milton and Rose Friedman Foundation commissioned their own survey last year, using Harris Interactive, another well-known polling firm. In this survey, they contacted 1,000 adults, asked 500 of them the PDK question verbatim, and asked the other 500 a slightly modified version.

The results, with changes in wording underlined:

PDK Question: “Do you favor or oppose allowing students and parents to choose a private school to attend at public expense?”

• 37% favor
• 55% oppose
• 5% neither favor or oppose
• 2% don’t know/refused
  

Adjusted Question: “Do you favor or oppose allowing students and parents to choose any school, public or private, to attend using public funds?”

• 60% favor
• 33% oppose
• 5% neither favor nor oppose
• 2% don’t know/refused

That’s right – a 23-point jump in support, based on a slight rewording of the question. And while the Friedman Foundation is wholly pro-voucher, I’ve got to consider their question the fairer of the two.

It seems clear to me that PDK’s work, at least on this issue, has been compromised by their agenda (they describe themselves as “a dedicated advocate for the public schools” on their site). As a result, it’s hard to take any of their findings very seriously – just one more example of education research that can’t be taken at face value.