Wednesday, August 23, 2006

Public Schools "Success" Study: Not Necessarily So

When a study released by the Federal Dep't of Education indicated that public school students were doing just as well as private school students, eyebrows were raised.

They should have been. It ain't really so.

Test scores from the 2003 National Assessment of Educational Progress (NAEP) indicated that public-school students enjoyed a 4.5% advantage over private school students in 4th-grade math and were competitive with them in 4th-grade reading and 8th-grade math. They were, though, at a decided 7.3% disadvantage in 8th-grade reading.

These results, mixed though they were, prompted Reg Weaver, the head of the nation’s largest teachers’ union, the National Education Association, to boast that “the results were nothing more than we expected” and that public schools were “doing an outstanding job.” The Palm Beach Post weighed in with a mean-spirited editorial titled “Bush-suppressed study dispels voucher myth.”

Weaver may want to re-cork his champagne. In a meticulous 56-page critique, distinguished education expert Paul Peterson of Harvard lambasted the government study for its flawed methodology. The study, he found, fell short of routine academic standards by improperly boosting the performance of public-school students relative to their private-school peers.

The Department of Education researchers “adjusted” the raw NAEP data to account for differences in the socioeconomic status between the two groups. They did this, however, by classifying students according to whether they received federal aid such as subsidized school lunches or Title I assistance, rather than by a more even-handed approach.

Public schools receive federal aid on a “school-wide” basis (that is, the assistance flows to the entire school based on the percentage of poor students enrolled) but reaches private-school students on an individual basis only. Once a public school qualifies, therefore, every student at the school, regardless of poverty level, is technically counted as a recipient of those services.

In other words, the Feds adjusted "the curve" to pump up public schools' grades.

Peterson substituted better criteria, such as the parent’s level of education and whether another language is spoken in the home, and found that “when student characteristics are estimated consistently across school sectors, a private-school advantage relative to public schools is evident at all grade levels in both math and reading in all estimations but one.”

S'pose Folkbum will take back all that stuff he said about P-Mac?


Jay Bullock said...

I won't take back anything.

The feds held on to the study for as long as they did not because they were trying to make public schools look better, but because they were embarassed by the results and wanted to bury the fact that private schools are not the panacea the right wing believes they are.

Peterson, by the way, is, while not necessarily a hack, a man with a clear pro-voucher agenda who has ethical issues in his past research. He's responsible for training actual hacks like Jay Greene, who will distort any statistic to fit his pro-voucher agenda.

In other words, I wouldn't trust Peterson if he told me it was raining and I was soaking wet.

Dad29 said...

Ummmnnn...Jay, the question was rhetorical...

Seems that you have no fact-based comments about the study's methodology.

Or are you holding those in reserve?

Jay Bullock said...

Part of a pattern. Peterson takes the peer-reviewed work of others and twists the data so that it makes vouchers or private schools look better. The subsequent research then either fails peer review or goes straight to the popular press and conservative outlets.

He did it with a study of Milwaukee, of Cleveland, of DC, of Dayton, and on and on until now.

Peterson is full of crap on this issue. Period.

Dad29 said...

Jay, Peterson's complaint is that DoE "twisted" the data.

I'm well aware that his work is controversial; but data is data. Either 2+2=4, or it doesn't.

As a reminder, I also agree with the proposition that PARENTS have extraordinary influence on the success or failure of children in schools--more than teachers except in the 'outlier' cases.

So if Peterson merely points out that DoE's data is poorly generated by "curving" the 'poverty level' inaccurately, how is that "twisting?"