Education Week: Why the GAO Should Play Powerball

Education Week

December 20, 2010
By Rick Hess

On Friday, Andrew Kelly and I penned an Inside Higher Ed column that called out the Government Accountability Office (GAO) for the fact that its influential and scathing report on for-profit colleges turns out to be riddled with errors. Even worse, the GAO did its damndest to keep it under the radar--quietly posting an errata online on November 30, without so much as a press release to document the changes. Senator Mike Enzi expressed concerns in a December 7 letter to the GAO, and then it took enterprising journalists like theWashington Post's Nick Anderson to bring the issue to the public's notice. And then, when questioned, the GAO denied that the errors mattered and refused to alter any of its conclusions.

In our piece, Andrew and I observed: "The authors of the Government Accountability Office's for-profit secret shopper investigation pulled off a statistically impressive feat in August. Let's set aside for the moment that on Nov. 30, the government watchdog quietly revealed that its influential testimony on for-profit colleges was riddled with errors, with 16 of the 28 findings requiring revisions. More interesting is the fact that all 16 of the errors run in the same direction -- casting for-profits in the worst possible light. The odds of all 16 pointing in the same direction by chance? A cool 1 in 65,536." This is all especially troubling given that the report was seized upon this summer by Senator Tom Harkin to attack for-profit lenders and by the Department of Education to justify its controversial "gainful employment" regulations.

Andrew and I noted, "The GAO [is] the $570 million-a-year organization responsible for ensuring that Congress gets clean audits, unbiased accounting, and avowedly objective policy analysis... Its report-vetting process entails GAO employees who are not involved with the project conducting a sentence-by-sentence review of the draft report, checking the factual foundation for each claim against the appropriate primary source."

Yet, the mistakes were manifold. The original report claimed that a financial aid officer purposely ignored an undercover applicant's supposed $250,000 in savings when calculating eligibility for financial aid, yet it turns out that the financial aid representative did so "upon request by applicant." This doesn't exonerate the financial aid officer, but it sure changes the equation. Another applicant was actually informed that he "could" take out the maximum in student loans, whereas the original report said that he was told he "should" do so. A for-profit official supposedly told an applicant that massage therapists could earn up to $100 an hour--turns out that the official really said that the applicant could expect to earn up to $30 an hour (below the Bureau of Labor Statistics' estimate of $34 for therapists in California).

Not surprisingly, our column roused the ire of those who have no use for for-profit colleges. In a particularly high profile response, Senator Harkin fired back in a December 20 Inside Higher Ed piece that we had been too hard on the nature of the revisions, suggesting that there were even a handful that went in the opposite direction. Without any illustrative examples, this claim does little to bolster one's faith in the edits. Especially disheartening is that Harkin never expressed disappointment or concern with the GAO's handiwork.

Instead, Harkin's apologia was echoed by Education Sector analyst Chad Adleman'senthusiastic defense of the GAO. Adleman dismissed many of the GAO corrections as "minor things like changing a 'should' to a 'could.'" (Seriously? It's small beer, after savaging a loan officer for telling potential students they "should" take loans, to find out she was answering a query about whether a student "could" take loans?)

Behind the scenes, Harkin's office responded to our column by circulating on Capitol Hill a three-page memo titled "Setting the Facts Straight - GAO and the For Profit Investigation." Tantalizingly, the memo lists five "new instances of deceptive or misleading conduct" from the GAO secret shopper tapes. My favorite nugget? "The admissions representative talks about what the applicant could expect with a college degree, asking her if she could imagine having a car and home of her own along with a job that she enjoyed." The sleazy horror... But wait a minute, haven't administration officials and higher education advocates also been touting college as a pathway to the middle class?! Those deceptive, misleading sonovaguns!

Most amusingly, both Harkin and Aldeman seemed to take great comfort in noting that the GAO revised 12 reports last year. They both fail to note that this is a dozen out of the 1,000-odd reports issued in that period, which, as Andrew and I noted on Friday, makes corrections "almost unheard of." O-kay. That 12 out of a thousand breaks down to one out of a hundred. So, let's recap. There's a 1-in-65,000 chance that the corrections would all point the same way, and then a 1-in-100 chance that the GAO has to correct a given report. The combined odds mean we'd expect this kind of situation once out of every 6.5 million reports. Coincidental that it happened just when a powerful Senate chair and like-minded administration are pushing so aggressively for controversial policies. If those eager to apologize for the GAO's troubling behavior want to stand on those odds, more power to them.

Me, I find those probabilities damning. Either there's a problem here, or the GAO staff ought to be playing a whole lot of Powerball. And, when it comes to transparency, the GAO has failed miserably by even the most rudimentary measure.