Portfolios and Accountability

Comments on Portfolios and Accountability by Richard G. Innes
Education Analyst
Bluegrass Institute for Public Policy Solutions

August 27, 2007


Portfolios can have a place in school, but they do not work well in accountability programs. Interest has been shown by some to put portfolios into NCLB accountability. This would not be a good decision.

As a Kentucky resident, I have had a front row seat to one of the most extensive statewide writing portfolio programs ever since I got concerned about education in 1994.

The program here has remained fuzzy and contentious throughout. In general, I don't think the inter-rater reliability is good enough to support a high stakes accountability system, and plenty of others here would agree. In fact, portfolios are under regular attack in the Kentucky legislature, and teachers have often been involved on the "let's get rid of them" side of the discussion.

In Kentucky the writing portfolio program actually hampers the teaching of writing because any piece a child creates can be selected later as a portfolio item. As a result, teachers are constrained on the types of correction comments they can make on student papers. The constraints severely impact the effective teaching of punctuation and grammar. Spelling, of course, has not been a key requirement in Kentucky ever since our fad-laden reform was enacted in 1990.

Kentucky's writing portfolio audits have always shown some inflation in teacher-awarded portfolio grades, thought there has been some improvement in recent years. Perhaps more compelling, the scores received for writing portfolios are very, very different from the scores received for on-demand writing in the Kentucky assessment program. For example, in the 2006 Kentucky Performance Report for the entire state it shows elementary writing portfolio proficiency at 57.53 percent. The elementary level on-demand writing proficiency (which is graded by independent scorers at a commercial testing organization) was just 5.31 percent, less than one tenth of the proficiency rate claimed in the portfolios! All for the same group of fourth graders.

For middle schools, the proficiency rates were 30.53 percent and just 14.62 percent, so portfolio proficiency rates were more than double the on-demand writing performance. This was for the same group of seventh graders.

For high schools, the proficiency rates were 36.58 percent and just 13.9 percent, so portfolio proficiency rates were almost three times the on-demand writing performance. In high schools, both assessments are given to 12th graders.

Believe it or not Kentucky at one time also had a separate mathematics portfolio. This one was so egregious that it was thrown out after four years of a very unsuccessful experiment. Writing about math does not help one to do it.

Anyway, it is clear that powerful interests aligned against any real accountability for schools certainly support these fad ideas for NCLB. These fad assessments have worked very well in Kentucky, creating an illusion of tremendous progress. Sadly, that illusion isn't too hard to dispute using other external measures like the NAEP and the ACT, and even using other data from Kentucky's assessments such as the on-demand writing results.