Finding the Pony in the ACTAnalysis of 2006 ACT Results
An Illinois Loop Analysis
Kevin C. Killion
January 4, 2000
Where the "good" news came from
Here are the key "good" stories that the ISBE managed to extricate from the ACT:
Now, the REAL story
It turns out that the ACT scores are up nationally, not just in Illinois.
The new composite ACT score is 21.1, up from 20.9, an "increase" of 0.2. So, Illinois' "increase" by the same 0.2 (from 20.3 to 20.5) turns out to be merely keeping pace with the national score!
So, can Illinois at least claim that it's improving along with the rest of the nation?
Well, we'd like to think that the whole country is improving. But we have to wonder: has the country improved, or has the test gotten a little easier? A rising tide raises all boats. If you see boats going up, that tells you more about the tide than about boats.
I looked at results for the 50 states plus the District of Columbia. It turns out that 45 -- 45!-- of them had ACT composite scores that were either improved (37) or unchanged (8). Only 6 states had drops!
Wow -- that means that just about EVERY state had something to crow about!
What about specific tests?
But again, this "rising tide" MUST be compared to the national changes:
The modest Illinois math "improvement" turns out to be the same as the national change.
Ah, but what about those English and Reading scores, that are better than the national changes? For that, we really need to look at longer trends.
In Reading, the Illinois "improvement" this year (+0.3) is largely a recovery from a drop (-0.2) the year before. The English score is a more consistent increase. Here's the trend:
So, the trends are up, and Illinois is at its best score in the five-year series. But once again, notice the "tide" effect: the national scores are up as well. Is the nation as a whole improving? Or is the ACT getting easier?
Finding Good News
I explored this a little further ...
As reported above, a mere 6 states had declines in their composite scores, so everyone else gets to announce "Overall gains" or at least "Keeping pace".
When we break it out by subject, the spinnable good news (largely thanks to the national score increases) multiplies rapidly. We find:
In another way of looking at the data, I found that 44 states would be able to announce "increases" in one or more of the test results! Only two states (Alaska and Rhode Island) would be unable to announce even a "keeping pace" in at least one of the results.
Finding the Pony in the Bad News: