Illinois Loop
Your guide to education in Illinois
  Bookmark and Share
 
The Illinois Loop website is no longer updated on a a regular basis. However, since many of the links and articles have content and perspectives that are just as valid today, we are keeping this website online for parents, teachers and others researching school issues and solutions.
Broken links:If you encounter links that no longer lead to the desired article, it's still often possible to retrieve them. Most of the linked items include a sentence or more from the original. Copy a section of that text, and type it into Google surrounded by quotes. More often than not, Google will find the article at a revised location.

 

Finding the Pony in the ACT

Analysis of 2006 ACT Results
An Illinois Loop Analysis

Kevin C. Killion
January 4, 2000



    WOW -- Look at what a good PR effort can produce! Here's how the this year's release of ACT scores was headlined this morning:

    Tribune:

      "Illinois ACT Scores Up Across Board"

    Sun-Times:

      "City, State ACT Scores jump, Still Lag"

    Daily Herald:

      "A Tough ACT to Follow / Test Scores Rise Across State ..."

Where the "good" news came from

    Yesterday, the ISBE, our state's education bureaucracy, sent out a carefully worded press release selecting specific numbers from the ACT results, which they got to see in advance of the official release. As all good bureaucrats do, the news was spun to highlight the best stories possible.

    Here are the key "good" stories that the ISBE managed to extricate from the ACT:

    • Illinois' "composite" score was higher than the previous year, going from 20.3 to 20.5
    • Illinois' Reading score is 20.6 (which is up from 20.3 the previous year)

Now, the REAL story

    But today the results are available to everyone, and the story about Illinois is less impressive than those headlines suggest.

    It turns out that the ACT scores are up nationally, not just in Illinois.

    The new composite ACT score is 21.1, up from 20.9, an "increase" of 0.2. So, Illinois' "increase" by the same 0.2 (from 20.3 to 20.5) turns out to be merely keeping pace with the national score!

    So, can Illinois at least claim that it's improving along with the rest of the nation?

    Well, we'd like to think that the whole country is improving. But we have to wonder: has the country improved, or has the test gotten a little easier? A rising tide raises all boats. If you see boats going up, that tells you more about the tide than about boats.

    I looked at results for the 50 states plus the District of Columbia. It turns out that 45 -- 45!-- of them had ACT composite scores that were either improved (37) or unchanged (8). Only 6 states had drops!

    Wow -- that means that just about EVERY state had something to crow about!

What about specific tests?

    Illinois educrats will (no doubt) be cheering about results in each of the areas of the ACT. Illinois scores are up in English, Math and Reading. Only in Science were results flat.

    But again, this "rising tide" MUST be compared to the national changes:

       CompositeEnglishMathReadingScience
      National+0.2+0.2+0.1+0.10
      Illinois+0.2+0.3+0.1+0.30

    The modest Illinois math "improvement" turns out to be the same as the national change.

    Ah, but what about those English and Reading scores, that are better than the national changes? For that, we really need to look at longer trends.

    In Reading, the Illinois "improvement" this year (+0.3) is largely a recovery from a drop (-0.2) the year before. The English score is a more consistent increase. Here's the trend:

       ReadingEnglish
      Report YearIllinoisNationalIllinoisNational
      200220.321.119.420.2
      200320.421.219.620.3
      200420.521.319.720.4
      200520.321.319.920.4
      200620.621.420.220.6

    So, the trends are up, and Illinois is at its best score in the five-year series. But once again, notice the "tide" effect: the national scores are up as well. Is the nation as a whole improving? Or is the ACT getting easier?

Finding Good News

    The ACT results are a playground for eager public relations people looking for "good" news to crow about. With four subject areas plus a composite score, and then a "rising tide" of scores nationally in the ACT, wow, just about any state can find something good to announce!

    I explored this a little further ...

    As reported above, a mere 6 states had declines in their composite scores, so everyone else gets to announce "Overall gains" or at least "Keeping pace".

    When we break it out by subject, the spinnable good news (largely thanks to the national score increases) multiplies rapidly. We find:

      Test componentStates that get to announce
      "improvements" or "keeping pace"
      Composite45
      English43
      Math46
      Reading44
      Science41

    In another way of looking at the data, I found that 44 states would be able to announce "increases" in one or more of the test results! Only two states (Alaska and Rhode Island) would be unable to announce even a "keeping pace" in at least one of the results.

Finding the Pony in the Bad News:
The Woebegone Effect

    A good PR effort doesn't stop even when it's hard to find the good news.

    I wondered about that in looking at the results for our neighbors in Indiana. Poor Indiana! Down in English (despite national increases), flat in Reading and in the Composite (despite national increases), and math and science track at the national rate. Yikes, what can the Hoosier educrats crow about?

    Well, I searched, and found that Indianapolis did indeed pull a rabbit from an empty-looking hat:

      "Indiana ACT Scores Top National Average for 16th Year"

    (See their PR release here.)

    Yup, when all else fails, shout, "We're better than average!"

    So, who IS better than average in the ACT? In investigating this, I found that Lake Woebegone is thriving at the ACT! There are 32 states that have composites better than the national average. Moreover, 36 states get to claim they are at the national average or better in Reading!

All the Details

Copyright 2012, The Illinois Loop. All Rights Reserved.
Home Page     Site Map     Contact Us