Illinois Loop
Your guide to education in Illinois
  Bookmark and Share
 
The Illinois Loop website is no longer updated on a a regular basis. However, since many of the links and articles have content and perspectives that are just as valid today, we are keeping this website online for parents, teachers and others researching school issues and solutions.
Broken links:If you encounter links that no longer lead to the desired article, it's still often possible to retrieve them. Most of the linked items include a sentence or more from the original. Copy a section of that text, and type it into Google surrounded by quotes. More often than not, Google will find the article at a revised location.

 

"Chicago" Math

by Mary Damer
July 24, 1996

Analysis of studies detailed in the booklet,
"Perspectives of Everyday Mathematics:
University of Chicago Math Program
Student Performance Data"

STUDY #1:
Northwestern University Study
Researcher: Karen Fuson
Group studied: First graders (20 classes using Everyday Mathematics program)

1. This study has a serious SELECTION problem and can not be considered a credible research design. Because no information is given concerning the 20 classes of first graders using Everyday Mathematics, the test score differences could be attributed to preexisting differences in prior math achievement, intelligence, motivation, social class, parental involvement, etc. We have no way of knowing. When the authors talk about the comparison classrooms using traditional math , they provide no information about what texts are used in these classrooms. It appears that "traditional math" means any math other than Everyday Mathematics.

2. This study has a serious HISTORY problem. The reported differences could be due to the Everyday Mathematics classes spending more time on mathematics, having more money spent on instructional materials, having better trained teachers. Because these variables have not been addressed, we have no way of knowing.

3. This study easily could have an INSTRUMENTATION problem. This would occur if some of the same test questions on the final assessment instrument were items where Everyday Mathematics students would be expected to perform better than other students, and questions not used were ones where we would expect other students to do better than the Everyday Mathematics students. Without information on the selection of the final assessment, we have no way of knowing. No information on the reliability or validity of the AERA-Stigler quest final assessment instrument has been given.

4. No test of statistical significance was ever made. Any differences between the performance of students in these two groups could just as easily be a result of chance.

STUDY #2:
University of Chicago School Mathematics Project
Researcher: William Carroll
Group studied: students in fourth grade who had completed the K-3rd grade Everyday Mathematics program

1. This study has a serious SELECTION problem and can not be considered a credible research design. Because no information is given concerning the 20 classes of first graders using Everyday Mathematics, the test score differences could be attributed to preexisting differences in prior math achievement, intelligence, motivation, social class, parental involvement, etc. We have no way of knowing. In addition, there is a another SELECTION problem so we do not know about the 11 classes selected for the final performance based assessment. When the authors talk about the comparison classrooms using traditional math , they provide no information about what texts are used in these classrooms.

2. This study easily could have an INSTRUMENTATION problem. This would occur if some of the same test questions on the final assessment instrument were items where Everyday Mathematics students would be expected to perform better than other students, and questions not used were ones where we would expect other students to do better than the Everyday Mathematics students. Without information on the selection of the final assessment, we have no way of knowing. No information on reliability is provided for either the 7 selected test items or the 16 selected items.

3. More information is needed to convince a reader that the 4 schools selected from the 7 would be expected to score about the same as the national average than the one sentence used by the author of this study to substantiate this claim; ..one rural, one urban, and two small-city with a sizable low-income population and with ethnic and racial diversity. Information showing how the scores of these students in other subjects compare with the national averages in those subject areas would provide some substantiation for this claim.

4. No test of statistical significance was ever made. Any differences between the performance of students in these two groups could just as easily be a result of chance.

STUDY #3:
Silver Ridge Elementary School
Researcher: John Woodward
Group studied: third graders who had used the Everyday Mathematics program since kindergarten

1. This study has more credibility than the first two studies because reliability is reported and statistical tests are used. In fact, this is the only credible research design presented. The use of a pretest-post test design mitigates selection problems. However, since Silver Ridge students started at the 71st percentile on ITBS whereas the comparison group started at the 58th percentile, one would expect them to show greater gains on the TOPS which unlike the ITBS is not grade level normed.

It is interesting to note that on the ITBS raw scores, Silver Ridge had higher pre treatment means on all three subtests and therefore would be expected to show greater gains on all subtests. However, the comparison group made greater gains on two of the three subtests! What can this mean about the effectiveness or ineffectiveness of the Everyday Mathematics program?

2. This study could have a serious HISTORY problem. Perhaps while Silver Ridge was very concerned about math, the comparison school was more concerned with reading. Several teachers report spending more time on Everyday Mathematics. This is a serious history problem as any improvement in achievement could be due to increased time spent on math in the Everyday Mathematics classes. Because these variables have not been addressed, we have no way of knowing.

3. The graph on pages 23 and 24 are meaningless, because with a non-random comparison group, differences in gain scores may be due to any number of variables.

STUDIES #4 and #5:
Barrington and Indiana Areas
Barrington researcher: Herb Price
Barrington group studied: third and fourth graders
Indiana researcher: Walter Kealey
Indiana group studied: second graders

1. Because both of these small studies are one group pre treatment - post treatment studies, HISTORY is a problem. It could be that between the pre and post tests the Barrington or Indiana schools became more interested in math, spent more time and money on it, or provided more training to their teachers. The improvement in the student's achievement test scores could just as easily be a reflection of this. We have no way of knowing.

Advice to Parents

On a mother-to-mother aside, I would advise that next year you go to school and look at your children's "journals" that accompany the math program at least once a month. The home links that you will receive at home will not provide enough information to tell you what your child is doing during the school day. Two friends of mine had to hijack the journals from their children's classes (the teacher wouldn't let them leave the room) and xerox them in an overnight copy center. When they looked at the journals, they realized that their third graders had been incorrectly adding with all carrying problems and raised this concern with the teacher. She replied that it was important for the children to work out their own strategy independently and that if they didn't get it during this spiral, they would during the next one. My friends who were apalled to realize that it was Spring and their children had learned nothing in math all year now have their children on a waiting list at a private school (All the private schools in this suburb have had long waiting lists for the past two years.) They dearly wish that they had made more school visits and studied those journals. Right now they are madly tutoring their children in morning in third grade math.

Copyright 2012, The Illinois Loop. All Rights Reserved.
Home Page     Site Map     Contact Us