

"Chicago" Math
by Mary Damer
July 24, 1996
Analysis of studies detailed in the booklet,
"Perspectives of Everyday Mathematics:
University of Chicago Math Program
Student Performance Data"
STUDY #1:
Northwestern University Study
Researcher: Karen Fuson
Group studied: First graders (20 classes using
Everyday Mathematics program)
1. This study has a serious SELECTION problem and can not be considered a
credible research design. Because no information is given concerning the
20 classes of first graders using Everyday Mathematics, the test score
differences could be attributed to preexisting differences in prior math
achievement, intelligence, motivation, social class, parental involvement,
etc. We have no way of knowing. When the authors talk about the comparison
classrooms using traditional math , they provide no information about
what texts are used in these classrooms. It appears that "traditional math"
means any math other than Everyday Mathematics.
2. This study has a serious HISTORY problem. The reported differences
could be due to the Everyday Mathematics classes spending more time on
mathematics, having more money spent on instructional materials, having
better trained teachers. Because these variables have not been addressed,
we have no way of knowing.
3. This study easily could have an INSTRUMENTATION problem. This would
occur if some of the same test questions on the final assessment
instrument were items where Everyday Mathematics students would be
expected to perform better than other students, and questions not used
were ones where we would expect other students to do better than the
Everyday Mathematics students. Without information on the selection of the
final assessment, we have no way of knowing. No information on the
reliability or validity of the AERAStigler quest final assessment
instrument has been given.
4. No test of statistical significance was ever made. Any differences
between the performance of students in these two groups could just as
easily be a result of chance.
STUDY #2:
University of Chicago School Mathematics Project
Researcher: William Carroll
Group studied: students in fourth grade who had
completed the K3rd grade Everyday Mathematics program
1. This study has a serious SELECTION problem and can not be considered a
credible research design. Because no information is given concerning the
20 classes of first graders using Everyday Mathematics, the test score
differences could be attributed to preexisting differences in prior math
achievement, intelligence, motivation, social class, parental involvement,
etc. We have no way of knowing. In addition, there is a another SELECTION
problem so we do not know about the 11 classes selected for the final
performance based assessment. When the authors talk about the comparison
classrooms using traditional math , they provide no information about
what texts are used in these classrooms.
2. This study easily could have an INSTRUMENTATION problem. This would
occur if some of the same test questions on the final assessment
instrument were items where Everyday Mathematics students would be
expected to perform better than other students, and questions not used
were ones where we would expect other students to do better than the
Everyday Mathematics students. Without information on the selection of the
final assessment, we have no way of knowing. No information on
reliability is provided for either the 7 selected test items or the 16
selected items.
3. More information is needed to convince a reader that the 4 schools
selected from the 7 would be expected to score about the same as the
national average than the one sentence used by the author of this study to
substantiate this claim; ..one rural, one urban, and two smallcity with
a sizable lowincome population and with ethnic and racial diversity.
Information showing how the scores of these students in other subjects
compare with the national averages in those subject areas would provide
some substantiation for this claim.
4. No test of statistical significance was ever made. Any differences
between the performance of students in these two groups could just as
easily be a result of chance.
STUDY #3:
Silver Ridge Elementary School
Researcher: John Woodward
Group studied: third graders who had used the
Everyday Mathematics program since kindergarten
1. This study has more credibility than the first two studies because
reliability is reported and statistical tests are used. In fact, this is
the only credible research design presented. The use of a pretestpost
test design mitigates selection problems. However, since Silver Ridge
students started at the 71st percentile on ITBS whereas the comparison
group started at the 58th percentile, one would expect them to show
greater gains on the TOPS which unlike the ITBS is not grade level normed.
It is interesting to note that on the ITBS raw scores, Silver
Ridge had higher pre treatment means on all three subtests and therefore
would be expected to show greater gains on all subtests. However, the
comparison group made greater gains on two of the three subtests! What can
this mean about the effectiveness or ineffectiveness of the Everyday
Mathematics program?
2. This study could have a serious HISTORY problem. Perhaps while Silver
Ridge was very concerned about math, the comparison school was more
concerned with reading. Several teachers report spending more time on
Everyday Mathematics. This is a serious history problem as any improvement
in achievement could be due to increased time spent on math in the
Everyday Mathematics classes. Because these variables have not been
addressed, we have no way of knowing.
3. The graph on pages 23 and 24 are meaningless, because with a
nonrandom comparison group, differences in gain scores may be due to any
number of variables.
STUDIES #4 and #5:
Barrington and Indiana Areas
Barrington researcher: Herb Price
Barrington group studied: third and fourth graders
Indiana researcher: Walter Kealey
Indiana group studied: second graders
1. Because both of these small studies are one group pre treatment  post
treatment studies, HISTORY is a problem. It could be that between the pre
and post tests the Barrington or Indiana schools became more interested in
math, spent more time and money on it, or provided more training to their
teachers. The improvement in the student's achievement test scores could
just as easily be a reflection of this. We have no way of knowing.
Advice to Parents
On a mothertomother aside, I would advise that next year you go to
school and look at your children's "journals" that accompany the math
program at least once a month. The home links that you will receive at
home will not provide enough information to tell you what your child is
doing during the school day. Two friends of mine had to hijack the
journals from their children's classes (the teacher wouldn't let them
leave the room) and xerox them in an overnight copy center. When they
looked at the journals, they realized that their third graders had been
incorrectly adding with all carrying problems and raised this concern with
the teacher. She replied that it was important for the children to work
out their own strategy independently and that if they didn't get it during
this spiral, they would during the next one. My friends who were apalled
to realize that it was Spring and their children had learned nothing in
math all year now have their children on a waiting list at a private
school (All the private schools in this suburb have had long waiting lists
for the past two years.) They dearly wish that they had made more school
visits and studied those journals. Right now they are madly tutoring their
children in morning in third grade math.
