Christian Schools Calgary: AKCS Principal Points

Past Principal Points

Standardized Tests—Part 3

Over the years I have written several times about standardized testing. There are two other articles at our website under Past Principal Points if you wish to get further information. The three articles should be taken together if one wants to get an accurate understanding of how to interpret standardized test results in general and to understand how we use them here at AKCS. For the latest information about AKCS results, please visit our Tests Page which is always up to date.

One fairly recent development that is garnering a lot of attention is the publication by the Fraser Institute of school rankings based on Achievement Tests. The newspapers have done something a bit similar in the past, but they have used the Alberta Learning data which indicates the percent of students achieving at the acceptable (generally 50%) or excellence (generally 80%) level. The Fraser Institute has used the actual percentage scores that students obtained (on just Language & Math) plus additional material related to difference in achievement for boys and girls to come up with an overall score which they use to rank the schools. They also have what they call a “Value Added” indicator which shows the improvement from grade three to six.

You will no doubt read about the considerable uproar that publishing these rankings has caused. The Fraser Institute defends their report by stating that parents need additional, objective information to help them assess the quality of their children’s schools. The Alberta Teachers Association attacks the report as an unfair and very limited representation of the learning that actually happens in schools. I take a middle ground in this debate.

First, I do agree with the ATA that the published ranking of schools is quite unfair. There are schools with scores of students who hardly can speak any English. They will obviously do poorly on the Language Arts tests and, quite possibly, the math tests also. This does not mean that the school is doing a bad job. There are schools in poverty stricken areas of the province where attendance at school is very sporadic. The teachers who work in these schools deal with many children from broken homes with all kinds of social problems, e.g. alcoholism. They deserve medals for what they do. One other thing to note is that there is a clear tendency for student results to track the level of their parents’ education. The greater the education level of the parents, the better students performs, and vice versa. Schools shouldn’t take too much credit or blame for this factor.

Furthermore, I agree with the ATA that achievement tests do not tell the whole story about a school. They never test Bible knowledge or character development which are so important to us. And, of course, they tell us nothing about students’ artistic, musical, dramatic, or leadership abilities. However, to say that these tests do not tell us everything is not the same as saying that they tell us nothing. They are useful guides to how well a school is implementing the curriculum in the core subjects of language and math. Schools do much more than this, but this is pretty important! Every parent has a right to know that schools are doing this well, and if a school cannot do these basics well, how can we be confident that they are teaching moral development properly?

We have always used standardized tests and have always sought to teach our students and parents how to interpret them properly. Both good scores and bad scores can be completely misleading at times. It’s important to read the data carefully and interpret it in light of a school’s social make up and historical experience. Please read the other two articles for a better understanding of this.

So what are we to make of AKCS’s “ranking” ? Well, for one thing one year’s scores are not a valid means of evaluating the strength of a program or school, especially when the numbers are small. We have always posted at our website and in our annual reports our five year average because it gives a much more realistic picture of what is happening. Neither very strong nor very weak classes can skew such an average by much. Second, when I look at all the other Christian schools in Alberta, I see that there are only a few that have better five year averages than AKCS: Glenmore in Calgary, and Immanuel in Lethbridge. I am particularly impressed with Immanuel because the average education level of their parents is actually a bit lower than ours whereas Glenmore’s is a good bit higher. They are clearly doing something right. And, by the way so are we! We should all rejoice in what God has done here at AKCS. We have a five year average that puts us in the top 10% for all schools in the province. That is not too shabby, as they say!

So, overall, I am quite satisfied with how our children performed this past year, given their particular abilities. I am confident that our program remains solid, and our teachers are doing an excellent job. I am confident that we are maintaining, and will continue to maintain, the very high standard that we have always had.

On the other hand, while our five year average is very strong, it is in a bit of a decline, so I will be keeping an eye on that. We haven't changed our program, but perhaps we are enrolling a different kind of student and need to make better adaptations to meet their needs. Perhaps, our program no longer matches the recent changes to the Alberta curriculum as well as it has in the past. We know this is the case with our middle school mathematics program and have been working on addressing this specifically. It will be interesting to see how that plays out this year and next. We will continue to keep an eye on the trends.

Another feature of the Fraser Report which I think is good in concept but flawed in execution is the “Value-Added” indicator. What they are trying to do is determine what additional input has occurred between grade three and grade six that is the result of the school’s program but not home environment. They acknowledge in their report (see their website) that there are a number of flaws with this approach, but they still feel it has some merit. One of the flaws, for example, is that a school that does very well at the grade three level is not going to be able to “add much value” by grade six because there is very little room for improvement. We almost always do very well in grade three and so (apparently) “add little value” by grade six.

A second flaw is that the students tested in grade three are not the same as those in grade six because students transfer in and out of schools. Most of our grade threes have gone to grade one and two at AKCS. Many of our grade sixes have transferred in later. I have written to the Fraser Institute and suggested that the best value-added indicator would be to simply compare performance with the socio-economic status of the parents. This would not only be fairer for us, but especially for those public schools that operate in very difficult situations and are doing great work against all odds. We’ll see if they take my advice. J

Finally, a number of people have asked about how the Fraser Institute gets their information about the education of our parent body. I asked them, and they said that they get it from census data. Alberta Learning has a record of the addresses of students and which schools they attend, and this is used with census data to get a rough profile of income based on housing values.

Finally, a rule of thumb for standardized tests is “Don’t ignore them, but don’t treat them as ‘the truth, the whole truth, and nothing but the truth.’”