From the Law of Large Numbers to the Rule of Ten: Why Culling Your Classroom Assessment May Be a Good Idea

Art Lightstone

I'm one of those guys who was forced to take statistics back in university. I didn't want to... I'm not a math guy. But I had to. It was a required course for my psychology degree. While I might not have wanted to take that stats course, I've always been glad I did. It was tough - don't get me wrong - but I really learned some valuable lessons about numbers... as well as the things those numbers have to say about the world around us. 


The Law of Large Numbers

Again, it bears repeating that I am not a math guy. However, by my understanding of it, the law of large numbers states that the accuracy of a statistical inference will increase as we draw more samples from a given population. In other words, the more data you collect, the more accurate your statistical inference will be. This is the law that I have traditionally employed in my classroom. I've historically believed that the grades my assessment mix generates for my students will be more accurate if I collected more data on that student.  


The Composition Error

We commit a composition error, at times, when we assume that what is true for a part is true for the whole. (Most of the time we're probably pretty safe in making such an assumption... but not always. That's why it's such a darned tempting error to make.) Let's take a look at how a composition error could impact assessment or measurement. Imagine, for a moment, that you're attempting to measure the average speed of a tennis player's serve. If we get that player to make a single serve, and we record the speed of the serve using a radar gun, will we get an accurate depiction of the player's typical serving speed? Probably not. Well then... how about if we measure two serves? How about five? Maybe Ten? As you might guess, the accuracy of our little statistical inference will probably increase as we record more data. 

However, let's try pushing this example just a little further. What if we make our tennis player show us - all in a row - twenty of his best serves? How about fifty? How about... a hundred? What do you think is going to happen to our statistical inference? You probably guessed it. The average speed of the serve would go down. The tennis player will get tired, and the tennis player won't be able to give us his best serves in such an arduous and unnatural set of circumstances. We would invariably commit a composition error in assuming that the benefit of a little more data would be amplified by gathering A LOT more data. As we gather more data, the data itself becomes skewed by a measurement effect, where the attempt to measure something exerts an influence on the very thing being measured. 


Enter the The Rule of Ten

The composition error and measurement effect described above speaks, in part, to the potential benefit of judiciously planned - and methodically limited - course assessment. As teachers, it's probably a good idea to always multiply our total number of assessments by the total number of courses that our students take at any one point in time. Then divide that number by the number of weeks in the semester or school year. That's the theoretical total number of assessments that your students might have to do per week. Take a good long look at that number, and ask yourself where you are along the statistical measurement road. Are you still within the region wherein you can extract further benefit from the law of large numbers? Are you possibly approaching the composition error zone? Have you perhaps long passed into the dark and tumultuous region of the measurement effect? Crunch the numbers, and have a good honest discussion with yourself and your teaching partners or department colleagues about where you think you stand with respect to your course assessment. 


A Change in Goals

Applying the Rule of Ten this year has forced me to change the goals I have for myself and for my students. I'm not necessarily lowering my expectations... but I'm changing my objectives. As a Grade 12 teacher, I've traditionally had a goal of teaching my students every little thing they would need to know for their associated first year university course. If I taught economics, I wanted my students to learn everything they would need to know for first-year economics. If I taught law, I wanted to prepare my students for law school. I mean, for crying out loud, my students hadn't even graduated from high school and I was trying to prepare them for law school! 

The bottom line is this: I'm a high school teacher, and my students are high school students. What they really need, I think, is a good high school teacher. Not a good professor... and certainly not a bad one. Another thing they don't need is their lives so crammed full of deadlines, assessment, and stress that they can't learn the things they really need to learn, think about the things they need to think about, or have the high school experience that they would not only enjoy, but perhaps even benefit from the most. I know it's quite trite to say, but there are times when less is more... there really are.

---