Pre-test surveys: The next big tool for the next big test

Art Lightstone

The first test in a course is like the first goal in a hockey game: an early success can create a lot of positive momentum for the rest of the game. 

Effort directed toward ensuring that a class is ready to write a test - especially the first one - can definitely be time well spent. Here's a little idea that might just be the ticket for improving student results on your next test. 

Pre-Test Surveys

For many years now, I've made comprehensive review sheets and practice tests for my students to use while studying. I've also tended to use online surveys to see how my students are feeling about my teaching and my courses in general. Yet, it never really occurred to me until quite recently that I could use online surveys to see how my students are feeling about the material within a given unit of study.  

As soon as I got the idea, online pre-test surveys quickly became a part of my blended learning toolkit. The process associated with administering this style of survey can be broken down into four parts: i) creating, ii) distributing, iii) analyzing, and iv) responding. 

Creating the Survey:

These surveys do take some time to create. However, if you've already done some work up front to develop a review sheet, then you can use your review sheet to generate the pre-test survey fairly quickly. 

I tend to simply copy the requirements outlined within a test review sheet into a new survey. I then add a simple little introductory phrase to each requirement, such as, "Rate your ability to..." This generates a survey question that might look like: 

Rate your ability to define and explain the following concepts: elasticity of demand, "elastic" demand, "inelastic" demand, and "unitary" elastic demand.

I should point out that I do tend to lump together a few thematically related concepts into each survey question. Otherwise, these surveys might end up containing fifty or more questions and take a really long time to complete.

I then have my students choose a response from a five-point Likert Scale, ranging from "1: I'm really confused," to "5: I'm very confident." As the survey progresses, I tend to come up with increasingly outlandish polemics for each Likert Scale, such as, "I have no idea what you just said," versus "That's about as easy as opening my fridge." (I'd like to think that these humorous options tend to encourage greater student response.) 

The survey ends up looking like a collection of questions similar to the ones illustrated below. The really great thing about this style of survey is that it asks the student to engage in an extremely valuable style of assessment that we as educators rarely pursue: namely, assessment AS learning. This is the style of assessment where students are asked to reflect upon their learning. Research tells us that this style of reflection is an extremely powerful style of assessment in terms of its ability to promote learning, and it is actually one of the mechanisms that gives rise to Arthur Gates' famous "testing" effect.   

Distributing the Survey:

I tend to use Google Forms to facilitate this type of survey (as opposed to an anonymous survey tool like Survey Monkey) because I do want to collect user information on the respondents. This information allows me to provide help to specific students who might require some additional support ahead of the test.

I then send out an email to the entire class that explains the purpose of the survey and provides a hyperlink to it. I also ask students to provide their most accurate and honest response to each question. I tell them that the survey will help us both: it will help them identify any areas of weakness, and it will help me understand how to best support my students in their efforts to prepare for the test. Moreover, a pre-test survey tends to show students exactly how much material they actually do know, and that in of itself can help to calm nerves and build confidence ahead of a test.

---

Hello Grade 12 Economics Students,

Thanks so much for your feedback on the Unit #1 Pre-Test Survey. It's pretty clear that you really do enjoy a good old fashioned lecture... because the one thing we haven't discussed in class is one of the main things that you're feeling a bit fuzzy on. I'm talking about "Types of Goods."

I've done a couple of things to support you in your efforts to prepare for our upcoming unit test: 

1. I've made sure to trim down the "types of goods" on the Unit #1 Review Sheet to just the goods that have to do with elasticity: either income or price elasticity. 

2. I've created and launched a quiz on these different types of goods in the Mastery Learning Lab. Please go check it out and give it a try! (... and then another try, and then another, and another...)

As mentioned in a previous email, I have a really nice resource on this topic. If you haven't used it already, please see the Types of Goods handout on the course website. 

Cheers,

Mr. L

Analyzing the Survey:

The first lesson I learned about analyzing these survey results is to not expect perfection.

In a five-point Likert Scale, where the number 5 represents perfect understanding of the requirement, and a number 1 represents complete lack of understanding, then we basically want to see responses that are skewed around the number four. Even on tests where the class average ends up in the 90% range, I have never seen a result on a pre-test survey that consists entirely of fives.

Responding to the Survey:

Now, if the results are skewed around a number two or three, then that should raise a red flag. I've definitely seen such results from time to time, and they will prompt me to do one of two possible things: i) review the material associated with such a response in class, and/or ii) send out a class-wide email addressing the issue.   

In the blended learning environment, the extra support required to address a weak area can often be facilitated with a quick email that points students to some online resources. A typical class email regarding a low-scoring survey question is illustrated below: 

---

I'll remind you of the fact that I do choose to use Google Forms for these surveys because they allow me to collect user information. Thus, if necessary, I can provide a little extra support to individual students who scored themselves particularly low in a given area. Generally, a quick email pointing the student to some specific online resources and offering a little extra help is enough to do the trick. A typical response to an individual student is illustrated below:

Hello Ted,

Based on your pre-test survey, you should probably review this video on demand shifters: http://www.youtube.com/watch?v=LwLh6ax0zTE

Also, be sure to look at the resources I sent around yesterday about the Production Possibility Curve. 

I'll be in my room for extra help after 3:05 PM tonight if you'd like to drop by.

Cheers,

Mr. L

Without a doubt, pre-test surveys represent an additional time requirement for those teachers who might wish to incorporate them into their programs. In some cases, teachers may very well be pursuing some form of pre-test introspection already. The decision to use such a tool or not is invariably one that must be made based on a cost-benefit analysis. I will say this: just like most everything else related to edutech, these initiatives are often front-end loaded. Pre-test surveys will definitely take some time and effort up front, but once you have them in your kit bag, they'll pay some nice dividends down the road.