I'm one of those guys who was forced to take statistics back in university. I didn't want to... I'm not a math guy. But I had to. It was a required course for my psychology degree. While I might not have wanted to take that stats course, I've always been glad I did. It was tough - don't get me wrong - but I really learned some valuable lessons about numbers... as well as the things those numbers have to say about the world around us. 


The Law of Large Numbers

Again, it bears repeating that I am not a math guy. However, by my understanding of it, the law of large numbers states that the accuracy of a statistical inference will increase as we draw more samples from a given population. In other words, the more data you collect, the more accurate your statistical inference will be. This is the law that I have traditionally employed in my classroom. I've historically believed that the grades my assessment mix generates for my students will be more accurate if I collected more data on that student.  


The Composition Error

We commit a composition error, at times, when we assume that what is true for a part is true for the whole. (Most of the time we're probably pretty safe in making such an assumption... but not always. That's why it's such a darned tempting error to make.) Let's take a look at how a composition error could impact assessment or measurement. Imagine, for a moment, that you're attempting to measure the average speed of a tennis player's serve. If we get that player to make a single serve, and we record the speed of the serve using a radar gun, will we get an accurate depiction of the player's typical serving speed? Probably not. Well then... how about if we measure two serves? How about five? Maybe Ten? As you might guess, the accuracy of our little statistical inference will probably increase as we record more data. 

However, let's try pushing this example just a little further. What if we make our tennis player show us - all in a row - twenty of his best serves? How about fifty? How about... a hundred? What do you think is going to happen to our statistical inference? You probably guessed it. The average speed of the serve would go down. The tennis player will get tired, and the tennis player won't be able to give us his best serves in such an arduous and unnatural set of circumstances. We would invariably commit a composition error in assuming that the benefit of a little more data would be amplified by gathering A LOT more data. As we gather more data, the data itself becomes skewed by a measurement effect, where the attempt to measure something exerts an influence on the very thing being measured. 


Enter the The Rule of Ten

The composition error and measurement effect described above speaks, in part, to the potential benefit of judiciously planned - and methodically limited - course assessment. As teachers, it's probably a good idea to always multiply our total number of assessments by the total number of courses that our students take at any one point in time. Then divide that number by the number of weeks in the semester or school year. That's the theoretical total number of assessments that your students might have to do per week. Take a good long look at that number, and ask yourself where you are along the statistical measurement road. Are you still within the region wherein you can extract further benefit from the law of large numbers? Are you possibly approaching the composition error zone? Have you perhaps long passed into the dark and tumultuous region of the measurement effect? Crunch the numbers, and have a good honest discussion with yourself and your teaching partners or department colleagues about where you think you stand with respect to your course assessment. 


A Change in Goals

Applying the Rule of Ten this year has forced me to change the goals I have for myself and for my students. I'm not necessarily lowering my expectations... but I'm changing my objectives. As a Grade 12 teacher, I've traditionally had a goal of teaching my students every little thing they would need to know for their associated first year university course. If I taught economics, I wanted my students to learn everything they would need to know for first-year economics. If I taught law, I wanted to prepare my students for law school. I mean, for crying out loud, my students hadn't even graduated from high school and I was trying to prepare them for law school! 

The bottom line is this: I'm a high school teacher, and my students are high school students. What they really need, I think, is a good high school teacher. Not a good professor... and certainly not a bad one. Another thing they don't need is their lives so crammed full of deadlines, assessment, and stress that they can't learn the things they really need to learn, think about the things they need to think about, or have the high school experience that they would not only enjoy, but perhaps even benefit from the most. I know it's quite trite to say, but there are times when less is more... there really are.

Field studies describe an organized, well-considered effort to collect data from a real-world environment. Field studies generate primary research data through the use of recorded observations, interviews, or surveys. Field studies are not to be confused with experiments, wherein data is generated from within a controlled setting - such as a lab.

The objective of any field study is to make inferences about what happens in the real world.


The Field Study Extended Project

In the Field Study Project, each student (or pair of students) would propose, design, and conduct some form of original field study. The field study must explore a phenomenon that is related to a course's overall topic of study (ex. law, sociology, or economics). The actual field study (i.e. the data collected) would form the basis of a scholarly journal article that can later be published within a journal that the class produces and posts online.

The Field Study extended project includes the following phases: 

I) Learn

Students familiarize themselves with the idea of field studies by reading field study articles curated by the course instructor. 

II) Design

• Students then design an original field study. Students should be reminded that simple and elegant designs for field studies are generally held in high regard. A critical factor to bear in mind when designing a study is the issue of validity. Students must ensure that they are actually studying what they are seeking to study - not some unanticipated factor. 

III) Ethical Review

• All students must submit their proposed study to an ethical review board comprised of their peers. All students must also volunteer to serve on an ethical review board. (Ethical review boards must be comprised of at least two students.) Each board must review at least two study proposals. (Download an example of an Ethical Review Application Form below.

IV) Conduct 

• This is the fun part. Once the students receive approval from their ethical review, they may proceed with conducting their field study. Conducting a field study may consist of distributing and collecting surveys, or just recording observations made at a certain location.

V) Write-up

• This is an informal piece of writing that will be assessed on a formative basis. Nonetheless, this piece of writing will serve as the foundation for both the presentation and the journal article: both of which will be included within the summative assessment for the course. (Please see below for the structure and the length of this component.)

VI) Present

• Students present their field study to the class. The presentation can use audio-visual aids or presentation software to assist with this presentation; however, there is no substitute for the ability to engage an audience solely with one's voice, knowledge, and speaking ability. (Please see below for the structure and the length of this component.)

VII) Publish

• The study will be published as an article within a scholarly journal produced and posted online by the class. Such a scholarly journal would generally require the use of APA citation. The front matter of the article must include:
  
  
  • a title,
  • a subtitle,
  • a 100 word abstract, and
  • three to five keywords.
    
    
  Following the front matter, the article should follow the structure outlined below. The article should also adhere to the length stipulated below.

Structure and Length

Structure:

Each field study write-up, presentation, and journal article should:

1. Outline some form of relevant issue.
2. Describe and quantify the parties involved or affected by the issue. (i.e. Who are they? How many are they?)
3. Explain why this issue is of concern to Canadian society; include any long-term implications the issue might hold.
4. Discuss the methodology used to conduct the study. (This can be as simple as observing the number of people who exhibit a certain behaviour at a certain time in a certain location.)
5. Outline the findings of the study.
6. Discuss possible interpretations or explanations for the study's findings.
7. Suggest possible implications of the study's findings.
8. Suggest possible directions for further research regarding the issue.
9. Present the raw data generated by the study within charts or tables. (This can be presented in an appendix.)

Length:

Write-Up: There is no prescribed lower or upper word limit for the study write-up. However, this study should be well considered and well implemented so that it can serve as the foundation for both your presentation and your scholarly article.

Presentation: Ten minutes.

Scholarly Article: Maximum of 1500 words, including the front matter. (Front matter for a journal article generally consists of a title, abstract, key words, and the names of the author or authors.) This is an article intended for publication within a scholarly journal, so it should be representative of the student's very best writing. Single spaced, at font size 12, the article should take up three pages within the scholarly journal.  

Looking for an Icebreaker for your first social science class? 

I've used a certain icebreaker exercise in my economics classes for years, and it never ceases to amaze me. I first go over the basics of how field studies and investigative science work: how scientists will propose hypotheses, gather data, identify correlations, and then attempt to explain causal relationships. 


I then ask the students (alone or in groups, depending on the class size) to develop a hypothesis that they might be able to examine by just studying the students in our class. Each student (or group) must then interview each and every student in the class in order to collect the two variables that they wish to examine. Do their classmates have any siblings? How tall are they? Do they wear corrective lenses? Do they wear a watch? How many languages do they speak? The possibilities are endless. 

The students plot the data they collect on a graph, and then present both their hypothesis and their findings to the class. The study / icebreaker portion of this exercise can take place in a single 80-minute class, and the presentations can generally be completed in the next class. 

Google Spreadsheets Serve Up Excellent Scatter Graphs in Three Easy Steps  
 
Google spreadsheets provide a particularly quick and easy way to illustrate a correlation between variables. If you have access to Google Apps in your school, then a Google spreadsheet can plot the data points and illustrate correlations in three easy steps:

i) Set up three adjacent columns: the first column being for the names of the students interviewed (so the interviewer can track who she has and hasn't interviewed), and the next two columns being for the two variables that are being analyzed.

ii) Highlight just the two columns of data (without any names), and then click on the "Chart Wizard" button. You will see a variety of chart options, but click on "more" chart options to find the scatter graph option. (You must select the "scatter" graph option to plot correlations between variables.)

iii) Click on the "Customize" tab and then scroll down to exercise the options of setting your chart title, naming your X and Y axes, and even generating a line of best fit. (The line of best fit is a particularly handy feature of Google spreadsheets that Google had previously been criticized for not including. As you can see, Google Apps are constantly evolving.) 

Over the years, I've seen my students make so many amazing discoveries right before my eyes. For example, did you know that people who wear watches tend to enjoy greater academic success in school? How about this one: Did you know that blue-eyed people tend to wear corrective lenses less than brown-eyed people? Finally, would you believe that people who speak two languages tend to do better in school than people who speak one - or even three - languages? These are just a few of the incredible findings that my students have unearthed during this exercise. While these are just correlations, not causations, they are still pretty amazing discoveries.

These mini-studies are a great way for students to meet and learn about each other while also exploring the tools of investigative science. Every class will inevitably find themselves exploring issues of correlation, causality, sample bias, split effects, and even post-hoc fallacies. 

Try this the next time you're looking for a way to break the ice in your social science course, and let me know how it goes. 


Please note: While computers help, you don't need computers to do this exercise. I did this exercise for years before my school became a laptop school. You can download a PDF below that will facilitate a pen and paper version of this exercise.