I know I've said it before, but teachers in Ontario are still scratching their heads over the question of how to collect evidence of "observations" and "conversations," and then integrate that evidence into a final grade for a student. 

The confusion stems from a paragraph found on page 39 of the Ministry of Education's Growing Success policy document. The paragraph states:

Evidence of student achievement for evaluation is collected over time from three different sources – observations, conversations, and student products. Using multiple sources of evidence increases the reliability and validity of the evaluation of student learning.

Sadly, teachers have not found much guidance from the Ministry on the implementation of this directive. In particular, teachers are left wondering about two critical questions:

1. How might one actually record or collect evidence of conversations and observations?
2. Were it to be collected, how would one use this data to inform a student's final grade? 

Perhaps even more troubling is the fact that teachers cannot find a great deal of consensus, even amongst themselves, about one crucial point. Namely, how do we actually differentiate between a "conversation," an "observation," and a "product"?

To be sure, in our own minds we are all quite clear on the distinction between all three, and we would scarcely imagine any particular debate in the matter. It is only when we subject our individual perceptions of these elements against the scrutiny of our colleagues that we come to realize that there is indeed a great variety of perspectives regarding products, conversations, and observations.

Invariably, most teachers probably like the spirit of triangulation, but just because we like it, doesn't mean we do it. In fact, a recent NewLearnerLab Twitter survey revealed that most teachers do not collect evidence of observations or conversations at all.

Personally, I have developed and experimented with two distinct triangulation systems that would both collect conversation and observation data, and then integrate this data with a student's final grade. I called my first system the Ongoing Triangulation Index, and my second system the Classwork Portfolio. 

I have discussed the logistics of each method in separate articles, but after spending a good deal of time implementing both systems in my program, I felt I was now in a better position to present a comparative review of both methods. 

Ongoing Triangulation Index 

Description:

The Ongoing Triangulation Index is a value that is based on recorded observations and conversations that are generated in class. The OTI depends on the ClassDojo application to capture observations and conversations and then curate these observations within a single record for each student.

Each observation or conversation is time-stamped, dated, and accompanied by a brief explanation of the observation and the curriculum associated with the observation. Within this index, students can accumulate both positive and negative observations, the sum of which will determine their overall Index value.

The OTI counts for 5% of the overall mark in a course. Thus, a number of OTI observations are recorded each term.

Strengths:

Allows the teacher to capture demonstrations of learning, understanding, appreciation, etc. that occur spontaneously within the classroom.

Generates a comprehensive record of observations and conversations for each student.

This record can contain detailed information regarding the specific learning goals associated with each observation. Example: “Demonstrated a deep understanding of how Keynesian fiscal policy could be used to fill a fiscal gap using the balanced budget multiplier.”  

The above record is updated instantly and can be made accessible to both parents and students.  

Challenges:

The OTI system uses the ClassDojo app, which lends itself to capturing evidence of observable behaviours. Thus, this system can potentially be criticized as assigning a mark for learning skills and/or work habits.

This system must record both negative and positive observations in order to generate a meaningful value that can then be translated into a mark.

While the OTI facilitated the recording of both conversations and observations, conversations never figured into the index value.

Classwork Portfolio                

Description:

The Classwork Portfolio uses Google Classroom, Kahoot, and Edvance (an online student information system and grade manager) to generate a portfolio of the various activities, exercises, and check-ins that are pursued in class on a day-to-day basis. Each activity is assessed and given a mark between zero and four based on relative quality.

For those in-class activities and exercises that require deeper qualitative assessment, a streamlined Canada and World Studies, 2015 Achievement Chart is used to assess the varying levels of achievement.

The Classwork Portfolio counts for 5% of the overall mark in a course. Thus, a number of portfolio marks are recorded for each student within each reporting period.  

Strengths:

The CWP allows the teacher to capture progress that occurs in class on a daily basis, and then record the observation of that progress in a way that impacts the student’s mark immediately.

Students come to realize that they can effect a positive influence over their mark during each and every class.

The CWP tends to focus more on the manifest result of a student’s learning than it does on student behaviour or learning skills. (Although, depending on one's perspective, this could be seen as a weakness.)

The teacher can include detailed information regarding the specific learning goals associated with each observation. Example: “Achieved the 80% threshold on today’s check-in on using Keynesian fiscal policy to fill a fiscal gap using the balanced budget multiplier. Silver medal... well done!”

As this information is recorded in an online grade manager, it is made accessible to parents and students immediately.

Regular check-ins are designed to more readily allow students to effect a positive influence on their mark, while limiting the potential for a negative impact.

Challenges:

The CWP system can generate a large quantity of micro-assessments that can rather quickly outpace a teacher’s ability to mark. Teachers are therefore advised to limit CWP items to activities and exercises that can be assessed rather quickly: preferably, during the class in which they are assigned.  

The CWP generates a rather disconnected record of observations and conversations for each student. This record is brought together, somewhat loosely, within our online grade manager.

If you're like me, you often find yourself wishing the Ontario Ministry of Education provided a more user-friendly version of their curriculum documents. The Achievement Charts found in our curriculum guidelines are certainly no exception. These charts tend to be excessively wordy and repetitive, to the point where they become so dense and cluttered that students and parents can find it difficult to comprehend the relationship between the various categories of knowledge and skills, the achievement levels, and all of the associated descriptors. Moreover, the document is long (usually two pages), and is published as a Adobe Acrobat file, which is dreadfully difficult to copy and paste from.

Thus, I often take it upon myself to produce a streamlined version of the achievement charts found in our curriculum documents so that I can use the charts in my work.

Linked to the bottom of this page, you will find such a streamlined version of the 2015 Ontario Curriculum, Grades 11 and 12, Canadian and World Studies achievement chart. This version of the chart is a one-page document that teachers can readily use for assessing their students. 

A Few Points to Note: 

The original achievement chart in the 2015 Ontario Curriculum, Grades 11 and 12, Canadian and World Studies did not include a Level 0 to specifically address a non-existent level of skill or application. I have added this level because, otherwise, when entering an actual mark that is between 1 to 4 in a grade manager, the lowest level of achievement (a Level 1) would be calculated as a 25% grade. I found that adding a Level 0 to be necessary when actually recording these marks in a grade manager for my Classwork Portfolio, and so I highly recommend employing the five-scale version of the Achievement Chart.  

I should also point out that Page 41 of the 2015 Ontario Curriculum, Grades 11 and 12, Canadian and World Studies states that specific “qualifiers” are used with the descriptors in the achievement chart to describe student performance at each of the four levels of achievement – the qualifier “limited” is used for level 1; “some” for level 2; “considerable” for level 3; and a “high degree of” or “thorough” for level 4. Hence, achievement at level 3 in the Thinking category for the criterion “use of planning skills” would be described in the achievement chart as “[The student] uses planning skills with considerable effectiveness.”

Examples of the various categories of knowledge and skills:

The original achievement chart in the 2015 Ontario Curriculum, Grades 11 and 12, Canadian and World Studies contains a number of examples within the chart itself. Those examples were removed from the version of the achievement chart that I have made available here in an effort to both streamline the chart and to more clearly delineate the connection between the various categories of knowledge and skills and the descriptors associated with the various levels of achievement.

Nonetheless, I do present the various examples that were originally contained within the achievement chart. I have simply separated them out from the chart itself and then presented them on the back of the page. The examples I am speaking of are presented below:

• Knowledge of content: facts, terms, definitions
• Understanding of content: concepts, ideas, theories, interrelationships, procedures, processes, methodologies, spatial technologies
• Use of planning skills: organizing an inquiry; formulating questions; gathering and organizing data, evidence, and information; setting goals; focusing research
• Use of processing skills: interpreting, analysing, synthesizing, and evaluating data, evidence, and information; analysing maps; detecting point of view and bias; formulating conclusions
• Use of critical/creative thinking processes: applying concepts of disciplinary thinking; using inquiry, problem-solving, and decision-making processes
• Expression and organization of ideas and information: clear expression, logical organization) in oral, visual, and written forms
• Communication for different audiences: (e.g. peers, adults) and purposes: (e.g., to inform, to persuade) in oral, visual, and written forms
• Use of conventions: mapping and graphing conventions, communication conventions), vocabulary, and terminology of the discipline in oral, visual, and written forms
• Application of knowledge and skills: concepts, procedures, spatial skills, processes, technologies) in familiar contexts
• Transfer of knowledge and skills: concepts of thinking, procedures, spatial skills, methodologies, technologies) to new contexts
• Making connections within and between various contexts (e.g., between topics/issues being studied and everyday life; between disciplines; between past, present, and future contexts; in different spatial, cultural, or environmental contexts; in proposing and/or taking action to address related issues; in making predictions)
• Use of conventions: mapping and graphing conventions, communication conventions, vocabulary, and terminology of the discipline, either in oral, visual, or written forms.

Thus, I have modified the achievement chart in an effort to produce a highly useable, one-page achievement chart that acknowledges all levels of skill and achievement from 0% to 100%. Moreover, I have distilled this modified version of the achievement chart into both PDF and MS Word files. These files are linked below. Please feel free to use and modify them as you see fit. 

​
Downloadable Files:

By definition, formative assessment is supposed to provide data that teachers use to identify needs, modify instruction, and provide support. 

But how many teachers actually do this on a regular basis? How many teachers can? I'm not talking about a token event that occurs just every now and then. I'm talking about consistent formative assessment that is conducted throughout the year and across the entire curriculum of a course. That proposition takes a lot of time, and time isn't something that teachers have in great abundance. Nonetheless, I am here to report that, with the right tools, consistent and comprehensive formative assessment can not only be done, but it can be fun and motivating as well. 

The tools I'm going to discuss in this post include: Google Classroom, Kahoot!, and Google Email. Let's get started.

Google Classroom

To implement this system, you will need to use Google Classroom to post comprehensive lesson plans, complete with objectives and instructions, as well as links and attachments for any required resources. It is important to understand that the "Assignment" is the best option for a lesson plan, as it can be associated with a Google Calendar date. (Although it will appear as a "due date," which isn't necessarily ideal.) It is also important to note that you can quickly obtain and copy a URL that will send students to a specific Google Assignment post (i.e. lesson plan). More on that later. 

Kahoot!

Anyone who knows me or who follows my blog knows how much I love Kahoot. I have developed a Kahoot for virtually every concept I teach. (I won't just have a big Kahoot for a unit review. Rather, if a unit has eight concepts, then that unit will have eight Kahoots.)

I therefore use a Kahoot to provide a quick check-in at the end of each topic. I usually run my Kahoots at the end of a class or at the beginning of the next class. They take about 15 to 20 minutes, and they provide both the students and the teacher with excellent and immediate formative feedback. They are also well designed to allow the teacher to provide timely, targeted remediation on any points of confusion revealed by the check-in. After the results for a given question have been revealed, the teacher can even expand an image that may have been used in that question in order to examine the issue before moving onto the next question. 

At the end of the Kahoot, the teacher can download a comprehensive spreadsheet of the class results. For this reason, it is imperative that students sign in using their real names. I make a habit of downloading the Kahoot results, and then quickly seeing who might have failed the Kahoot. Anyone who failed a given Kahoot will receive an Early Warning Radar Bulletin from Google Email. 

Google Email

An email can be sent out from within the Google Classroom, or from with Google Mail. I simply copy, paste, and modify a version of the email presented below, and send it out to anyone who failed the Kahoot. If the circumstances warrant it, I might also carbon copy the parents. I've also been known to send out motivational emails to students (and, at times, their parents) who demonstrate exceptional brilliance within a Kahoot. I should point out that the email will contain live links back to the lesson plan that resides within Google Classroom. If you recall, that lesson plan will contain attachments to any resources associated with the lesson.

In Summary

As I have often pointed out, setting up technical infrastructures such as Google Classroom lessons and Kahoot check-ins do indeed require a lot of time up front. However, the dividends they pay are great, and the teacher will get to draw those dividends for years to come. After that, this outstanding form of formative feedback, timely remediation, and student/parent communication can become a practical, realistic, and regular part of a teacher's program.

​Give it a try, and let me know how it goes.

If you are receiving this email it is because our recent Kahoot check-in indicates that you could benefit from reviewing the material on consumer and producer surplus, socially optimal outcome, and market intervention (price ceilings and price floors).

I'm going to suggest that you review this material at your earliest convenience, and then try doing the Market Intervention Quiz in the Mastery Learning Lab. [Unit #1: Economic Theory & Consumer Behaviour --> The Mechanics of the Market System --> Market Intervention Quiz].

Cheers,

Mr. L

References:

​Formative Assessment Definition - The Glossary of Education Reform." 29 Apr. 2014, http://edglossary.org/formative-assessment/. Accessed 11 Oct. 2017.