The New Learner Lab
  • Home
  • Blog
  • Surveys
  • NewLearner
  • Contact

Deconstructing two approaches to triangulation: The OTI and the CWP

4/12/2018

Comments

 
Art Lightstone
I know I've said it before, but teachers in Ontario are still scratching their heads over the question of how to collect evidence of "observations" and "conversations," and then integrate that evidence into a final grade for a student. 

The confusion stems from a paragraph found on page 39 of the Ministry of Education's Growing Success policy document. The paragraph states:

Evidence of student achievement for evaluation is collected over time from three different sources – observations, conversations, and student products. Using multiple sources of evidence increases the reliability and validity of the evaluation of student learning.

Sadly, teachers have not found much guidance from the Ministry on the implementation of this directive. In particular, teachers are left wondering about two critical questions:

  1. How might one actually record or collect evidence of conversations and observations?
  2. Were it to be collected, how would one use this data to inform a student's final grade? 

Perhaps even more troubling is the fact that teachers cannot find a great deal of consensus, even amongst themselves, about one crucial point. Namely, how do we actually differentiate between a "conversation," an "observation," and a "product"?

To be sure, in our own minds we are all quite clear on the distinction between all three, and we would scarcely imagine any particular debate in the matter. It is only when we subject our individual perceptions of these elements against the scrutiny of our colleagues that we come to realize that there is indeed a great variety of perspectives regarding products, conversations, and observations.

Invariably, most teachers probably like the spirit of triangulation, but just because we like it, doesn't mean we do it. In fact, a recent NewLearnerLab Twitter survey revealed that most teachers do not collect evidence of observations or conversations at all.

Personally, I have developed and experimented with two distinct triangulation systems that would both collect conversation and observation data, and then integrate this data with a student's final grade. I called my first system the Ongoing Triangulation Index, and my second system the Classwork Portfolio. 

I have discussed the logistics of each method in separate articles, but after spending a good deal of time implementing both systems in my program, I felt I was now in a better position to present a comparative review of both methods. 

Ongoing Triangulation Index 

Description:


The Ongoing Triangulation Index is a value that is based on recorded observations and conversations that are generated in class. The OTI depends on the ClassDojo application to capture observations and conversations and then curate these observations within a single record for each student.

Each observation or conversation is time-stamped, dated, and accompanied by a brief explanation of the observation and the curriculum associated with the observation. Within this index, students can accumulate both positive and negative observations, the sum of which will determine their overall Index value.

The OTI counts for 5% of the overall mark in a course. Thus, a number of OTI observations are recorded each term.

Strengths:

Allows the teacher to capture demonstrations of learning, understanding, appreciation, etc. that occur spontaneously within the classroom.

Generates a comprehensive record of observations and conversations for each student.

This record can contain detailed information regarding the specific learning goals associated with each observation. Example: “Demonstrated a deep understanding of how Keynesian fiscal policy could be used to fill a fiscal gap using the balanced budget multiplier.”  

The above record is updated instantly and can be made accessible to both parents and students.  

Challenges:

The OTI system uses the ClassDojo app, which lends itself to capturing evidence of observable behaviours. Thus, this system can potentially be criticized as assigning a mark for learning skills and/or work habits.

This system must record both negative and positive observations in order to generate a meaningful value that can then be translated into a mark.

While the OTI facilitated the recording of both conversations and observations, conversations never figured into the index value.

Classwork Portfolio                

Description:

The Classwork Portfolio uses Google Classroom, Kahoot, and Edvance (an online student information system and grade manager) to generate a portfolio of the various activities, exercises, and check-ins that are pursued in class on a day-to-day basis. Each activity is assessed and given a mark between zero and four based on relative quality.

For those in-class activities and exercises that require deeper qualitative assessment, a streamlined Canada and World Studies, 2015 Achievement Chart is used to assess the varying levels of achievement.

The Classwork Portfolio counts for 5% of the overall mark in a course. Thus, a number of portfolio marks are recorded for each student within each reporting period.  

Strengths:

The CWP allows the teacher to capture progress that occurs in class on a daily basis, and then record the observation of that progress in a way that impacts the student’s mark immediately.

Students come to realize that they can effect a positive influence over their mark during each and every class.

The CWP tends to focus more on the manifest result of a student’s learning than it does on student behaviour or learning skills. (Although, depending on one's perspective, this could be seen as a weakness.)

The teacher can include detailed information regarding the specific learning goals associated with each observation. Example: “Achieved the 80% threshold on today’s check-in on using Keynesian fiscal policy to fill a fiscal gap using the balanced budget multiplier. Silver medal... well done!”

As this information is recorded in an online grade manager, it is made accessible to parents and students immediately.

Regular check-ins are designed to more readily allow students to effect a positive influence on their mark, while limiting the potential for a negative impact.

Challenges:

The CWP system can generate a large quantity of micro-assessments that can rather quickly outpace a teacher’s ability to mark. Teachers are therefore advised to limit CWP items to activities and exercises that can be assessed rather quickly: preferably, during the class in which they are assigned.  

The CWP generates a rather disconnected record of observations and conversations for each student. This record is brought together, somewhat loosely, within our online grade manager.
Comments
comments powered by Disqus

    The New Learner Lab

    Exploring the ever-changing, often challenging, and always controversial world of teaching.

    Archives

    March 2020
    May 2019
    January 2019
    August 2018
    June 2018
    May 2018
    April 2018
    March 2018
    January 2018
    December 2017
    November 2017
    October 2017
    August 2017
    May 2017
    October 2016
    September 2016
    July 2015
    March 2015
    February 2015
    January 2015
    December 2014
    November 2014
    October 2014
    September 2014
    August 2014
    July 2014

    Categories

    All
    Activities
    Assessment
    Brain Research
    Civic Engagement
    Classroom
    Collaborative Learning
    Community
    Computers
    Educational Policy
    Edutech
    Evaluation
    Formative Assessment
    Gamification
    Logistics
    Mastery Learning
    Metacognition
    Parent Communication
    Pedagogy
    Personalized Learning
    Project Based Learning
    Psychology
    Realification
    Social Media
    Social Responsibility
    Teaching
    Teaching Strategies
    Testing
    Triangulation
    Writing

    RSS Feed

The NewLearner Lab
Real ideas for real teachers
Picture
  • Home
  • Blog
  • Surveys
  • NewLearner
  • Contact