Improving Teacher Effectiveness through Competency-Based Classroom Observations tool

Default image, no image supplied by the user.

Teachers are the single most powerful resource in a classroom, especially in classrooms where resources are limited. Often the teacher is the only resource in the room, and learning solely depends on the teacher’s instruction. Ensuring teachers deliver high quality lessons is therefore central to improving student-learning outcomes. Finding out what makes effective teaching is the key to driving successful learning, and classroom observations provide an opportunity to identify best practice, share innovations, and inform teachers’ ongoing professional development.

At WUSC, a competency-based classroom observation tool has been developed to monitor, evaluate and inform the effectiveness of teachers delivering remedial classes. The development of the tool was an iterative process, which required close coordination between the monitoring and evaluation team, the project implementation team, and, most importantly, the remedial teachers and classroom observers. The tool is a guide to support the observation of various aspects of a remedial class, including lesson structure, pace, planning, teaching methodologies, use of resources, classroom routines, classroom management, assessment for learning, and student engagement. Feedback sessions following each observation are critical to support and guide teachers in areas that need improvement.

 

Default image, no image supplied by the user.

In order to create an effective classroom observation tool, the WUSC team took the following six steps:

Step 1: Talk to the Teacher First!
Beneficiary led interventions are crucial in the process of developing an instrument that can be useful to them. In fact, it is ideal to find out from the person who is being observed what makes sense to them in the observation process. We held meetings with remedial teachers and Community Mobilizers in Kakuma and Dadaab on ‘What makes a great remedial teacher?’ which included a brainstorming activity to explore all the things that a great remedial teacher does to drive student learning. When asked to identify the qualities of a great remedial teacher, the field teams came up with the following:
 

Default image, no image supplied by the user.

Step 2: Building the Classroom Observation Tool
The WUSC M&E team, with support from the AIR team, reviewed an existing classroom observation tool that was used in previous projects by WUSC. The primary advantage of using an existing observation tool is that it saves time and resources that would otherwise be required into developing an instrument with minimal levels of reliability and validity for predicting outcomes that we are interested to find. Using the information gained in the workshop with the teachers and community mobilisers, the tool was adapted to fit the needs for the current project. First, we categorized the responses of what makes a great remedial teacher to come up with our observation framework. We then reviewed existing tools, and decided that have a 4-point scale and differentiated criteria against each competency would help to ensure that observations were as objective as possible.
 
Step 3: Moving from Paper to Digital Data Collection
WUSC has collaborated with UNHCR to use their Kobo-Tool-box, an online platform that builds questionnaires in a digital form, and includes basic analysis in the form of dashboards. This is a unique approach in classroom observations, as it will enable collating real time data on the classroom interactions as well as identify teaching practices that are both innovative or that require additional training. The classroom observation process will contribute immensely towards generation of data on how remedial learning takes place. Thus, timely relay of the data will be crucial for the program. As well, the Kobo tool will allow the observer to recall past observations sessions that they have had with the teacher, and identify areas that have been improved, and document what needs to be emphasized in the subsequent trainings.
This would allow the observer to have a quick coaching with the teacher directly after an observation and together they could come up with a better lesson plan for the following week. This information is also important for the WUSC evaluation team to support the programing aspects that are evidence based.

Default image, no image supplied by the user.

Step 4:  Test the Tool
An essential component of observing classrooms is ensuring that the observer understands how to record his observations to facilitate comparisons across observers and classrooms. This is done by having clear instructions for use. This process helps address how different observers are likely to use different methods, which severely limits the potential for agreement between observers when making ratings, and thus impedes system-wide applicability.

There are three main components of standardization that we considered evaluating the classroom observation instrument:
1. Training protocol; training on how to go about using the tool.
2. Observation protocol; guidelines on what we want to observe and the sequence to be followed in the observation instrument
3. Scoring options; what weighting is given in each of the sections on the observation? This is crucial in helping us know how to evaluate the performance of the teacher and what are some of the areas that need to be focused for mentorship and coaching. 

This is an important step to pilot an instrument to find out what works and what needs to be changed. WUSC did pre-test the instrument in a live remedial class where we regrouped again afterwards to consolidate our notes and share lessons learnt.  This process confirmed that having guiding descriptors against each observation point (competency), using a 4-point scale would be beneficial and would help to standardize observations. Having guiding descriptors also helps to build the observer and project based education officers capacity to effectively conduct lesson observations. The guidance descriptors will also help to inform the teacher’s professional development, by giving a roadmap for progression against each competency.

Step 5:  Rolling Out the Observations
For the purpose of the initial classroom observations, the project team decided to hire 1 expert classroom observer in Dadaab and Kakuma who has experience in curriculum development and large classroom management. Project staff who will then be able to undertake the observations themselves in the future will shadow the consultant.
The consultant will aim to have at least one classroom observation for each remedial class in the programme. With 10 remedial classes running (for class 7&8 high achieving learners) across five centers this will take approximately 3 months. This will culminate in a 3-day training for all remedial teachers to address challenges as well as promising practices on their observations of the different teaching styles they saw over the three months.
In future, the tool will continue to be used and refined by project support staff, who will do frequent observations on a continuous basis, so that the teachers can have support in managing their classrooms and teaching engaging content

Step 6: Next steps; feedback loops for continuous improvement
As the same classroom observation instrument is being used in Kakuma and Dadaab, a very full picture will be analyzed of the different teaching styles across two very different contexts. The two expert classroom observers will also have a joint debrief with the WUSC team to understand similarities in the programme and challenges faced in each location. The WUSC team believes that basing both teacher training and classroom observations on a competency-based framework supports the notion that changing teachers’ values and attitudes will lead to changes in their behavior and practice, which in turn influences students’ knowledge, attitudes and behaviors and ultimately impacts student outcomes as outlined in schema below:

Default image, no image supplied by the user.

It is important to note that the goals of conducting observations is not solely for gathering information on the quality of classroom processes but use that information to help teachers improve their practices (and, eventually, student outcomes) as envisioned in the figure above. The information collected will be translated into professional development planning moving forward. The Evaluation team expects to identify the norms as well as threshold scores on various levels of practice that will be extremely useful to have. Though the classroom observation tool was designed to improve the remedial classes on an ongoing basis, it will also address the overall research question for the HEA impact evaluation that is ongoing over the next two years: “to what extend does the remedial program impact the girls’ school attendance, performance and attendance in grades 7 and 8 in Kakuma and Dadaab refugee camps,” and specifically, investigate the following characteristics of the remedial classes:

  1. Do remedial teachers apply the pedagogical techniques learned in the training?
  2. Do remedial teachers effectively engage with girls in the classroom?
  3. Do remedial teachers cover the expected content topics in the classroom?
  4. Do remedial girls participate and engage in the remedial classroom?

 
These questions will be crucial to inform the process of implementation and scalability of the programme.

Through this inclusive process, with participation from all stakeholders (including the girls in the remedial classes themselves!) the WUSC team was able to develop a classroom observation tool that was tailored to the needs of the remedial programme. In a similar vein, the project will also roll out a tool to improve the way the Community Mobilizers are able to track and look at absenteeism from the classes, and assist them in their community outreach work.

This will be outlined in our next blog: Watch this space!
 

Discuss