How does the WhatWorked evidence base work?
We work with teachers to test interventions they have developed for their pupils and then release these onto our website. The interventions are free for other teachers to use and are structured so they are easy for you to test if they have a positive impact in your classroom. The anonymous data is then aggregated to create a live evidence base to help inform teachers to decide which interventions will be best to use for your pupils.
Our evidence model uses a prospective cumulative meta-analysis (PCM) approach, which allows small scale mini-trials to be set up and run in schools. The key to this is the creation of a trial protocol to support teachers set up and implement an intervention in their school. The protocol is to ensure the same thing is tested in each setting. As the same research design, eligibility criteria for learners, resources and assessments are used, we are able to aggregate the results together to get a bigger picture of the results.
Figure 1 shows the model we use for setting up the PCM for a particular intervention.
The use of a protocol allows teachers to set up an intervention so that they can evaluate if this has been effective in their own school. Then the results are collated by our stats wizards to aggregate the results in the PCM. This is a graphical method to show each the effect size for each cohort of schools and also the cumulative effect size for the intervention.
The first example of this approach investigated the effectiveness of online peer tutoring using year 8 pupils to peer teach year 6 pupils. The trial compared 1:2 and 1:4 online group learning with 1:1 to demonstrate the use of a PCM.
Figure 2 shows the first example of this approach to a cumulative evidence base in education. Now, we are looking to support teachers and schools to test if the model can be scaled to form a global evidence base for education.
The evidence base uses a range of different research designs (not only trials) but the design used will impact the strength of the evidence of the intervention.