Does blended work? – The 4Ps of evaluating your blended program’s effectiveness
by Kim Loomis
I often get asked about the effectiveness of blended learning. Educators, from classroom teachers and building administrators, to district curriculum directors and superintendents want to know, “Does blended learning work?” What they are really asking is, “Can I be confident that adopting blended classrooms and/or digital curriculum will improve student learning?” I don’t blame them. Budgets are tight. Time is precious (including addressing professional learning needs). Yet, no one would ever ask, “Is learning in a K-12 environment actually effective?” However, in today’s highly technological world, digital learning is a life skill that schools must adopt. So, the better questions are, “What can we do to teach effectively in a blended learning environment?” and “How do we know it’s working?” The answers will be subject to the interplay of media, method, and modality (great video!).
When seeking a return on investment (ROI) for blended learning, one must first have a vision for what blended classrooms should look, act, and feel like. Like the multifaceted education system and the numerous teaching strategies deployed in countless classrooms across the nation, the answer lies in this very same systems complexity and variability in performance. The Carnegie Foundation reminds us that it takes a multi-layered approach of using improvement science to accelerate learning and address problems of practice.
When seeking an answer to “How do we know blended learning is working,” consider breaking the measures of evaluating blended learning effectiveness into four different data sources, which I like to call the 4Ps:
Performance (outcome measure) – Are student outcomes improving (and for whom)? Student achievement data.
Pedagogy – (process measure) Is classroom instruction changing (and how)? Classroom walkthrough data using an in-house created rubric focused on self-selected priority practices and instructional strategies.
Perception – (balancing measure) How are students and teachers feeling about the changes? Survey data from in-house created tools for staff and stakeholders geared towards important implementation factors.
Partnerships – (balancing measure) How often is digital content being used? Usage data from vendor and other digital content partners.
The question of “does blended learning work?” can only be answered by looking at various complex variables. Beyond student achievement are variables in degrees of implementation, change in practice, and meeting expectations (e.g. priority practices, such as small groups or student choice). Blended learning has numerous models and abundant classroom implementation strategies; this is why as a school/district it is important that you identify priority practices, create a classroom walkthrough tool, conduct stakeholder surveys, and monitor digital content usage to ensure that your vision of blended learning is working.
To determine the success of your blended learning program, gather data, and analyze the results in the 4Ps: 1) Performance – achievement data, 2) Pedagogy – walkthrough data, 3) Perception – survey data, and 4) Partnerships – digital content usage data. Like the blended learning classroom implementation, the evaluation process will require new thinking, a change in practice, and continuous data analysis. And as always, iteration is the key.
About the Author
Kim Loomis has 20 years of experience in digital learning; she recently retired as the Director of Digital Learning in Nevada’s Clark County School District, the 5th largest in the nation, where she provided leadership in the growth of digital learning in the K-12 setting, from teacher training, school admin consultation, secondary course development, software and systems contracting, to designing deployment models.