Why ed tech’s 95% problem doesn’t apply to digital learning

The scene:       A doctor’s office

Patient:           Doctor, my new medication doesn’t seem to be working

Doctor, looking at her chart:
Hmm, the prescription is for one dose twice a day. Remind me when you started that?
 
Patient:           Oh, I never did that. I just take it once a week

Doctor:            (Blank stare)

Patient:           I’m really frustrated that it doesn’t work.

That is the medical version of ed tech’s 95% problem, from the perspective of ed tech providers and advocates. Our stuff works when used as intended, they say; the problem is that it’s often not used correctly.
 
Of course, that’s not how teachers and school leaders would describe it. Instead, they talk about how technology products are built for unusually motivated students, and how the products often don't reflect an understanding of how teachers really teach, or how the use of the product would fit into typical class time. Or, to use one often quoted line, the students they benefit the most, are the ones who least need the help.
 
It’s an issue that is getting increased attention, and rightly so. But it doesn’t apply in most online and hybrid schools and courses, and that’s important for digital learning advocates to understand as the issue gains salience.

The 95% problem explained
Writing in Education Next, Laurence Holt explains what he calls the 5% problem (you can call this the 5% problem or the 95% problem, because math). He delves into math products including DreamBox, i-Ready, and Khan Academy. All of these programs have reported impressive results that improve student outcomes. Why then, if these products are available and effective, are math scores in US schools not improving?
 
“A clue is in those wiggle words “students who used the program as recommended” [show improved outcomes]. Just how many students do use these programs as recommended—at least 30 minutes per week in the case of Khan Academy? The answer is usually buried in a footnote, if it’s reported at all. In the case of the Khan study, it is 4.7 percent of students. The percentage of students using the other products as prescribed is similarly low.
 
Imagine a doctor prescribing a sophisticated new drug to 100 patients and finding 95 of them didn’t take it as prescribed. That is the situation with many online math interventions in K–12 education today. They are a solution for the 5 percent. The other 95 percent see minimal gains, if any.
 
Worse, some studies report that the 5 percent who do see results skew towards higher income, higher performing students.”

Dan Meyer picks up the thread from a tech-skeptic perspective in The Kids That Edtech Writes Off. He focuses on the 95% of students who don’t appear well served by many tech products, delving into what these students, and their teachers, need. He points out that many tech developers were among the 5% of highly motivated students, and were disproportionally surrounded by other such students, so they have little exposure to and understanding of the other 95%. His overall argument is that technology companies need to better understand the wide range of students in terms of motivation, and the real-world ways in which teachers engage all students.

Writing in Forbes, Michael Horn expands on these ideas, writing that “EdTech must pay attention to the learning model.”
 
“it’s not the presence of technology alone that will move learning. It’s the use of technology to support a novel model of learning that will move the needle. What matters most is the model.
 
A central reason why technology isn’t a silver bullet in education is that when it’s crammed into the existing classroom model, at its best it can only serve as an additional resource to bolster that model’s existing processes and priorities. That means it can make an operation more efficient or allow it to take on additional tasks, but it can’t reinvent the model in and of itself. It also means that in many cases it will conflict with the organization’s processes and priorities and therefore go largely unused.”
 
Why digital learning succeeds while ed tech struggles
Horn gets at one of reasons why the 95% problem doesn’t apply to online, blended, and hybrid schools.
 
These schools don’t start with a traditional classroom model and add technology. They also don’t start with technology (at least not the good schools). Instead, they start with ideas about how students and teachers can flourish when instruction is freed from the usual time and space constraints. Then, they figure out what combination of online content, tools, and resources can be used by teachers and other professional adults, to meet the needs of a wide range of students.
 
A second reason that digital learning is different is that in nearly all cases, students and parents are choosing the school or course. They might not like every technology being used, but overall, they like the school’s approach, or they would switch schools. That’s not the situation with the most common use cases in ed tech, in which technology is being added to a traditional public school that students are most likely attending due to geographic proximity.
 
Why it matters
This distinction matters because the online, blended, and hybrid learning world is still new to so many people. The uninitiated often believe that these schools start with technology and are built on tech, instead of starting with new models of instruction and how to meet the needs of students who were not well served in conventional schools. That mistaken starting point then leads some people to think that technology problems like the 95% issue must be far greater in digital schools than in traditional schools. As these tech-related issues become more widely understood, the misperception impacts online and hybrid schools.
 
But that misperception is exactly backwards. By and large, digital schools don’t have these problems—because they don’t start with technology, and they do start with designing for students who have chosen a new learning option.