Digital transformation is being heavily evangelized amongst school reformers.  Heck, I’m one of them.  As I write this I’m attending a summit hosted by Digital Promise, focused on school level digital transformation stories.  I’m a believer in the tremendous potential of personalized learning, student adaptive content, and competency based assessment systems.  I know it can accelerate student learning because I’ve seen it in practice.  But there is a three-headed monster in the closet.

In our district, we call it a unicorn, actually.  And we’re all on the hunt.

High Quality Digital Content

First is our search for high quality digital content.  A few districts develop their own online content.  As a large school district, we have some capacity to do that in-house.  Of course it takes time and resources to either train current curriculum writers or hire new curriculum specialists to develop high quality digital content.  Regardless, developing talented educators with both the content expertise and digital literacy acumen is a daunting undertaking.

Many schools come to the conclusion that they don’t have the capacity to develop their own team of digital content creators, so they buy content from external vendors.  ST Math, Lexia, Achieve 3000, and hundreds of other programs offer content across the market.  New content-driven companies come online every day.  They’re constantly looking for willing districts and schools to help them test and vet their developing content platforms.  To survive, textbook publishers have moved into digital spaces, and quality and pricing of content is as variable as it was when everything was paper-based.

In other words, the content question is not easy, and let’s not even get into the professional development required to build the capacity of staff to utilize the content once you acquire it.  Transformation to the digital environment has not made the challenge of providing high quality, rigorous content any simpler.  Perhaps it’s even more complicated.

Standards & Assessment

You might normally think that the conversation about standards should be addressed in the conversation about content.  Fair enough.  Certainly content and assessment are closely intertwined – or at least should be.  Yet content standards often come to life (or go to die) in the assessment system.  We have to ask ourselves the basic question about how we come to understand what students currently understand and what they’re ready for next.

Competency-based learning, personalized learning, and student adaptive learning systems all rely on an ability to adequately assess student performance and then marry content to the unique learning needs of each student.  To be frank, even our most talented classroom teachers struggle mightily to do this.  How do you develop a learning experience tailored to each individual student?  Many teachers skip it altogether, and instead move their entire class through a standardized scope and sequence at the same time.  Assessment for grading trumps assessment for learning, as there is little real intention of adjusting content and pedagogy based on what individual students are actually learning.  You cover the curriculum using the best teaching strategies you’ve got, and move on.  Or, you use the scale of the school to create tracks based on grouping “similar” students.  That’s not really personalizing learning either, and the structures of tracking often create inequitable barriers that make it difficult to move from one track to another.  Once a student get’s “locked in,” it becomes an identity that is hard to shake.

The hero of the story is supposed to be digital content providers, who have adopted the language of personalized learning, and proclaim to offer a student adaptive system to match.  That automatically implies a robust assessment system – a system that can accurately assess student work, provide quality formative feedback, and then provide the next level of content. This can work when the content and skills being assessed are relatively simple.  Most current digital content delivery platforms live in this space, if they address assessment at all.

But more complex thinking and understanding requires more complex assessment.  The Common Core State Standards aspire to move learning to a deeper, more rigorous place. Districts and content vendors alike must face the challenge of developing reliable and valid items that give us the learning data that we need.  We don’t typically hold vendors accountable for their assessment systems because frankly most educators aren’t comfortable with the technicalities of assessment, nor do we usually have the resources to adequately field test items or calibrate our scoring. Complex understanding is hard to measure.

It’s one thing to know what we want to teach. It’s quite another to understand what students have learned.  Good assessment is expensive and hard to find.

The Dashboard

Okay, so we know nothing is perfect.  But let’s say that over time you curate a portfolio of internally developed systems and external providers that gets you a package of high quality, rigorous content, with assessment instruments that do an arguably good job letting you know what students know and can do.

That’s really the starting point for any blended learning model, at least in terms of the digital learning component of your system.  If you’re running a rotation model in your classroom, for example, you need to have a basic degree of trust that the computer-based learning that happens when you are pulling small groups or guiding collaborative learning is providing a meaningful, standards-based learning experience for your kids.

Assuming you’ve come this far, now you want to pull all of your systems together in one place where students, parents, and teachers can easily monitor and reflect on progress and areas for growth.  A data dashboard that brings all the assessment data across content providers and assessment systems together into one gloriously accessible place.  The unicorn.

Some vendors say they already have it.  They’re lying.

Okay, maybe lying is too strong a word here.  Certainly there are dashboards available that have the ability to take data from multiple sources and put them together into one place.  You can even automate the inputs.  But some content and assessment vendors don’t want to play.  Their programs offer a closed system that doesn’t want to talk to an external dashboard.  In fact, I think the higher the quality of the content, the more likely the vendor won’t want to share their data.  You might think that a district owns the data that is generated by their own students, and could ask for access to that data however they want it.  Not the case.  There is little financial incentive for established vendors with internal progress data dashboards to make their systems compatible.  And yes, I’ve sat in dozens of meetings and conference sessions where the proposed solution is to band together as districts to force vendors to meet our request for data output or refuse patronage.  I call that the Walmart strategy – to force our suppliers to comply or walk away from the table.  The problem is districts are desperate for the content and vendors know it.

The most promising systems have found ways to patch their systems together.  But it’s usually an analog solution.  At our personalized learning school Advanced Learning Academy, for example, students do the heavy lifting and use the transfer of data from disparate locations into a common digital dashboard as an opportunity for reflection and dialogue.  But the dashboard can’t talk back to the systems.  It’s just a container for data.  And it’s tremendous work for the teachers to manage all of the data transfer.

Bringing It All Together

We know what the unicorn would look like if we found it.  High quality, engaging content across subjects with a trustworthy assessment system that all feeds into a dynamic data dashboard. Opportunities for peer feedback and collaboration would permeate the system.  Access to expert insight would be no further than the touch of a button.

I guess that true disruptive innovations don’t come neatly packaged in a box.  It’s the organizational pursuit that builds our capacity and refines our abilities.  Perhaps we should just embrace the hunt.