Panning for gold: using NSRF Data Protocols to unearth hidden treasure from the muck

By Luci Englert McKean, NSRF Assistant Director and International Facilitator

Participating in NSRF data protocols reminds me of tourists panning for gold and gemstones on vacation: we’re squatting along a stream with our sieves, but it’s hard to know if that shiny thing is Fool’s Gold, a piece of common quartz, or something really valuable. Swirling around the muddy water, most of what we pull from our pan is just plain rocks, but we really don’t know, it’s not ours to judge. Other than finding something “pretty” that we want to save as a souvenir even if it has no financial value, we have to depend upon an outside expert to determine what’s what. And in the case of the data protocols, that “expert” is the person presenting the data. (Later, we’ll likely discover that the “expert” has learned a few things from our responses in the course of the protocol, but they have inside information that we do not, so they definitely are far more expert than we are, even as a collective.)

Data protocols are unique in the NSRF protocol library because the presenter is instructed to tell VERY LITTLE about what the data reports, or even what the column heads might mean, in some cases. Obviously identifying details are obscured, such as the names of students or teachers. The presenter limits the amount of data offered for review, and the written protocol asks participants to act as “data detectives” and look for details that catch their attention. Following my metaphor, they’re looking for “shiny objects” amid just a single “panful” of mud and rocks. We’re not looking at the whole stream or even a gallon of goop, but just a small selection of content that can be sifted through in a limited period of time.

Because the presenter doesn’t say what they think the data means, we aren’t front-loaded with what the “obvious outcomes” might be (“Oh no! Have our standardized scores gone down?” Or worse, “Whose job is at risk this time?”). Because we have no preconceptions, we’re free instead to look at data in an entirely different way. Without protocols, data is too easily wielded as a weapon against educators, students, or schools in general.

In NSRF data protocols, we are encouraged to look for outliers, sets, ranges, connections, possible threads, etc. While presenters often bring spreadsheets into CFG meetings, data protocols are equally effective with surveys and other non-numeric data sets. In those cases, participants might notice particular words or phrases, trends over time, differences from one cohort to another, etc.

Often, participants worry that what they contributed during the discussion rounds was minor or totally insignificant. And that’s why we ask the presenter to rejoin the group and reflect on what they learned from listening, what new perspectives they’re considering now, how the outcome of the discussion might be helpful and useful to them as presenter. The key way to measure success, as in every NSRF protocol, is what the PRESENTER thought about and learned from the experience. And while the participants may not glean particular content from data drawn from an entirely different classroom than theirs, we hope that the insights the presenter reveals at the end will at least reinforce the power of using data protocols for every member of a Critical Friends Group, and that they’ll feel more open to bringing their own data to their trusted CFG collaborators to see what treasures it contains.

Watch for another post soon with some helpful tips for coaches facilitating data protocols.

And let us know your experiences with data protocols, no matter whether you presented, coached, or “simply” participated. What treasures were you able to unearth from the muck?