">EnglinConsulting LLC

Measure More than Inputs

This year we’ve been obsessed, along with many others in the non-profit and advocacy communications space, with getting better at measurement and evaluation. For us, that’s meant helping our clients move away from measuring only inputs to measuring outputs and their impact on outcomes. When it comes to changing the world, it’s incredibly tough to figure out how to measure outcomes in ways that are constructive in terms of making decisions for future efforts.

Elections are won and lost based on a wide variety of factors, many if not most of them impervious to the efforts of individual organizations trying to influence elections. Major social problems – hunger, illiteracy, unaffordable health care, joblessness, etc. – are complex, long term, and decidedly not the responsibility of a single organization’s or coalition’s efforts. Legislating takes a long time and is a messy undertaking under the best of circumstances – advocacy and lobbying can’t change that reality.

So, what should organizations, coalitions, and the sector at large measure in the context of unrelenting complexity, widely dispersed responsibility, and, often, a virtual blackout when it comes to timely, reliable data? We’re working on some larger pieces on this for the coming weeks, but in the meantime, three things to get the ball rolling:

1) Measure progress against an articulated strategy.
If your issue depends on key legislators taking a leadership role and you’ve heard from their staffs that they fear retribution from their constituents, then it makes sense to generate support – and grassroots actions demonstrating that support – in those legislators’ districts. That’s your strategy.

The ultimate measurement is whether you successfully swayed the legislators to take the action you wanted them to take, but that might be a larger ballgame than your at-bats can reasonably deliver. You can measure how well you delivered on your strategy. How many new advocates in the district did you move to action, how many in-district alliances did you build, how many and which grasstops influencers did you bring on board, how many of the legislators’ donors did you bring on board, etc.?

“Was your strategy the right one?” is an important question that’s hard to measure. Knowing with certainty how well you delivered against that strategy is vital, and you can and should be measuring it.

2) Measure capacity building.
Changing the world is a long term endeavor. After you win the current battle, the next one’s probably coming up right behind it and you’re more likely to win that battle if you’re expanding on capacity you’ve already built and tested. Identify the two or three critical capacities you’d like to see extended, and measure how well your efforts build on them. Examples of important capacity questions: how many partners did you coordinate with and what types of coordination happened, how many advocates did you move up the engagement ladder, how many new relationships did you build, how did your structures for communications or accountability work?

3) Measure learning.
Particularly when an organization is taking on something new – new partnerships, new tactics, new tools, new messages – lessons learned are a critical and measurable outcome. When you tested message A versus message B to your email list, which one worked better for which segments? Are there any larger lessons learned from that? Maybe the lesson is just that it’s important to test, because the results were counter to your expectations. Did you bring on new staff to manage elements of a campaign, or assign existing staff new roles? What worked and what didn’t about the arrangement?

It’s easiest to measure inputs: how many emails did we send, how many people signed the petition, etc. We’d never argue that organizations should stop measuring those inputs, but that outputs and outcomes are important, too. In our view, changing the world demands it.