NextBillion - An Initiative of the William Davidson Institute at the University of Michigan

Hal Varian, chief economist at Google, has said that the sexy job in the next 10 years will be statisticians. It was clear statistics, data, and metrics – in this context to measure, report, and track outputs and eventually outcomes of our work – were indeed sexy topics throughout the first two days of SoCap.

Believe it. Nearly every panel in some way brought up the need to increase transparency and credibility in how organizations define, track, and report on the social and environmental performance of their investments – whether they be in nonprofits or for-profits. When some of the leaders of this metrics movement gathered for a panel to discuss our emerging solution, the people came in droves. The room, set to fit 50-60, had at least twice that many people in the audience when the session started with managing directors at foundations, social enterprise founders, and nonprofit executive directors watching from the back without seats.

The panel they came to see included Brad Presner of Acumen Fund, Steve Wright from Salesforce.com Foundation, Kevin Starr of Mulago Foundation, Laura Callanan of McKinsey, and Margot Brandenburg of Rockefeller Foundation. Lucy Bernholtz of Blueprint R&D was charged with the task of letting each speaker talk about their role in the evolving impact metrics field (while limiting long-winded self-promotion) and leaving enough time for a vibrant Q&A (while cutting off speeches masked as questions). She led by calling metrics “the carbon in the ecosystem of impact investing space” (oxygen being the policy), before passing it off to her panelists.

Presner talked about Acumen’s portfolio data management system, called Pulse, that helps impact investors manage their performance more effectively and efficiently. Built with Google and Salesforce, Pulse was created at first to replace spreadsheets as a better way to track impact metrics over time across the Acumen portfolio from hospitals in India to bed-net manufacturers in Tanzania.

While the system was valuable internally, what was missing was a way of seeing how the Acumen portfolio was benchmarking vs. peers. To allow for side-by-side data management, they encouraged others to track their data using the Pulse, which is now available for free download from Salesforce.com. Presner told the crowd that leaders of the field who had spoken throughout the day, including Root Capital and E+Co, were early adopters.

Pulse is fully integrated with Impact Reporting and Investment Standards (IRIS), the project Margot Brandenburg of the Rockefeller Foundation has been working on with B Lab, Acumen Fund, PricewatershouseCoopers, Deloitte, and others to build a common set of definitions used to measure social and environmental impact of investments.

When Rockefeller interviewed impact investors across sectors at the launch of their initiative on Impact Investing last November, Brandenburg explained that reoccurring issue that investors highlighted that would enable them to implement more money, more efficiently in impact investments what as consistent set of social metrics and common definitions for nonfinancial terms. A common taxonomy was needed to as the base for a number of projects – from a single rating system for investors to a market intelligence database. Most importantly Brandenburg highlighted, was this taxonomy needed to be consensus-driven and adaptable. IRIS and Pulse have both been driven by a transparent and inclusive governing process that has been driven by feedback from the actual users of this data from the start.

Wright from Salesforce talked about his project to help nonprofits to use effective data management tools to know whether or not their projects are working. Wright noted that “improving efficacy of the nonprofit sector, starts from driving efficiency at the single organizational level.” Callanan at McKinsey is similarly trying to help individual organizations more efficiently track data but building a database called TRASI (Tools and Resouces for Assessing Social Impact). TRASI, due out this fall, is an open clearinghouse for tools that is easily searchable and filterable. Starr, from Mulago, said current attempts to measure impact are “too complicated and not rigorous enough.

The heat in the room rose as the questions came in from the audience. The questions and answers dealt with the obvious tensions in the room from those being rated and those doing the rating, from those seeking funding to those with dollars to disperse, from those in offices in Manhattan to those running social enterprises in Rwanda.

When Starr was questioned about the “slippery slope” of trying to measure attribution from an organizations work, he responded, “Why would you fund something that you don’t think causes outcomes?” This followed with a headed discussion distinction between measuring “outputs” rather than “outcomes.” While some advocated random controlled experiments to drive to drive to better answers, others questioned the efficacy of these studies.

David Bonbright pushed the group to instead think about a range of methodologies that is triangulating to causation. Brandenburg talked about the need to understand the nuance of data and circumstances on the ground, while also delivering data at the appropriate level for different audiences, including investors who don’t want social science data. She noted, “We don’t want to abandon obligation for intellectual honesty, but need to make data useful. We can’t overwhelm people so they throw up hands.” In other words, if we want the investors to pour in the private capital money that is needed to scale market-based solutions, we need to deliver the breadth and depth of data that investors need.

Other questions focused the cost and feasibility of evaluation for bootstrapped organizations. Callahan highlighted how the TRASI tool will let organizations find the right studies for their cost and stage of their growth – and they need to “apply rigor within reason.” Panelists acknowledged the costs of reporting, but talked about various ways to minimize cost by working as sector to do the heavy lifting on technology and taxonomy tools. By unifying a network of funders along a common set of reports and metrics, organizations reporting burden can move down and ultimately more dollars can drive to high performing organizations.

The panel responded to the related concern that metrics can be used as a tool to de-invest in organizations. Wright responded with the broader comment that regardless of sector or geography, metrics will help investees become better at measuring what they do and in the end more effective at driving outcomes. Money will flow more efficiently to projects that are working. The danger of course is we are measuring and therefore rewarding the wrong set of metrics. Presner and Acumen Fund kept this concern front and center when rolling out Pulse to their investees, “Organizations will see it as a cost if it is not helpful to them to achieve their mission. They won’t get a benefit if what is being measured isn’t what is applicable to them.”

As Rob Katz predicted before the conference, this was not a 101 panel that stayed at the level of introduction or hyperbole. Metrics are sexy, but metrics are contentious. It was obvious that we still have a long way to go to drive coherence to the marketplace and consensus to drive scale. Ultimately everyone in the room wants to drive more dollars to more solutions to fight big world problems and see the world we want to see. As Andrew Kassoy of B-Lab so eloquently put it later that afternoon, “In this space, we are so naturally fragmented by people’s passions. We need to put aside some agenda for the greater good and work together.” Amen.