The Lean Data Approach: Measuring Impact

The Lean Data Approach: Measuring Impact

By Kate Reid

We’ve discussed Lean Experimentation Methodology in previous blogs, both as an intro to the methodology and to introduce it as the strategy that The Inspiring Enterprise employs to help efficiently launch startups focused on solving a social or environmental issue. Let’s talk now about the Lean Data Approach, drawing on points from a recent article from Stanford’s Social Innovation Review. Piggy backing off the major principles of Lean Experimentation, lean data can help social enterprises tackle one of the most difficult aspects of this area – impact measurement.

It’s a common problem for organizations with a social mission, the inability to gather usable impact data. Why? Impact data is complex – how can you accurately measure social good? The result is that for organizations seeking to gather useful impact data, the process is normally time-consuming and costly. Employing a Lean Data approach can alleviate some of these issues. The approach relies on two key aspects:

  1. A move away from a mindset of reporting and compliance and towards creating value for an organization and its customers.
  1. The use of data collection methods and technologies that favor efficiency and speed while maintaining accuracy

Impact Measurement in the Investing Field

According to a survey conducted by JPMorgan Chase and the Global Impact Investing Network, 95% of impact investors say that they measure and report on the social impact of their investments. Despite this figure, the data that is usually gathered is far from thorough, as most impact reporting targets output figures like, “number of jobs created or lives reached”. Investors (and entrepreneurs) rarely look to understand on a fundamental level how customers experience services/goods that an organization provides, and data rarely takes into account demographic factors (such as income level or ethnicity) that are vital in understanding if a social enterprise is reaching those who most need their services/goods. While there has been a move in the impact-investing field towards building standardized performance metrics (IRIS – the Impact Reporting and Investment Standards) impact investors still tend to collect data on just financial and operational metrics, rarely devoting resources to tracking social metrics.

Social Enterprises: Problems with Traditional Impact Measurement

Because of the unique attributes of social enterprises, neither standard business metrics nor M&E’s (monitoring and evaluation methods commonly used by international aid agencies for impact measurement) are conducive to gathering valuable impact data for social enterprises. Standard business metrics can tell us how an organization is doing financially, but not if they are making an impact from a social standpoint. M&E’s and RCT’s (randomized control trials), which are often considered the gold standards for impact measurement are costly, can take years to complete, and require significant previous knowledge and experience. These are aspects that most start-up enterprises/social enterprises are unable to meet, as they typically operate with: dynamic environments, financial constraints, limited human capital, and limited or poor data management systems.

These issues are leading to an accountability gap for social enterprises. Organizations often focus only on collecting data that meets upward accountability (data needed for their investors) while neglecting downward accountability (making sure the organization is using data to improve the lives of their customers/beneficiaries).

How Lean Data Works

 Social enterprises and their investors require an approach to impact measurement that combats the issues mentioned above. Author’s Dichter, Adams, and Ebrahim (of the SSIR article mentioned at the start of this blog) offer the acronym BUILD, to remember the core properties of an effective approach:

Bottom-up: Nurture the habit of listening to customers in order to provide actionable insight on their needs and interests. 

Useful: Yield data that is of sufficient quality to support decision-making.

Iterative: Allow for learning, adaptation, and replication.

Light-touch: Use low-cost tools and technologies that require a minimal investment of time and money.

Dynamic: Enable rapid data collection within a fast-changing environment.

Two key developments make The Lean Data approach feasible: the prevalence of mobile phones, and new and improved customer feedback tools. Mobile technology makes it easy to communicate cheaply and effectively even in extremely rural areas, and customer feedback tools make it easy to gather data without asking for significant time/attention of customers.

By design, The Lean Data approach is clear and simple, and typically once an organization has gone through the process once they can easily repeat it or augment it to suit additional needs. See the below graphic for the five phases of a Lean Data project:

Lean Data

  1. Impact Question: In the first phase of the Lean Data Approach leaders of an organization will define the specific thesis they wish to test, gathering feedback from customers/beneficiaries on the impact of the product/service.
  1. Design: In the design phase, leaders will identify an enabling technology (such as SMS) and enabling instrument (such as a pre-tested survey) that they will deploy in their project.
  1. Execution: In this phase the organization will develop a concrete plan for gathering data from people in it’s target market. During this phase the project managers will train staff members on how to use the enabling technology and instrument, and test their plan via rapid prototyping.
  1. Learning: After development comes the learning phase, where an organization will analyze the data and take away any lesson learned, then determine how to apply them to company operations.
  1. Action: In this last phase, leaders will determine how to use the knowledge they acquired from their project, and decide how (and whether) to apply the lean data approach to other impact questions.

Concluding Thoughts on the Lean Data Approach

Case studies of organizations who have used The Lean Data process prove that it can yield meaningful and timely results. The SSIR article on the Lean Data Approach cites that companies using this process were able to conduct projects quickly and at low cost (direct cost per engagement ranging from $500-$15,000, and the duration of data collection ranging from 10 days to 4 months).

Of course, as with any process, things don’t always go smoothly. The approach can require iteration to ensure data quality, and The Lean Data approach is still in the early days of development. The results though, warrant giving the approach a go. The Lean Data Approach offers the power to close the accountability gap and for social enterprises, beget a culture of measurement – one where gathering data becomes more than just an obligation to their funders, and instead a way to immediately inform strategic decisions and help measure and deliver social good.

For more information on the Lean Data Process visit: http://ssir.org/articles/entry/the_power_of_lean_data

Written by Kate Reid

Leave a Reply

Your email address will not be published. Required fields are marked *