Impact Management Practice

01. Could you please elaborate on your impact framework which captures – direct and sector level impact, and what are the specific components under these broad headings which you capture impact data on?

We measure impact at two levels:

Direct impact: Impact of the products and services offered by our investees or grantees on their own customers or beneficiaries.

Sector impact: Catalytic change spurred at the sector level, beyond the direct customers of our investees e.g., we are an early-stage investor in EdTech companies with live learning models. Could these investments create a demonstration effect leading to a paradigm shift in how EdTech is deployed by others beyond our investees (e.g., governments).

Within Direct impact, we assess three main metrics:

Reach i.e., cumulative number of beneficiaries, or end-customers, a metric tracked and reported by nearly every investee.

Depth i.e., quality of life improvement. We measure this through customer surveys that determine the % of end-customers who say their life ‘very much’ improved by the product or service of our investee.

Inclusion i.e., socio-economic status of end-customers. We measure this through customer surveys that determine the % of end-customers who belong to what we call as the ‘Next Half Billion’, typically consumers who earn less than Rs. 21,000 per household per month (our analysis shows this is ~60% of India).

Sector Impact is harder to measure, but we now have a good handle on how to do this. We measure three things:

Imitators We believe initial pioneers can spark changes in the way business is done and products are offered by others. So we measure ‘number of organisations that have adapted the business model of our investees’.

Follow-on capital raised since ONI funding, indicative of the interest that our investees have crowded in from other funders.

Contribution to impactful policies i.e., the extent to which the work of our investees is contributing to positive policy changes in the sector. We have developed a 5-level framework to measurer this.

In addition to measuring these six metrics, we also track specific impact indicators that are unique to each investee, which we identify at the time of investment You can read more about our framework and the targets we have set here. Finally, as a guiding light, we also track how income for the Next Half Billion (our target customer, falling withing the bottom 60% of India’s income pyramid) is growing compared to GDP per capita.

02. ONI uses a rating framework to assess their investee performance. Basis the framework, how do you decide which enterprises qualify to be ‘impact stars’ and which don’t?

Based on the framework above, we are able to classify investees into three groups:

Runway/taxiing: Typically, younger investees in our portfolio who are still to reach a superior level of impact

Cruising altitude: Investees who have achieved some degree of impact, and have great potential to advance this further

Stratosphere: Investees who demonstrate significant direct or sector impact

This provides a ‘cheat sheet’ of impact performance at a portfolio level and also some answers to where we need to support investees to do better.We have made this part and parcel of our business, use this in our portfolio reviews, along with a review of financial performance.

03. How often does ONI collect data on various parameters of their impact framework, and what tools are used to collect them? Is the data collected by ONI, the investee company, or a third party? Do we see cost of impact measurement and management to be on a major issue particularly in India, where the sector is still emerging in terms of size and scale ?

We collect data on ‘reach’ on a quarterly basis and all other impact data on an annual basis. We collect data directly from investees, but we also partner with external organisations to conduct surveys that capture customer or beneficiary insights

When we think about costs of impact measurement and management, first and foremost, we believe that a paradigm shift is required from the current perception of impact measurement as difficult, burdensome and costly, towards the recognition that it helps entrepreneurs do even better in serving their customers and creating impact. It’s a value conversation, not a cost conversation.

There are now several innovative technology systems and lean approaches that are, in fact, making data collection and aggregation more affordable. Phone-calls, IVR and online data collection methods can often be quite cost-effective.

The key is to create value for the investees, and not treat this as merely a reporting exercise. At ONI, we do it in a way that’s directly helpful to our investees. For instance, each organisation who signs up to participate in customer surveys gets to select some questions customised to their operating model; at the end of the exercise, each organisation gets a customised report and a call with our survey partners to dive into the learnings. Our entrepreneurs have found tremendous value in these customer insights!

04. ONI has a diverse portfolio, with investments in sectors like Digital Identity, Education, Emerging Tech, Financial Inclusion, Governance & Citizen Engagement and Property Rights. Is the impact measurement framework modified or adapted to capture data for various sectors? And if so, is it challenging to do so?

As we mentioned in response to Q1, we have a strong set of six metrics that we capture across the portfolio; these help us assess the core of our impact thesis as a firm, across both for-profit companies and non-profit organisations.

In addition to tracking these six metrics, we also track specific impact indicators that are unique to each investee, which we identify at the time of investment. The investment thesis and specific metrics are a key discussion for us before we make the investment decision, and at any follow-on funding discussions.

This is part of our operating model as a firm because we recognise that unlike financial returns, ‘social returns’ do not have a natural ‘common currency’ for measurement, and we therefore have to take a more nuanced approach.

05. Going beyond impact measurement, what are the governance and oversight mechanisms to manage impact at ONI : In terms of impact reviews at an enterprise/portfolio level, tracking delivery of impact performance parameters and so on ?

We take an impact-oriented lens throughout the investment process.

At the outset, there is a ‘strategic fit discussion’ that occurs when a deal is first considered, long before it is brought to the investment committee (IC) for a decision. For all our investments, we consider the impact on the Next Half Billion (NHB), whom ONI considers as our ‘target customer’. This is an important frame for us, and all our portfolio organisations must have a pathway to impact the NHB.

The investment thesis and specific metrics are a key discussion for us before we make the investment decision, and at any follow-on funding discussions.

Once the IC approves the investment, the investment team and the Strategy & Impact (S&I) team agrees on an impact management and measurement plan before funds are disbursed.

Impact reviews are then conducted through-out the lifecycle of the investment – regularly by investment leads, on a quarterly basis by our investment management team, and annually by the S&I team.

06. What are your thoughts on tying employee compensation (portfolio management team) with success of the investee companies on their impact and other performances? Do you believe that holding employees responsible will improve ‘social and environmental’ impact performance of the impact enterprise?

At ONI, employee compensation is directly linked to performance on social impact as well as financial returns. This is done on an annual basis and at a fund level.

Beyond compensation, we try to seed a culture of impact orientation in the organization, right from hiring to organisational strategy.

07. What are your views on responsible exits ? what processes do organizations like yours which build strong impact centred enterprises have in terms of ensure that exits are almost managed from an impact perspective such the new owners to carry on with the impact journey ?

Setting investees towards a path to continued impact beyond the holding duration of an impact investor is an important question. This is in fact one of the key questions many firms are grappling with as they get to the next frontier of impact creation and measurement.

There is some good early thinking around this, e.g., the Global Impact Investing Network’s (GIIN) report, ‘Lasting Impact: The Need for Responsible Exits’, and the Consultative Group to Assist the Poor (CGAP) and the Center for Financial Inclusion (CFI)’s paper, 'The Art of the Responsible Exit in Microfinance Equity Sales’.

In practice, there are many considerations investment firms have to keep in mind as they navigate exits - for instance, ownership stake, investment duration and vintage, promoter preferences, co-investor motivations, number of potential buyers or realistic exit opportunities. So the question of responsible exits is not just a conversation to be had at the time of exit, but across the lifecycle of an investment.

We are starting some new work to advance our own perspective on ‘responsible exits’, and to see what we can learn.

08. ONI has a unique impact measurement and management framework (that captures impact at an enterprise and sector level). What would be your advice to new impact fund managers in terms of building a impact framework for their fund ?

The first piece of advice is to get started with some impact measurement, even if it is basic at first. There is always an opportunity to improve one’s frameworks and it is important to iterate as firms learn more about what works best for them.

The second thing to keep in mind is to anchor one’s impact measurement and management on a particular theory of change, rather than simply bean counting for the sake of reporting. We constantly ask ourselves, “For what will we use this data? What decision will this guide?”

The last thing we would say is to ensure that the impact framework is fit for purpose, i.e., one must balance rigour and practicality. i.e. metrics should be robust, without imposing an excessive reporting burden on investees, considering the plethora of demands on their time.

The good news for those getting started on this journey is that there is an increasing body of work, and tools out there that can help. Helping the ecosystem learn and evolve is one of the reasons why we have published our framework publicly too. Please do send us any feedback or thoughts; we’d love to be able to improve it from the sector’s shared experiences and wisdom.

09. ONI published an India centric impact report to showcase impact generated between 2008 to 2018. What are your thoughts on disclosing impact data in the public domain on a periodic basis?

At ONI, we think that disclosing impact data advances the field because it serves as an example from which others can learn and improve upon.

It also helps us reflect on our own systems and processes, learn from others, and get more nuanced about our impact approach going forward.

I hope some of you will give us feedback on our framework based on this interview and the more detailed notes here