Introduction

The global community is struggling to achieve the Development Agenda 2030,^[1] and development cooperation has become even more crucial in addressing the resultant policy challenges and forging collective action for sustainability.^[2] In 2011, the creation of the Global Partnership for Effective Development Co-operation (GPEDC) was supposed to create synergies between the traditional Development Assistance Committee (DAC)^[a] donors and the new non-DAC donors, including India, which engage in South-South cooperation. However, India differed with the basic principles of GPEDC which it saw as contravening its ‘demand-driven’ model of development cooperation, and exited the grouping before the first High-Level Meeting could be held in 2014.^[3] (China and Brazil too, left GPEDC along with India.)

India has become a pivotal development partner in recent years, and its cooperation projects are paving sustainable pathways that offer alternative modes of financing.^[4] While the Western model of development partnerships is largely donor-driven, India’s norms follow demand-driven principles and are not attached to traditional conditionalities. In 2012, the country created a Development Partnership Administration (DPA) under the Ministry of External Affairs (MEA).^[5] The DPA has set up an online Performance Dashboard that gives details on its partnership projects under three heads: (a) Lines of Credit; (b) Grants and Loans; and (c) Capacity Building.^[6] However, the DPA does not provide any public assessment of these initiatives. The imperative for India is a gap analysis of its cooperation initiatives to evaluate progress in its development partnerships chart over the last few decades. To conduct such analysis, however, India will need specific indicators that it can measure over time. Although there are indicators in existence,^[7] they are designed to monitor the traditional or Western model of development cooperation and may not be suitable for India. There is a need, therefore, for specific indicators that could suitably assess India’s development cooperation.

India emphasises that its development cooperation model is “demand-driven and Southern-owned with respect for sovereignty.”^[8] The country should therefore conduct regular and stringent monitoring of its partnerships and ensure that these are indeed abiding by India’s development cooperation principles. Regular monitoring can also ascertain whether the objectives of its development partnerships have changed over time or are static. The larger goal is to create a unique development cooperation index (DCI) that will track the DPA’s initiatives.

This brief attempts to outline a framework of indicators. There are no pre-defined indicators that can serve as a blueprint for fulfilling the priorities of each partner country or the implementation of a particular project. Choosing the correct indicators is an uphill task for policymakers.

The brief begins with an overview of the definition, relevance and types of indicators utilised by traditional donor agencies, such as the United Kingdom (UK) government’s Foreign, Commonwealth and Development Office (FCDO), Global Affairs Canada, the European Union (EU), and the United States Agency for International Development (USAID). Although some of the indicators may not be specifically quantifiable, they can be computed on an indicative basis. The effectiveness and reach of a large number of cooperation initiatives can be gauged primarily from their performance against a host of indicators. Such ‘development indicators’ become more relevant when they facilitate developing country-specific or country-owned policies or institutions.

The Relevance of Indicators

Indicators are defined in a variety of ways, some of them generic in nature and others, more contextual. In 1993, the Organisation for Economic Cooperation and Development (OECD) defined an indicator as “a parameter or a value derived from parameters, which points to/provides information about/describes the state of a phenomenon/environment/area with a significance extending beyond that directly associated with a parameter value.”^[9] In other words, indicators give precise information about an intricate scenario in a streamlined manner that is easy to communicate to the different stakeholders involved. However, their utility is limited to assessment; they cannot explain why certain circumstances arise or a change takes place at a given time. In 2009, the DAC of the OECD further elaborated that an indicator, especially in the context of development cooperation, is “a quantitative or qualitative factor or variable that provides a simple and reliable means to measure achievement, to reflect the changes connected to an intervention, or to help assess the performance of a development actor.”^[10] Indicators are vital to ensure adequate and targeted allocation of resources and monitor development cooperation over a period of time.^[11]

Most international cooperation agencies follow the Results-Based Management (RBM) approach to monitor and evaluate their projects. As defined by the United Nations (UN), RBM is “a way of working, a broad management approach which focuses primarily on achieving results. Going beyond the actual processes, activities, products and services, RBM looks at the real benefits of a particular programme or project, concerning the beneficiaries.”^[12] It follows a specific results chain consisting of input-activity-output-outcome-impact, thereby giving it the shape of a logical framework. Figure 1 illustrates the links of the chain in the context of a development cooperation activity. The RBM comes into play when the indicators are carefully ascertained so as to assess the level of performance of a development project.

Figure 1: Results Chain in Results-Based Management for Development Cooperation

Many agencies are now also using the term ‘Managing for Development Results’ (MfDR) to complement the RBM model.^[13]  Looking at development cooperation from the recipients’ perspective, MfDR underscores the demands of public accountability and to what extent a particular project has led to real-time results.

The UN has noted that it is more oriented towards human development and is less effective for the agencies’ internal performance assessment.^[14] RBM has been endorsed and utilised considerably by traditional donors to deal with the financial aspects of a project, while partnership entities of the Global South have often looked at these through the MfDR lens. Considered as an offshoot of RBM, MfDR does not have a separate set of indicators. Since the development cooperation objectives and ideologies of OECD/DAC members and India often vary widely, MfDR is preferred by India. It facilitates the assessment of the pace of development, and promotes stronger focus on sustainability by enhancing national ownership and building the capacity of recipients.

Classifying Indicators

The typology of indicators varies according to the function they perform or the criteria they fulfil. Analysts like Sarah Holzapfel from the German Development Institute are of the view that indicators vary depending on what they intend to assess: an agency, performance within a country, or a project/programme. Depending on their function, indicators can be classified as follows:

a) Performance-based Indicators: These give an idea of the extent to which a particular parameter/target has been achieved (or else not achieved). USAID utilises them to “detect progress towards the results as part of the Results Framework.”^[15] Such indicators lay down the pathway for a Country Development Cooperation Strategy (CDCS) consisting of Development Objectives linked to Intermediate Results (IRs) and Sub-IRs (see Figure 2).

Figure 2: USAID Results Framework

For its part, the OECD defines a performance indicator as “a variable that allows the verification of changes in the activity and shows results relative to what was planned.”^[16] In 2014, the UK’s FCDO^[b] developed a four-level results framework to monitor, manage, and track the progress of its development partnerships. Following are the Key Performance Indicators (KPI) utilised at the four levels (see also Figure 3).

• Level 1: Progress on Key Development Outcomes: These relate to the global development outcomes that FCDO aims to fulfil along with other international partners, such as the Sustainable Development Goals (SDGs) which constitute Development Agenda 2030.
• Level 2: DFID Results: These measure outputs and intermediate outcomes directly related to its development cooperation activities.
• Level 3: Operational Effectiveness: These deal with the functional or operational ability of the agency for delivering better results.
• Level 4: Organisational Effectiveness: These relate to improving the internal functioning of the agency for better delivery.

Figure 3: FCDO Four-Level Results Framework

b) Contextual Indicators: Descriptive in nature, these are employed in addition to performance indicators. They provide a broader perspective, taking into account the specific circumstances or exogenous aspects affecting the ability of an agency, a project or a national government, to fulfil its targets.^[17] The European Commission defines a context indicator as a “datum which provides simple and reliable information describing a variable relative to the context. It offers information about a situation and its evolution in a country, or an area relevant to the assistance policy.”^[18] From a development cooperation perspective, the type of cooperation, the funds disbursed, the allocation of funds according to sector/domain, qualify as contextual indicators. These indicators are also helpful in monitoring risks and unintended consequences that may impede development, be they economic, social or political in nature.

c) Standard or Common Indicators: These are used to compare data and produce analytical results across regions and agencies. They utilise a common layout, methodology, measuring variables and interpretation. USAID, for instance, uses such indicators to report externally on the results of its development investments and projects. Such indicators aim to measure both the outcomes the US government is directly invested in, and those to which it contributes.^[19] They follow a standardisation procedure allowing for a fair amount of data aggregation.

d) Custom Indicators: As opposed to standard indicators, these are tailored to meet particular information demands, priorities and requirements relating to an agency, a project or a programme at a given time. They evaluate the performance of specific development cooperation projects in special circumstances. They complement the standard indicators, being suitable for situations where no standard indicators exist.

e) Results Chain-based Indicators: These are based on the components of the results chain. One of the prominent and largest development partners, the European Union (EU), following the OECD, utilises the Input-Output-Outcome-Impact typology to categorise its indicators (see Figure 4).

Figure 4: European Commission’s Typology of Indicators

f) Sustainability Indicators: These move in tandem with efficiency and effectiveness factors, relating more to judging the efficacy of an activity rather than measuring real-time development. They pertain to outcomes: if an intervention is linked to capacity-building of teachers in schools, for example, such indicators reveal how far these teachers have helped to meet the goal of quality education. These indicators can play a vital role in monitoring how development cooperation is working towards meeting SDGs.

To be sure, these different indicators overlap. The OECD, for example, reported in 2014 that most of its DAC members found it difficult to select appropriate indicators to measure results at a particular level.^[20] Thus, several cooperation agencies, such as Global Affairs Canada or USAID, develop and employ their own indicators. Global Affairs Canada^[c] has created its unique criteria for selecting a performance indicator, devising a “how to” guidebook as part of its RBM strategy. The criteria include ways of mapping elements such as validity, reliability, sensitivity, simplicity, utility and affordability. As noted earlier, therefore, there is no template indicator that India’s DPA can duplicate.

Figure 5 illustrates Global Affairs Canada’s model.

Figure 5: Global Affairs Canada’s Results Chain

Other countries like Ireland consider it essential to train their staff to understand the rationale, concept, terminology, processes and frameworks involved in developing indicators. The correctness of the indicators also depends extensively on the availability of comparable, reliable and timely data. Development partnership agencies need to build capacity to identify precise indicators, train staff in statistical systems, and keep themselves updated by knowledge sharing between DAC actors, Southern donors, and multilateral forums.

While India does not abide by the GPEDC norms, it did create its own dashboard in 2018, comprising 10 indicators, to assess the effectiveness of its development cooperation.^[21] These include factors such as untied aid, involvement of civil society, reliable disbursements of finance, and the partner country’s own profile. The indicators are not in consonance with India’s norms concerning development cooperation, which are demand-driven. 

Indicators for Measuring India’s Development Cooperation

India’s DPA can take its cues from USAID, FCDO and Global Affairs Canada, all of which have their own model of essential indicators to track development cooperation. India need not replicate these blindly. It would be ideal if India creates its own set of indicators, unique to its cooperation projects. Based on the theoretical typology, certain indicators for measuring, monitoring and reporting of data related to development cooperation projects can be identified. They can also facilitate the creation of a more relevant DPA dashboard, and ultimately, a development cooperation index.

a) Level of engagement of India’s DPA in the project

This will measure the extent to which India’s DPA is aligned with the recipient countries’ own results frameworks and tools.^[d]  The indicator should be quantified as much as possible, comparing for instance, the consonance of Indian Technical and Economic Cooperation (ITEC) projects with the objectives, priorities, results frameworks, and more, of the African countries it is working with. It should also be able to show what percentage/share of India’s funding goes towards fulfilling the national priorities of the recipient nations.

b) Accountability of the disbursed fund guaranteed by India’s DPA

This should measure whether the flow of finance in a development cooperation initiative is being disbursed as planned—i.e., it measures accountability, in other words. It should also assess the predictability of the project over a period of time, evaluating the DPA’s reliability. It can be quantified by measuring the level of finances channelled by India through the recipient countries’ public sector.

c) Ratio of allocated budget to disbursed funds

This will assess how much of the funds allocated by India under any programme(s) have been disbursed at any given point during the project.

d) Extent of knowledge-sharing and capacity-building in science, technology and innovation

This will focus on efficacy. With capacity-building and sharing of technical expertise for the developing countries being prioritised as a global collective action by the UN, it will assess whether India is paying adequate attention to this goal. It can be quantified in terms of the number of projects taken up or the number of days allocated. However, this is only indicative.

e) Extent of involvement of civil society and local agencies

This will assess the involvement of local/grassroots agencies and civil society in the project. It will evaluate whether and how the Indian DPA is providing an enabling environment to the concerned agencies to implement the project. It can be computed from the percentage of participation or the diversity index, in terms of the civil society and local agencies’ involvement.

f) Quality of training of DPA staff

This will evaluate how well the staff are being trained to undertake the responsibilities of a cooperation project. The DPA staff should understand the recipient’s perceptions, background and expectations. It can be calculated by the number and frequency of workshops, seminars and other skills building programmes conducted by India.

g) Availability of open-access statistical data systems

This will measure the extent of transparency of the programme since accountability is vital to effective development cooperation. The quantifiable component of this indicator will be (a) the proportion of data systems and (b) the percentage of the total number of variables available for open access.

h) Ratio of tied and untied grants/loans

This will examine the ratio between tied and untied grants/loans offered by India. Those ‘tied’ restrict the recipient countries’ ability to spend the funds for a particular sector or domain. ‘Untied’ grants/loans do not impose any geographical or sectoral restrictions on utilisation. 

Table 1 provides the classification of the development cooperation indicators based on the typology mentioned earlier in this brief.

Table 1: Classification of Indicators of Development Cooperation

Conclusion

The indicators suggested in this brief will be useful in tracking the trajectory of DPA projects, how they have progressed on different parameters, whether their purpose has been served and the changes, if any, in their policy formulations. They are by no means exhaustive. They can only be indicative of the development cooperation vision pursued by India at a particular point of time. This is a crucial limitation, as India’s development vision is bound to evolve.

What is certain, however, is that this framework can help in three crucial ways:

a) Creating a development cooperation indicator dashboard for project assessment and a development cooperation index (DCI)

It is essential that the DPA create a dashboard of indicators focused on assessing and monitoring its cooperation projects at regular intervals. This would help in reviewing its policy targets, streamlining fund allocation, and appraising the partner country’s reception of a particular project. The dashboard would also help in designing a development cooperation index (DCI) to measure the progress achieved by DPA in a particular sector or project or time period.

b) Utilising development cooperation as a geostrategic tool

With increasing geopolitical competition in its neighbourhood and beyond, New Delhi needs to look at its development cooperation projects through a geostrategic lens. It can utilise the DCI for decision-making and negotiating in the geostrategic space of the development cooperation architecture. India is keen to contribute to the global sustainability narrative. It should explicitly employ the DCI to assess and monitor development objectives.

c) Assessing development partnerships from a demand-driven perspective

The indicators suggested in this brief can also be used by the recipient nations to assess their development partnerships with both the global South and the global North from a demand-driven rather than a donor-driven perspective. The variables mentioned in the indicators can be contextualised by any nation to assess its development partnerships across sectoral and geographical areas and projects.

Endnote

^[a] The Organization for Economic Cooperation and Development (OECD) has a forum called Development Assistance Committee (DAC) that brings together nearly 30 countries that provide development assistance, primarily dominated by Western donors. India is not part of the DAC.

^[b] It was then called the Department for International Development (DFID).

^[c] It was previously called Department for Foreign Affairs, Trade and Development (DFATD) Canada.

^[d] SDG 17.15 emphasises the need to do so: ‘Respect national leadership or country’s policy space to establish and implement policies for poverty eradication and sustainable development goals’

^[1] United Nations, The Sustainable Development Goals Report 2021, New York, United Nations Statistics Division Development Data and Outreach Branch.

^[2] United Nations Department of Economic and Social Affairs, “Development Cooperation, a Powerful Lever for Integrated SDGs Implementation,”  February 22, 2016.

^[3] Gerardo Bracho, “Failing to Share the Burden: Traditional Donors, Southern Providers, and the Twilight of the GPEDC and the Post-War Aid System”, In: The Palgrave Handbook of Development for Achieving the 2030 Agenda, December 2020.

^[4] Malancha Chakrabarty, “Development Cooperation Towards the SDGs: The India Model,” ORF Occasional Paper No 369, September 2020, Observer Research Foundation.

^[5] Ministry of External Affairs, Overview of India’s Development Partnership.

^[6] Ministry of External Affairs, MEA Performance Dashboard.

^[7] Sarah Holzapfel, “The Role of Indicators in Development Cooperation,” German Development Institute (DIE), 2014.

^[8] Ministry of External Affairs, Overview of India’s Development Partnership

^[9] OECD, “OECD Core Set of Indicators for Environmental Performance Reviews,” 1993.

^[10] OECD-DAC, “Glossary of Key Terms in Evaluation and Results Based Management,” 2009.

^[11] Eurostat, “Statistics in Development Cooperation- Development Indicators”.

^[12] UN Habitat, “Results Based Management Handbook: Applying RBM Concepts and Tools for a Better Urban Future,” 2017.

^[13] UNDP, “Handbook on Planning, Monitoring and Evaluating for Development Results,” 2009.

^[14] United Nations Development Group, “Results-based Management Handbook,” October 2011.

^[15] USAID, “Performance Monitoring Indicators“.

^[16] OECD-DAC, 2009

^[17] Holzapfel, “The Role of Indicators in Development Cooperation”

^[18] Capacity4Dev, “Context Indicators”.

^[19] US Department of State, “Standard Foreign Assistance Indicators”.

^[20] OECD, “Measuring and Managing Results in Development Co-operation: A Review of Challenges and Practices among DAC Members and Observers,” 2014.

^[21] GPEDC.

• International Affairs
• Developing and Emerging Economies
• China
• European Union
• India
• USA and Canada
• DEVELOPMENT PARTNERSHIPS
• STRATEGIC STUDIES

The views expressed above belong to the author(s). ORF research and analyses now available on Telegram! Click here to access our curated content — blogs, longforms and interviews.