INTRODUCTION

The Sustainable Development Goals (SDGs) – a set of 17 goals and 169 targets – will direct global development efforts for the next 15 years. While the goals and associated targets have been adopted, it is still unclear how they will be rolled out in practice. For instance, there is the question of how effectively each country will adopt the SDGs in a manner that is globally relevant while making sure that such goals are aligned with their respective national priorities. The National Institution for Transforming India (NITI) Aayog is currently formulating a 15-year vision document that is expected to provide a roadmap for India's achievement of the 2030 development goals.1 However, the exact mechanisms are yet to be finalised, and the ambiguity is even more pronounced for the monitoring and evaluation aspect. The proposed indicators under each target are still on the negotiation table.

Indicator frameworks are being developed at three levels – global, regional and national. While the Inter-Agency and Expert Group on Sustainable Development Goal Indicators (IAEG-SDG) constituted by the UN Statistical Division (UNSD) is responsible for creating the global framework, the United Nations Economic and Social Commission for Asia and the Pacific (ESCAP) is tasked with the creation of the indicator set for the Asian region.

The development of the national framework, meanwhile, is a country-led process. Each country is given the flexibility in designing a reporting template that reflects its own national priorities. Through an analytical study of India's past efforts to track the earlier Millennium Development Goals (MDGs) and its initial preparations for the SDGs, this paper seeks to identify specific hurdles that the government must overcome in building a national framework that is in line with India's national development agenda. It makes an assessment of the quality and availability of the national data pool in the context of the reproductive, maternal, newborn and child health (RMNCH) aspects of SDG 3, and outlines the fundamental challenges that must be addressed for effective tracking.

The monitoring and evaluation (M&E) exercise of the MDGs provides valuable insights for implementing the secondgeneration development goals. The global MDG monitoring and accountability system had been generally considered weak.2 Despite a cursory mention in one of the earlier UN Secretary General's (UNSG) reports charting the MDG roadmap3, detailed discussions did not take place until as late as 2002.4 The global M&E framework thus evolved on an ad hoc basis, resulting in the lack of accountability and missed opportunities for mid-course corrections.5

At the country-level as well, there are broad strengths and weaknesses that serve as important lessons. For instance, the MDG national indicators were disaggregated by gender and residence (rural-urban) – for societies as diverse as India, tracking these additional layers is crucial in determining overall progress. A recent paper by Saikia and Kulkarni6 identifies health inequality as the primary impediment to the implementation of public health policies. Thus, the new framework must incorporate such strengths and further build on them. On the other end, the Indian MDG framework lacked an effective evaluation system. While the nodal agency for tracking the MDGs – the Ministry of Statistics and Programme Implementation (MOSPI) –produced annual progress reports, they were not supported by corresponding evaluation efforts looking at why certain indicators under-performed.7 The secondgeneration framework needs to fill this void.

India has had limited success in realising the child and maternal health MDGs, missing the mark on both MDG 4 (child mortality) and MDG 5 (maternal health). According to MOSPI's 2015 report on the MDGs, India is 'moderately on-track' in reducing child mortality and 'slow or off-track' in improving maternal health. This time, for the SDGs, ensuring timely achievement of targets requires crafting an effective strategy that is able to routinely track progress and make the necessary mid-course corrections.

The first section of this paper maps existing national databases that collect RMNCH data and analyses their performance. The second section outlines the fundamental issues facing the Indian statistical system in monitoring the RMNCH SDG targets, and the paper concludes with key suggestions towards building a national follow-up and review structure.

DATA SOURCES FOR RMNCH

In terms of follow-up and review, the spotlight is on the national-level process. The synthesis report of the UNSG, titled The Road to Dignity 8 by 2030, states that country-led monitoring – as it is closest to the people – will be critical. The first step in this process would be ensuring timely availability of reliable data. For this, the UNSG report recommends national efforts to “build on existing national and local mechanisms and processes, with broad,9 multi-stakeholder participation.” Reliance on existing information systems is a sound strategy: India already has multiple such data collection systems, and these should be leveraged for the SDG tracking exercise.

a) Administrative Statistics

Death and birth registration records are a vital source of a country's demographic information, especially on mortality and fertility. In India, this is carried out through the civil registration system (CRS): the Registration of Births and Deaths Act 1969 mandates registration of all births and deaths via the CRS. However, the CRS is hobbled by significant coverage issues. According to the Office of the Registrar General of India – the same agency responsible for the CRS – the incompleteness of the data system makes it ineligible for national statistical tracking.10 The sample registration system (SRS) – although technically not an administrative record – was introduced as a temporary arrangement to fulfil the role of the CRS until it could produce robust statistics.11 Based on representative sample units, the dual recording system has been providing Pan- India data since 1971. Annual estimates, available mainly through its annual reports, are widely used by both government and civil society. In fact, it was the primary source for tracking most of the RMCH-related MDG indicators: MOSPI employed SRS data to monitor the Under Five Mortality Ratio (U5MR), Infant Mortality Rate (IMR) and Maternal Mortality Ratio (MMR).12

The SRS was conceptualised as an interim system; today, it has become the chief source for demographic statistics. The SRS, however, has certain fundamental limitations making it a poor substitute for a competent CRS. The sampling design of the SRS only allows for estimations at the national and state level. While the 2004 report included regional estimations – based on the National Sample Survey Organisation (NSSO) sub-state regions – data on district level is still absent. Another drawback is the rising underregistration issue. One of the major strengths of the SRS has been the completeness of registration for the sample units. This has been questioned for recent estimates. While completeness stood at about 90 percent up until the 1980s, this figure is said to have declined considerably in the 1990s and the 13 2000s, though no official evaluation efforts have been conducted post-1990 to either support or refute this claim. A more regular evaluation framework for the SRS would help improve and maintain the accuracy of the registration system.

The third major source of administrative information is the Health Management Information System (HMIS), a web-based system that collects district and sub-district level data to monitor various health programmes especially on RMCH.15 Launched in 2008, it captured only district-consolidated data in the beginning. It has since been expanded to allow more disaggregated monitoring, and now employs 'facility-based reporting' in 640 districts (out of 676) across 35 States/Union Territories (UTs).16

Leveraging ICTs for HMIS not only allows collection of data at each facility, but it also enables easy dissemination of data through the HMIS web-portal. The data processing mechanism is also expedited, as an online system speeds up communication among different levels of the data collection and management structure such as the field offices and the management agencies at the district, state and central levels. Despite the wide range of advantages offered by the HMIS, it has not emerged as a popular data source due to its limited data coverage. The HMIS data collection is restricted to public institutions – given the growing role of private healthcare institutions in India, the selective coverage of the HMIS means that its utility is currently limited.17

b) Surveys

The Indian government conducts multiple surveys focusing on health statistics. One of the most comprehensive surveys for RMNC health is the National Family Health Survey (NFHS), which is equivalent to the Demographic Health Surveys conducted in many countries. Consecutive rounds have attempted to expand the scope of the survey: the second round included additional parameters such as the haemoglobin level for women and children, and the third round further extended the scope by including unmarried women as opposed to the earlier restriction to married women. HIV testing for the adult population was also introduced in this round.

While the NFHS has continuously been augmented to increase its relevance, the survey has suffered from a long-standing critique of its sample design. Much like the case of the SRS, the first three rounds of the NFHS do not provide district-level data. Certain alternative health surveys, such as the Annual Health Survey (AHS) and the District Level Health Surveys (DLHS), have thus been used to supplement RMNCH analyses. Given the distinct advantage of a unified augmented survey – as opposed to the existence of multiple parallel ones – the government has decided to subsume the AHS and the DLHS into the NFHS umbrella. Thus, the sample size of NFHS-4 – the latest survey round – has been increased to provide robust district-level 18 data. It will now produce estimates of most indicators for all 640 districts based on the 2011 census geographical classification. Perhaps now the NFHS could be employed more effectively for decentralised tracking of the SDG indicators.

Another survey pertinent to monitoring RMNC health is the National Sample Survey Office's (NSSO) morbidity and healthcare rounds: sample-based surveys that collect data on areas around morbidity and the utilisation of private and public healthcare in India. The NSSO health surveys are generally separated by large and uneven intervals: for instance, the last three health rounds (2014, 2004 and 1995-96) were carried out with 10- year gaps in between. Thus, while the NSSO data can be used to supplement the SDG monitoring process, it is hardly qualified to serve as a primary source.

c) Census

The decennial Indian Census – the largest monitoring exercise in the country – is the 19 third tool for compiling demographic data. The time-tested statistical system provides one of the most reliable data sets, and it can also be stratified to monitor geographical and sectoral differences. While the census data can be employed for certain indirect estimations for the inter-round duration, the decade-long interval between successive rounds and the long time lag between the collection of data and disclosure of report, makes it a poor candidate for an effective monitoring mechanism for the RMNC health targets.

TRACKING RMNCH INDICATORS: KEY CHALLENGES

Effective tracking requires the timely availability of reliable and adequately disaggregated data. Despite the abundance of data sources in India, there is an undeniable absence of data collection systems that make the cut. With the introduction of district-level data in NFHS-4, perhaps the data set can be now be more effectively employed to track the RMNCH targets. Through an analysis of the above data sources, this section identifies five chief issues that must be addressed in order to build an effective national SDG monitoring framework for RMNCH.

a) Data Gaps

A fundamental challenge in the MDG tracking process was the lack of reliable sub-state data particularly at the district-level: the 2015 India Country Report explicitly mentions this as a 20 major impediment. Most of the sample-based databases are statistically significant only up to the state level; neither the NSSO nor the SRS provides district-level data. Similarly, the first three rounds of NFHS did not include districtlevel estimates. The inclusion in the latest one is a welcome change.

The size of most Indian states, coupled with the heterogeneous characteristic of these populations, call for a follow-up mechanism with effective district-level tracking. Kulkarni points out that the “failure to have indicators for diverse regions within states is a major handicap in carrying out demographic 21 analysis and making forecasts”. Statistical tracking of the child and maternal health MDGs was predominantly carried out using 22 two data sources – the SRS and the NFHS. While the monitoring process did highlight large inter-state disparities, the absence of district-level data meant that a more in-depth analysis was not possible. For instance, as better performing states like Kerala recorded a maternal mortality rate (MMR) of 61, the corresponding figure for Assam was almost 2 3 f i v e - f o ld (300) . Ta r g e t ed he a l th interventions to rectify such variations demand more disaggregated data.

Even at the state level, certain major indicators suffer from the issue of data gap. For instance, SRS-sourced data for the Maternal Mortality Ratio (MMR) – the primary indicator for MDG5 and most likely for the first target for SDG 3 – is available only 24 for 15 Indian states. Data on under-5 Mortality Rate (U5MR) is also limited only to 25 Delhi and some of the larger states.

Plugging the gaps will be crucial in tracking the RMNCH indicators and targets. The strengthened NFHS is expected to address this to a considerable degree, especially with regard to the provision of district-level data. For specific indicators such as the MMR, there is still some ambiguity – the first batch of NFHS-4 factsheets is silent on the MMR. However, it would be premature to comment on this, given that the Ministry of Health and Family Welfare has so far released data for only the first phase. Augmenting populationbased, rather than sample-based data collection systems such as the CRS and the HMIS, can also play a large role in effective tracking of the RMNCH indicators.

b) Periodicity

A second challenge is that data on most indicators are available at irregular intervals, which prevents systematic tracking of progress. For example, the four rounds of the NFHS – one of the principal sources for monitoring health indicators – were conducted with erratic frequency. The second round was conducted six years after the first, the next round with a seven-year gap, and the latest will be published after a break of at least nine years. The re-designed NFHS, starting with the fourth round, addresses this issue: the survey will now be carried out every three years.26

The NSSO morbidity surveys are another example: Because the NSSO has a much broader mandate, each round focuses on a different subject. As mentioned earlier, the latest health round was conducted in 2014 (the 71st round), exactly 10 years after its predecessor in 2004 (the 60th round).27 In addition to the non-availability of district level data, periodicity issues mean that the NSSO cannot be primarily relied upon to track the RMNC health targets regularly. However, given the credibility of NSSO sample estimates, the survey's results can be used for triangulation to check the accuracy of the primary SDG data sources.

Fluctuating Inter-Round Frequency

Source: *National Family Health Survey, India, http://rchiips.org/nfhs/about.shtml **71st NSSO Round Report

c) Coverage Issues

Issues of incomplete coverage related to administrative data systems, particularly the CRS, means that there are large variations for the same indicators. The discrepancy between the CRS and the SRS estimates illustrates this point: Calculations (made by the author) as per the latest CRS report show that India's infant mortality rate (IMR) in 2013 stood at 28 eight as opposed to the SRS estimated figure, 29 which was 40.

Apart from general incomplete coverage, certain administrative data sets also suffer from selective coverage. Given that the HMIS is based on a facility-based monitoring system, it has immense scope: the absence of data for private institutions, however, limits its usability to a considerable degree.

d) Equity-Sensitive Tracking

The issue of inequality is considered a priority at the global SDG platform as well as in India's national priorities. The UNSG Synthesis Report emphasises how the SDGs should 30 “leave no one behind”. In line with this, the issue has been taken up as an independent goal (SDG 10), and the concept has also been applied in individual SDGs. This also aligns with the domestic priority on ensuring universal health coverage with equitable access for all sections of the population under 31 the 12th five-year health sector plan.

The size and diversity of India's population mean that extreme disparities on health outcomes exist across geographies, social groups, religious groups, income-levels and genders. For instance, the 2015 MDG country report highlights the vast rural-urban divide in terms of child mortality: in 2013, the U5MR for rural India (55) was almost double that of urban India (29). Inter-state indicators show a similar trend. Kerala – the best performing state – recorded a U5MR of 12; on the other end of the scale, the statistic for Assam was 73. This holds true for most indicators – an earlier example on inter-state MMR progress also reflects this point.

The MDG tracking framework for India provided gender-wise and residence-based 32 analyses. Given the ambitious nature of the SDGs, the incorporation of even more stratifiers would definitely facilitate a more inclusive realisation of the SDG RMNCH targets. This is particularly true for India: the existence of multiple social and religious divisions, and fast but non-inclusive economic growth, has created deep socio-economic inequalities and a significantly hierarchical 3 3 society. Thus, it is likely that the development story of one section of the population is drastically different from that of another. For instance, NFHS-3 shows that there were large variations in the IMR figures for different social groups – the IMR for scheduled castes was 17 points more than that 34 of the general class. The IMR also displayed considerable variations from one income level to another. A nuanced understanding of RMNCH issues requires a tracking system that addresses this aspect. As most data sources, such as the NFHS and the NSSO, already collect information for these stratifiers, incorporating them into the SDG framework can be carried out with relative ease. To further strengthen the equitysensitive tracking process, death registration systems – such as the CRS and the SRS – must incorporate information on socio-economic 35 aspects in the mortality records.

e) Supplementary Indicators

Primary indicators for the RMNCH targets must also be accompanied by auxiliary indicators, which can help maintain steady progress. There is a range of factors that can either accelerate or inhibit the realisation of any primary indicator, and tracking these factors will enable the monitoring agency to pinpoint specific problem areas. For instance, analysing India's failure to achieve the maternal MDG by a wide margin, Kapur Mehta and Arora state that addressing this challenge will require a nuanced understanding of 36 mortality causes. This would ensure targeted remedial measures and timely realisation of the RMNCH targets.

CONCLUSION

There are considerable challenges in monitoring the RMNCH indicators. The government recognises these obstacles and is already in the process of addressing them. For one, the MDG experience highlighted the importance of geographically disaggregated data, particularly robust, district-level data. The recent overhaul of the NFHS data collection system deals with this issue to a large extent. NFHS-4 addresses the additional issue of periodicity. The existence of multiple, fairly detailed, data sets also means that equity-sensitive tracking can be carried out easily. The country, however, needs to strengthen its administrative data systems to build an effective indicator framework. Coverage must be enhanced. Incorporating information on private sector provisioning will also be required to make a comprehensive country assessment.

Apart from monitoring, designing a competent SDG follow-up and review framework calls for a well thought-out evaluation mechanism. While raw tracking and data analysis are important functions, decision-making for mid-course corrections requires a further step: evaluating the underlying causes of recorded data shifts at regular intervals. While MOSPI's annual India reports for the MDGs presented in-depth statistical analyses, they were lacking in any meaningful review of the reasons behind the observed trends.

The participation of various actors – the government, private sector and civil society – in the review process can further enhance the evaluation mechanism. This is also in line with the promotion of the multi-stakeholder 37 approach in the global SDG process. Including non-governmental evaluations will lead not only to additional scrutiny but to the creation of independent reviews by non-state bodies. This requires information that is easily accessible for public consumption; the creation of an e-repository, which serves as a focal point for all reliable data sources and analysis reports, would definitely ease access issues.

Timing is crucial in this process: a streamlined monitoring and evaluation (M&E) mechanism must be worked out right at the formulation stage. Junctures for statistical tracking and review must be clearly specified. According to the UN Office of Internal Oversight Services (OIOS), while monitoring should be carried out at smaller intervals, 38 evaluation can take place every five years. NITI Aayog, in collaboration with the MOSPI, is currently developing the Indian M&E system. The exact structure is yet to be finalised. Despite the challenges ahead, the government is better prepared this time, and concerted efforts now could lead to the creation of a rigorous system that is able to track the county's health goals more effectively.

ENDNOTES

1. Remya Nair and Jyotika Sood, “NITI Aayog to replace 5-year plans with 15-year vision document,”
Livemint, May 14, 2016.
2. See: Shannon Kindornay and Sarah Twigg, “Establishing a workable follow-up and review process for the
Sustainable Development Goals,” ODI Report, April 2015; “Review of the contributions of the MDG
Agenda to foster development: Lessons for the post-2015 UN development agenda,” Discussion Paper, UN
System Task Team on the Post-2015 UN Development Agenda, March 2012; “Regional Monitoring and
Review Mechanism for Effective Implementation of Post-2015 Agenda,” Expert Paper for Asia-Pacific
Forum on Sustainable Development 2015, Economic and Social Commission for Asia and the Pacific, May
20, 2015
3. “Road map towards the implementation of the United Nations Millennium Declaration,” Report of the
Secretary-General, UN General Assembly, September 6, 2001, Paragraph 83.
4. “Statistics and indicators for the post-2015 development agenda,” United Nations System Task Team on
the Post-2015 United Nations Development Agenda, July 2013.
5. “Thematic evaluation of monitoring and evaluation of the Millennium Development Goals: lessons learned
for the post-2015 era,” Report of the Office of Internal Oversight Services, United Nations Economic and
Social Council, March 18, 2015,Pg 15.
6. Nandita Saikia and P. M. Kulkarni, “Data for Research into Health Inequality in India. Do We Have
Enough?” Economic and Political Weekly 51, nos. 26 & 27 (2016).
7. “Millennium Development Goals, India Country Report 2015,” Ministry of Statistics and Programme
Implementation, Government of India, February 2015.
8. “The Road to Dignity by 2030: Ending Poverty, Transforming All Lives and Protecting the Planet,”
Synthesis Report of the Secretary-General On the Post-2015 Agenda, United Nations, December 2014, Pg
40, Paragraph 149 (i)
9. Ibid
10. “Vital Statistics of India Based On the Civil Registration System 2013,” Office Of The Registrar General,
India, Pg 5.
11. Prasanta Mahapatra, “An Overview of the Sample Registration System in India,” Prince Mahidol Award
Conference & Global Health Information Forum 2010.
12. See: “Millennium Development Goals, India Country Report 2015,” Ministry of Statistics and Programme
Implementation.
13. Prasanta Mahapatra, “An Overview of the Sample Registration System in India,” Prince Mahidol Award
Conference & Global Health Information Forum 2010.
14. Ibid
15. Charu C. Garg, “Availability, Quality and Data Gaps in Health Statistics in India,” Ministry of Statistics and
Programme Implementation, Government of India, (2014).
16. “Manual on Health Statistics in India,” Central Statistical Office, Ministry of Statistics and Programme
Implementation, New Delhi, (2015).
17. Charu C. Garg, “Availability, Quality and Data Gaps in Health Statistics in India,” Ministry of Statistics and
Programme Implementation, Government of India, (2014).
18. “Manual on Health Statistics in India,” Central Statistical Office, Ministry of Statistics and Programme
Implementation, New Delhi, (2015).

19. “Census of India,” http://www.censusindia.gov.in/2011-common/aboutus.html
20. “Millennium Development Goals, India Country Report 2015,” Ministry of Statistics and Programme
Implementation, Pg 13.
21. P. M. Kulkarni, “Issues of Data Requirements in the Context of the Demographic Transition in India,”
Country Paper for India, 15th Conference Of Commonwealth Statisticians, February 7-10, 2011, New
Delhi, India, Pg 10.
22. See: “Millennium Development Goals, India Country Report 2015,” Ministry of Statistics and Programme
Implementation
23. Ibid
24. Aasha Kapur Mehta and Dolly Arora, “Base Paper on Availability of Data and Data Gaps for Situation
Analysis of Well-being of Children and Women,” IIPA, New Delhi, (2014).
25. “Discussion Paper on Health Statistics,” Ministry of Statistics and Programme Implementation, India,
Februa ry 2015, Pg. 6, ht tp: / /mospi .ni c . in/Mospi_New/upload/Di s cus s ionPape r -
CoreIndicatorsHealthStatistics-27feb15.pdf
26. Rukmini Shrinivasan, “National health survey not nixed,” The Times of India, Jul 30, 2012
27. “National Data Bank for Socio-Religious Categories,” Ministry of Statistics and Programme
Implementation, India, http://mospi.gov.in/national_data_bank/ndb-rpts.htm
28. “Vital Statistics Of India Based On The Civil Registration System 2013,” Office Of The Registrar General,
India
29. “Statistical Report 2013, Executive Summary,” Sample Registration System, (2013),
http://www.censusindia.gov.in/vital_statistics/SRS_Reports_2013.html
30. “Road map towards the implementation of the United Nations Millennium Declaration,” Report of the
Secretary-General, UN General Assembly, September 6, 2001.
31. “Twelfth Five Year Plan (2012–2017), Social Sectors,” Planning Commission, Government of India, 2013,
http://planningcommission.gov.in/plans/planrel/12thplan/pdf/12fyp_vol3.pdf
32. “Millennium Development Goals, India Country Report 2015,” Ministry of Statistics and Programme
Implementation
33. Milind Deogaonkar, "Socio-economic inequality and its effect on healthcare delivery in India: inequality
and healthcare," Electronic Journal of Sociology 11 (2004).
34. IIPS and Macro International, “Chapter 7 – Infant and Child Mortality,” in National Family Health Survey
(NFHS-3), 2005-06: India, Volume I, (2007), http://rchiips.org/nfhs/NFHS-3%20Data/VOL-
1/Chapter%2007%20-%20Infant%20and%20Child%20Mortality%20(313K).pdf
35. Nandita Saikia and P. M. Kulkarni, “Data for Research into Health Inequality in India. Do We Have
Enough?” Economic and Political Weekly 51, nos. 26 & 27 (2016).
36. Aasha Kapur Mehta and Dolly Arora, “Base Paper on Availability of Data and Data Gaps for Situation
Analysis of Well-being of Children and Women,” IIPA, New Delhi, (2014).
37. “Road map towards the implementation of the United Nations Millennium Declaration,” Report of the
Secretary-General, UN General Assembly, September 6, 2001, Paragraph 149
38. “Thematic evaluation of monitoring and evaluation of the Millennium Development Goals: lessons learned
for the post-2015 era,” Report of the Office of Internal Oversight Services, United Nations Economic and
Social Council, March 18, 2015, Page 23

• Developing and Emerging Economies
• Sustainable Development
• India

The views expressed above belong to the author(s). ORF research and analyses now available on Telegram! Click here to access our curated content — blogs, longforms and interviews.