Data disputes: Have some clothes to shed

Not everything that can be counted counts, and not everything that counts can be counted, said Albert Einstein. But counting does matter, as pointed out by Joseph Stiglitz, Amartya Sen and Jean-Paul Fitoussi in their report for the Commission on the Measurement of Economic Performance and Social Progress, “What we measure affects what we do and if our measurements are flawed, decisions may be distorted… if metrics of performance are flawed, so too may be inferences we draw.”

There is a new buzz in town after the United Nations Secretary General (UNSG) called for a data revolution for sustainable development. This was in response to the growing angst about the diluted definition of data—where it has been restricted to numbers and the naive assumption that technology and private sector will provide the silver bullet solution to the data-deficit. The absence of accountability, the bedrock of public data for public action, is also another cause for concern.

An era of increasing censorship where disclosure policies and sunshine laws are rescinding is another worry and so is the watering down of whistle-blowers’ protection legislation, India being a case in point. What is even more discomforting is the inadequate attention to the bottom billion who are impacted by the digital divide. Therefore, the onslaught of censorship, quasi-censorship on inconvenient surveys deserves attention. India has acquired the ignominy of being one of the countries at the forefront of proscribing books. But in this din of freedom of speech and expression, not much has been written about the constant censoring of inconvenient survey findings.

The 3rd National Family Health Survey (NFHS), 2005-2006 showed the malnutrition percentages in India had gone up, compared to the 2nd NFHS. This was in the midst of the “India Shining” narrative when India’s national Gross Domestic Product was growing at almost double-digits. The increased malnutrition numbers dealt a body blow to the pro-liberalisation, orthodox economists’ theory of growth amounting to good. The result was predictable: some states like Madhya Pradesh challenged the methodology and wanted to conduct their surveys to show their “better” performance.

It is pertinent to note that NFHS is one of the most credible health surveys in the world with a globally benchmarked methodology. The NFHS exercise was halted by a governmental action for ten years and was revived only after much advocacy by public health activists, nutritionists, intellectuals and heterodox economists. The 4th round of the NFHS is expected to be out in 2015-2016.

In 2011, the World Bank came out with its seminal work, Poverty and Social Exclusion in India. It was a multiple data decomposition and regression analyses to see how four socially marginalised groups were performing economically through the prism of their discrimination. The four groups were Dalits, Adivasis, women and Muslims. But the Union government allowed the findings of three identity groups to be published, barring the chapter on Muslims. The bank acknowledged this in its executive summary too.

UNICEF conducted a Rapid Survey of Children for the Union government in 2013-2014. The publication of the survey findings was delayed. One of the reasons cited in informed circles is that while the pan India averages are good, Gujarat has shown a considerable decline, and hence the gag. This is because the survey findings give a body blow to the Gujarat development model narrative.

These are big stories and need as much public attention and media scrutiny as any film ban or cancellation of a comedy performance. The public discourse around data deficit and data dishonesty and the need for greater and better investments in data-generation has been at a peak, both from health, economics and development practitioners and policy wonks. But what happens when some of the most robust data-sets are either not asking the key questions around disaggregation or not releasing the unit-level data in a time when they are relevant for effective planning? For example, while the NFHS trend analysis was possible, in cases like frontline workers’ home visits and access to safe drinking water, a comparison was impossible because the questions had changed from NFHS 2 to NFHS 3. Poverty measurement exercise is also marred by the same arcane language and methodology challenges. In 2014, the Rangarajan Committee report, the erstwhile Planning Commission and the Asian Development Bank gave three different sets of the number of poor people in India.

The UNSG’s call for a data revolution does not take into account the inadequacies in data collection, analysis, and its grey areas in the proposed “data revolution”. For instance, the UNSG-appointed International Expert Advisory Group submitted its recommendations on October 24, 2014, and opened the feedback/comments window only till October 26, 2014. This smacks of arrogance. That they expected the world to be glued to their computers and give comments over a weekend is appalling. It is essential to pivot data on accountability and to stop using statistical institutions as public relations agency or survey findings as public relations advertisements by country governments, including India. As philosopher-mathematician Bertrand Russell stated, “There is no nonsense so arrant that it cannot be made the creed of the vast majority by adequate governmental action.”

It will be a tragedy if surveys become that arrant nonsense backed by adequate governmental action. Precisely the reason to resist, whenever essential surveys and their findings are censored, each time, every time!