How randomised controlled trials hollowed out Indian education

Anurag Shukla

I used to think randomised controlled trials (RCTs) were the voice of reason.

When I first entered the development evidence circuit, the RCT was treated as an antidote to ideology. Politics was framed as noisy, qualitative work as “soft”, and theory was a “nice to have”. Randomisation, I was told, would make everyone accountable. It would stop governments from chasing fads. It would stop NGOs from selling stories. It would force humility on policy. It would only count what works, compare it cleanly, learn, repeat.

Then I spent years in Indian education systems where the school is not an externally-funded programme, and the classroom is not a lab.

In tribal regions across India, I observed that child attendance had as much to do with forest produce seasons as it did with “teacher effort”. Language was not merely a delivery channel but closely linked to identity and dignity. The teacher was not a “resource” but state’s face in a village that might have known the state mostly as a police station or a ration card.

The textbook was not a neutral object but an archive of what a society considers worth knowing. If you enter this world carrying a treatment-control mindset, you begin by simplifying. If you stay long enough, you realise the simplification is the intervention.

That is the first harm RCTs do in education. They train us to see what is measurable as what is real.

Also read Over 7 lakh Kendriya Vidyalaya students assessed via education ministry’s TARA app, 1.46 lakh on career tool

New priesthood of ‘hard evidence’

The RCT’s rise in Indian education coincided with the rise of a new professional class – the evidence broker. They should not be villainised as many are sincere, hard-working, technically gifted. But then a system grew around them, with its own incentives.

Donors want attribution. Ministries want quick wins. Vendors want procurement. Academic careers want publishable results. RCTs fit this ecology beautifully because they offer something irresistible – a number that looks like certainty.

Angus Deaton has been warning for years about the “hierarchy of evidence” framing, and about the way randomisation can become a rhetorical shortcut rather than an intellectual discipline. I saw this happening in Indian education where effect size began to stand in forunderstanding. Mechanisms that were fundamental to education or educational outcomes, became footnotes. The problems with measurement got pushed into appendices. The politics of schooling and education was treated like background noise. Not realising that it is often this background noise which is the real story in education.

Also read Less bias, more risk? CBSE on-screen marking system leaves Class 12 students, teachers cautious but optimistic

Legible to markets

But if you want to understand why RCTs have done damage in Indian education, stop thinking about methods and start thinking about markets.

Once “impact” becomes a short-term, test-score outcome that can be demonstrated quickly, education becomes a product category. A vendor can now sell a “proven intervention”. An NGO can now claim “evidence-based scale”. A platform can now promise “learning gains” in a quarter. Government systems, under pressure to show results, start treating schools like distribution networks for packaged solutions. A whole new political economy is in place.

The typical RCT-friendly intervention in education has a familiar shape: a discrete input, delivered to an identifiable unit, yielding a measurable output within a project cycle. Tablets. Apps. Workbooks. Scripted pedagogy. Teacher nudges. Remedial modules. Each of these can be trialed.

But Indian education often needs what cannot be trialed easily – teacher formation over years, curriculum coherence, local language ecosystems, institutional trust, the slow repair of school culture, the credibility of the public system, and the capacity of districts to learn and adapt.

Martin Ravallion’s critique of “randomista” dominance is useful here. He warns about the crowding out the effect of these randomised experiments. When one method becomes the gold standard of legitimacy, whole classes of questions stop getting asked. In Indian education, it is often the crowded-out questions that threaten procurement pipelines and managerial comfort.

Also read Private employees in government schools, Assam vocational teachers want 3rd-party agencies out of their jobs

External validity trap

In a country like India, the most dangerous sentence in education policy is: “This intervention has evidence.”

Evidence where? With what teachers? In what language setting? Under what administrative conditions? With what political support? Through which implementation chain? With what unintended effects?

Lant Pritchett has been blunt about this problem: even if a local effect is correctly identified, it can mislead at scale when context and state capability differ. Education systems are not frictionless arenas. They are high-friction institutions with uneven capacity, contested authority, and fragile legitimacy. A trial can show that a neatly delivered package produced a modest gain in a district where an NGO held the implementation tight. Scale it through a bureaucracy that is already overburdened, and the “same” intervention becomes a different creature.

Also read Economic Survey 2026 proposes NIRF-like school ranking, PISA-type Class 10 test, more composite schools

Yet the RCT economy rewards the opposite impulse. It rewards portability. It rewards the clean story. It rewards the policy brief that can be read between meetings.

The consequence is not just that this reduces public education to plug-and-play learning process but turns it into a national addiction.

The black box problem

James Heckman’s work is a long argument against treating programmes as black boxes. People have differences. Incentives and mechanisms matter. In education, heterogeneity is the core, but RCTs treat it merely as a detail. The same “intervention” can lift one child while humiliating another. It can raise test performance while shrinking curiosity. It can help a school’s averages while pushing out the least advantaged.

Most RCT experiments do not even have the space, the method mix, or the institutional mandate to see these harms, especially when the evaluation window is short. And short windows are not an accident. They perfectly match grant cycles, procurement calendars, and the impatience of reputational competition.

Schooling is not lab science

Nancy Cartwright’s critique cuts through the mystique here: there is no automatic gold standard. Experiments warrant causal claims only under demanding assumptions, and they do not automatically travel. Julian Reiss extends this into the policy question that matters: “Will it work here?” cannot be answered by “It worked there.”

Also read Pre, Post-Matric Scholarships for minorities disbursed to thousands of ineligible or fake beneficiaries: CAG

Schooling in tribal areas, for example, is a dense social field: language politics, migration patterns, local histories of displacement, gender norms, the moral economy of households, distrust of state institutions, and the everyday experience of being “taught” in a tongue that is not your own. You can randomise a teaching aid across schools, but you cannot randomise the historical relationship between a community and the state.

Development technocracy that loves RCTs

There is also a political story hiding inside the scientific story.

Dani Rodrik has argued that “one experiment at a time” cannot address the hard problems of institutional change, structural transformation, or context-heavy governance. Education reform is a governance problem before it is an input problem. It involves unions, bureaucratic routines, curriculum authorities, local politics, parent expectations, language movements, and ideologies of merit and discipline. It is messy and slow. It requires diagnostics and theory-guided learning.

The RCT regime often sidesteps this by narrowing ambition. It searches for what can be tweaked, not what must be rebuilt. This is where the method becomes politics as it selects a world that is manageable for experts. It remains focused solely on “what works”, while ignoring how it works and why.

Agnès Labrousse’s critique of the rhetorical status of experiments helps name what is happening here. The RCTs, as she argues, should not be just seen merely as a method, but as a scientific authority. It does its job supremely well. It flatters donors. It promises neutrality. It creates a moral hierarchy in which any dissent or disagreement looks like anti-science.

And more than anything, it helps the market grow.

Also read 3.5 lakh lack uniforms, books a year late in Odisha schools as half of Samagra Shiksha funds lie idle: CAG

What RCTs keep missing

In India, the largest educational harm in recent decades has not been the absence of clever interventions. It has been the steady weakening of the public system’s moral and institutional authority.

Teachers are increasingly treated as implementers of external scripts. Curriculum is subordinated to test scores. Digital platforms have become central to procurement with classroom life becomes dependent on corporate maintenance cycles. As NGOs are nowparallel systems of “delivery”, governments have only learnt dependency rather than capability. And this goes on to become worse when RCTs focus on short-run gains, instead of approaching education as a long-term game.

A serious alternative

A serious evaluation culture for Indian education would look less like a lab and more like a diagnostic craft:

• Begin with theory about learning, institutions, and incentives, and treat measurement

as a moral act, not a technical detail.

• Combine methods: experiments where they fit, longitudinal designs where time matters, ethnography where meaning matters, political analysis where power matters.

• Treat implementation capability as part of the causal story, not a “scaling challenge”

to be dealt with later.

• Value teacher knowledge and agency as evidence, not merely as anecdote.

• Refuse the conversion of education into a product category certified by short-term gains.

• Make room for disagreement without branding critics as anti-evidence.

If the RCT turn claimed to be a revolution, its real legacy in Indian education has often been a narrowing: of questions, of imagination, and of what the state feels permitted to do.

The tragedy is not that RCTs exist. The tragedy is that they became the passport to legitimacy in a field that cannot survive on passports alone.

Anurag Shukla is director at Brhat, a public policy think tank. He holds a PhD from IIM Ahmedabad and has previously worked with Pratham, Azim Premji Foundation, and the Centre for Advocacy and Research (CFAR). He has also served as visiting faculty at several institutions, including TISS Mumbai.