The role of evidence in better public-sector decisions

Public institutions are not short of documents that look like evidence. Evaluations, expenditure reviews, facility assessments, household surveys and routine service reports are commissioned because a statute, a financier or an annual planning cycle requires them. Volume is easy to mistake for use. A paper has entered a decision only if it was in a position to change the choice in front of the institution: the intervention, the allocation, the delivery model, or the standard by which performance will afterwards be judged. Citation in a preamble, or a finding that arrives once the choice is closed, is a record of analytical work. It is not an input.

The two products are designed differently, which is why the confusion persists. A record can be comprehensive, retrospective and addressed to a general reader. An input has to be specific to a decision rule. It has to say what would count as a reason to continue, to adapt or to stop, and it has to arrive while that rule can still be applied. Much of what is commissioned as evidence for decision-making is specified as a record, and is then judged as though it had been an input. The study is described as unused. Often it was never in a position to be used.

This is not an argument for shorter reports, or for setting rigour aside. It is an argument for placing rigour where it can bind. A careful estimate that cannot enter the budget calendar, the procurement decision or the supervision meeting will not improve the choice, however well it is estimated. A narrower analysis, tied to an open choice and to a source that can support the inference, will often do more than a larger study addressed to no one in particular. The first task is therefore to say which product is being commissioned, and to stop describing a record as though it were an input.

Unused evidence is often explained as a failure of demand: leaders are said to ignore analysis, or to prefer politics to facts. That happens. It is not the usual mechanism. More often the supply was never capable of informing the decision that was actually open. Three failures account for most of the gap. They are technical before they are political, and each points to a different correction.

The first is timing against the decision cycle. Budget ceilings, annual work plans, procurement packages and delivery models are fixed on a calendar. A review commissioned after those choices are locked can explain why performance looks as it does. It cannot reopen them. The finding may still be a legitimate lesson for a later round. That is a different use from the one the terms of reference usually claim. If the decision of interest is this year’s allocation, the analysis has to be sequenced against the budget call, not against the date at which a final report would be convenient to write. A design that cannot meet that date should be narrowed or declined. Allowing it to slip produces a history of a choice that has already been taken.

The second is fitness of the source to the question. Sources are not interchangeable, and treating them as such produces a precise and misleading number. Routine service statistics can describe volume and, where visit dates are reliable, continuity of contact. They rarely carry a population denominator, so they do not by themselves measure coverage, and they are a weak guide to quality of care or to outcomes. A household survey can speak to coverage, equity and some outcomes, and it cannot manage a programme month by month. A facility assessment can describe readiness and, sometimes, process. It does not say who in the population was reached. Budget-execution data can show whether funds moved, not whether the service was delivered. An impact evaluation can, under stated assumptions, speak to whether an intervention caused a change. It is seldom available in time for the next operational decision, and it does not substitute for monitoring. Asking one of these sources to do the work of another does not make a decision more evidence-based. It gives a weak inference the appearance of measurement.

The third is transmission to the officer who can act. A finding that remains inside the unit that produced it, or that arrives as a long narrative when the meeting requires a judgement, has not been made available for the decision. The relevant officer is not management in general. It is the person whose authority covers the thing the evidence might change: a budget line, a protocol, a deployment of staff, a decision to continue or end a contract. If that person cannot see the result that would alter the choice, circulation of the report is not use. None of the three failures is repaired by commissioning another document of the same kind. A late study, a mismatched source, or a finding that never reaches the decision-maker will fail again for the same reason.

Evidence work should be specified from the decision backwards, not from a preferred method forwards. Before a study, a review or a fresh analysis is worth commissioning, four conditions ought to be stated in the same conversation. Separating them is how evidence becomes a parallel activity: busy, technically competent, and disconnected from management.

The choice has to be open. If the intervention, the ceiling or the delivery model cannot change, the honest product is a record of the decision already taken, or a lesson reserved for a later cycle. Describing that record as analysis to inform the closed choice misstates what the work can do. The decision-maker and the date have to be named as well. The terms of reference should say which meeting, submission or approval the work is meant to inform, and the timetable should be built back from that date. A finding that reaches the file after the committee has sat is an input to next year’s process, and should be described as such.

The decision rule has to be sayable in advance. What result would lead the institution to reallocate, to change a protocol, to stop, or to hold the present course for a stated reason? If no such result can be described, the study has no decision to serve. It may still be worth doing as research. It should not be presented as an input to management. Stating the rule before the tables exist also disciplines the analysis. It forces a choice about the comparison, the population and the threshold, instead of leaving the interpretation to be invented once the numbers are in.

The source then has to be able to identify that result. This is a question about identification, not about the prestige of the method. Routine information, a survey, a facility assessment, expenditure data and an evaluation answer different questions, on different clocks, with different errors. The design note should say what the chosen source can support, what it cannot, and which error would be most damaging if it were ignored: treating reported service contacts as population coverage, for example, or treating budget release as delivery. A more elaborate method applied to the wrong source does not repair the inference. If the four conditions cannot be held together, the right recommendation is often not to proceed. That is an analytical conclusion, not a failure to be helpful.

A ministry or a programme is rarely short of views. What it usually lacks is a routine in which a finding is expected at a known point, examined against a decision rule, and then either followed or set aside for a reason that is written down. Without that routine, evidence remains an event: a workshop, a launch, a report. Events do not accumulate into a practice. The next decision is taken as if the previous finding had not been produced.

The routine is ordinary management, not a separate research function sitting beside it. It needs a place in the calendar before the decision it is meant to inform, a short statement of the finding and of the inference it does and does not support, and a record of what was decided in the light of it. The record is the part most often omitted, and it is the part that makes the next round possible. If no one can say whether the previous finding changed an allocation, a protocol or a stop decision, the institution learns that analysis is ceremonial. Staff then invest in producing what will be accepted, not in producing what would be awkward and useful.

Disagreement belongs inside that routine. If the only findings that survive are those that confirm the preferred option, the evidence function is being trained to avoid the cases in which it would matter. A review that a manager can contest, on the source, the comparison or the implication, is doing more institutional work than a review received in silence and filed. The standard is not that evidence should dictate the decision. Public choices involve mandates, resources and politics that a study does not settle. The standard is that when evidence is set aside, the reason is explicit, so that the next decision is not taken in the pretence that the finding was never there.

An advisory team is often asked to bring the evidence, as though the scarce thing were another estimate. The more useful brief is prior to method. Name the decision that is still open. Appraise whether the information the institution already holds can inform it, including where that information is being asked to answer a question it cannot. State what is missing, and whether the missing piece can be produced before the decision date. On that appraisal, the right advice is sometimes that a further study should not be commissioned. The institution should use what it has, or stop treating a weak source as if it were a strong one.

Where new evidence is justified, it should be designed so that the essential part can be repeated by people who will still be in the institution after the advisers have left. A definition, an extract, a comparison and a decision rule that live only in a consultant’s workbook will be stale within a cycle, and will then be ignored for a good reason. Repeatability is a design requirement, not a training session added at the end. It means the source is one the institution can reach, the specification is written where the responsible team can maintain it, and the product is short enough to re-enter the same meeting next quarter or next year.

The test of the work is therefore not the volume of material produced. It is whether a named officer, facing a choice that was still open, had a finding from a fit source in time to respond to it, and whether that response can be seen afterwards. That is a higher standard than commissioning a study. It is also a more accurate description of what evidence can do for a public decision.