Impact assessment

Ex post impact assessment

Ex post impact assessment in research refers to the use of specialized methods to estimate changes in selected development parameters and the extent to which these can be attributed to defined research activities, interventions or innovations. Quantitative approaches typically seek to compare a “treatment” group with a “control” or “comparison” group that represents what would have happened if there had been no intervention (i.e. the counterfactual). These approaches can be challenging to apply in the complex, multistakeholder systems in which FTA works, and with the kinds of technical and social interventions FTA provides. MELIA deals with these challenges in three ways.

First, where possible, ex post impact assessment is integrated into the research design itself. This involves testing interventions using specific experimental and quasiexperimental designs to scientifically document not only what works but also where it works, for whom, how, and at what cost. The effectiveness and cost-effectiveness of technical interventions and even social interventions can be evaluated, for example, by randomly assigning groups to specific treatment conditions or, at a minimum, comparing impact-related measures and other data on both the treatment and comparison group, before and after the intervention. Opportunities are generally more limited with unplanned (retrospective) impact assessments, but it is possible to establish plausible comparison groups through, for example, ex post village-level matching, combined with the reconstruction of baseline data and the application of appropriate econometric modelling approaches.

Second, FTA does ex post impact assessment work at appropriate (generally small) scales and/or with a focus on proximal (intermediary) impacts. Impact assessment results can then be used to support and inform scaling up and out processes and provide key inputs into ex ante assessments that seek to estimate the impacts of larger scale adoption.

Third, ex post impact assessment is integrated with other monitoring and evaluation tools and approaches in an effort to develop a full understanding of change processes, outcomes and the role of FTA research in those processes.