Plan for an adequate preparatory or inception phase of the evaluation that will take place before the actual project activities begin. Since capturing initiatives’ impacts will often require comparison of the situation before (ex-ante) and after (ex-post) the project activities, or among two groups (counterfactual), establishing sound baseline data is essential. Experiences of Athens Curing the Limbo and Utrecht U-RLP show that a lack of data collection tools being developed and deployed prior to project activities makes measuring and accounting for change difficult at the individual level of the beneficiaries receiving services but also the broader communities.
The baseline was compromised so we were constantly playing catch up from the word go. We would have wanted to have been there at the beginning, exploring what the tenants’ aspirations and hopes were for the project, how much contact they had before with any refugees or asylum seekers, so that we could then build up a picture over the whole project.
Source: U-RLP project representative
At the same time, since Athens Curing the Limbo opted for evaluation based on the action research paradigm (which favours ongoing adjustments to the project implementation based on knowledge accumulated through monitoring and analysis activities), having a shared, commonly agreed understanding of the evaluation’s aim and key concepts appeared more important than methods and data collection planned in detail at the very beginning of the project.
If a city is choosing action research as a method, they cannot have the evaluation planned very accurately in the very beginning. They can have some ideas and discussions but in order to develop sound evaluation, they first need to start implementing activities. This is sort of a strategic acupuncture, where you do the activity first and then try to see what the consequences of this action are.
Source: Curing the Limbo representative
Whatever the content and purpose of the inception or the preparatory phase of evaluation (development of a detailed data collection plan and indicators matrix or establishment of shared direction and guiding principles), it is important to provide strong foundations for the whole process. It is particularly important in evaluations based on theory of change where data collection and analysis should clearly reflect and follow its logic. In any of the approaches selected, establishing the baseline scenario and accounting for the situation prior to the intervention’s launch is essential for discussing the project’s impact at the end of it.