The analysed UIA M&E case studies show the importance of designating a person, team, partner or subcontractor responsible for overseeing the evaluation. The case studies had either a designated evaluation focal point (e.g. CURANT, Steam City, BRIDGE, CALICO, U-RLP, CoRE) or a number of partners tasked with various aspects of the evaluation from the project initiation (e.g. B-MINCOME, OASIS). In Curing the Limbo, the evaluation was coordinated by the quality assurance team comprised of representatives of all partners. The Utrecht U-RLP project benefitted not only from the involvement of its lead evaluator, but also the presence of the academic Advisory Board. While in Vienna CoRE, the subcontractor leading the evaluation effort set up a think tank for this purpose.
The presence of a designated evaluator, evaluation team or partner adds weight to this component in the overall project scheme. As seen in the case studies, a committed evaluator can guarantee that evaluation receives enough attention in complex project environments. This can, for example, take the form of giving impetus to the development of a common approach (e.g. in U-RLP, Steam City), making sure that enough data is available from the start (e.g. in B-MINCOME, BRIDGE) or that results are communicated to the partnership and discussed (e.g. in CURANT or CALICO).
“Of course the project partners are focused on the implementation of their activities. […] And we are focused on the impact and monitoring all these activities.” (Source: STEAM City representative)
The managers or implementers will naturally focus on carrying out the activities, so the evaluation will likely not be their top priority. A designated evaluator will be there specifically to set up good research conditions and learn about whether, how and why the project worked to make that knowledge available to others.
“When you’re focused on executing the project, an evaluator can help you to continuously see the bigger picture. And you can only benefit from that.” (Source: BRIDGE representative)
This is not to say that evaluators’ work is or should be fully separate from that of project managers and implementing partners. On the contrary, the experiences of analysed M&E case studies, as discussed in other lessons, suggest that cooperation in evaluation pays off. Yet, some projects saw benefits in separating the function of project execution from that of the evaluation (e.g. B-MINCOME, CoRE, CURANT).
“We need to separate the team that executes the project from the team that is evaluating the results of the project. It does not mean that they do not communicate. In fact, it’s very important that the evaluators’ team were in contact with the executing team […] But the competences, the functions of both parts should be separated.” (Source: B-MINCOME representative')
'We rightly decided […] that we need somebody to – while we implement the activities – have an eye on how we are performing, where we have to adapt […] we need a kind of outside view too to tell us if we’re still in line or if we need to turn at some point and go another way.” (Source: CoRE representative)
The M&E case studies reveal an integrative potential in having a dedicated evaluation partner. This is particularly the case when the evaluation adopts a more participatory approach whereby implementation partners are also involved in evaluation. In the initial phases, an evaluator can assist in bringing together the multitude of perspectives and ideas of what the project tries to accomplish, contributing to the creation of a common vision (e.g. in CURANT, Curing the Limbo, U-RLP, Steam City, CALICO).
During the later stages, a designated evaluator collects data from all of the project partners, enabling them to capture a holistic view of the project, adding a fresh outsider perspective, one that can surely be valuable for managers throughout the project.
“While in a partnership every partner had some expertise […] nobody had a helicopter view. And that’s what CeMIS had, CeMIS had a helicopter view from all of the partners.” (Source: CURANT representative)
In complex and multidimensional evaluations, where more than one partner is involved in the evaluation, an ‘evaluation integrator’ may be necessary to pull together all of the evaluation’s results. In Barcelona B-MINCOME, the integration of results was undertaken after project completion. It was considered important enough to receive additional financing from the municipality beyond UIA support.
Since Athens Curing the Limbo did not have a clearly designated evaluator from the start, its experience is illustrative of the important role an evaluation focal point can play. The project was implemented by several partners with strong expertise and individual understanding of evaluation.
'But then we realised that we were not measuring the common goal of the programme.” (Source: Curing the Limbo representative)
It took time for the Athens Curing the Limbo partners to arrive at a shared interpretation of the project’s goals and how their achievement could realistically be evaluated. The role of recreating a common framework was undertaken by the quality assurance team composed of representatives of all partners. This was a new task assigned to this team during the project’s implementation.
It is hard to deny the value of a leader in any type of endeavour. This proves true in evaluation efforts in large-scale innovative urban interventions. It comes as no surprise that this role is particularly pronounced when evaluation is distributed across a larger number of project partners, when the integrative process proves crucial for generating a final understanding of the whole experience.