Monitoring and evaluation practices: UIA lessons learnt
UIA considers monitoring and evaluation (M&E) one of its operational challenges. In 2020, it commissioned Ecorys to conduct the ‘Capitalisation activity on good practices for monitoring and evaluation of results in Urban Innovative Actions (UIA) projects’. As well as identifying good M&E practices, the capitalisation activities aimed to explore why and in what way these could be seen as such. They were also to highlight the necessary elements that make successful M&E. Importantly, evaluation rather than monitoring formed the main focus of the research.
During the capitalisation activities, Ecorys reviewed basic documents for 55 projects from UIA’s three calls for proposals. It created a long list of 20 projects for further analysis. After additional desk research, a short list was compiled consisting of 11 projects. Between August and October 2020, representatives of these projects participated in online meetings with Ecorys to discuss the details of their approaches and practical observations from implementation.
Eventually, nine projects were further analysed and described as M&E case studies: Antwerp CURANT, Athens Curing the Limbo, Aveiro Steam City, Barcelona B-MINCOME, Brussels CALICO, Paris OASIS, Rotterdam BRIDGE, Utrecht U-RLP and Vienna CoRE. Since the projects were in different stages of M&E development and implementation, some started quite recently, we refer to them collectively as case studies rather than best practices. They also offer clear examples of good practice with respect to different M&E elements.
Our report is divided into three main parts: theory, practice and case studies. The theoretical section offers a few general observations on evaluating innovation based on academic and grey literature. The practical section contains cross-cutting lessons learnt while analysing the UIA M&E case studies. We divided the lessons into four pillars – evaluation governance, evaluation approaches, data collection methods and horizontal issues. In the final section, we describe the M&E approaches implemented in the selected UIA case studies with particular emphasis on evaluation.
The online platform was released on 11 March 2021 during a dedicated web session to which 200 attendees participated. Related information (presentation, video recording) are available here.
UIA projects are to go beyond traditional policies and services – they need to be bold and innovative. Consequently, monitoring and evaluation of such projects presents its own set of challenges. First, a distinction needs to be made between these two elements. Interwoven and interconnected, monitoring and evaluation often get confused, and the lines between them blurred.
