Evaluation has been strongly embedded in the implementation of the OASIS project from the start and this has been considered an advantage. Being ‘in the middle of things’ allowed evaluators to feel at ease with those designing and implementing activities.
The evaluation didn’t only take place at the end, but throughout the process, which means it was an interactive evaluation that could lead to the adjustments.
Source: OASIS project hearing
Being part of the project team and the process provided evaluators with a good sense of what is feasible, in terms of suggesting changes and improvements, and helped with the design of activities with impact indicators in mind.
Evaluation has been managed by two entities; the Laboratory for Interdisciplinary Evaluation of Public Policies of the Sciences Po (LIEPP) for the behavioural aspects, and the Interdisciplinary Laboratory of Energies of Tomorrow (LIED), associated with ESIEE Paris, for the climate impact. The evaluation has also been strongly supported by Météo-France, the country’s key weather monitoring and forecasting entity. Météo-France was in charge of climate modelling and defining the climatic and micro-climatic evaluation protocol for quantifying the cooling effects created by the OASIS playgrounds at neighbourhood level.
The fact that the entities brought significant expertise in evaluating urban policies was considered important for the quality of the service. Importantly, LIED’s researchers have been involved in similar data collection activities at other sites in Paris, and this expertise facilitated the evaluation of OASIS. Evaluation governance has been characterised by clear definition of the responsibilities assigned to the involved entities and the scope of their engagement, as well as effective communication. This approach was helpful, for instance, when LIEPP worked on adjusting and fine-tuning its data collection instruments (questionaries for children). While LIEPP strived for simplicity of questions, its researchers consulted LIED and Météo-France to make sure that the instruments remained true to the scientific nature of the project.
Cooperation between the evaluation team and the project implementation stakeholders has been very active and enhanced learning loops for the project. The project implementing team was already able to efficiently use suggestions from the evaluators and quickly adjust activities as they evolved. For instance, thanks to monitoring inputs, the protocol initially developed for ensuring active participation of local community members was adjusted to better reflect the character of the groups.