The project was implemented by the City of Rotterdam as the Main Urban Authority, supported by the Metropolitan Region Rotterdam - The Hague, Stichting Economisch Onderzoek Rotterdam (SEOR), the Rotterdam University of Applied Science and the RebelGroup Executives BV. SEOR, an independent research organisation linked to the Erasmus School of Economics, was the main partner responsible for monitoring and evaluation. Some decisions regarding the monitoring and evaluation design were made by the project Steering Committee, composed of two directors from the city of Rotterdam and one representative from each of the project partners.
At the design stage of the project, SEOR was not envisaged as an evaluation partner. The original idea for the BRIDGE project was to gather an evidence base for long-term continuation, with public or private funding – a sort of social impact bond scheme. Within this approach, a consultancy experienced with working on social bond projects was considered as an evaluation partner. However, the idea of a social bond scheme was abandoned at the writing stage of the project, due to the lack of opportunity to gather in-depth data (better understanding of the contributing factors in the south of the city of Rotterdam).
Once the idea of the project changed, the partnership changed as well, and an evaluating partner with a more scientific background was sought. The evaluation was assigned to SEOR and planned as a distinct project component. This was managed by SEOR together with the City of Rotterdam. Data collected through this work package was intended to be enriched by qualitative data from the NPRZ programme, collected in a different project component, dealing with the implementation of the Career Start Guarantee.
The change of the evaluation framework put extra pressure on the evaluation partner, who had to be present at every project meeting and explain to the rest of the partnership what the effects of the interventions were in terms of the broader picture. The evaluators became more central to the project than it was intended at the proposal stage, and their presence in project meetings also facilitated communication between partners; it gave SEOR the opportunity to discuss information gaps and engage the other partners in collecting information and reaching out to relevant stakeholders, such as schools.
They had a very decisive role in explaining to us, continuously, what our project was doing […] what the effect was in the broader picture. They became much more the centre of the project than it was planned in the proposal stage.
Source: BRIDGE project hearing
The project adopted a participatory approach to evaluation where the partners were involved in the design of some parts of the evaluation and the Steering Committee decided on the level of application for the interventions. The involvement of both teachers and parents proved to be a valuable choice, as they are important for the career guidance of the pupils. However, it could be argued that an increased role in the design of the evaluation itself would benefit the evaluation team, providing valuable insights.
The evaluation team had to face several challenges, leading to the need to adapt to several deviations from the final intervention plan due to practical reasons. In one case, the team had to renounce the idea of using personal portfolios for the pupils involved (longitudinal intervention) because this would have meant obtaining permission from each participating pupil, which (due to the size of the project) was problematic. This would have been a complex situation to handle, since there is no one system used in every school and, as a result, the format of the data would have varied. Using portfolios would be an option better suited to countries where the education system is more centralised and the systems align. The next-best solution was to implement surveys among pupils.