Project news
Edit 03 October 2022
by Hannu Karhunen, Research Leader, Labour Institute for Economic Research LABORE Hanna Virtanen, Chief Research Scientist, ETLA Economic Research

Impact evaluation in the Urban Growth Vantaa project

eui vantaa
The Urban Growth Vantaa project (also known as the GSIP Vantaa project, an acronym for the Growth and Social Investments Pacts) was developed to find innovative ways to boost employment, productivity, and growth in the Vantaa region. The project develops and tests service and incentive models that aim at improving the competitiveness and technological transition of local firms, while simultaneously encouraging social responsibility.

A central idea of the project is to create and test innovative models that provide incentives and tools for both firms and labour force to adapt to technological change. The focus of the project is on firms with higher shares of routine jobs and on low-skilled individuals. The Urban Growth Vantaa includes three distinct stages, Growth pacts 1-3, which each provide service and incentive models that focus on different targets: 1) recruiting and educating new employees, 2) updating skills and competences of employees, and 3) digitalization and adoption of new technologies. Each growth pact stage begins with developing and testing new service and incentive models. Next, these models are offered to companies in the treatment group. Finally, the data and experiences are evaluated and used to improve the developed models as well as in designing services and incentives for the next growth pact.

The impact evaluation

The Urban Growth project is making a substantial effort to measure and evaluate the impact of the project. Firstly, all partners take part in collecting and assessing the data on the quality and success of the service and incentive models. Moreover, all stages and activities of the project are analysed critically by an outside consultant. Finally, two project partners, Etla and Labore, evaluate the effects of the models using experimental design that provides credible causal evidence.

This article focuses solely on the impact evaluation carried out by Etla and Labore. This impact evaluation differs from the other evaluation activities of the project in that it utilizes a randomized control design to overcome potential issues present if the participants were chosen merely based on their willingness to participate.

A common challenge of any impact evaluation is, how to design a project that enables learning from it.  How to make sure that treatment is not credited for a normal regional development or results are not inflated by selection? For example, it is very common that the most motivated and better performing firms and individuals participate in available services. Hence, better results of the treated agents may be merely due to this positive selection. In this project, we attempt to overcome these challenges by comparing program participants to a randomly assigned control group.

In practice, companies are randomly divided into two groups: the treatment and control groups. Only the companies in the treatment group are offered the service and incentive models. Due to the randomization, treatment and control groups are on average similar, and thus, comparing the changes in the main outcomes of the groups, provides reliable information on how well the developed models work. Both extensive administrative data and multi-round survey data and are used in the analysis.

The findings

The numerical targets set for participation were clearly achieved. A total of 65 companies took part in the project services, which surpasses the goal set in the project plan by 8 percent. A large share of the participating companies was involved in more than one growth pact, indicating high level of satisfaction among the treated companies. In line with prior literature, growing firms and firms that were making also other investments simultaneously were more likely to participate in the project services.

We do not find any systematic evidence that the Urban Growth Vantaa succeeded in its core targets and improved the growth potential of the firms or the local economy in general. There are some positive signals in particularly with respect to the targets of Growth pact 2. However, there are also many indicators where the development in the treatment firms is worse than in the control firms. The evidence is too ambiguous to draw any definite conclusions. The results are much more cohesive regarding the targets of Growth pact 1. For most of the indicators, the firms in the treatment group are outperformed by the firms in the control group. Based on the impact evaluation, the project did not increase new recruitments. Finally, the evidence regarding Growth pact 3 is mixed.

There are several disadvantages and limitations to our analysis that may prevent the impact evaluation capturing any potential effects. In particular, due to the small number of target firms and treated firms the actual effect of the service and incentive models would have needed to be almost unrealistically large to be visible in the average statistics. The problem is further magnified when using the survey data. Although the survey attracted a decent response rate, unfortunately, it was still too low for reliable inference.

On the other hand, the most reliable and extensive data, namely administrative registers, can take several years to be available for researchers, and thus, we were not able to observe all outcomes for the full project period. This is a substantial drawback. The innovative and developmental nature of the project meant that there was a great deal of learning that took place during the project, and thus, it is very likely that the quality and effectiveness of the project services improved towards the end of the project. Hence, missing this information for the latter stages of the project may lead to significantly underestimating the effects of the project.

Another crucial challenge for the impact evaluation is that companies have long-term strategies for their employment and production choices, and they do not alter these decisions lightly. Achieving changes in these fundamentals is a challenging and a long process. We have attempted to capture early signs of any improvements by including outcome indicators that measure the attitudes of the companies towards increasing the skills and competences of their employees or adopting new technologies. However, here again the data suffers more from the lack of statistical power to draw reliable conclusions.

Due to these challenges, the businesses and the Vantaa region may have experienced improvements, even when this analysis fails to detect them. A major lesson for any future development project must be to take into account also the requirements of an impact evaluation when planning the project. This is the only way to increase our understanding of what works.

Share

Other news from this project

cover picture

The use of impact evaluation in local jobs and skills initiatives – The paradigm of the Vantaa GSIP Project

This is the third Zoom-in of the EU (Urban Innovative Actions - UIA) funded GSIP (Growth and Social Investments Pacts) Project, one of the main instru...

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

Jobs and skills in the local economy
Vantaa people in action

Management and monitoring models of local jobs and skills initiatives – Lessons from the Vantaa GSIP Project

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

Jobs and skills in the local economy
vantaa

Inclusion through the workplace – Vantaa upskilling solutions

The Urban Growth Vantaa project (Urbaania kasvua Vantaa in Finnish) co-created and piloted upskilling solutions in Vantaa-based small- and medium-size...

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

Jobs and skills in the local economy
Local services in action

The contribution of municipal companies to local jobs and skills agendas - Lessons from the Vantaa GSIP Project

This is the web-article No. 8 of the GSIP (Growth and Social Investments Pacts) Project, an initiative of the City of Vantaa (FI) to develop an innova...

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

Jobs and skills in the local economy
Welcome to Vantaa!

The GSIP Project story during 2021 - Journal 3

This is the third Journal of the EU (Urban Innovative Actions - UIA) funded GSIP (Growth and Social Investments Pacts) Project, one of the main instru...

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

Personal Digital Coach

Personal Digital Coach - could an application help the competence development of the worker?

Often the worker is solely responsible for their own competence development (Erkkilä & Kortesalmi 2020; Erkkilä, Lamberg & Kortesalmi 2021). In order ...

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

Jobs and skills in the local economy
The power of plain science

How could independent Research Organizations contribute to the development of local jobs and skills agendas? Lessons from the Vantaa GSIP Project

This is the web-article No. 7 of the GSIP (Growth and Social Investments Pacts) Project, an initiative of the City of Vantaa (FI) to develop an innova...

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

Jobs and skills in the local economy
UIA competence growth

Growth through competence management

One part of the Urban Growth in Vantaa project included in spring 2021 a coaching programme for competence management and development called "Competen...

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

Jobs and skills in the local economy
Revitalizing urban labour markets

Municipalities as drivers of change in the regeneration of urban labour markets – The paradigm of the GSIP Project “Updating skills for employees” model

This is the second Zoom-in of the EU (Urban Innovative Actions - UIA) funded GSIP (Growth and Social Investments Pacts) Project, one of the main instr...

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

Jobs and skills in the local economy
Youth from Vantaa in action

“Communication and dissemination strategies of local jobs and skills agendas - Lessons from the Vantaa GSIP Project Communication Plan”

This is the web-article No. 6 of the GSIP (Growth and Social Investments Pacts) Project, an initiative of the City of Vantaa (FI) to develop an innova...

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

Jobs and skills in the local economy
Vantaa

Competence development and management support

It all started with a recruitment event in winter 2020. The GSIP project organised a tailor-made recruitment event for A-Kumppanit, which is a company...

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

Jobs and skills in the local economy
GSIP2 Learnings

Personnel growth development– GSIP2 Learnings

Project Urban Growth - GSIP Vantaa addresses a socially important issue when it encourages companies in Vantaa to make social investments and grow res...

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

Jobs and skills in the local economy
GSIP Journal 2: get an update on Vantaa's project

GSIP Journal 2: get an update on Vantaa's project

Despite the strong challenges of the COVID-19 pandemic and their effect on the broader Vantaa economy and social fabric, the GSIP Project continues it...

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

The need for skilled workforce has not disappeared

The need for skilled workforce has not disappeared

The Covid-19 pandemic has influenced greatly on companies across the world and Finland is not an exception....

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

Optima Real Estate Services – maintenance solutions in a flexible and high-quality manner

Optima Real Estate Services – maintenance solutions in a flexible and high-quality manner

The sales coaching and management coaching provided by the project have sharpened the sales of Optima Real Estate Services and streamlined its busines...

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

Local active employment programmes with strong social impact

How did the Vantaa GSIP Project achieve to develop active employment programmes with strong social impact? The GSIP 1 model case study

This is the web-article No. 4 of the GSIP (Growth and Social Investments Pacts) Project, an initiative of the City of Vantaa (FI) to develop active em...

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

Jobs and skills in the local economy
How could European cities promote a solid active inclusion model – Zoom-in 1

How could European cities promote a solid active inclusion model

In his 1st Zoom-In, GSIP expert Gabriel Amitsis, focuses on a rather neglected issue in the overall debate about the development of local community jo...

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

Vantaa

”A learning platform encourages the creation of new material and the development of existing ones”

Description and user experiences of a digital learning platform selection process Visual and motivating in appearance, versatile in content, clear ...

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

Committed activity for the benefit of the company

Committed activity for the benefit of the company

In YPR Yleinen Pohjarakennus Oy, common opinion is that competence grows through work....

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

GSIP Project Team

Modern models are needed for a certificate of competence

Certificates of competence have been of clear benefit in employment. But what would be the certificate model of the future and how would it be obtaine...

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

The road to knowledge

How Universities participate in the development of local jobs and skills agendas? Lessons from the Vantaa GSIP Project

The evolving links between the university and the local government are becoming a major focus of European regional policies in specific areas, as the ...

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

Jobs and skills in the local economy
Picture from Pexels

Urban Growth in Vantaa – Creating Guidance Programs for Secondary Education

The project has progressed to the second stage. The target of this phase is to update skills of employees. During the spring, we had designed and pilo...

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

Jobs and skills in the local economy
GSIP Expert Journal 1: Get to know the project and what happened in the first 6 months!

GSIP Expert Journal 1: Get to know the project and what happened in the first 6 months!

In his 1st Journal, Gabriel Amitsis, UIA expert of the GSIP project, discusses all key activities that you need to know about it, what the project is ...

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

A tale of growth

A tale of growth

Information technology service provider, InfoCare Oy piloted a concept for itself and capital city region companies in Vantaa with which success in th...

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

Enjoying life in Vantaa

How may policy discourse influence local jobs and skills agendas? The case of the Vantaa GSIP Project

Transnational and comparative social policy research shows that integrating individuals with complex social problems (such as low qualifications, outd...

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

Jobs and skills in the local economy
Jaakko's story from Vantti: A permanent job as a Christmas present

Jaakko's story from Vantti: A permanent job as a Christmas present

Food service worker Jaakko Toljander is a cheerful and nice man even more when he has a good reason for a broad smile. In December, Toljander got a pe...

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

Innovative ways to promote local jobs and skills agendas

Live from the Growth and Social Investments Pacts Project! An interview with the City of Vantaa Mayor Ritva Viljanen

The City of Vantaa Mayor is now serving her first term. Ritva Viljanen talks about “Jobs and skills in the local economy” and reinforces the commitmen...

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

Jobs and skills in the local economy
Urban growth – GSIP Vantaa committed to finding out what works

Urban growth – GSIP Vantaa committed to finding out what works

GSIP-Vantaa is exploiting a research design to test innovative ways to improve the skills and competencies of the local labour force, and to boost emp...

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland

Project Manager Elina Taponen from Metropolia, Development Manager Sini Temisevä from Laurea and Project Designer Elina Salo from the City of Vantaa

Growth and social investment pact 1 combines labor recruitment and general growth trainings

The fourth largest city in Finland, Vantaa, has a special problem: there are many vacancies and unemployed jobseekers. The strange equation is because...

Urban Growth-GSIP Vantaa - Growth and Social investment Pacts for Local Companies in City of Vantaa

Jobs and skills in the local economy

Vantaa - Finland