Artem Korzhenevych (p)

Lisa Sella

In the recent years, urban experimentation in its diverse forms has become popular among both researchers and policymakers in search of new insights for sustainable urban development. Real-world labs, community-based initiatives, innovation experiments, and pilot projects have attracted a lot of public attention and produced active local and international discourses. However, simultaneously, a number of methodological issues arose that sparked a debate on suitable experiment design. In this paper, we aim at addressing one particular aspect of that debate, which in our view currently remains understudied, namely the issue of (possibility of) causal inference through urban experiments as well as challenges and risks related to the quest for distinguishing cause and effect in this context. We start by carefully examining the concept of causality, in particular from the points of view of the variance theory and the process theory. We then carry out a review of a selection of widely cited examples of urban experimentation (especially in the energy, transport and resource management domains), and categorize them with respect to their approach to causal inference. We also review examples of field experiments in the domain of urban economics. Based on the review, we gather the common features of experimental design that allow various studies to come up with statements related to effectiveness of urban experiments in terms of certain outcomes or outputs. Particular attention is drawn to the aspects of complexity and uncertainty. We conclude with recommendations regarding scientific experimental designand the role of causality in it.

Arjen Edzes, Richard Rijnks, Viktor Venhorst (p)

Artem Korzhenevych

In the Netherlands, dissatisfaction with the current scheme of social assistance benefits (SAB) led various Dutch cities to run randomized controlled trials (RCT) to test out new ways of delivering support for re-integration to its citizens. Among them is the City of Groningen where the study goes by the name ‘Bijstand op Maat’ (English: Tailored Social Assistance Benefits). The Groningen RCT has a number of unique features. Firstly, the RCT follows a Zelen’s design, which allows us to shed light on the internal and external validity of our results. Secondly, besides outflow to work, our study includes outcome measures in the domains of individual wellbeing, health and trust. Third, the Groningen RCT uniquely features a “choice” treatment. For the duration of the two-years trial, job search requirements and SAB sanctions tied to SAB will be eliminated to make room for more autonomy (treatment 1). Other treatments include a more tailor-made intensive coaching approach in supporting SABSAB recipients (treatment 2) and increasing financial incentives to take up paid work (treatment 3). As a final treatment, unique relative to similar RCTs (4), one group may actually choose between the first three treatments which allows us to investigate the effect of freedom of choice on the outcomes. The study is set up as a randomized controlled trial (RCT) with interventions running from November 2017 to November 2019. The RCT includes 900 recipients of SAB, for whom we measure differences in the uptake of paid work (full time and part time) and societal participation and social activation as well as participants’ health and well-being. Overall, we conclude that the Groningen RCT demonstrates that there are various work and wellbeing related positive effects among SAB clients that choose and are in a position to comply with options that are available. An overall population effect could not be demonstrated. Considering the substantial heterogeneity amongst SAB clients in terms of skills, personal situation, individual effects could still be a ground to implement policies along the lines of our interventions.

Elena Ragazzi, Lisa Sella (p), Alfonso Langastro, Nga le

Viktor Venhorst

No one, neither in politics nor in sciences dares to argument against the necessity to fight to reduce the impact of working conditions on worker’s health. Perhaps such a wide consensus on the importance of this goal is one explanation for the general lack of evaluation studies (The European Agency for Health and Safety at Work 2013) on occupational safety and health policies (OSH). Nevertheless, acquiring more information on the effectiveness of the different tools available for the policy maker could help to afford those situations that are refractory to change and move towards a more effective policy mix. At present this last is heavily unbalanced, with the greatest majority of interventions concentrated in regulation and related enforcement mechanisms (inspections and sanctions).
The Italian case offers us an interesting case study, because in 2008, a system of economic incentives has been introduced, providing grants to SMEs that invest in the OSH field. This represents a sort of revolution, because it leverages the corporate social responsibility towards their workers. This experimental policy, called the “ISI calls” (bandi ISI) is the most long lasting (12 years), richest in terms of money granted (2 billion euros) and widest in terms of firms involved OHS policy based on incentives. Despite this, at the policy level the balance is still leaning in favor of sticks rather than carrots.
Up to now, the evaluation analyses performed by Inail on the ISI calls mainly concerned implementation processes and performance monitoring, reporting, and accounting. Recently, the need for an ex-post evaluation of the impacts emerged. Therefore, Inail is funding a research project aimed at identifying appropriate models to assess the impact of ISI incentives and to highlight their strengths and criticalities as an economic support to SMEs. The ISI call, implemented through the mechanisms of the click-day, represent a case of natural experiment, nevertheless, many evaluation challenges (previously discussed in Colagiacomo et al. 2018) are present for this type of policies, in which the literature is practically non-existent.
In this paper we will present the first results of the impact evaluation of the ISI calls, based on administrative data on participating firms. A profiling of the several subgroups of firms (selected or non-admitted, funded or not funded) will be provided. Moreover, if already available, we will integrate this analysis with data on the safety profile of the firms