Original Research••

Effectiveness, cost-effectiveness and return on investment of individual placement and support compared with traditional vocational rehabilitation for individuals with severe mental illness in the Netherlands: a nationwide implementation study

Abstract

Objectives To assess the effectiveness, cost-effectiveness and return on investment of individual placement and support (IPS) implemented through a reimbursement strategy on a nationwide scale compared with traditional vocational rehabilitation (TVR) regarding sustainable participation in competitive employment in individuals with severe mental illness receiving sickness or disability benefits.

Methods An observational study was conducted using administrative data regarding all Dutch individuals receiving sickness or disability benefits in the period 2012–2019. Exact matching and difference-in-difference fixed-effect estimations were performed to handle the non-randomised nature of the data. The matched sample consisted of 863 IPS and 16 466 TVR participants. The primary effect measure was the proportion of individuals having worked for at least 48 hours per month in competitive employment (ie, for 12 hours or more per week); the proportion of individuals having worked in competitive employment for at least 1 hour per month was also evaluated. Cost-effectiveness and return on investment were assessed from the societal perspective (intervention, sickness/disability benefit and healthcare costs) and payer perspective (sickness/disability benefit costs).

Results IPS led to a statistically significant higher probability of being competitively employed for at least 12 hours per week of 3.7% points (95% CI 0.8% to 6.7%) to 7.5% points (95% CI 3.8% to 11.3%) and of being competitively employed for at least 1 hour per month of 4.7% points (95% CI 1.6% to 7.7%) to 8.9% points (95% CI 5.2 to 12.6%) from 6 to 36 months after starting the intervention. From the societal and payer perspective, IPS was—on average—less costly and more effective than TVR and return-on-investment estimates showed that IPS was—on average—cost saving (eg, societal perspective: ∆C: −364 (95% CI −3977 to 3249); ∆E: 0.104 (95% CI 0.046 to 0.164); benefit–cost ratio: 2.1 (95% CI −14.8 to 19.1)), but the uncertainty surrounding these estimates was large.

Conclusions IPS implemented through a reimbursement strategy on a nationwide scale is more effective and potentially cost-effective than TVR in people with severe mental illness receiving sickness or disability benefits. Based on these results, the implementation of IPS by a wide scale reimbursement strategy could be promoted to enhance sustainable participation in competitive employment in these individuals. Future economic evaluations should strive for a more robust sample size and a long follow-up period.

What is already known on this topic

  • Worldwide work participation of people with severe mental disorders is low. In randomised controlled trials, it is shown that individual placement and support (IPS) is an effective and cost-effective method to help individuals with severe mental disorders achieve competitive employment. Despite its effectiveness, implementation of IPS in usual rehabilitation care is limited, highlighting the need for effective implementation strategies to implement IPS on a wide scale.

What this study adds

  • The results of this study show that IPS implemented on a nationwide scale through a reimbursement strategy is effective and potentially cost-effective and could be cost saving from both the societal and payer perspective.

How this study might affect research, practice or policy

  • The results of this economic evaluation can motivate decision-makers on the national level to reimburse IPS for people with severe mental disorders. A nationwide reimbursement strategy will improve the implementation of IPS and work participation of people with severe mental disorders.

Introduction

A commonly used definition of severe mental illness (SMI) in the Netherlands is a psychiatric disorder that requires treatment, for which coordinated support from professional care providers in care networks is indicated to realise the treatment plan.1 The disorder is accompanied with serious impairments in social and/or societal functioning and is persistent over time; the impairment is the cause and result of the psychiatric disorder.1 The majority of individuals with SMI have a psychotic disorder, such as schizophrenia or a bipolar disorder.1 Other diagnosis in these individuals include autism, a severe depression or a personality disorder.1

Although most individuals with SMI prefer to work and working contributes to their recovery, their employment rates are low and they often rely on disability benefits.2–6 Hence, the economic burden of SMI for society is significant and includes both direct costs to the healthcare system and indirect costs, which are mainly borne by employers and the social security system.2

Individual placement and support (IPS) is internationally recognised as an evidence-based, effective and cost-effective method to help individuals with SMI to obtain and maintain competitive employment.7–12 IPS services aim to get all clients who want to work, rapidly into competitive jobs without pre-employment training, and are provided by employment specialists who are integrated in mental health services.13 In contrast to this ‘place and train’ approach, traditional vocational rehabilitation (TVR) services for individuals with SMI use a stepwise approach, by training clients before placing them, in often sheltered or volunteer, work (‘train and place’).14 15 Research shows that IPS is more than twice as effective as TVR in getting individuals with SMI into competitive employment.7 Regarding cost-effectiveness, a European multisite randomised controlled trial conducted between 2003 and 2005, demonstrated that IPS was more effective at a lower cost compared with TVR in all countries except for the Netherlands.11 At the Dutch site, the quality of IPS implementation, as measured by the IPS fidelity scale,16 was low, which may have had a negative impact on the cost-effectiveness outcomes.11 12

In both the Netherlands and other countries, one of the major barriers to a successful implementation of IPS has been inadequate funding.15 17–25 Reimbursement for IPS by Dutch health insurance organisations was not, or only partially, provided because the Health Insurance Act excludes reimbursement for vocational rehabilitation. Consequently, the Ministry of Social Affairs and Employment requested that the Dutch Social Security Institute: the Institute for Employee Benefits Schemes (UWV) offer reimbursement on a trial basis and evaluate its effectiveness and cost-effectiveness. To improve the implementation of IPS in the Netherlands, UWV began to reimburse IPS from 2012. The aim of UWV was not only to improve work participation of individuals with SMI but also to reduce the costs of benefits. The underlying hypothesis for this policy change was that the benefit costs for clients with SMI may exceed the reimbursement costs for IPS, as has been demonstrated in other countries.11 26 This nationwide reimbursement consisted of IPS funding, offered by UWV to all mental health agencies that provided IPS services and achieved a fair or good IPS fidelity score,24 25 27 28 as measured by the IPS fidelity scale.16 Only clients with SMI who received sickness or disability benefits from UWV were eligible for this funding.24 25 27 28 Given the positive association between high model fidelity and improved employment outcomes, this funding was expected to enhance the implementation of IPS, and consequently its outcomes,26 28–31 as compared with those found in the aforementioned randomised controlled trial.11 A detailed description of the reimbursement policy for IPS is provided in the methods section.

Despite the recognised importance of adequate funding in improving the implementation of IPS within usual mental healthcare, little is known about the impact of funding on the outcomes of IPS. Only one study investigated the effectiveness of IPS after the introduction of the nationwide reimbursement of IPS, and showed that IPS leads to a higher probability of being competitively employed compared with TVR in individuals with SMI who receive disability benefits.27 Since administrators and policy-makers are increasingly investing a considerable amount of time and resources in IPS implementation,19 22 25 28 it is important to not only evaluate the effectiveness, but also the cost-effectiveness and return on investment of IPS to ensure that resources are being allocated optimally.

This is the first study assessing the effectiveness, cost-effectiveness and return on investment of IPS implemented through a reimbursement strategy on a nationwide scale compared with TVR regarding sustainable participation in competitive employment in individuals with SMI receiving sickness or disability benefits. The results of this study have significant implications for national policy-makers, potentially strengthening the argument for continuing IPS reimbursement for individuals with SMI.

Methods

Study design and data

This study was carried out using administrative data from Statistics Netherlands and UWV, regarding all Dutch individuals receiving sickness or disability benefits in the period 2012–2019. The data were anonymised before access.

Patient and public involvement

It was not appropriate or possible to involve patients or the public in the design, or conduct, or reporting, or dissemination plans of this research.

Context information

Benefits and vocational rehabilitation for individuals with SMI

In the Netherlands, individuals with SMI who are not able to work due to their illness can apply for different types of benefits at UWV.32–34 See online supplemental file 1 for detailed information on these benefits. Generally, individuals who receive a disability benefit are more likely to face greater challenges in participating in the labour market compared with those with a sickness benefit, who typically have a shorter duration of illness. UWV is an autonomous administrative authority and is responsible for the vocational rehabilitation of individuals receiving sickness or disability benefits.27 28 For a significant period, UWV primarily offered TVR services to individuals with SMI receiving these benefits. In 2012, UWV also began to reimburse IPS, provided by mental healthcare agencies. Since then, an increasing number of mental healthcare agencies have implemented and started offering IPS services.

In the Netherlands, healthcare and vocational rehabilitation are strictly divided in terms of insurance and providers. Healthcare insurance will in general not cover any care focusing on return to work. As such, financing vocational rehabilitation executed by a mental health provider is considered rather unique from a Dutch perspective. It needs commitment from both the healthcare insurer for the first few consultations with the IPS employment specialist, and from the income insurer (UWV) for the actual long-term guidance for return to work from the same mental health provider and employment specialist.

Interventions

Individual placement and support

IPS is a clearly described model of vocational rehabilitation and includes the following eight principles: (1) focus on competitive employment, (2) zero exclusion and eligibility based on client choice, (3) rapid job search, (4) attention to consumer choice, (5) integration of mental health and employment services, (6) personalised benefits counselling, (7) systematic job development and (8) time unlimited and individualised follow-along support.13

To be effective, IPS should be implemented according to the model, as a high model fidelity is associated with better employment outcomes.29–31 In the Netherlands, mental healthcare agencies that wanted to offer IPS to their clients received a model fidelity assessment after having professionals trained in providing IPS services. This fidelity assessment was conducted every 2 years by independent reviewers to ensure quality of the IPS services.29 To receive funding from UWV, the mental healthcare agencies needed to have a fair or good IPS fidelity score and only individuals with SMI receiving sickness or disability benefits from UWV were eligible.24 25 27 28 Only professionals trained as IPS employment specialists were allowed to provide IPS services within these agencies. During the study period (2012–2019), around 15–20 mental healthcare agencies in the Netherlands offered IPS services more than incidentally.

During the study period, there were two IPS funding regimens:

  1. Old regimen (January 2012—February 2017): Funding was provided for a maximum duration of 36 months, excluding job coaching. Mental healthcare agencies received additional funding on successfully placing participants in competitive jobs. Data were available only for IPS participants receiving a disability benefit.

  2. New regimen (March 2017–December 2019): The maximum funding duration remained at 36 months, but a higher amount was introduced, which included coverage for job coaching expenses. Data were available for all IPS participants receiving a sickness or disability benefit.

Traditional vocational rehabilitation

The goal of vocational rehabilitation services provided to UWV clients is to help individuals overcome barriers to employment and successfully find work. The process involves regular meetings with UWV professionals to assess the client’s needs and determine which additional services may be required, such as social activation, employee skills enhancement, determining labour market position, job hunting and matching, and job coaching services. These services are financed by a designated budget.

Study population and matching procedure

Administrative data were used to select both intervention and control group individuals, of whom their baseline was defined as the start of their intervention. The intervention group consisted of individuals receiving sickness or disability benefits, who participated in IPS services, funded by UWV, between 2012 and 2019 and were not employed at the start of the intervention. The control group consisted of individuals who participated in TVR, funded by UWV, during the same time period and were not employed at the start of the intervention. These individuals were matched to the intervention group individuals on sex (male/female), age (<25, 25–35, 35–45 and >45 years), region (agglomeration of big cities in West-Netherlands and rest), work history in 12 months before intervention (0–1 month and ≥2 months), type of benefit (disability, other), presence of disability benefits 12 months before start of the intervention (yes, no), number of months of disability benefits over the last 12 (1–9 months, 10–12 months) and over the last 36 months (1–24 months, 25–36 months), medical costs in the last 12 months before start of the intervention (≤€1200, €1200–€6000, €6000–€12 000 and ≥€12 000), mental healthcare costs in the last 12 months before start of the intervention (≤€1200, €1200–€12 000 and ≥€12 000), size of the mental health institute (no institute, ≤49 interventions, ≥50 interventions for IPS and TVR interventions during the study period) and presence of other interventions 36 months before start of the intervention (yes, no). Individuals were also matched on SMI criteria. For the operationalisation of SMI in administrative data, an individual is usually considered to have SMI in a specific year if he or she meets one of the four following criteria: (1) at least one health insurance claim for treatment related to schizophrenia in the current year or one of the previous 2 years; (2) a record of both open and closed longer-lasting mental health insurance claims in each year for the current year and the previous 2 years, excluding claims for diagnostics, indirect time and treatments related to alcohol, addiction, dementia or delirium; (3) an indication for living in sheltered or supported housing and (4) expenses for medications related to bipolar disorder or psychosis.27 These criteria were developed by a Dutch business intelligence centre for health insurance providers (Vektis), in cooperation with mental healthcare providers. All four SMI criteria have been included as binary (0/1) variables in the matching.

To avoid selection effects and provide a clearer comparison with the intervention group, the control group was limited to individuals who received treatment from mental healthcare agencies that do not offer IPS on a large scale (ie, ≥30 IPS trajectories over the total research period). This was done to ensure that the control group was not influenced by the availability of IPS and to reduce the likelihood that individuals who receive IPS differ from individuals who receive TVR on unobservable characteristics. Individuals who had been competitively employed within the month prior to starting the intervention and those who were permanently disabled and unable to work were excluded from the analyses. Additionally, participants who had received IPS within a 2-year period before or after the start of TVR were also excluded. The same sort of interventions (ie, IPS or TVR) that started during the same 2-year time frame were treated as one intervention with the starting date of the first intervention.

Due to the large sample size (ie, 1787 individuals in the IPS group and 103 439 individuals in the TVR group) it was possible to apply exact non-parametric matching. Regarding observed characteristics, exact matching outperforms propensity score matching, ensuring that individuals in the TVR group have the exact same characteristics as their counterparts in the IPS group. When using a propensity score, matched pairs have similar scores but may differ in terms of observed variables of interest, which may introduce a bias in the results. While exact matching ensures that the TVR and IPS group share identical characteristics, it could limit the number of matches due to a possible shortage of exact matches for every individual receiving IPS.

Please note, however, that an individual in the TVR group can serve as a control for multiple IPS participants. In the analysis phase, every individual in the TVR group was weighted by the inverse of the number of controls for the specific individual receiving IPS he or she was matched to.

Effect measures

The primary effect measure was the proportion of individuals having worked for at least 48 hours per month in competitive employment (yes/no). This corresponds approximately to having worked for at least 12 hours per week and may be considered as a measure for sustainable employment (eg, these individuals work more than 1 day). A commonly used effect measure in literature on IPS effectiveness was also evaluated7: the proportion of individuals having worked in competitive employment for at least 1 hour per month (yes/no). Competitive employment was defined as having positive non-zero wage and number of hours worked per month; data on self-employment were not available in this study. Both effect measures (ie, (1) monthly competitive employment ≥12 hours per week and (2) monthly competitive employment ≥1 hour) were assessed every month after baseline (ie, the start of IPS/TVR) during a 36-month period, using data from Statistics Netherlands, which contained precise information on the start and end dates of jobs, the type of employment (competitive or sheltered) and the hours worked in those jobs for all Dutch citizens.

Resource use and valuation

Costs were measured from a societal and payer perspective. From the societal perspective, intervention costs, sickness and disability benefit costs, and healthcare costs were included. The societal perspective is generally recommended when various stakeholders may be affected by an intervention (eg, by the Dutch Manual of Costing).35 36 This is also the case for IPS, as UWV invests in IPS and may benefit from its implementation through reduced spending on sickness and disability benefits, whereas the government and health insurance companies may benefit from it through reduced medical costs. From the payer perspective, only UWV costs were included. This perspective was added as it may provide valuable information to possible payers of the intervention. Information on the individuals’ intervention and benefits costs was collected from the database of UWV. Information on the individuals’ healthcare costs was collected from a National database containing information on all health insurance claims of Dutch citizens that are part of the Basic Health insurance package of all adults (≥18 years). This package covers visits to all medical specialists, emergency services and medically necessary procedures, such as surgery, X-rays, obstetrics and prescription medications. All costs were standardised to 2019 Euro rates using consumer price index numbers for each year.

Statistical analyses

Effectiveness

First, the baseline characteristics of both matched cohorts were descriptively analysed. For the effectiveness analysis, a difference-in-difference fixed-effect estimation to the matched sample was applied, which made it possible to estimate the causal effect of IPS. The difference-in-difference estimation together with exact matching corrects for potential preintervention differences between the IPS group and the control group. A similar approach has been followed by De Graaf-Zijl et al.27 See online supplemental file 2 for the specification of the analysis model.

The differences in effect measures were analysed for the total IPS and TVR group. For all analyses, a two-sided significance level of 5% was used and 95% CIs for the coefficients (βt) were calculated. Effectiveness analyses were performed in STATA v16.

Cost-effectiveness

The cost-effectiveness analysis was performed from the societal and payer perspective and related the difference in total costs between the IPS group and the control group during the 36 months after baseline (∆C) to the differences in effects (∆E: having worked for at least 12 hours per week and having worked for at least 1 hour). These differences were estimated using independent sample t-test on the matched cohorts. Please note that these analyses were performed on complete cases only; that is, those with complete cost and effect data for the full duration of 36 months. As data were gathered from databases and data incompleteness was mainly due to the fact that some individuals started with the intervention less than 36 months ago (ie, they could not have completely observed data), data incompleteness was assumed not to bias the outcomes.

Incremental cost-effectiveness ratios (ICER) were estimated by dividing the differences in costs by the differences in effects (ICER=∆C/∆E). Incremental net benefits (INBs) were estimated for three different willingness to pays (ie, ʎ=the maximum amount of money decision-makers are willing to pay per worker returned to sustainable employment), being €0, €10 000 and €25 000, using the formula INB=∆E×ʎ−∆C. Then, cost-effectiveness acceptability curves were developed indicating the probability of IPS being cost-effective versus TVR for a broad range of willingness to pays.37 38 Cost-effectiveness analyses were performed in R.

Return on investment

The return-on-investment analysis was performed from the societal and payer perspective. Costs were defined as the difference in programme costs of the IPS and TVR intervention. Benefits were defined as the mean difference in benefits costs (payer perspective) and benefits and healthcare costs (societal perspective). Positive benefits indicate a cost saving, while negative benefits indicate a monetary loss.

Two metrics were determined: (1) net benefit (NB)=benefit–costs; (2) benefit cost ratio (BCR)=benefits/costs. An NB of more than 0 and a BCR of more than 1 indicate a positive financial return.38 Return-on-investment analyses were performed in R.

Sensitivity analyses

Effectiveness

Three sensitivity analyses were performed evaluating the effectiveness of (1) the old funding regimen (2012–2017) for individuals receiving a disability benefit, (2) the new funding regimen (2017–2019) for individuals receiving a disability benefit and (3) the new funding regimen for individuals receiving a sickness benefit.

Cost-effectiveness and return on investment

Two sensitivity analyses were performed: (1) excluding sickness and disability benefits costs and (2) stratified for funding regimen (ie, old vs new funding regimen for individuals receiving a disability benefit). The first sensitivity analysis was performed for the cost-effectiveness analysis only, as sickness and disability benefits costs have some level of overlap with the effect outcome (ie, competitive employment).

Results

Characteristics

The characteristics of the IPS and TVR groups, prematching and postmatching are shown in table 1. The matching strategy resulted in a TVR group of 16 466 controls for 863 persons who received IPS, that is, for 48% of the IPS group at least one control was found. On average, there were 19 controls per individual receiving IPS. The matched IPS sample had more or less the same characteristics as the total IPS group, with a few differences: the matched sample scored lower on all SMI criteria and had lower medical costs per month compared with the total IPS group.

Table 1
|
Characteristics of IPS and TVR groups, prematching and postmatching

Effectiveness

Figure 1 and online supplemental file 3 present the estimated coefficients per month for the effect of IPS over TVR with 95% CI for the total group. The first plot shows that IPS leads to statistically significant higher probabilities of being competitively employed for at least 12 hours per week of 3.7% points (95% CI 0.8% to 6.7%) to 7.5% points (95% CI 3.8% to 11.3%) from month 6 until 36 after the start of the intervention. The second plot shows that IPS also leads to statistically significant higher probabilities of being competitively employed for at least 1 hour per month of 4.7% points (95% CI 1.6% to 7.7%) to 8.9% points (95% CI 5.2% to 12.6%) from month 6 until 36 after the start of the intervention.

Figure 1
Request permissionrequest permission icon
Figure 1

Estimated coefficients per month for the effect of IPS over TVR with 95% CI for the total group. (A) Monthly competitive employment ≥12 hours per week. (B) Monthly competitive employment ≥1 hour. IPS, individual placement and support; TVR, traditional vocational rehabilitation.

Costs

Information on the mean costs per group and corresponding cost differences can be found in table 2. Intervention costs and healthcare costs were higher for IPS compared with TVR, while sickness and disability benefits costs, total societal costs and total payer costs were lower. Of those differences, only the difference in intervention costs was statistically significant.

Table 2
|
Mean costs per group and corresponding cost differences

Cost-effectiveness

Cost-effectiveness results can be found in table 3 and figure 2. In the main analysis, IPS dominated TVR for both outcomes and both perspectives, meaning that it was—on average—less costly and more effective. INBs were—on average—positive for all willingness to pays. Cost-effectiveness acceptability curves indicated that if decision-makers are not willing to pay anything per individual extra returned to competitive employment, the probability of IPS being cost-effective compared with TVR is 0.58 for the societal perspective and 0.63 from that of the payer. These probabilities gradually increased with increasing values of willingness to pay, for example, to 0.9 if societal decision-makers are willing to pay €19 000 per individual extra returned to competitive employment.

Table 3
|
Differences in mean costs and effects (95% CI) and ICERs
Figure 2
Request permissionrequest permission icon
Figure 2

Cost-effectiveness acceptability curves for outcomes from the societal and payer perspective. (A) Monthly competitive employment ≥12 hours per week from the societal perspective. (B) Monthly competitive employment ≥12 hours per week from the payer perspective. (C) Monthly competitive employment ≥1 hour from the societal perspective. (D) Monthly competitive employment ≥1 hour from the payer perspective.

Return on investment

Return-on-investment results can be found in table 4. These results show that implementing IPS instead of TVR resulted in positive financial return estimates from the societal and payer perspective. To illustrate, the NB and BCR from the societal perspective show that—on average—implementing IPS instead of TVR resulted in net saving of €364 (NB) and a return of €2.1 per euro invested (BCR). All return-on-investment estimates, however, were associated with high levels of uncertainty.

Table 4
|
Costs, benefits, net benefits (NB) and benefit cost ratios (BCRs)

Sensitivity analyses

Effectiveness

Online supplemental file 4 presents the estimated coefficients per month for the effect of IPS over TVR with 95% CI for (1) the old funding regimen for individuals receiving a disability benefit, (2) the new funding regimen for individuals receiving a disability benefit and (3) the new funding regimen for individuals receiving a sickness benefit. These analyses showed that IPS is effective, on average, for both effect measures in individuals receiving a disability benefit, with the highest impact in the old funding regimen. In addition, IPS was not found to be effective in individuals receiving a sickness benefit.

Cost-effectiveness and return on investment

Table 4 shows that when disability costs were excluded from the cost-effectiveness analysis, IPS was—on average—more costly and more effective than TVR, meaning that it no longer dominated TVR and that its cost-effectiveness will more strongly depend on the willingness to pay of decision-makers. The stratified results showed that the cost-effectiveness and return on investment of IPS compared with TVR were more favourable for the old funding regimen compared with the new funding regimen.

Discussion

Main results

The aim of this study was to assess the effectiveness, cost-effectiveness and return on investment of IPS reimbursed on a nationwide scale compared with TVR regarding sustainable participation in competitive employment in individuals with SMI receiving disability benefits. IPS was found to be more effective than TVR in helping individuals obtain and sustain competitive employment from 6 to 36 months after starting the intervention. From both the societal and payer perspective, IPS dominated TVR, meaning that it was—on average—less costly and more effective. Its eventual cost-effectiveness, however, will depend on what decision-makers are willing to pay per individual extra returned to competitive employment and what probability of cost-effectiveness they perceive as reasonable. For that, they can use the results of the current study. Return-on-investment estimates showed that IPS was—on average—cost saving, but the level of uncertainty surrounding these estimates was large. The latter is likely due to the high level of uncertainty surrounding the cost estimates, which is typical for cost outcomes, as they tend to be heavily right skewed, and hence relatively large sample sizes are required for obtaining significant results. The explanation for the more favourable return on investment of IPS in the old funding regimen compared with the new funding regimen lies in the higher healthcare costs incurred by the IPS group in the new funding regimen.

Comparison with literature

The finding that IPS outperformed TVR in helping individuals with SMI obtain competitive employment is consistent with numerous previous studies on the effectiveness of IPS.7–9 A Dutch randomised controlled trial, for example, found a 19%-point higher probability of IPS participants having worked after 30 months compared with TVR participants.39 The estimates in this study were slightly lower (ie, 9%-point higher probability of having worked for at least 1 hour per month after 30 months), which may be attributed to the following factors: (1) all individuals in the TVR group of the current study actually received treatment, compared with only an 80% take-up rate in the aforementioned trial; (2) the matched IPS population in the current study likely had less severe mental health problems (ie, few individuals received treatment for schizophrenia in the last 3 years) compared with the population in the previously mentioned trial, while research has shown that IPS is relatively more effective for individuals with SMI and schizophrenia spectrum disorders.40

Although IPS aims to rapidly place individuals in paid work, its effects were found to be significant only from month six after the start of the intervention. This is in line with previous research, reporting significant effects after 6 months of participation in the intervention.27 39 An explanation for this delayed impact may be that it takes time for IPS employment specialists to establish a trusting relationship with their clients, and to create opportunities for them to gain work and learning experiences.24 In addition, participating in activities focused on obtaining competitive employment may improve clients’ work motivation over time, resulting in better work outcomes.24 41 42

This study found that IPS was generally effective in individuals receiving a disability benefit, but was not effective in those receiving a sickness benefit, who typically have more working experience. This finding is consistent with previous research, suggesting that individuals with a greater distance from the labour market benefit more from IPS compared with TVR.40 43

The finding that IPS is on average less costly and more effective than TVR is in contrast with a previous European multisite trial, conducted between 2003 and 2005, in which IPS dominated TVR in all countries except for the Netherlands.13 The improved impact of IPS in the Netherlands may be due to the introduction of nationwide reimbursement of IPS, where a fair or good IPS fidelity score is a prerequisite. Given that high model fidelity is linked to improved employment outcomes, this reimbursement may have resulted in improved implementation of IPS, and consequently improved outcomes.26 28–31 Another reason may be the larger sample size and longer follow-up period in this study.12 As this study is conducted in the period 2012–2019, it is also possible that the improved outcomes are influenced by changes in economic conditions in the community, greater acceptance by community employers and legislation stimulating work participation.25 28

The finding that healthcare costs were slightly higher (but not statistically significant) in IPS participants was not in line with previous economic evaluations of IPS10 12 and was unexpected since having a competitive job is associated with improved mental health, quality of life and global functioning,44 45 which in turn can lead to a reduced need for treatment and lower healthcare costs in the long term.10 12 An explanation for this finding may be that in the Netherlands, employment specialists regularly claim expenses from health insurance companies for follow-up support to IPS participants at risk of losing their job after their trajectory ends.24 These claims require approval from psychiatrists they are collaborating with. This approval process can lead to a divergence in the follow-up support offered, influenced by the psychiatrists’ personal opinions of the IPS approach or their impressions of the participants involved.24

Strengths and limitations

This study represents a natural experiment conducted on a nationwide scale to implement IPS. It is the first study examining the effectiveness, cost-effectiveness and return on investment of IPS implemented by reimbursement on a nationwide scale, compared with TVR regarding sustainable participation in competitive employment in individuals with SMI receiving sickness or disability benefits. The use of administrative instead of self-report data and the long follow-up period are also strengths of this study. Moreover, the ‘common trend assumption’, necessary for valid difference-in-difference estimates, along with the presumption that incomplete data would not cause significant bias in the results, were confirmed. This was evident from the fact that the results of the cost-effectiveness and return-on-investment analyses, which were based on complete cases only, were in line with those of the effectiveness analysis, for which all data were used. There are also several limitations. To reduce the influence of confounding, exact matching was performed with many different matching criteria, resulting in a control group with exact the same characteristics as the intervention group. However, it is plausible that unobserved factors that were not used as matching criteria may also have influenced employment outcomes, such as educational level, diagnosis, quality of life and self-esteem.46–48 Exact matching may also have the limitation that fewer matches can be made due to a lack of exact matches for every individual receiving IPS (ie, the common support is smaller). This may have led to potential selection bias and limited generalisability of the findings, as it restricts the number of individuals who can be included in the analyses. However, considering the relatively similar results obtained from a post hoc analysis using propensity score matching (data not shown), it is unlikely that selection bias exerted a significant impact. Data on additional interventions participants received from their mental healthcare providers were also not available. Although the control group was limited to individuals who received treatment from mental healthcare agencies that do not offer IPS on a large scale, selection bias could not be ruled out. The criteria used for the operationalisation of SMI in administrative data are in line with a commonly used definition of SMI in the Netherlands.1 It is, however, that not all participants in this study have an SMI according to this definition.1 In addition, the matched IPS sample scored lower on all SMI criteria and had lower healthcare costs per month compared with the total IPS group. Therefore, the analyses may be based on a subgroup with relatively more favourable characteristics. The societal perspective did not include some other relevant costs, such as presenteeism (ie, lost productivity while being at work) and productivity losses from unpaid work. In addition, the healthcare costs only included those covered by the Dutch basic health insurance package, and hence excluding over-the-counter medications healthcare insured through additional healthcare packages and out-of-pocket expenses.

Implications for practice and research

The results of this study can help organisations involved in IPS, such as mental healthcare agencies, benefit agencies and health insurance companies, make better-informed decisions about the implementation and funding of IPS. Since inadequate funding has been one of the major barriers to successfully implement IPS in practice, a significant amount of resources has been allocated for its implementation in the past few years.22 24 25 49 This research shows that reimbursement for IPS on a national scale is effective and could be rewarding for both individuals with SMI and society as a whole. This unique domain-transferring approach, therefore, yields a lot for both individuals with SMI and society as a whole.50 Because it was a temporary reimbursement for IPS, the Ministry of Social Affairs and Employment has now instructed UWV to make this reimbursement permanent for individuals receiving benefits from UWV. This change is expected to stimulate more mental healthcare agencies to start offering IPS services, improve the fidelity of IPS programmes and enhance the expertise of the professionals involved. Additionally, individuals with SMI receiving benefits from municipalities in the Netherlands are currently eligible for temporary IPS reimbursement. In order to also increase the labour participation of this group of people, the findings of this research could influence the Ministry’s decision to make this compensation permanent.

Although the results of this study appear promising and seem to justify implementation by nationwide reimbursement, there are still other barriers that need to be addressed, such as making competitive employment financially more appealing than receiving benefits for participants and reducing the impact of the so-called benefit trap.11 15 24 25

To avoid divergence in follow-up support in IPS participants at risk of losing their job after their trajectory ends, health insurance companies could consider adapting their IPS reimbursement to current practice (ie, reimburse more than only the IPS intake for a maximum of 8 hours).24 25 Another approach to offering follow-up support to IPS participants at risk of job loss after completing their trajectory could involve close collaboration with occupational physicians and insurance physicians. These specialists can provide advice to participants on addressing health-related challenges at work, helping them to stay employed.

Future economic evaluations should be conducted with a larger sample size, additional cost categories (eg, presenteeism) and use quality-adjusted life-years as an outcome measure. In addition, employment outcomes concerning long-term, sustainable employment should be evaluated.

Conclusions

IPS implemented through a reimbursement strategy on a nationwide scale is more effective and potentially cost-effective than TVR in people with SMI receiving sickness or disability benefits. Based on these results, the implementation of IPS by a wide scale reimbursement strategy could be promoted to enhance sustainable participation in competitive employment in these individuals. Future economic evaluations should strive for a more robust sample size and a long follow-up period.

Overview

Article metrics
Altmetricopen-url
Dimensionsopen-url