Assessing the impact of employment support in Manchester30 January 2018
Learning and Work Institute conducted an impact assessment for the Department of Work and Pensions of the early impacts of Greater Manchester’s Working Well pilot programme, which has recently been published.
Working Well provides locally-designed employment support to ESA claimants who had received support through the Work Programme for two years but were not in work at the time of entry to Working Well. This is described in the Greater Manchester Combined Authority’s own evaluation.
Greater Manchester are using the ‘Working Well’ name as a brand-name for their wider set of locally designed employment programmes. Our findings relate only to the pilot we analysed, and not to other programmes under the Working Well banner.
DWP asked Learning and Work Institute to identify impacts on:
- Job starts
- Time in work
- Time off benefit
This was in comparison to other similar ESA claimants – who we identified statistically from DWP-supplied data. The matched comparison group had Jobcentre Plus support budgeted at 88 minutes of Work Coach time a year.
What we found:
- Early figures showed 4.7% of both Working Well and a matched comparison group had started work.
- 9.8% of the comparison group had been off benefit for 13 weeks or more, compared to 8.1% of Working Well participants.
- 23.1% of the comparison group had any time off benefit compared to 21.6% of Working Well participants.
- We wanted to know if the weakness of the Greater Manchester labour market, compared to the rest of Great Britain, made a difference to the outcomes of Working Well – it did, and this accounted for those headline differences that were negative, such as those noted above.
- There was a significant difference in favour of Working Well for jobs lasting for 26 weeks or more.
- The positive employment effect of Working Well came via helping participants who had entered work to stay in work for longer than the comparison group rather than by increasing the rate at which participants moved into work when compared to non-participants. We found that participation in Working Well increased weeks in work for those who entered it by 4.6 weeks.
- There were no significant differences in other job measures or off-benefit measures.
- The DWP-supplied data on benefits and jobs meant that we could closely match the characteristics, and work and benefit histories of Working Well participants.
- DWP should help local partners find out if their innovations to help benefit claimants into work actually do work – by either making data available or by running a datalab service as the Ministry of Justice do.
Using DWP Administrative Data is a powerful approach to evaluation
- For this report, we had access to DWP administrative data linked to HMRC tax records. This provided information on the subsequent employment outcomes of participants in Working Well and the matched comparison group of non-participants.
- We used a Propensity Score Matching (PSM) approach to matching participants and non-participants to estimate the net impact of Working Well. Matching approaches, including PSM, can only match programme participants against non-participants on the basis of observed factors, such as age, gender, length of time on out of work benefits etc. We included in the matching previous work and benefit receipt histories. This ensures that participants and those they are compared with have similar prior labour market experiences. This reduces the chances of there being relevant unobservable differences between them. This is because if there were such differences between the two groups, which affected labour market outcomes, then we would expect to see differences in their work and benefit histories.
- But, while it is not possible to rule out that a matched comparison group may have unobservable differences from participants that make them more or less likely to move off out-of-work benefits or into work than participants, we think we have reduced this problem to the minimum attainable.
- It was the access to DWP administrative data linked to HMRC data that allowed us to match on work and benefit receipt histories. This shows the importance of access to DWP administrative data for estimating the impact of labour market interventions.