The team approached the study by examining the implementation and impact of the Act through its five principles – voice and control, well-being, co-production, multi-agency working, and prevention and early intervention – and the financial implications of each.
Each of these principles were evaluated by an academic ‘theme lead’ supported by an expert adviser. This combination was crucial to ensuring that skills and knowledge of the academics was complemented by the practical expertise and experience of making and implementing policy, delivering and managing social care, working with citizens across organisational boundaries, and putting people at the centre of ‘what matters’ to them. More on the approach can be understood by watching this film.
The IMPACT study used Michael Patton’s ‘Principles-Focused Evaluation’ (P-FE) approach as the overarching framework. P-FE focuses on evaluating how principles guide the implementation of interventions or programmes in contexts which are complex, uncertain and ‘turbulent’, and what happens as a result. There are three central questions that are answered in a P-FE evaluation, and these framed the work done across this study:
1. To what extent have meaningful and evaluable principles been articulated? (Conceptualisation)
2. If principles have been articulated, to what extent and in what ways are they being adhered to in practice? (Implementation)
3. If adhered to, to what extent and in what ways are the principles leading to the desired results? (Optimisation)
The IMPACT project was a programme of work which constituted of 11 individual studies which were undertaken and reported on from November 2018. In all, we heard from more than 450 study participants from across Wales, all of whom have taken their time to provide detailed and comprehensive accounts of their experiences under the Act, from a range of perspectives.
The Study Expert Reference Group reflected the overall model of engagement and partnership underpinning the development and implementation of the Act, ensuring the evaluation reflected the ‘on the ground’ experiences of key stakeholders including people who use services and their carers. The group did not impact the outcome of the independent research undertaken but rather advise and offer a steer on the programming, methodologies and engagement developed by the team, acting as a critical friend.
The group was a sounding board on overcoming potential barriers and on developing creative opportunities. The group enabled the development of strategic and sustainable linkages between researchers and key stakeholders, and shared information about approaches and services being developed locally and nationally to inform the researchers and the study.
The group was jointly chaired by three lay people: a lay expert to link to the academic team, a young person to ensure the rights of children and young people were reflected in the study, and an adult to ensure the rights of service users and carers are reflected in the study.
