Justice Social Work - CI National Review - CHAPS 19 08 25
Renfrewshire Council · 19 August 2025
Back to Communities and Housing Policy Board — 19 August 2025 and its agenda
Text extracted from the council's PDF. Layout and reading order may differ from the original.
Page 1
___________________________________________________________________ To: Communities and Housing Policy Board On: 19 August 2025 ___________________________________________________________________ Report by: Director of Children’s Services ___________________________________________________________________ Heading: Justice Social Work - Care Inspectorate National Review Self-evaluation of performance, quality and outcomes ___________________________________________________________________ 1. Summary 1.1 On 13 May 2025 the Care Inspectorate published a national self- evaluation review Report on performance, quality and outcomes of community sentences in Justice Social Work. 1.2 To inform this national review all 32 local authority justice social work services, including Renfrewshire Council, submitted its own self-evaluation during November 2025. These returns were augmented with on-site validation activities in six, selected local authority areas, early in 2025. Renfrewshire Council was not a selected site. 1.3 The Report found that Justice Social Work (JSW) services are committed to driving improvement and generally are confident and capable in planning and implementing self-evaluation. It noted however that across Scotland quality and outcomes data are not systemically gathered, consequently limiting the ability to demonstrate the effectiveness of community based sentences. 1.4 Both local and national barriers were highlighted in the Report. Locally, these included resource constraints, staffing shortages and challenging reporting requirements. Nationally, it highlighted strategic gaps including a need to overcome infrastructure limitations within current national data gathering and information sharing activities.
Page 2
The report concluded a need to establish a shared, national framework that employs nationally agreed measures and tools which link better current quantitative reporting demands with the gathering and collation of qualitative data. 1.5 The Care Inspectorate highlighted how these conclusions contrast the consistently positive messages it heard from service users about their experiences of their involvement with JSW. 1.6 The Report’s general conclusions are not unexpected and reflect some of the challenges already identified locally within Renfrewshire JSW. Developing a more structured approach to quality assurance for JSW was a priority identified within the wider children and justice social work service’s redesign programme. 1.7 A quality assurance and improvement framework for JSW was produced in January 2025. This provides a systemic and structured approach for reviewing local performance and effective service development. It embeds different evidence gathering systems to better assure, evidence and develop practice quality. Underpinning the framework is a principle to learn better from the experience of those who use and directly practice within the service. This learning will be integrated within the evidence used to inform future service improvement and effective resource management. ___________________________________________________________________ 2. Recommendations 2.1 To note the Care Inspectorate’s recently published review report on performance, quality and outcomes of community sentences in Justice Social Work. 2.2 To note the progress to date made locally by Justice Social Work against the national Report’s conclusions. _________________________________________________________ 3. Background 3.1 Findings from the Care Inspectorate’s previous scrutiny work of JSW have highlighted its view that performance management and quality assurance were areas for national improvement. In 2021, Social Work Scotland’s (SWS) Justice Standing Committee recognised this. It established a working group with a remit for performance, quality assurance, service user feedback and continuous improvement. The Care Inspectorate advised that its national review sought to build on previous scrutiny activity and add value to the work of the SWS sub-group by: evaluating the extent to which justice social work services were able to evidence performance, quality and outcomes in relation to community- based sentences.
Page 3
exploring the factors that impacted justice social work services’ ability to confidently and robustly demonstrate the effectiveness and impact of community support and supervision. using a self-evaluation approach to building capacity for improvement across the sector. 3.2 The Care Inspectorate state its conclusions are intended to “meaningfully inform specific improvement work planned by the national SWS Justice Standing Committee, wider national stakeholders, and improvement priorities of wider national and local leadership”. Nationally, the progression of the report’s conclusions is under active consideration by SWS. 3.3 Within Renfrewshire, the Children’s Services’ redesign process had identified an improvement priority for JSW to more systematically develop quality assurance and performance activities. Similar to the findings in the Report, the redesign process found that ongoing local planning and improvement activity did take place. However, this could be ad hoc and specific activities were often in response to a current emerging national or local policy priorities. The service redesign identified specific limitations in strategic management capacity in JSW as a key barrier. 3.4 During 2024 JSW incrementally implemented its service redesign. This brought a tangible increase in strategic management capacity to lead on a more robust sustainable approach to quality assurance, best value and improvement work. In implementing this there was no reduction to frontline staff delivery which has continued to perform very well throughout the change period of the redesign. 3.5 During this same period JSW started to progress and complete its improvement framework. Shaped by national / local priorities, available data, staff and service user views, it frames a planning structure that methodically approaches quality assurance to better inform service improvement and resource management. This framework is now embedded within the service. It will be reviewed regularly and adapted as necessary, to any specific areas including those that may evolve from the national Report’s conclusions in the future. 3.6 The improvement framework’s application is supported by an oversight group and two ongoing delivery groups. It is directed by a current JSW service plan. Specialist sub team plans also have their own plans that correlate to the same shared priorities. As staff connectivity and a learning culture, embracing shared values and goals are integral to improvement, officers of all grades are actively involved. 3.7 Learning from the service users and other stakeholders about their experiences and what works for them is a key component underpinning the framework. Activity is now underway to better gather this information more systematically and directly influence future service planning.
Page 4
Some innovative examples have recently been undertaken in partnership with two third sector providers, Open Aye and Braveheart Industries. Working together but independently these small projects have provided both support and learning experiences for service users whilst simultaneously collating their observations and views of both JSW and the wider justice system it works within. The qualitative information this offered is being integrated into future service planning. 3.8 In terms of the broader nationally based conclusions Renfrewshire JSW will continue to actively participate and support the work of SWS and other national stakeholders in the activities required to achieve a national, shared framework. ___________________________________________________________________ Implications of the Report 1. Financial - None. 2. HR & Organisational Development - None. 3. Community/Council Planning • Building strong, safe and resilient communities – Effective justice social work achieving the best outcomes for people in the justice system helps build safer and more resilient local communities. 4. Legal - None. 5. Property/Assets – None. 6. Information Technology - None. 7. Equality & Human Rights - (a) The Recommendations contained within this report have been assessed in relation to their impact on equalities and human rights. No negative impacts on equality groups or potential for infringement of individuals’ human rights have been identified arising from the recommendations contained in the report as it is for noting only. 8. Health & Safety – None. 9. Procurement – None. 10. Risk - None. 11. Privacy Impact - None. 12. Cosla Policy Position – None.
Page 5
13. Children’s Rights – None. 14. Climate Risk – None. _________________________________________________________ List of Background Papers (a) Appendix A. “Justice Social Work – Self-evaluation of performance, quality and outcomes. A national review” by the Care Inspectorate ___________________________________________________________________ Author: David Wilson, Justice Services Manager Email : david.wilson@renfrewshire.gov.uk
Page 6
Justice Social Work Self-evaluation of performance, quality and outcomes A national review
Page 7
Page 1 of 38 Justice social work self-evaluation Contents Introduction 2 Key messages 6 National context 8 Findings - Performance - Quality - Outcomes - Organisational drivers - Learning from a self-evaluation approach 11 11 14 17 21 29 Conclusions 31 Considerations for the sector 32 Appendix 1 - Quality Improvement Framework 33 Appendix 2 - The terms we use in this report 36 Published: May 2025
Page 8
Page 2 of 38 Justice social work self-evaluation Introduction The Care Inspectorate is the independent body tasked with undertaking scrutiny, providing assurance and supporting improvement in justice social work. Between 2018 and 2021 we focused on the inspection of Community Payback Orders (CPOs) delivered by justice social work services. In recent years, often in partnership with other scrutiny bodies, we have undertaken broader, thematic work focused on other justice priorities such as Throughcare, Diversion from Prosecution and Prison-Based Social Work. Informed by the current national aims and objectives of the National Strategy for Community Justice, and in light of the current pressures created by prison overcrowding, the time is now right to renew our focus on the delivery of community sentences by local authority justice social work services. Aim 2 of the National Strategy for Community Justice is to “Ensure that robust and high-quality community interventions and public protection arrangements are consistently available across Scotland”. In relation to community sentences, there is an associated priority action to “Ensure that those given community sentences are supervised and supported appropriately to protect the public, promote desistence from offending and enable rehabilitation by delivering high quality, consistently available, trauma-informed services and programmes.”1 Key to delivering on these intentions, and the overarching aim, is the ability of justice social work services to demonstrate that the supervision and support offered to those on community sentences is of a high quality. Building confidence in community sentences is also a vital component in addressing the systems issues relating to prison overpopulation. The findings of this review are deliberately detailed to meaningfully inform the considerations of all stakeholders on these issues. Background The findings from our previous justice scrutiny work undertaken between 2018 and 2021 highlighted that performance management and quality assurance were key areas for improvement 2. They revealed that, in general, there was no consistency in the format of performance frameworks, the measures being captured, or the frequency of reporting. Some local authorities had yet to clearly define a range of strategic priorities for the justice service or fully embed approaches to measuring performance. Accessing timely and reliable data to aid analysis of performance was often challenging as information management systems could be difficult to interrogate. Services were often limited in their ability to demonstrate the difference that CPOs were making in people’s lives. Most services had yet to agree a set of clearly identified, person-centric outcomes against which progress or change could be demonstrated. This was exacerbated by an absence of consistent data or standardised mechanisms to capture the necessary information. Services were not consistently gathering feedback from stakeholders or people using their services. In 2021, Social Work Scotland’s (SWS) Justice Standing Committee recognised this as an area of priority which prompted the establishment of a sub-group with a remit 1 National Strategy for Community Justice, Priority Action 5, p12 2 Justice Overview Report 2018-2021
Page 9
Page 3 of 38 Justice social work self-evaluation for performance, quality assurance, service user feedback and continuous improvement. Aim and approach of the review This review sought to build on previous scrutiny activity and add value to the work of the SWS sub-group by: • evaluating the extent to which justice social work services were able to evidence performance, quality and outcomes in relation to community-based sentences. • exploring the factors that impacted justice social work services’ ability to confidently and robustly demonstrate the effectiveness and impact of community support and supervision. • using a self-evaluation approach to building capacity for improvement across the sector. We developed a two-part self-evaluation approach in order to gain a national picture of what was working well and where improvement was required. In the development phase, we engaged with national stakeholders and piloted the approach with two local authorities: South Lanarkshire and City of Edinburgh. Phase 1 took place between September and November 2024 during which all 32 local authorities completed a structured self-evaluation template in which they considered: • their current approaches to gathering, reporting and responding to performance, quality and outcome data • the extent to which organisational drivers were supporting the effective gathering, reporting and use of data to provide assurance and drive improvement For each element, services were asked to rate themselves using a scale of ‘fully’, ‘mostly’, partially’ and ‘not at all’. All 32 submissions were analysed to identify key themes, strengths and areas for improvement.
Page 10
Page 4 of 38 Justice social work self-evaluation Phase 2 of the review took place between January and March 2025. Informed by the emerging themes from the national submissions, we used a validation approach to better understand the strengths and challenges at a local level. A further four local authorities - Aberdeenshire, East Dunbartonshire, Fife and Perth and Kinross - were chosen to ensure the review included a range of service delivery models, governance structures and a broad geographic spread. We deliberately chose areas that had not previously been subject to justice scrutiny activity. Selection was not based on concerns regarding any perceived service delivery risk. Throughout the review a mix of methods were used. These included: • semi-structured interviews with national stakeholders • a review of documentary evidence referenced within the six local authority self-evaluations • in-person focus groups with leaders, managers, staff and people using services in each local authority, held over two days During the phases of the review we spoke to and heard from a total of 193 people and four national stakeholder organisations: 47 people using justice services 56 front-line staff 46 operational managers 22 strategic managers 12 senior leaders 10 performance staff Scottish Government Justice Analytical Services Social Work Scotland Justice Standing Committee leaders Community Justice Scotland Risk Management Authority
Page 11
Page 5 of 38 Justice social work self-evaluation Acknowledgements We are extremely grateful to all 32 local authorities and their staff who made time to analyse and reflect on their practice during their self-evaluation. In addition to the people and organisations noted above we would like to thank: • the people using justice services who took the time to share their views and experiences with us • the pilot areas for their proactive interest in testing new ways of working • the co-ordinators and support staff from the six local authority areas for the extremely well-organised in-person validation activities All six services that participated in the validation activities took full ownership of the process and staff at all levels actively engaged in discussions.
Page 12
Page 6 of 38 Justice social work self-evaluation Key messages Justice social work services are committed to driving improvement. • Services have a clear sense of their strengths and areas for improvement. • Services are working hard to consistently meet their statutory reporting responsibilities and are making good use of frameworks, where these exist. Important organisational drivers are supporting improvement across services. • Services demonstrate strong leadership, a reflective learning culture and robust governance structures. • Services are investing in developing and sustaining specialist knowledge and expertise. People using justice services consistently report that the support and supervision they receive is having a positive impact on their lives. • People describe the experience of being on an order as ‘transformational’ and ‘life-saving’. They feel supported, safe, and encouraged to progress. • Staff are noted to be responsive to needs, appropriately challenging of offending behaviour, and non-judgemental in their approach. Services are not systematically gathering and reporting quality or outcomes data. This limits capacity to demonstrate the effectiveness of community sentences. • There is a disproportionate focus on gathering and reporting performance metrics. Much of what is gathered and reported is not informing local improvement. • Many services lack a systematic quality assurance framework. • There is no agreement or shared language about quality and outcome measures. Tools are not consistently applied to support reliable data gathering and reporting. Significant barriers are hindering services’ efforts to evidence the quality and impact of their service delivery. Local barriers include: • Restrictive information management systems which limit and complicate data gathering. • Resource constraints, staffing issues and onerous reporting demands which limit capacity for quality assurance and improvement activities. National barriers include: • The lack of a shared strategic approach which articulates agreed measures, tools, processes and priorities for data gathering and reporting
Page 13
Page 7 of 38 Justice social work self-evaluation • The lack of shared leadership and clarity regarding collective roles and responsibilities • The need to overcome the limitations of the national data gathering and information sharing infrastructure
Page 14
Page 8 of 38 Justice social work self-evaluation National context Justice Social Work Justice wocial work services across Scotland are enabled by Section 27 of the Social Work (Scotland) Act 1968. Although services are delivered as part of a local authority or Heath and Social Care Partnership (depending on the governance arrangements) the bulk of justice social work service funding is in the form of a ‘ring- fenced’ grant from the Scottish Government. Additional money is also transferred to local authorities from the Scottish Government Community Justice budget. Local authorities may also support justice-related activities from their overall budget allocation. The Grant Aided Expenditure details the allocation of funds across the 32 local authority areas using the justice social work funding formula. Understanding the type and amount of work undertaken by the justice social work service is crucial to this process. Key to determining these figures is the production of statistical returns required by the Scottish Government at regular points throughout the year. The production of this data is a significant undertaking for every local authority and it is essential to get it right in order to avoid being financially disadvantaged. Each April Scottish Government Justice Analytical Services issues all the data forms to be returned by the local authority during the course of the coming year. The annual collection is split into two delivery processes. The aggregate return includes data in relation to: • Diversion from prosecution • Fiscal work orders • Bail supervision • Justice social work reports • Structured deferred sentences • Statutory and voluntary throughcare • Pre-release reports • Home detention curfew assessments • Court-based services The second part of the process, the unit level return focuses on: • Community payback orders • Drug treatment and testing orders Significant checks and balances are incorporated into these processes to ensure accuracy, with Scottish Government Justice Analytical Services and local authorities communicating routinely throughout the process. The Local authority social work
Page 15
Page 9 of 38 Justice social work self-evaluation statistics (LASWS) justice group chaired by Scottish Government Justice Analytical Services is crucial to supporting the efficient and effective collection and monitoring of justice social work data. The data requested by the Scottish Government is analysed and assured by Justice Analytical Services and then published as Justice Social Work Statistics. Following a comprehensive review of Justice Social Work Statistics in 2024 the publication was split into two which allows certain data to be in the public domain earlier than was previously the case. With more than half of the data indicators within the National Community Justice performance framework reliant on this national data-set, the statistics are critical to informing national policy, strategy and direction as well as supporting local performance reporting. The Scottish Government Justice Analytical Services team valued the significant efforts made by local authorities to submit the required data by the due dates. They recognised that the size and complex nature of the submissions can cause difficulties for local authorities. Additional challenges arise if experienced staff leave or are unavailable. The most significant issue impacting efficiency is the lack of standardisation, with different local authorities using a variety of information management systems, some of which are more effective than others. Any updates to often outdated systems can be costly financially and in terms of staff time and expertise. These challenges become more pressing as the Scottish Government attempts to shift the balance from custody to community and the range of community options managed by justice social work services increases. Other reporting systems The Level of Service/Case Management Inventory (LS/CMI) is the comprehensive general offending assessment and management planning method used by all community and prison-based justice social work services in Scotland. It is used to aid decisions on the level and focus of intervention with people (aged 16+) who have been involved in offending. The LS/CMI method was adapted for use in Scotland by the Risk Management Authority (RMA) to enable an evaluation of the pattern, nature, seriousness and likelihood of offending and helps structure professional decision-making in a manner that is consistent and understandable regardless of the nature or complexity of the case. As such the method aligns with the principles of FRAME (Framework for Risk Assessment, Management and Evaluation). Training in the use of the method is delivered by Community Justice Scotland with additional training provided by the RMA to support risk assessment and management practice with individuals who present a risk of serious harm Owned by the Scottish Government, the electronic LS/CMI portal is used by justice social workers in the community and in prison to record assessment and case/risk management information. Additionally, the portal is used to record key data about accredited programme delivery. The national electronic system for supporting the efficient, effective and consistent application of the LS/CMI method is hosted and supported by an externally commissioned supplier. While LS/CMI is invaluable to professional decision-making in individual cases, local aggregated reporting from this aspect of the system is limited, and there are no national reporting requirements currently in place to utilise the rich data within the system to inform national reporting of performance, quality and outcomes.
Page 16
Page 10 of 38 Justice social work self-evaluation Community Justice The Community Justice (Scotland) Act 2016 provides the legislative framework for the model of community justice in Scotland. A National Strategy for Community Justice was published in 2022 followed by a revised Community Justice Performance Framework (CJPF) comprising nine nationally determined outcomes and ten national indicators used in measuring local and national performance against the national outcomes. As statutory partners within community justice partnership arrangements, local authorities, including justice social work services, contribute performance data as required by the Act. This reporting is expressed in their respective Community Justice Outcome Improvement Plans, aligned with the CJPF, with reports returned to Community Justice Scotland annually for assessment and analysis of performance. There is also a statutory requirement for local authorities to report specifically on the delivery of CPOs by justice social work services in their area. This reporting is delivered using a national Community Payback Order Annual Returns template and returned to Community Justice Scotland and the Scottish Government. Community Justice Scotland collates these returns and summarises them in a report which is laid before the Scottish Parliament. This report is then published nationally as the annual Community Payback Order: Summary Of Local Authority Annual Reports. Public Protection Local authorities are a defined Responsible Authority, amongst others, within the Multi-Agency Public Protection Arrangements (MAPPA) in Scotland. Justice social work services are responsible for the assessment and management of certain categories of people in accordance with the national MAPPA guidance. Section 11 of the Management of Offenders etc. (Scotland) Act 2005 sets out the statutory requirement for each MAPPA region in Scotland to produce and publish an annual report, and to provide information to the Scottish Government, on the delivery of MAPPA in their area. Justice social work services report information on performance via their local MAPPA Co-ordinator and MAPPA Strategic Oversight Group which informs annual reporting. The Scottish Government compiles and publishes an annual National Overview Report informed by the annual reports from the ten MAPPA regions in Scotland.
Page 17
Page 11 of 38 Justice social work self-evaluation Findings We asked local authority justice social work services to evaluate the efficiency and effectiveness of their arrangements for gathering and reporting performance, quality and outcome data. This was done using a four-point scale: ‘fully’, ‘mostly’, ‘partially’ and ‘not at all’. Using a structured template, services were also asked to evaluate the extent to which key organisational drivers were impacting their ability to gather and report data. These included a culture of learning and improvement, leadership, governance, knowledge/expertise and resources. The findings that follow are reported against the key elements of the self-evaluation template. They include a summary of the themes arising from the 32 self-evaluation reports and validation activity in the six local authority areas. Related findings from discussions with national stakeholders are included at relevant points. To guide the reader, key themes are highlighted in bold throughout. Performance: Learning from the 32 local authority self-evaluations Services rated how effectively they were measuring their performance in delivering support and supervision for people on community sentences. In reaching their conclusions they were asked to specifically evaluate the extent to which they: • had a clear and comprehensive performance management framework • had a set of clearly articulated key performance indicators (KPIs) relating to their core functions • had processes and tools in place to ensure routine gathering and collation of data relating to their core functions • were able to evidence improvement initiatives informed by performance data From the 32 responses, two local authorities rated their performance management arrangements as ‘fully’, 15 as ‘mostly’, and 15 as ‘partially’. This indicated that almost half of the services identified the measurement of performance as an area for improvement. This finding reflected the fact that the majority of services indicated they did not have a clear justice performance framework in place. While some services made reference to elements of a framework, only two services were ‘fully’ confident they had a robust framework in place. Most services recognised the need to ensure that performance improvement priorities were clearly identified, actioned and monitored. Encouragingly, almost all services had clear key performance indicators (KPIs) relating to the core functions of justice social work, though the range and number of indicators varied. Some KPIs were limited to statutory reporting requirements for the Scottish Government returns. This statistical overview was widely agreed to be
Page 18
Page 12 of 38 Justice social work self-evaluation providing a helpful, common national baseline and services were noted to be working very hard to provide the required data. Where additional local indicators were tracked, these related to Serious Incident Reviews, staff sickness absence and complaints. Services that rated themselves ‘partially’ for this question were keen to extend their range of reported KPIs and acknowledged a need to review their current indicators to ensure these were aligned with local and national priorities. Over half of services were confident they had processes and tools to support routine performance monitoring. The range of processes and tools to support local and national performance monitoring varied significantly. They included a mix of locally developed data platforms, dashboards and Excel spreadsheets and some nationally developed checklists on the delivery of accredited programmes. Although all services provided the same data to meet Scottish Government Justice Analytical Service’s reporting requirements, there was no common approach to generating that data at a local level. Although the majority of areas were confident they had tools to monitor performance data, challenges were noted in relation to time, resource and data quality. These were compounded by laborious manual data extraction methods, a lack of training in data interrogation and a reliance on social work information management systems which some areas felt had limited functionality. Almost half of all services noted significant issues with the reliability and functionality of their IT and information management systems. Some services had adopted manual and cumbersome ‘workarounds’, and over a third of services were investing in new information management systems. Those in the early stages of implementation expressed optimism that the new systems would support better data gathering and reporting. However, several services indicated significant problems following implementation of new systems which meant they were no longer able to reliably report on data at all. All services indicated that supporting the development of new systems significantly limited capacity for service delivery and improvement work. Positively, the majority of services felt able to evidence improvement initiatives informed by performance data. This was particularly noted in services where staff had been trained in quality improvement and systematic processes and the monitoring of trend data was supporting improvement activity. In other services, performance improvement initiatives were more reactive; driven by developments in national strategy or practice, rather than local performance data. Performance: Learning from local validation activity The self-evaluations for the six validation areas reflected mixed confidence levels in performance management arrangements, ranging from ‘partially’ to ‘fully’. Arrangements to capture and report on performance varied. Where a performance management framework was in place it was not always underpinned by a coherent system for gathering, analysing, reporting and evaluating performance data. In other instances there was a data dashboard but no corresponding framework.
Page 19
Page 13 of 38 Justice social work self-evaluation Spotlight on Practice – Performance Services that had developed a data dashboard had also established a corresponding forum in which key staff reviewed, interrogated, and analysed performance data to consider practice implications. Although operational leaders and managers acknowledged varying levels of confidence in analysing and interpreting data, these arrangements worked best where staff with expertise in performance management and data analysis were included. This suggested that a structured approach to the review of data, which involved dedicated time, relevant expertise and focused discussion was helpfully contributing to the identification and development of evidence-led improvement initiatives. As we found nationally, beyond the measures in the Scottish Government annual and quarterly returns, there was little consistency in the key performance indicators being gathered and reported, despite the relatively standardised functions of the justice social work role. There was very limited data on performance metrics in relation to the completion of LS/CMI assessments and case management plans. Approaches to monitoring the delivery of statutory reviews in accordance with national outcomes and standards varied. Most services gathered some form of workload monitoring data, although the focus was not consistent. Very little of this wider performance data was routinely reported through governance structures or wider national forums. Managers and leaders conveyed a clear appetite for greater national clarity and consistency around meaningful performance measures for their service. The tools, processes, systems and staff generating performance data also varied significantly. Most services acknowledged that, in order to overcome the limited functionality of local information management systems, they had developed an often complex and sometimes cluttered set of spreadsheets, checklists and manual processes to enable them to capture and collate performance data. To successfully develop, populate and maintain these systems, services were often critically reliant on dedicated or specialist performance or business support staff. While these staff were considered a vital resource, the specialist nature of the skill-set required meant they also represented a single point of failure in the event of absence or diminishing resource. All local validation areas used performance data to drive improvement initiatives which included some excellent examples of innovative and person-centred practice development. However, no service had consistent processes in place to capture performance improvement activity and track progress and outcomes.
Page 20
Page 14 of 38 Justice social work self-evaluation Quality: Learning from the 32 local authority self-evaluations Services rated how effectively they were measuring the quality of their work to support and supervise people on community sentences. In reaching their conclusions they were asked to specifically evaluate the extent to which they: • had a clear and comprehensive quality assurance framework • had a set of clearly articulated quality measures relating to their core functions • had processes and tools in place to ensure they routinely gathered and collated data regarding the quality of their work • could evidence improvement initiatives informed by quality assurance data Thirteen local authorities gave themselves an overall rating of ‘mostly’ for their quality assurance arrangements, with 17 rating themselves as ‘partially’ and one as ‘not at all’. Only one service felt they were ‘fully’ confident in their quality assurance arrangements. In services where quality assurance activity was robust, common elements included: a clear articulation of expected quality standards; policy guiding the tools to be used, frequency of use, and by whom they should be applied; and agreed processes for the collation, review and oversight of quality assurance data. However, the majority of services lacked a systematic quality assurance framework, relying instead on ad hoc or reactive activities. This limited their ability to promptly identify and respond to areas for improvement. Encouragingly, almost all self-evaluations identified the development of a robust and systematic quality assurance framework as an improvement priority. The range of tools and frequency of their application to provide local assurance about core elements of practice varied significantly. The majority of areas reported using national templates produced by the Care Inspectorate and Scottish Government to audit court reports, case files and serious incidents. A smaller number were either using, or intending to use, other national tools to review risk assessments and plans. A very wide range of locally developed tools was used to review the quality of core functions including statutory reviews, CPO closure and completion processes and breach proceedings. There were also varied approaches to the review of MAPPA, Drug Treatment and Testing Orders and the delivery of accredited programmes. While many services had clear performance targets, they were not routinely setting or monitoring equivalent targets in relation to the quality of practice. Although quality standards, linked to national standards and local policies and procedures, were in place, there were no clearly articulated examples of the high-level quality measures that services were aiming to achieve. Instead, consideration of quality measures was dominated by reference to performance indicators relating to the completion, frequency and timeliness of core functions.
Page 21
Page 15 of 38 Justice social work self-evaluation The self-evaluations provided limited information about how quality assurance data was being collated and reported to inform local service development. A small number of services generated specific operational management reports but these were not routinely reported to senior managers. All services provided annual reports on the delivery of CPOs which were shared with governance groups, however, the data within those reports was not routinely being analysed to inform local improvement activity. Services recognised the importance of quality assurance, and almost all referenced how audit activity and feedback from stakeholders contributed to improvement initiatives. However, a third noted that staffing pressures and wider system demands made meaningful quality assurance activity ‘unfeasible’. Quality: Learning from the local validation activity Validation activities around quality assurance broadly mirrored the themes from the national self-evaluation reports. Across the six services, levels of confidence about quality assurance arrangements varied from ‘partially’ to ‘mostly’. None of the six services had a clear quality assurance framework which outlined their process, tools, standards, measures, frequency or reporting arrangements for assuring service quality. All services highlighted this as an improvement priority. In spite of the absence of clearly articulated quality measures, staff were clear and confident about the standard of practice expected of them and most were able to access clear policies, procedures and guidance on the quality standards for their service. Strategic and operational leaders recognised an absence of agreed quality measures within existing national reporting frameworks and noted annual reporting on CPO delivery asked for selective qualitative examples of the impact and benefits of services for individuals and communities. Given the lack of, or demand for, nationally agreed quality measures, they found it hard to justify the development of quality assurance mechanisms as a priority. Almost all services noted quality assurance activity had been paused or interrupted by the impact of the Covid-19 pandemic. Efforts to resume routine quality assurance had been hindered by widespread staff shortages, the backlog resulting from the pandemic and wider system demands including efforts to support the early release of prisoners. There was a clear appetite to undertake more quality assurance activity, but an honest admission that the capacity to undertake the necessary development and consistently apply tools was very limited. Across services, a range of routine processes helped first-line managers to feel assured about the quality of support and supervision. These included staff supervision, counter-signing of reports and informal case discussions with staff. Statutory reviews on the progress of people subject to community orders provided an important opportunity for managers to check the progress and quality of work and gather feedback from people using services. That said, these were not consistently taking place in line with national outcomes and standards across all services. Other than some periodic reviews of case files and other ad hoc activities, formal and consistent audit and quality assurance activity, supported by the use of tools was intermittent. Given that supervision requirements are a key feature of many CPOs, not reviewing the content and quality of this aspect of practice presented a
Page 22
Page 16 of 38 Justice social work self-evaluation significant missed opportunity to better understand and build on the strengths of community-based sentences and to evaluate their effectiveness. Stronger audit performance was evident in a few services. Most were reviewing the quality of court reports, although a variety of tools were being used despite the recent development of a national template. Processes around MAPPA audit and the reporting of Serious Incident Reviews were much more robust, suggesting that, when applied consistently, agreed national tools, frameworks and reporting structures were providing helpful clarity around quality assurance practice. Spotlight on Practice – Quality Assurance A number of services had developed clear internal processes to maximise learning and drive improvement following the completion of Serious Incident Reviews (SIRs). SIRs were undertaken collaboratively with staff and any identified learning, including feedback from the Care Inspectorate, was shared with staff across the service. Where improvement actions were identified, processes were in place to capture and monitor the progress in delivering these. By involving staff in the learning process and establishing a process to follow up on learning arising from serious incidents, services were able to demonstrate that they were both quality assuring practice and building their culture of shared continuous learning and improvement. Services were alert and committed to improving quality. Gathering qualitative feedback from stakeholders and people using services enabled services to take timely action to address quality concerns and respond to suggestions for improvement initiatives. While there were examples of this feedback being shared in local and national reports, this data was not thematically analysed to inform local or national improvement.
Page 23
Page 17 of 38 Justice social work self-evaluation Outcomes: Learning from 32 local authority self-evaluations Services rated how effectively they measured the impact of their work and the difference their delivery of services was making for people on community sentences. In reaching their conclusions they were asked to specifically evaluate the extent to which they: • had a set of clearly articulated outcome measures which reflected local and national priorities. • had a set of clearly articulated outcome measures to capture improved wellbeing and life chances for people who use their services. • had processes and tools in place to ensure they routinely gather and collate data regarding the difference their services are making. • could evidence improvement activity informed by outcome data. Ten local authorities rated their outcome gathering arrangements as ‘mostly’, 21 as ‘partially’, and one as ‘not at all’. None felt they were ‘fully’ measuring the outcomes of their work. Collectively, the self-evaluation responses suggested a lack of clarity and confidence across services on local, national and individual outcome measures relating to the provision of community sentences. Most services referenced national outcomes and indicators within the Community Justice Performance Framework. The language used to describe this framework and the associated reporting requirements was very inconsistent suggesting a lack of shared understanding and consistent terminology across the sector. A number of services gathered and reported against wider national outcomes informed by National Outcomes and Standards (NOS), national practice guidance, and delivery standards for accredited programmes. However, this was not universal and, again, there was no consistency in the language used to describe these wider national outcomes. Together this created the sense of a confused and disjointed understanding of the national outcomes that services were working to deliver. Local outcomes measures also varied significantly and reflected a cluttered, and at times, outdated reporting landscape. Services variously reported against measures from historic Community Justice Authorities, the previous national Outcomes, Performance and Improvement Framework for Community Justice (2016), Local Outcome Improvement Plans, HSCP strategic plans, local performance frameworks, local strategic needs assessments, justice service plans, individual outcome tools (e.g. Justice Outcome Star), LS/CMI indicators and the indicators that are derived from Scottish Government statistical returns. While a number of services had updated their local outcomes to better reflect the Community Justice Performance Framework this was not consistent.
Page 24
Page 18 of 38 Justice social work self-evaluation This pattern was mirrored in relation to capturing outcomes for people on community sentences. Fewer than half of services had a clearly defined set of outcome measures to capture improved wellbeing and life chances for people using services. While services clearly recognised the importance and value of capturing the outcomes and experiences of people using services, the collection, analysis and reporting of personal outcomes was variously described as ‘difficult to achieve, ‘challenging’, ‘ad hoc’, ‘inconsistent’, ‘limited’ and ‘disparate’. This lack of a shared, national focus on outcomes and the experience of people using services was also noted as a gap in our discussions with national stakeholders. They reflected that nationally reported KPIs did not have a clear focus on outcomes. Equally, the outcome data submitted by local authorities to Community Justice Scotland in the annual Community Payback Order reports was narrative in form. This made it difficult to aggregate and analyse to identify themes and patterns to support national learning around outcomes. In recognition of this, in 2021 the SWS Justice Standing Committee established a sub-group jointly chaired by Community Justice Scotland, with a remit for performance, quality assurance, service user feedback and continuous improvement. Although there were aspirations to develop shared approaches to gathering and reporting on data, there was an honest acknowledgement that very little progress had been made in this area. Across Scotland, standard processes were in place which offered the potential to gather outcome data. These included exit interviews, statutory reviews and the LS/CMI assessment process. However, these processes were not providing consistent or reliable outcome data. Nationally, there were no standard tools for gathering outcome data. Although the case management progress record within LS/CMI had capacity to capture and report on changing outcomes, most self- evaluations were silent on the use of this, suggesting it was not used as originally intended. Moreover, services reported issues with the functionality and accessibility of reporting from the LS/CMI system indicating that even where data was being captured, it could not be easily or routinely extracted to inform monitoring or improvement. Services also expressed concern that disjointed oversight arrangements and lack of ongoing investment in the system threatened the sustainability of the tool, and risked diluting the underlying FRAME practice principles embedded in the approach. While statutory reviews were referenced in all self-evaluations, there was no nationally consistent template, nor agreed set of outcomes routinely considered in statutory reviews. This limited opportunities for benchmarking. Additionally, the use of exit interviews was inconsistent and most services noted low response rates. A small number of services had linked entry and exit questionnaires which enabled them to capture and demonstrate change over time. Many services used manual or ‘paper-based’ forms for these processes which limited capacity for easy analysis. Gaps in information management systems, business support and staff capacity also limited services’ ability to collate, analyse and report on any data captured. Despite these challenges and limitations, a number of services highlighted efforts to innovate and invest in improving how outcomes were gathered and reported. Almost half of services referenced use of Justice Outcome Star to gather data on a small, but clear set of individual outcomes. Other services were taking steps to implement locally developed tools, but these were in the early stages of testing. Several services were exploring opportunities to utilise technology to help them
Page 25
Page 19 of 38 Justice social work self-evaluation more effectively gather outcome data through the use of applications like Microsoft Forms and QR codes. In the absence of reliable systems and processes to gather outcome data, most services were limited in their ability to demonstrate how they used outcome data to drive improvement initiatives. A small number of services noted examples of how they had used feedback from exit interviews and statutory reviews to identify and address staff training needs. Others had made improvements to their CPO unpaid work ‘Other Activity’ provision on the basis of feedback from people using services. However, many services acknowledged that, in the main, improvement activities were isolated projects prompted by individual and ad hoc feedback. One self- evaluation summarised this well, noting their improvement efforts were more reflective of “isolated decisions rather than a system informed by outcome data [that was] being routinely captured and actioned.” A number of services gathered feedback on outcomes through lived experience panels, service user participation groups or capturing people’s stories. Where feedback was gathered through direct engagement with people using services, it was effectively driving improvement activity. Almost all services noted that listening to the experiences and voices of people using services was a critical but underdeveloped aspect of their culture and practice and noted it as an improvement priority. Outcomes: Learning from local validation activity The key messages from the self-evaluation reports regarding the gathering and reporting of outcomes were confirmed through validation activities. Across the six areas, levels of confidence about their ability to evidence outcomes varied from ‘mostly’ to ‘partially’. All services reflected a good awareness of the National Outcomes and Standards, and the associated measures and it was clear that Community Justice Partnerships had taken steps to identify local actions mapped against the priorities of the National Strategy for Community Justice. Within justice services though, most had not yet aligned their service delivery plans to reflect the new national priorities and local outcome measures were not well defined. Leaders at all levels expressed some frustration about the Community Justice Performance Framework, noting that a number of the priorities had limited or no outcome measures representing a missed opportunity to capture the impact of services. They felt this conveyed a lack of clarity at a national level about what services are trying to achieve. There was a clear focus on identifying and measuring outcomes for people using services in all six services. Frontline staff had a good understanding of what constituted a positive outcome in the context of their work. They cited public safety, reducing offending and social inclusion as important outcomes alongside a clear commitment to supporting individual change and addressing wider support needs for people on CPOs. However, managers and leaders noted the complex nature of outcomes and, in the absence of a national ask, admitted a lack of clarity and confidence on the outcome measures they should be gathering and reporting on for people using services. This lack of clarity was reflected in the very wide range of tools and processes used to capture outcomes for people using services. Mirroring the findings of the national self-evaluation submissions, a wide range of exit questionnaires were used
Page 26
Page 20 of 38 Justice social work self-evaluation across services. Most were completed on paper, with no capacity to aggregate and report the data, and all services acknowledged very poor rates of return. None of the services had an associated ‘initial’ questionnaire to enable them to capture change over time. Although statutory review processes considered outcomes, they did not include a mechanism to allow services to reliably report on them. Only one service routinely expected staff to complete the LS/CMI progress record with others citing the need to ‘double-key’ as a barrier to its use. Some services had commissioned Justice Outcome Star, but in practice, this was inconsistently applied and the data not routinely reported anywhere. One service had developed a bespoke tool but did not yet have the capacity to collate and report on the outcome data it generated. Given that systems to capture and report on individual outcomes were underdeveloped, outcome data was not being routinely shared in local and national reports. This limited services ability to demonstrate the impact of their service delivery and celebrate success. However, services were much more able to report on individual stories and positive examples of feedback from stakeholders and people using services. This rich qualitative data was shared in Chief Social Work Officer (CSWO) reports and was being collated and reported to Community Justice Scotland via the Community Payback Order Annual Reports. While the subsequent national Summary Report highlighted examples of the positive impact of community sentences for both people and communities, services felt the limited thematic analysis of this data represented a missed opportunity to support collective learning and continuous improvement.3 The limited capacity to capture and report on individual outcomes stood in stark contrast to the very clear messages we heard from people about the positive impact of the support they received from services. They described the experience of being on an order as ‘transformational’ and ‘life-saving’. People described staff who ‘stick with you’, ‘support your progress’ and ‘help you feel safe’. Staff were noted to be responsive to needs, appropriately challenging of offending behaviour, but non- judgemental in their approach. This was confirmed in the person-centred values that characterised our discussions with staff across all services. Spotlight on Practice - Outcomes Some services had developed improvement initiatives to address needs and support compliance for people on community orders. These included the provision of specialist support to young men on CPOs, the development of additional support for people on unpaid work with high levels of need, and the creation of a peer navigator service for people engaging in substance use. In each instance, consideration of performance and outcome data enabled services to identify barriers to the successful completion of orders for people with particular needs. In turn, this prompted action to develop and implement evidence-led support initiatives, with the aim of improving outcomes for people on orders. National stakeholders agreed the lack of standardised, consistently applied tools related to key processes including exit interviews and statutory reviews limited the opportunity to gather and report on service wide and individual outcomes. All four stakeholders also reflected there were limits around their ability to gather, aggregate and report on outcome data for justice social work services. This was variously 3 It is noted that, to support shared learning, the Community Payback Order: Summary of Local Authority Annual Reports 2023 – 24 published in March 2025 contains a thematic summary of key improvements, persistent challenges, and innovative practices identified from local authority reports.
Page 27
Page 21 of 38 Justice social work self-evaluation connected to their lack of capacity, remit or the limits of their legislative authority. In the absence of a collective focus, means and mandate, there was no established or growing evidence base on outcomes associated with community sentences from which to identify and drive national development and improvement. This was a source of frustration for all stakeholders. Organisational drivers Services evaluated the extent to which the five organisational drivers enabled and supported their performance monitoring and quality assurance activity. The organisational drivers were: A Culture of learning and improvement; Leadership; Governance; Knowledge/Expertise and Resources. Culture of Learning and Improvement: Learning from 32 local authority self- evaluations Services rated the extent to which their organisational culture was characterised by: • reflective practice. • promotion of accountability at all levels. • investment in learning. • a focus on involving people who use services. • an evidence led approach to change. Most services were confident about their organisational culture. The extent to which the key elements noted above were present varied. Almost all services saw supervision and formal training as the key vehicles for learning and reflective discussion and it was clear that services had prioritised and invested in those opportunities for staff. Beyond the use of supervision and formal training wider learning and development opportunities were less consistent or clearly defined within self-evaluations. While staff training was a high priority, only a handful of services referenced specific investment in training for managers and leaders. Very few reflected on the important role that managers and leaders hold in modelling a strong culture and promoting shared ownership for learning. While reflective practice was noted as a critical learning tool, mechanisms to support this were not well articulated. Only a small number of services routinely hosted practitioner forums, or group supervision, and only two services specifically mentioned a ‘peer review’ process. Almost all submissions reflected that opportunities to gather and learn from the views of people using services was a critically important but underdeveloped aspect of the learning culture.
Page 28
Page 22 of 38 Justice social work self-evaluation Culture of Learning and Improvement: Learning from local validation activity A culture of learning and improvement was a clear strength across all six justice services. However, their self-evaluation ratings suggested that services had not taken full account of the positive elements of their organisational culture. Where the culture was strongest, there was a clearly articulated vision for the justice service which was communicated to, and understood by, staff at all levels. This fostered a clear sense of purpose and shared ownership across the staff group which was promoting shared accountability. Similarly, a strong culture of reflective practice was evident in all six services. This was underpinned by clear investment in regular professional supervision and investment in a range of formal and informal learning opportunities. A variety of helpful reflective practice opportunities were empowering and enabling staff to propose and develop change ideas. This included the provision of access to clinical supervision, although this was becoming much harder for services to resource. A widespread commitment to supporting student placements reflected the ethos of practice learning, and robust processes to support newly qualified social workers were in place across services. All services reflected a commitment to hearing from people who used services, and people we spoke to confirmed that feedback they provided was generally acted on. However, formal processes and forums to support the participation and involvement of people with lived experience in service review and development were limited. Leadership: Learning from 32 local authority self-evaluations Services rated the extent to which leadership within their service was characterised by: • a focus on performance and quality • ownership over QA processes • modelling a commitment to improvement • actively celebrating strengths and addressing performance issues Services were generally confident in their leadership skills and processes. Leaders used supervision, team meetings and staff appraisal processes to maintain oversight of performance and teams. Most services had regular processes in place to enable operational (and sometimes strategic) managers to review service-wide performance data. Quality assurance data was less consistently reported, indicating that not all aspects of service delivery were subject to ongoing interrogation by leaders. Because data
Page 29
Page 23 of 38 Justice social work self-evaluation gathering and reporting were not always systematic, efforts to promote and celebrate success tended to be ad hoc, although the importance of this as a means of boosting morale and service visibility was well recognised. Gaps in data gathering and reporting also meant that opportunities to consistently provide feedback to staff about performance, and engage them in discussions about improvement were not as robust as services would have liked. Leadership: Learning from local validation activity Leadership was a strength across the six services. Levels of confidence about leadership varied from ‘partially’ to ‘mostly’. Staff experienced oversight from operational managers as supportive, encouraging and empowering. This was reflected in the wide range of change ideas identified and led by front-line staff. This demonstrated services’ commitment to innovation and encouragement of leadership at all levels. Many of the improvement initiatives required creative leveraging of scarce resources reflecting leaders and managers ability to draw on strong partnership working to drive change. Staff understood their responsibilities and felt their work was valued by managers. Regular supervision provided routine opportunities for feedback on the quality of their work, as well as time to focus on professional development and personal wellbeing. A leadership commitment to improvement was clear in each service, but presented differently across services. Some services invested in dedicated roles to support development and improvement initiatives related to performance, quality and outcomes. Other, smaller services, created a culture in which staff had permission and freedom to test change ideas and adapt service delivery approaches in response to the specific and emerging needs of people using services. Governance: Learning from 32 local authority self-evaluations Services rated the extent to which governance arrangements within their service were characterised by: • Agreed reporting structures for performance, quality and outcome data • Engaged and informed senior leaders • Established levels of accountability • Data that is integrated into wider planning and performance systems
Page 30
Page 24 of 38 Justice social work self-evaluation Services were broadly confident about governance arrangements. Almost all services described reporting structures within services and upwards to relevant oversight groups although the range of oversight groups varied significantly across the country. Reported information predominantly focused on performance measures, with limited consideration of quality and outcome data. While justice data was captured in wider planning and performance systems in most services, the extent of the data reported was often limited. In contrast to other aspects of the self-evaluation submissions, reflections on governance were largely descriptive. This provided little insight into how well governance arrangements operated or the extent to which leaders and elected members were sighted and engaged in the work of justice services. This reflected the findings of our recently published social work governance report which noted that justice social work was not consistently given an ‘equal voice’ compared to adult and children’s social work.4 A number of services made positive reference to the oversight provided by Community Justice Partnerships and public protection forums. These groups provided more meaningful opportunities to reflect on the work of justice services beyond consideration of a small number of key performance metrics. Governance: Learning from local validation activity Validation activities presented a mixed picture in relation to governance arrangements. Across the six services, levels of confidence about governance oversight varied from ‘partially’ through to ‘fully’. While oversight arrangements differed across the six services, all had clear governance and lines of accountability in place. As noted nationally, the extent of formal reporting about the work of justice services varied. There were examples of a range of routine performance reports and service updates submitted to committees and governance groups which supported good visibility and oversight of justice services. In general reporting was limited. The focus of reporting was largely characterised by a narrative summary of activity. Where aggregated data was included, this was limited to a handful of high- level performance indicators, and in some services no performance data was shared. Beyond the inclusion of some individual case studies and practice examples, there were almost no instances of aggregated quality or outcome data reported in any governance report. While leaders felt assured about the performance and quality of services they acknowledged that much of this was reliant on self- report from managers rather than objective data reporting. Engagement from strategic leaders and elected members tended to focus on specialist provisions such as unpaid work and women’s services which were noted 4 Review of social work governance and assurance across Scotland, p16.
Page 31
Page 25 of 38 Justice social work self-evaluation to be more visible. Most services recognised scope to enhance oversight and visibility of service delivery – particularly in relation to CPOs – through more consistent and broad-ranging data reporting. Knowledge/Expertise: Learning from 32 local authority self-evaluations Services rated the extent to which the knowledge and expertise within their service reflected that: • Staff were equipped with the necessary skills, knowledge and behaviours • Dedicated performance and quality assurance expertise were in place • Clearly defined methodologies and metrics to support performance and quality assurance activity were evident More than half of services indicated that they were only ‘partially’ confident that their service had the necessary knowledge and expertise to support them to confidently gather and report on performance, quality and outcomes. Almost all services noted the critical importance of formal training in supporting staff to develop the required knowledge and expertise. Commitment to investing in this was clear across all services. However, the infrastructure supporting learning and development was less consistent. Only three services noted that they had (or were in the process of developing) a specific learning pathway for justice staff. Processes to identify and monitor staff training needs and attendance varied, and some services noted barriers in accessing national training. Leadership development pathways were not clearly articulated and approaches varied significantly across the country. A number of skill gaps for managers were also noted. Across services, managers were not confident or trained to undertake data gathering or analysis. One self-evaluation observed that 'services lack the level of knowledge or training in respect of interrogating systems and producing meaningful reports'. Reflecting this point, just under half of submissions indicated they had, or had recently created a dedicated resource to support performance reporting. A further third indicated that they accessed performance support from wider corporate services. While support around reporting was welcomed, services noted that it did not always include a sufficient level of analysis to enable them to drive improvement. There were conflicting messages about knowledge, expertise and training of managers in relation to quality assurance and quality improvement skills. Although most services identified that team managers were skilled in quality assurance, very few services referenced any formal training in quality assurance and audit. Additionally, a number identified that their plans to develop a quality assurance framework would include the need to provide additional quality assurance training to staff.
Page 32
Page 26 of 38 Justice social work self-evaluation Knowledge/Expertise: Learning from local validation activity Across the six services, levels of confidence about knowledge and expertise varied from ‘partial’ to ‘mostly’. Validation activities indicated that half of services underestimated the knowledge and expertise held within their service. There was a clear commitment to formal training, despite the associated resource challenges for services with a high level of staff turnover. Staff felt well equipped to undertake the core functions of their role. In particular, the impact of the local implementation of trauma-informed approaches was apparent in all services. While necessary access to core training was viewed as adequate, the move towards online training delivery had diminished opportunities for reflective discussion. Managers felt this had diluted the quality of training resulting in a noticeable impact on practitioners’ levels of confidence. As noted earlier, all six services had access to some form of specialist support to meet statutory reporting requirements and assist their gathering, reporting and analysis of data. Services were clearly wrestling with the tension between investing in dedicated roles versus embedding data management responsibilities within the service by upskilling existing staff. They were alert to the potential risk of a single point of failure in the event of staff loss or absence. As noted within the national self-evaluations, local validation activities confirmed that managers lacked confidence and training in quality assurance, data analysis, quality improvement and leadership of change. Leaders in some services had made concerted efforts to source suitable training for managers, with limited success. Where staff had been trained in quality improvement science they were introducing helpful systems and processes to support learning and improvement.
Page 33
Page 27 of 38 Justice social work self-evaluation Resources: Learning from 32 local authority self-evaluations Services rated the extent to which their access to, and use of, resources equipped the service with: • tools to support key processes • IT systems and technology to support data gathering and analysis • capacity to undertake performance and quality assurance activity Almost two-thirds of services indicated partial confidence that they had the resources needed to effectively undertake performance management and quality assurance activities. Services referenced a wide range of tools, templates and processes supporting the gathering of performance, quality and outcome data. Notably, the language used to describe these was very inconsistent which impacted our ability to reach clear conclusions on the extent to which tools are consistently and meaningfully applied. The development of national quality assurance templates to support the introduction of revised practice processes was welcomed and these were reported to be widely used. A significant number of services also used file reading templates developed by the Care Inspectorate suggesting an appetite for clear and consistent, nationally developed tools. The limited functionality of information management systems was the most commonly cited barrier to improving performance management and quality assurance activity. Almost half of services noted that current information management systems were not fit for purpose, and a third of services were investing in new systems. This was noted to be a significant resource drain. Services described little or no access to digital tools and platforms to support innovation and efficiency, and most services were resigned to pursuing ‘workarounds’ due to these digital and technological barriers. One service noted: “The general landscape of social work and justice information management systems, and the lack of communication between them presents a continued issue requiring staff to manually enter information gathered from one system to another…”. The impact of Covid-19 and budgetary pressures limited staff capacity and resilience to meet an increasing volume of demands. Additionally, work arising from critical practice developments, such as the early release of prisoners, had drawn managers away from strategic functions, into operational delivery. Considered in combination, these pressures had made meaningful quality assurance work unfeasible and limited capacity for wider service development. Despite these challenges, services worked hard to boost their data gathering and reporting capacity. For example, almost half of services indicated they had, or
Page 34
Page 28 of 38 Justice social work self-evaluation were in the process of appointing, dedicated staff to support performance reporting. Further self-evaluation will enable services to gauge whether the creation of dedicated posts has delivered the intended improvement. Most services reflected that a re-balancing of resources was required to support a focus on quality and outcomes. Resources: Learning from local validation activity Validation activities reflected varied levels of confidence across services about the adequacy of their resources with half rating themselves as ‘mostly’ and half as ‘partially’. In considering their resources, services drew confidence from their positive culture, and the presence of a strong cohort of dedicated and values-driven staff, committed to delivering high-quality services. Nevertheless, despite these strengths, wider system issues prevented services from consistently evidencing the performance, quality and outcomes of their service delivery, highlighting barriers to be addressed. In services where confidence in available resources was lower, two key issues were apparent. Firstly, recruitment and retention issues had limited capacity and resilience in a number of services. Although services were working hard to support and upskill new staff through robust induction and training programmes, they acknowledged that this would take time. In the interim, existing staff and managers carried increased workloads which limited capacity to undertake wider development and improvement work. Secondly, the impact of implementation of new information management systems had been seismic for some services. Staff time to deal with these issues diverted critical resources away from planned development and improvement work. In addition, services’ ability to accurately report data regarding key functions had been detrimentally affected. While services were hopeful the situation would improve, the impact on capacity over a period of several years had been enormous. All services reflected on a range of significant resource challenges largely outwith their control. Aspirations to develop digital tools to enhance data gathering capacity and improve efficiency were hindered by system issues relating to IT access and permissions. All services noted the impact of the increasingly complex nature of their work, both in terms of the presenting needs of the people they worked with and the evolving demands of justice policy and procedure. Services were frustrated that the complexity of need within a local authority area was not taken into account in the allocation of Section 27 funding. Additionally, all services reported the volume of reporting requests from across the sector had a material impact on their ability to take forward local improvement activity. National reporting demands from a range of stakeholders were noted to be onerous, cumbersome and duplicative. Strategic and operational managers felt that much of the data gathered was unnecessary and very little of it came back to them in any meaningful form that would support improvement. Despite the significant reporting demands placed on services, no resource provision was made to support data reporting within the annual funding grants, although each area has some autonomy to allocate spending to address local priorities. This burden of reporting demands was echoed in interviews with national stakeholders. This was despite significant efforts from Scottish Government Justice
Page 35
Page 29 of 38 Justice social work self-evaluation Analytical Services to streamline reporting processes. The limitations associated with local information management systems and staff capacity were noted as major barriers to expansion of the national data-set, with stakeholders reporting that it could take upwards of two years to implement changes, which also often incurred additional, unfunded costs for local authorities. Learning from a self-evaluation approach All 32 local authorities completed and submitted their self-evaluation within the allocated timescale. This reflected a sector-wide commitment to developing a shared understanding of the subject matter and to continuous improvement. The majority of submissions reflected proactive engagement and used the opportunity to reflect on and evaluate key areas of strength and opportunities for improvement. In the main, submissions were concise and presented a range of relevant information. Stronger examples demonstrated a reflective and analytical consideration of the available evidence, supported by useful reference to practice examples. A small number of responses were extremely limited or provided descriptive lists of activity, or unqualified statements about performance which were not supported by evidence. A number of services had elected to directly replicate the wording from the practice exemplar provided which limited their ability to showcase the uniqueness of their service. The documentary evidence submitted by the six services was relevant, appropriate and provided a helpful picture of practice. It was well organised and referenced against the key domains of the self-evaluation, demonstrating that services had undertaken a thorough review of their systems and records in support of the task. The volume of evidence submitted was broadly proportionate indicating that services knew themselves well and had a good sense of what was required. The in-person validation activity was characterised by strong representation and buy-in from staff at all levels of the service. Those who participated showed an interest in, and commitment to improving practice, and this was confirmed by the genuine and reflective nature of focus group discussions. Collectively, the self- evaluation activities demonstrated that most services were confident and capable in planning and implementing self-evaluation and that the appetite for reflection was well developed across the sector. Within the self-evaluation template, services were asked to rate themselves for each element using a scale of ‘fully’, ‘mostly’, partially’ and ‘not at all’. Although the validation activities did not highlight any significant discrepancies between the evidence and the locally identified ratings, there were some areas of deviation. While ratings around performance management arrangements were broadly found to be accurate, in several services, arrangements for gathering and reporting on quality and outcomes were less robust than services initially perceived. The opposite was true in relation to the evaluation of organisational drivers. We found that services consistently underestimated their performance in these areas. This was particularly true in relation to the culture of learning and improvement and levels of knowledge and expertise where half of services had underrated themselves. Two services had also underrated the strength of their leadership and governance arrangements.
Page 36
Page 30 of 38 Justice social work self-evaluation This review has demonstrated that self-evaluation is a robust approach to gathering relevant information on specific themes at a national level within a relatively short timeframe. Feedback from services indicated that although there was a time cost to undertaking the exercise, in the main it had provided a timely, focused and helpful opportunity to take stock of their current strengths and had usefully highlighted or confirmed priority areas for improvement. A number of services noted that the exercise had acted as a catalyst for improvement activity and had been an encouraging activity overall. Feedback on the approach from the six local authority areas was very encouraging with services noting the approach as proportionate and transparent. The approach had added value to their work and helped to build confidence in undertaking future self-evaluation. Engagement with justice staff had helped to reduce anxiety about the Care Inspectorate role, and the quality of engagement with people using services was highly rated within feedback forms. Respondents variously noted: ‘Self-evaluation... focused our attention on areas which we knew required work but hadn’t prioritised… We have now produced an action plan to implement and progress these areas.’ ‘(This) has given (us) the confidence to progress self-evaluation … using quality indicators…’ ‘We found the self-evaluation collaborative and supportive. The process was appropriately challenging, made us think, supported wider participation and engagement from service… This will be a real boost for all colleagues in the service.’ ‘This has been an excellent exercise in supporting us to develop our journey in performance management, auditing and outcomes. The areas highlighted were all known areas for change so it was reassuring this was also recognised by the review.’
Page 37
Page 31 of 38 Justice social work self-evaluation Conclusions This review set out to evaluate the extent to which justice social work services could confidently evidence the performance, quality and outcomes of community sentences. It also sought to explore the factors that were supporting or hindering services’ ability to demonstrate effectiveness and impact. These questions align with wider policy objectives for the justice sector, and the findings have implications for national stakeholders and justice social work services. The Scottish Government’s current strategic priorities include reducing the prison population by shifting the balance between custodial sentences and community disposals. However, as the Scottish Sentencing Council’s 2021 consultative exercise noted, this requires confidence in the availability, quality, and effectiveness of community sentences. Additionally, the Scottish Government recently announced an independent review of sentencing and penal policy 5 to identify the most effective ways of addressing offending behaviour, reducing crime, and lowering the number of victims. All this work requires a robust evidence base on the quality and effectiveness of community sentences. Our previous scrutiny activities found that justice social work services faced challenges in telling a cohesive, comprehensive story about how the services they deliver helped people to change and positively impacted communities. Performance management and quality assurance were identified as key areas for improvement. The findings from this review confirm this remains the case. This stands in stark contrast to the consistently positive messages we heard from people who use justice services about the impact of community sentences. There is a clear appetite across services to go beyond the numbers which inform statistical returns in order to demonstrate outcomes and the real difference community sentences make in people’s lives. There was therefore a sense of frustration from services that the collective efforts of the SWS Justice Standing Committee sub-group on Performance and Quality Assurance had not made progress in delivering improvements. The findings of this review are deliberately detailed to meaningfully inform specific improvement work planned by the SWS Justice Standing Committee, the digital intelligence and analysis work of Community Justice Scotland, and improvement priorities of wider national and local leaders. Reaching shared agreement about what justice services are seeking to deliver and how this will be meaningfully measured will be an important starting point. The development of a shared strategic approach, underpinned by consistent frameworks and systems to measure quality and outcomes, are indicated as clear priorities. However, delivering transformational change to robustly evidence the impact and value of community sentences is beyond the scope and best efforts of individual justice social work services working on their own. There are also risks and resource implications associated with 32 services developing ‘local’ solutions to national issues. A systematic and co-ordinated approach to working together offers greater opportunities for consistency and standardisation in identified areas of improvement. This will require a strong vision for justice, underpinned by investment and effective leadership at all levels. The Care Inspectorate remains committed to working with partners to support improvement informed by self-evaluation or other forms of scrutiny as appropriate. 5 Independent Review of Sentencing and Penal Policy
Page 38
Page 32 of 38 Justice social work self-evaluation Considerations for the wider justice sector The conclusions point towards a number of key areas of consideration for justice social work services and wider justice stakeholders: Developing a shared strategic approach to performance, quality and outcomes • Collective work to establish a shared framework for justice social work services, with agreed measures and tools is worthy of consideration. The current evidence base is limited to performance data, with outcome measures not well defined. Services are not consistently able to gather and report on quality and outcomes. Critically, there is no coherent national strategy or system to support them to do so. Building this robust evidence base is crucial for justifying shifts in investment from custodial sentences to community-based interventions. Clarifying leadership and stakeholder responsibilities • Development of a shared, strategic approach requires clear leadership and agreement about social work services and stakeholder responsibilities. Reflecting the findings of the recent National Care Service: Justice Social Work Research, services echoed the suggestion that “the Scottish Government could provide more national leadership in terms of the direction of travel for the sector.” 6 Respondents also expressed a lack of clarity about Community Justice Scotland’s role in supporting development in justice social work. Understanding the role and contribution of the forthcoming National Social Work Agency is also important. Optimising the national data infrastructure • Enhancing capacity for developing and sustaining the national data infrastructure is crucial. Services value the overview provided by the national justice statistics and support from Scottish Government Justice Analytical Services. However, there were requests to streamline quantitative reporting demands and address resource challenges linked to the gathering and reporting of qualitative data. • There was wide agreement that the LS/CMI system has the potential to provide valuable insights about the justice service population and the measures that are proving effective in supporting risk reduction and reduced reoffending. However services were concerned that disjointed oversight arrangements, inaccessible reporting functionality and lack of ongoing investment in the system threatened the tool’s sustainability, and risked diluting the FRAME practice principles which are embedded in the approach. • Building a robust evidence base for the effectiveness of community sentences will require access to broader and richer data than can be provided by justice services alone. Combining re-conviction data with unit-level data in the national justice dataset would allow for a much more meaningful understanding of the impact and outcomes of community sentences. At present, there is no mandate or framework to support this and services and stakeholders lack clarity about who has the responsibility or resources to address these gaps nationally. 6 National Care Service: Justice Social Work research, p34
Page 39
Page 33 of 38 Justice social work self-evaluation APPENDICES Appendix 1 - Quality Improvement Framework The Care Inspectorate team use this model to reach conclusions on the quality and effectiveness of justice social work services. This review focused specifically on Quality Indicator 6.4 – Performance Management and Quality Assurance.
Page 40
Page 34 of 38 Justice social work self-evaluation The report summarises the overall findings of the review in relation to the adapted version of QI 6.47. 7 Quality Indicator 6.4 – Performance management and quality assurance
Page 41
Page 35 of 38 Justice social work self-evaluation
Page 42
Page 36 of 38 Justice social work self-evaluation Appendix 2 - The terms we use in this report Assurance – processes for ensuring the quality and effectiveness of services. Case Management Plan – risk and needs are actively addressed through a case management plan of intervention in which the person actively participates. Any strengths identified by the assessment process (using LS/CMI) should be promoted within the plan. Community Justice Scotland – Non-Departmental Public Body (NDPB) with statutory responsibilities to promote, support and improve the provision of services by community justice partners, in line with the National Strategy for Community Justice. Community Payback Order (CPO)– a type of community sentence available to courts in Scotland. CPOs are supervised by justice social work services in accordance with Community Payback Order Practice Guidance. Culture – organisational culture – refers to values, beliefs, behaviours and norms shared by all members of an organisation or service. Data – a collection of statistical information that conveys quantity, quality or other units of meaning. Desistance - the process by which people who have engaged in a pattern of criminal or antisocial behaviour cease or reduce their involvement in offending behaviour. Focus group - a qualitative research method where a small group of people discuss a topic guided by a moderator. FRAME – Framework for Risk Assessment, Management and Evaluation (FRAME): a framework developed in partnership with justice agencies which aims to develop a consistent and evidence-based approach to risk assessment and management. Framework – a structured approach to monitor and improve efficiency and effectiveness. Governance - a system that provides a framework for managing organisations. It identifies who can make decisions, who has the authority to act on behalf of the organisation and who is accountable for how an organisation and its people behave and perform. HSCP - integrated arrangements for health and social care across Scotland. All partnerships are responsible for adult social care, adult primary health care and unscheduled adult hospital care. Some are also responsible for children's services, homelessness, and justice social work services. Information Management System - a software solution designed to help social work services manage and record case information, ensure compliance with regulations, improve communication, and enhance the delivery of social services. Common examples include Liquidlogic, Mosaic, and CareFirst. Indicator – specific and measurable standard used to assess performance, quality or effectiveness.
Page 43
Page 37 of 38 Justice social work self-evaluation Justice social work – local authority statutory justice social work services. Previous Care Inspectorate reports refer to justice services. Terminology varies across the 32 local authorities including justice social work services and community-based justice social work, to distinguish from prison-based social work. Justice Outcome Star - The Outcomes Star is an evidence-based tool for both supporting and measuring change. The Justice Star is designed for use with people serving a sentence, approaching release from prison or in the community. Level of Service/Case Management Inventory (LS/CMI) - a comprehensive risk/need assessment and management planning method for general offending. In Scotland, the approach has been developed to combine the robustness of an actuarial approach with an evaluation of the pattern, nature, seriousness, and likelihood of offending. The LS/CMI system supports the input of data. Lived experience - knowledge and understanding gained through direct, first-hand, and personal experiences, rather than through second-hand accounts or theoretical knowledge. MAPPA - Multi-Agency Public Protection Arrangements established to manage the risk posed by registered sexual offenders and other individuals who pose a serious risk of harm to people and communities. National Outcomes and Standards (NOS) – Known formally as National Outcomes and Standards for Social Work Services in the Criminal Justice System – they provide detailed Scottish Government quality assurance guidance for justice social work services. National Social Work Agency - The Scottish Government plans to establish a National Social Work Agency as part of the Care Reform (Scotland) Bill, aimed at improving social care support and workforce standards across Scotland. Other Activity - as part of a community payback order unpaid work requirement, a person can use a small proportion of their hours to undertake activities to help with the development of skills to support long-term desistance from offending. Outcome – the result or the effect of an action, an intervention or a specific situation or set of circumstances. Used in this report’s context to refer to impact on people, communities or the justice system. Outcome tools – tools used to assess the effectiveness of interventions or programmes by tracking changes in a person's health, well-being, behaviour or functioning over time. Key Performance Indicator – specific quantitative measures that evaluate performance. Quality assurance – a systematic process that ensures services meet or exceed established quality standards. Quality measures – measures by which services assess and quantify the effectiveness and impact of interventions and services. They typically focus on outcomes, processes, and organisational structures to ensure high-quality support.
Page 44
Page 38 of 38 Justice social work self-evaluation Risk Management Authority – A Non-Departmental Public Body (NDPB) providing expertise in risk assessment and risk management for the justice system in Scotland. Scottish Government Annual Aggregate Return – justice social work statistical return relating to various aspects of justice social work Scottish Government Justice Analytical Services – Justice Analytical Services within Scottish Government has multi-disciplinary analytical teams which include statisticians, social researchers, economists, operational researchers and performance analysts. The teams provide statistics and research support relating to a range of policy areas including police and community safety, court affairs and people accused or convicted of offences, victims, prisons and matters relating to civil and international law. Scottish Government Unit Returns – Justice social work annual statistical returns collection unit level data on CPOs and Drug Treatment and Testing Orders only. Self-evaluation – process of self-assessing, and monitoring performance and abilities. Serious Incident Reviews (SIRs) - are undertaken by local authority justice social work services when someone subject to statutory supervision has caused or been subject to serious harm. SIRs are submitted to the Care Inspectorate to evaluate the quality of the local authority’s review of the supervision of the person involved. Social Work Scotland (SWS) – social work professional leadership body for the social work and social care professions. Statutory reviews – National Outcomes and Standards determine that case management plans should be reviewed by the social work service and, where necessary, revised at regular intervals during the period of statutory supervision. Trauma-informed practice - a strengths-based approach grounded in an understanding of, and responsiveness to, the impact of trauma, that emphasises physical, psychological, and emotional safety for everyone, and that creates opportunities for survivors to rebuild a sense of control and empowerment. Validation – process of checking or proving validity or accuracy.
Page 45
Headquarters Care Inspectorate Compass House 11 Riverside Drive Dundee DD1 4NY Tel: 01382 207100 Fax: 01382 207289 Website: www.careinspectorate.com This publication is available in alternative formats on request. © Care Inspectorate 2025 I Published by: Communications I COMMS-0425-553 @careinspect careinspectorate