The history of the ‘What Works’ movement and Evidence Based Practice
- Pages: 13
- Word count: 3220
- Category: History
A limited time offer! Get a custom sample essay written according to your requirements urgent 3h delivery guaranteedOrder Now
This essay will give a brief developmental and historical overview of the history of the ‘What Works’ movement and Evidence Based Practice (EBP). It will then assess how this has contributed to the strengths and weaknesses of Probation practice as a modus operandi. Finally it will give some recommendations which could address the current weaknesses and build on the highlighted strengths.
From 1976 until the late 1990s, with very few exceptions, no government research was carried into the effectiveness of probation services in Britain and whether the methods used by Probation officers in supervising offenders were actually likely to reduce offending. In Britain (more than in several other countries) there was an official acceptance, particularly in the Home Office, of the conclusions of certain American and British research reviews (such as Martison (1974) and Brody (1976)) which were presented as proving that ‘nothing works’ in terms of the rehabilitation of offenders.
Such conclusions were exaggerated and were not an accurate summary of the research they reviewed. However it was believed that further research on the effectiveness of probation would not be a good investment. This feeling was confirmed by Britain’s own contributions to the ‘nothing works’ literature, the IMPACT study which showed that probationers supervised on specially low caseloads, who spent more time with their supervising officers, were no more likely to avoid reconviction than those supervised on normal caseloads (Folkard et al. 1976). Another reason for the decline in ‘effectiveness’ research was a shift in policy about the role of the Probation Service in criminal justice. During the 1980s it was seen as the critical agency for achieving reductions in costly custodial sentencing: the central tasks became to provide ‘alternatives to custody’ and to persuade sentencers to use them (Home Office, 1984) (Raynor 2004). However a revival of the rehabilitative model was brought about in the 1990s and the Nothing Works era began to fade.
Researchers and practitioners were soon beginning to benefit from the statistical analysis from the new statistical technique of meta-analysis; this combines the results from a number of studies by coding them to a common framework and applying a common measure of effect size. For example the extent to which outcomes for treated groups differ from those for control groups or matched comparison groups (Chui & Nellis, 2003). These methods were however criticised as they used reconviction rates as the main criterion for success.
In addition most of the research was conducted in the USA and Canada on white males and juveniles, thus questioning the results and how effective they will be in Britain on ethnic minorities and females (Chui & Nellis, 2003). Merrington and Stanley (2000) were cautious about using reconviction rates as an outcome measure of the offending behaviour programmes and felt that Probation success can be measured using various other measurements such as rate of compliance, improved attitudes towards offending and reduction in social and personal problems (Raynor 1996).
Robert Ross and his colleagues in 1985 developed the Canadian ‘Reasoning and Rehabilitative’ programme which was later adapted and named ‘Straight Thinking on Probation’ (STOP) by the Mid-Glamorgan Probation Service in 1991. The STOP programme indicated that when compared with offenders receiving custodial sentences, the STOP programme completers were reconvicted less and self reported changes in their thinking (Raynor and Vanstone 2001).
The Probation division of the Home Office began to promote the development, piloting and evaluation of particular initiatives as ‘Pathfinders’, usually with evaluation provided by the Probation Studies Unit in Oxford, and by the time more substantial research funds became available four priority areas had been identified for development and research. These were offending behaviour programmes, basic skills, and an enhanced version of Community Service and resettlement projects for short-term prisoners.
This was the first concentrated and targeted official research on the effectiveness of the Probation Service’s work with offenders since 1970s, and much the largest body of research on this subject ever undertaken in Britain (Raynor 2004). The Home Secretary incorporated ‘what works’ initiative into his Crime Reduction Strategy in July 1998 and supported it with i?? 250 million (Home Office 1999b). Guidance to probation managers and front – line manager practitioners was given in a number of publications including: Evidence Based Practice: A Guide to Effective Practice (Chapman and Hough 1998).
The ‘What Works’ probation experiment in England and Wales was the largest initiative of its kind in the world, and the expectation that the major elements of it would be implemented and show results in three years was not realistic. The pressure created by this time-scale led to short-cuts, some of which had little evidence available to support them at the time and have in retrospect proved damaging (Raynor 2004). This hastiness contributed to the weaknesses in EBP.
For example, the targets for accredited programme completions set in 1999, which drove the pace of the roll-out of offending behaviour programmes, were negotiated with Treasury officials without any systematic prior assessment of the characteristics of offenders under supervision and their suitability for programmes (Raynor, 2003a). The instrument which was designed to produce such assessments, the Offender Assessment System (OASys), originally promised for August 2000 was also delayed.
The targets quickly proved too high for most probation areas to achieve and were eventually reduced, but not before consuming time and effort and causing problems for staff and managers (Raynor 2004). As the implementation of EBP was not done gradually, this put internal pressure on probation officers and the system which contributed to its weakness, thus reducing its effectiveness. This caused the staff to feel disgruntled and contributed to a reduction in staff morale and dissatisfaction with their work.
Such low morale can reduce the effectiveness of the approach in working with offenders. (Rex)(1999) emphasises the role of probation officers promoting desistance. However Farrall (2002) in his study could not find much evidence linking desistance to specific probation intervention. The principal aim of ‘What Works’ is to reduce re-offending by ensuring that every aspect of Probation work with offenders is based on methods that have proven to be effective. EBP was adopted in response to Political and public response with regards to growing crime rates.
Vennard and Hedderman 1998 stated that there had been a 20% reduction in some of the research programmes since adopting ‘What Works’ principles in terms of risk, need, responsivity and programme integrity. Cognitive behavioural methods predominated as they scored highly in meta- analyses of what worked . Maruna (2001) suggests that cognitive based programmes do help offenders address their past mistakes however it doesn’t address an offender’s social needs in a way that desistance theory does. For instance, cognitive behaviour programmes alone are not going to address offender’s issues especially those that come from deprived backgrounds.
In my limited practice experience, it appears that cognitive behavioural programmes are the main focus rather than pertinent issues such as housing which is being neglected; the reason for this could have been because government were trying to shift their responsibility to the offenders as a result of resource constraints. However how do you retain people in programmes when they have other issues such as housing issues that needs to be addressed? Additionally it can be very easy for practitioners to adopt an approach that seeks to fit offenders to programmes in order to hit governmental targets instead of matching interventions to offenders.
This can cause conflict with regards to programme integrity and responsivity (Chui & Nellis, 2003) which may reduce the effectiveness of such an approach and contribute to its weakness. Underdown pointed to the crucial role of the case manager as ‘change agent’ in the context of cognitive – behavioural programmes (Gendreau, Goggin and Smith 1999). Underdown argued that by providing preparatory and motivational work prior to the commencing of programmes, the case manager would be uniquely placed to support the learning processes which cognitive-behavioural programmes promotes.
As a weakness it is now accepted that in the rush to implement programmes and achieve referral targets, this aspect of ‘effective practice’ has been neglected. Furthermore, it is increasingly being recognised that poor case management may well have compromised the effectiveness of such programmes (National Probation Directorate 2001; HM Inspectorate 2002b, 2003, Kemshall and Canton 2002). The high percentage of non-starters and non-completers of offenders on programmes may not necessarily be a reflection of programme ineffectiveness; instead it may also be due to poor case management.
Merrington and Stanley (2004) state that the success of a probationer in completing a programme, to a great extent depends on the preparation and support of case managers. However case managers can be faced with a number of constraints which can undermine their case management skills. Offenders who are placed on programmes are expected to start within six weeks of their order, as a case manager this can often prove difficult for me. Firstly because sometimes due to administrative problems I do not get to meet the offender until the second or third week of the offender’s order.
This leaves little time to complete the pre group work for the offender, get them motivated and address any problems that they may have. Once my offenders are placed on a programme I ensure that I have monthly one to one supervision with them to monitor their progression and address any problems. This ensures that the offender’s participation on the programme is integrated with overall supervision activity. This I feel has proved to be effective as these offenders go on to successfully complete the programme.
The ‘What Works’ risk assessment tool OASys determines the risk of harm and reconviction and the criminogenic needs of an offender. It is useful as the assessment forms the basis of effective offender supervision within the community. However OASys is said to be indifferent to diversity as the needs of ethnic minority groups and women are likely to be insufficiently recognised by this method (Shaw and Hannah-Moffat, 2000). The failure of OASys to accurately assess the criminogenic needs of these groups infers that there exists the possibility of mismatching offenders and programmes.
This could reduce effective participation and engagement of such offenders in the intervention provided as such intervention may not be relevant to addressing their problems resulting in programme ineffectiveness. This compromises the need and responsivity principle which underpins effective practice. Raynor (2003) argues that some risk factors for women overlap with men but accepts that there is a risk of over-predicting reconviction for women if the same risk instruments are applied without appropriate modifications.
This and the implementation on a large scale of many programmes which have not been subjected to reconviction studies appear to support the criticism that the implementation of What Works is running too far ahead of the evidence (Merrington and Stanley 2000). With regards to diversity I do not feel that programmes are always suited to ethnic origin: For example I am co-working a case where the offender is currently on the IDAP programme. He comes from an African background and due to this he has problems with what is taught on the programme.
The programme does not take into consideration cultural values because in the African culture the man is the head of the house, and there is no partnership. In IDAP when they talk about partnership and the man and woman playing equal roles he is not able to identify with this and this is a topic that I have to address with him often during supervision. However in response to the argument that EPB does not do enough to support diversity, Probation now has a programme designed only for women offenders to help them make positive changes in their lives and to reduce the risks of them re-offending.
This is a strength in terms of EBP in that it attempts to address problems that it is faced with. As stated earlier the ‘What Works’ research was conducted on groups of white men aged between 17 and 21 (Roberts, 1989), and their results have been put into practice by EBP and been applied to all offenders. Because one country has gained positive results it cannot be assumed that this will work well with another country. It may be inferred that the ‘What Works’ initiative may not be what it claims to be in that it is not objective and universal.
Consequently some consider EBP to be narrow and restrictive in its research base and practice Mair (2004). This contributes to the weakness as the approach appears to have failed to take into consideration that offenders are an eclectic in composition and would therefore have different needs. However what is clear is that although with flaws, meta-analyses have enabled us to draw some conclusions by collecting findings from smaller studies (Chui & Nellis, 2003). The provision of education, training and employment (ETE) for offenders is included in the ‘What Works’ agenda.
Research is unanimous that unemployment is closely linked with high rates of recidivism (May 1999). This is a strong point in the approach, as the approach is geared towards meeting the diverse needs of the offenders to reduce recidivism. A study comparing 12 month offending rates before and after ETE enrolment showed a 25% reduction in re-offending, however few of the studies were able to show that those improvements were linked to reduced re-offending (Merrington and Stanley 2004), thus making it difficult to evaluate effectiveness.
Basic skills is a community penalty scheme designed for offenders where they can acquire basic literacy and numeracy qualifications. It has been shown that offenders have limited numeracy and literacy skills and thus have limited employment opportunities. However other research evidence remains ambiguous as to whether improvements in literacy and numeracy actually reduce reconviction. An evaluation of the nine basic skills pathfinder projects (McMahon et al in Merrington and Stanley, 2004) showed very high attrition rates.
Improvement in basic skills was minimal due to the small amount of time spent on the course (Merrington and Stanley 2004). High attrition rates may be based on the fact that attendance is voluntary and not enforceable. Enforcing basic skills attendance may reduce attrition rates however I do not feel this will work well as offenders may not engage well with their tutors, thus limiting their learning. Hearden and Millie (2004) state that the best strategy for securing compliance should be reward based instead of punishment.
I agree and always encourage my offenders to attend basic skills, highlighting to them that instead of supervision with me they can use that time to attend basic skills on alternate weeks. This also ensures that those appointments are enforceable, however it is important to note that successful completion of basic skills depends highly on how motivated the offender is. Enhanced community punishment (ECP) is also part of effective practice. ECP combines unpaid work in the community and basic skills.
Ten community service pathfinder schemes were evaluated between 2000 and 2003, 54% of those enrolled for skills accreditation gained awards and 15% experienced an improvement in their employment status. The studies demonstrated a significant reduction in pro-criminal attitudes and in problem levels. Thus evidencing strength in terms of Probation practice (Rex et al, cited in Merrington and Stanley 2004). Another way EBP has contributed to the strengths of probation practice is that we can now review and assess the impact of our work with offenders.
In the past, the effectiveness of Probation work with offenders was not monitored, this proved to be a barrier with regards to government funding. However we are now able to enhance our practice by identifying our strengths and weaknesses, building on the strengths and addressing the weaknesses. EBP also enables us to evidence our work which will attract governmental funds. Research conducted in the past about how best to work with offenders can give workers knowledge to guide their practice and lead to improved outcomes.
Improved outcomes may not always apply in every case but overall those who follow Evidence Based practices are likely to have better outcomes Trotter (2006). However, there is the possibility of the researcher’s values and ethics influencing the research and thus its outcome and interpretation. Research is thus not without bias; this is a weakness for EBP. Cohen (1985) cited in Canton (2007, pg241) contends that the relationship between discursive accounts or research (in my view) and practice is not straightforward. Assessing ‘What Works’ and EBP have proved difficult as results have often proved inconclusive.
Problems with regards to research and evaluations is that they are often slow to be published and sometimes inaccessible (Knott 2004, cited in Burnette and Roberts 2004), this limits the effectiveness of the work that is being done. It is also becoming clear that the weight of evidence in support of cognitive-behavioural methods results partly from the fact that they have been subject to more systematic research than other approaches because they were among the first interventions with offenders that allowed researchers a degree of certainty about the content of services. This permitted greater clarity than usual about what practitioners were doing and therefore about the inputs associated with any observed outcomes. However, the approaches which have become characteristic of the ‘What Works’ movement do not claim to be a universal remedy but they continue to enjoy large and growing empirical support from internationally gathered evidence (Raynor 2004). It is important to consider the evidence thoroughly, and in the real world many of the managers and practitioners who drove forward the British version of ‘What Works’ will not have had time to do this.
Relying on summaries or extracts creates the risk that important qualifications, limitations and contextual issues will be missed, and this may help to explain some of the problems encountered. In particular, looking only at headline results will often mean that lessons about implementation are overlooked (Raynor 2004). Cognitive-behavioural methods based on social learning theory, often but not always delivered in group programmes; continue to emerge as a good option for assisting some offenders to stop offending.
Where there has been some shift in the emphasis of recent research, has been its increased recognition of the critical role played by sound implementation (Gendreau et al. , 1999; Bernfeld et al. , 2001, Cited in Raynor 2004) and by the personal skills of practitioners (Trotter, 1993; Dowden and Andrews, 2004, Cited in Raynor 2004). Increasing care is also being taken to recognize and control the element of judgment which is inevitably present in coding for meta-analysis, and experienced research reviewers (McGuire, 2002) warn against simplistic attempts to use the results of meta-analysis as recipes for programme development.
This is an aspect that will contribute to the strength of future Probation practice. Another significant point to remember is that offenders are all different and unique and what may be effective with one offender is not necessarily effective with another. What also needs to be address is specific improvements which need to be made with regards to programmes and programme completion. A standard methodology for evaluating programmes would be very beneficial. More research should also be conducted for other types of interventions other than cognitive behavioural intervention.
This is important as What Works literature reinforces the importance of a multimodal approach to address the diverse offending related needs (Chapman and Hough, 1998). This essay has given a brief developmental and historical overview of the history of the ‘What Works’ movement and Evidence Based Practice (EBP). It has assessed how this has contributed to the strengths and weaknesses of Probation practice as a strategy and has highlighted some suggestions that could enhance the effectiveness of the approach.