Purpose
Our Evidence Centre has published research that looks at the information available to social workers who are making decisions about appropriate care and protection options for children and young people.
The purpose of this report is to document the Enhanced Intake Decision-Making Project, which sought to answer the following question: could care and protection intake decision-making be improved by giving social workers access to a statistical risk tool? The current project involved the development of a model, pre-testing of trial materials, further model development, and a trial at the Child, Youth and Family (CYF) National Contact Centre.
Key Results
Results from the project demonstrate that a statistical risk model has the potential to improve the effectiveness of intake decision-making
The overall findings of the Enhancing Intake Decision-Making Project suggest that using statistical risk model information, in the form of a ‘Background Risk Indicator’, has the potential to improve care and protection intake decisions. Results from the trial highlight that when this information is used successfully, social worker decision-making is influenced in a safe and expected manner. These results suggest that the potential for more effective decision-making provided by the model could be realised, which may result in improved outcomes for New Zealand children, young people and whānau. While further development work is warranted, these results provide a clear platform for progressing towards an implementation phase.
The key findings of the project are summarised below.
Feasibility testing indicates that a statistical risk model can be developed from linked administrative data held by MSD, which has the potential to improve the effectiveness of care and protection intake decision-making
In past years, CYF has received notifications about care and protection concerns for 100,000 unique children and young people each year. In 2014, for a subset of this group with reliable data, about 63 percent were referred to local CYF sites for further investigation or services.
The project developed a measure of estimated concern in order to assess the accuracy of intake decisions. The measure used information about what happened after the referral decision (either further intervention on the part of CYF, or a further report of concern within the following two years) to estimate if there was a care and protection concern for the child or young person. This approach provides a reasonable measure, but will not be correct in all circumstances.
Based on the measure of estimated concern that was developed, existing intake decision-making is accurate in about 60 percent of cases.
Using historical data, the statistical risk model appears to be around 6 percentage points more accurate than existing intake decision-making. The model was able to refer an increased proportion of children and young people where there was an estimated concern, as well as fewer children and young people where there was not a concern.
As is good practice, ethnicity was not included as a variable in the model. When it was added in order to assess any remaining contribution, it added only a minor increase to the accuracy of the model’s decision-making.
The increase in accuracy provided by the statistical risk model was broadly comparable across different ethnic groups, although the model refers a higher number of Māori children and young people than under the status quo. The reason for this higher referral rate is currently unknown, and further work will be required to better understand this finding.