This two-part guest blog post is by Iain Matheson. Dr Matheson is the inaugural director of a soon-to-be-launched not-for-profit research centre for residential and foster care. He is a social sector management consultant, researcher and evaluator, with a background in statutory child welfare management in both New Zealand and Scotland; he started his post-qualifying social work career in residential care. His recent doctoral research was on the experiences of New Zealand university students who were formerly in state care. (Disclosure: Between 2002 and 2004 Iain was the Child, Youth and Family national manager for residential and foster care, and has since undertaken work for Child, Youth and Family and the Ministry of Social Development).
Part 1 comprised of an introduction and a discussion of the interim report’s strengths (the good). This post addresses its weeknesses of the report (both the bad and the potentially ugly)
Has the panel actually fulfilled the stated requirements for their first report, as specified in the original terms of reference document? The task was to provide “the Minster of Social Development with a programme level business case by 30 July 2015, and focused on the case for change, desired future state for CYF and a high level assessment of options for a future CYF operating model” (2015, p. 131). While there is certainly a considerable amount of material supporting the case for change, be it as acknowledged by the panel without an evaluative assessment of CYF’s current performance (discussed further below), the articulation of CYF’s future state is rather general, and there is no assessment of high level options at all. I always thought that the terms of reference, and timeframe, were overly ambitious. Nonetheless, any re-negotiation, or rescheduling, of what was to be delivered across the two reports, should have been clearly acknowledged here. Furthermore, given the magnitude of change that the interim report and Ministerial statements have signalled, the absence of any assessment of high level options, and thus the opportunity for the sector to respond/contribute ahead of the final December report, is concerning (e.g. including but not limited to outsourcing to international or New Zealand private companies or NGOs, use of Social Impact Bonds, strengthened role for iwi, reconfiguration of national state agency responsibilities or the establishment of an ACC-type Better Lives Agency model, the District Health and Social Board model or a new role for local authorities, and/or an expansion of the Children’s Team/Whanau Ora model).
While the report’s narrative rightly makes much of the number of CYF reviews since 1988, as well as the Office of the Children’s Commissioners (2015) State of Care report, it is worth noting here that the latter was, in comparison to earlier reports at least, remarkably positive about much of CYF’s work outside of care and protection residential and foster care. The State Services Commission, Treasury and Department of Prime Minister and Cabinet’s Better Public Services review (2011) went even further and graded MSD for effectiveness as being “well placed” for both care and protection (and adoption services), and youth justice services; i.e. second top grade on a four point rubric. Presumably, in the light of the reviewers commending the “significant fiscal savings [in relation to children in care, and being] well placed to deliver ongoing efficiency gains in this area” (2011, p. 24), MSD was also graded as being “well placed” for efficiency in relation to care and protection services (youth justice services were graded as “needing development” for efficiency). This central agencies’ report also goes on to make the surprising statement that “comment was made that the progress made by CYF has been such that it is now acknowledged as being one of the best care and protection agencies in Australasia” (2011, p. 23). Yet the panel appears to make no reference to this report. So has CYF’s performance deteriorated so badly since 2011? No. Indeed, my observation would be that CYF’s performance has probably improved since the Better Public Services review was undertaken in 2011. For me, this usefully serves to highlight that our reviews are always undertaken within a political context, and take a particular position on what is important, and to whom.
Having repeatedly emphasised the importance of the role of evidence, beyond the observations and judgements of panel members, there is remarkably little evidence presented in this report. There are the valuable qualitative interviews with young people and young adults with direct experience of the CYF system that were commended in part one of this blog article. There is a useful synthesis of inquiry reports, and as stated previously some reference to findings from the recent State of Care report (Office of the Children’s Commissioner, 2015), as well as references to some international statutory child welfare experts. However:
a) Throughout the entire report, I was only able to find one (unreferenced) mention (i.e. Atwool, 2011) to any published New Zealand empirical or theoretical social work orientated research on statutory child welfare. While I would be the first to point to a lack of direct or indirect government funding and support for statutory child welfare research in this country, and indeed a national failure to develop our evidence base, to have essentially ignored the New Zealand social work research literature in its entirety, is very concerning. While there is other New Zealand published research included, and some from overseas, this is still very limited, and in places the inclusion of particular sources (and sometimes the statements that they support) is rather random.
b) In contrast, a high degree of reliance is placed on unpublished administrative data from the MSD CYRAS system, and with no clear acknowledgement of the quality and reporting limitations associated with MSD’s administrative data. This data is also used as the basis for exploratory reports that seek to match administrative data on those who have ever had contact with CYF, across different government datasets, in order to produce some outcome metrics. We desperately need good outcome metrics and this material is undoubtedly a good start. However, this is essentially exploratory analysis on the art of the possible; i.e. what might the Ministry of Social Development’s Integrated Child Dataset system be able to tell us about outcomes for those who have ever had contact with CYF as children, rather than being question-driven research. As it turns out, this analysis of the integrated dataset tells us rather less than we might have hoped. That children who are reported to CYF over the course of their childhood (i.e. predominantly the poor) are much more likely than the general population to, for example, go onto a benefit as a young adult or go to prison, is hardly a revelation. The categories are also too crude to be useful, for example, the care category in Figure 4.3 includes children who were in state care for several years and care leavers, as well children who spent one night in care as babies. Furthermore, neither is it, as heavily implied, necessarily indicative of a performance issue with CYF or a problem with the current CYF operating model. Indeed, this analysis probably tells us more about the limitations of CYRAS as a data management system (it was after all developed as a practice management tool, and as the interim report infers is in need of replacement) than it does about either the outcomes of children, CYF or the current operating model.
As well as issues around the design of CYRAS, data quality and reporting, such data-matching exercises also throw up their own not inconsiderable issues, and the degree of successful matches can be surprisingly incomplete. For instance, the authors (some of whom appear to be referred to here) of the related published Treasury paper Using Integrated Administrative Data to Understand Children at Risk of Poor Outcomes as Young Adults (Treasury, 2015) went to considerable and commendable efforts to explain the limitations and caveats around their preliminary work in this area. However, no such statements were included in this interim report.
The interim report calls for the use of evidence-based approaches. In Appendix E, 14 programmes, predominantly from the US, are listed that it claims are all “well-tested and proven approaches that meet a high standard of evidence” (p. 84). We do need to learn more about such “manualized evidence-supported treatments” (Barth & Lee, 2014, p.60). However, to say that all of these programmes are well-tested and proven is overly simplistic. Barth and Lee (2014) instead, more conservatively, define an evidence-supported treatment as “one that, generally, has enough scientific evidence behind it to allow for a reasonable conclusion that the treatment is likely to provide some benefit to clients who are appropriate for the intervention and willing to engage in it” (p. 60). Or in other words, there is reasonably robust quantitative experimental or quasi-experimental evidence that they may, if implemented efficaciously, be effective as variously defined, with some subgroups of children and in some particular circumstances. As well as practical (and ethical) issues around running experimental design research programmes in statutory child welfare (Dixon et al., 2014), there are also significant challenges in transplanting programmes developed in one country to another; to date the results from international replications have been rather mixed (Sundell & Ferrer-Wreder, 2014). By way of illustration, multidimensional treatment foster care, with its recognition by both the National Registry of Evidence-based Programs and Practices in the US, and the Californian Evidence-Based Clearinghouse for Child Welfare, is widely regarded as an efficacious evidence-based programme for young offenders. Yet a 2009 Cochrane Review of MTFC studies found that, as none of the evaluations of the programme had at that time been undertaken independently, there were concerns about the generalisability of the results and the potential for bias (MacDonald & Turner, 2009); on that basis the review was only able to describe the programme as promising. Furthermore, when the first independent evaluations were done of the MTFC programme, English researchers found that the use of MTFC in their studies had little or no statistically significant effect (Biehal et al., 2010; Biehal, Ellison, & Sinclair, 2012).
Thus, given that such programmes require “considerable investment of resources in order to purchase these programs…and create the organizational infrastructure, such as training, support, supervision, required for their implementation” (Shlonsky & Benbenishty, 2014, p.12) great care needs to be taken on whether an individual programme, or combination of programmes, should be trialled. Thus, Barth and Lee’s (2014) argue that organisations also need to look at the utilisation of common elements and common factors, as part of a broader mainstream practice evidence-informed framework.
Problematically, the interim report generally conflates the ideas from Evidence-based Practice with Manualized Evidence-supported Treatments. For Evidence-based Practice context is everything, a wide range of evidence is used, and as originally conceived, equal weighting is also given to practitioner expertise, client values and expectations; i.e. “EBP is the union, or intersection of these three constructs (Shlonsky & Benbenishty, 2015, p. 4). In many ways these are two very different beasts, yet the interim report appears to use these emerging ideas on Evidence-based Practice as their rationale for promoting and privileging evidence that is derived from randomised controlled trials and quasi-experimental design, and placing even more reliance upon (low quality) administrative data. Well-run experimental and quasi experimental designs may have an important role to play, and while not discussed here as it is covered elsewhere (for example, Gillingham, 2015) the development of actuarial tools might potentially even have a place too if the quality of CYRAS data can be improved. However, what is imperative about Evidence-based Practice is that the collection and understanding of, and decision-making about, evidence is central to, and not peripheral to, social work practice and management. However, that is not what I am seeing in the interim report.
The interim report clearly demonises CYF, its managers and staff. While there are undoubtedly many issues that CYF, as a service delivery arm of MSD, does need to take responsibility for, this demonisation is most apparent around the degree of control and responsibility that CYF has in relation to the current operating model, and takes two forms as follows:
a) The report fails to make others in government accountable for their current and past decisions. While the ’89 Act has many positive features, there are clear deficiencies and a failure to rectify these over the course of the last 26 years; this is not the responsibility of current CYF managers or staff. The failure of other government agencies to sufficiently prioritise children in state care is similarly something that Ministers and the central government agencies should be taking up with them. Similarly, the current government’s failure to progress the Children, Young Persons, and Their Families Amendment Bill 2007 (No 6) (which addressed some of the current issues with the ’89 Act), our country’s continued reservations in relation to the United Nations Convention on the Rights of the Child (UNCROC) 26 years on and our lacklustre implementation, and our near bottom OECD ranking in relation to measures to protect children from unintentional (in addition to intentional) injuries and thus suggesting something of a national disregard for the value of children and their safety in comparison to other countries, are not the responsibility of CYF either. Then of course there is the issue of poverty and the widening gap in New Zealand between rich and poor. This context, as well as a broader understanding of why we are in the current situation, is critically important yet appears to have been entirely overlooked. As such, the quality of some of the report’s analyses, and, unless rectified, the forthcoming proposals, is somewhat compromised.
b) The interim report also appears to insulate MSD from any responsibility or criticism. Why? The presentation of CYF as a “largely autonomous service line within the Ministry of Social Development…[and] receives support from the corporate centre (legal services, human resource and research” (p. 72) is not a reality that I recognise. Such a description implies that CYF somehow chooses to make use of some of MSD’s corporate services that are offered to it, but not others? This is misleading. CYF and MSD merged in 2006. As such, many functions were as I recall stripped out of CYF, including for example, Finance, Legal Services, Human Resources, Training, Policy, Reporting, Property and IT. Furthermore, children in residential or foster care in New Zealand are in the care or custody of the Chief Executive of…MSD. The wider Ministry is also part of the problem.
The interim report claims that the operating model will be child centred. Yet, while making a passing reference to criticisms from the United Nations Committee on the Rights of the Child (UNCRC) in relation to the definition of a young person under the ’89 Act, the report makes no further reference to either UNCRC or UNCROC’s 40+ economic, social, cultural, civil and political rights, or our international obligations. Why not? For all children and young people who come to the attention of the state for care and protection reasons, Stevens et al. (2011) suggest that seven of the UNCROC articles have a particular relevance: article two (protection from discrimination or punishment); article three (best interests of the child); article four (rights and responsibilities of families); article six (the rights to life, survival and development); article nine (separation from parents only when it is in the best interests of the child); article 12 (the right to an opinion and for that opinion to be heard in all matters affecting the child); and article 18 (parents have the primary responsibility for raising children and the need for the support from the state to assist them in this). Those in the youth justice system also have various rights under UNCROC.
Internationally UNCROC has been particularly strongly profiled in relation to those in residential and foster care (Domanski, 2012). Article 20 specifically concerns “the right to special protection and support for children who cannot live with their parents” Office of the High Commissioner for Human Rights. (1989). In other words, the Convention makes it clear that the state has additional obligations to provide children in residential and foster care with special support and assistance. Article 25 also specifically relates to children placed away from home and provides children and young people with the right to have their circumstances regularly reviewed. Importantly, many of the other articles have a specific relevance to children in care, for example the right to: identity (article 8); freedom of expression (article 13); freedom of thought, conscience and religion (article 14); privacy (article 16); special care and education for those with a disability (article 23); the highest attainable standard of health care (article 24); an adequate standard of living (article 27); education (article 28); protection from sexual exploitation and abuse (article 34); help and support for abuse and maltreatment (article 39) and a range of protections for children accused of breaking the law (article 40). Beyond these individual rights there is also now greater recognition that under UNCROC, signatories have a number of broader responsibilities towards those in state care and to care leavers (Munro, Pinkerton, Mendes, Hyde-Dryden, & Herczog, 2011) arising from the Guidelines for the Alternative Care of Children (UN General Assembly, 2010); sections 131 to 136 of these guidelines specifically relate to preparation for leaving care and post-discharge (whether planned or unplanned) support.
As previously noted the report makes the statement that “we do not currently have a good picture of the organisational performance of CYF” (p. 105). Yet in the Executive Summary, the report purports to be “an assessment of the current operating model” (p. 7), much of which is about CYF performance. It is therefore surprising that the interim report does not make an evidence-informed evaluative judgement on how CYF’s current performance compares with the past, other comparable organisations overseas, or other broadly similar New Zealand government agencies. Furthermore, unlike State of Care (Office of the Children’s Commissioner, 2015), I could find no instance in the interim report where a single CYF strength was identified. The most rudimentary assessments of the current operating model required, at the very least, the identification of organisational strengths upon which the new operating model could be, in part, built. Do the panel really believe that CYF does not have any? Or is it intending to recommend that CYF be dismantled to such an extent that it considers that any organisational strengths are of little relevance to their deliberations?
Linked to this, in reaching its findings on the performance of the current operating model, the interim report unhelpfully conflates organisational performance with CYF’s legislation and Parliamentary intent. For example, while the need for more work on supporting the connection of children to their cultures and communities (p. 14) is clearly consistent with the ’89 Act, that the system does not reflect a high level of aspiration for vulnerable children is clearly not. I do agree that we need to be much more ambitious for children. However, while a commitment to biculturalism was a critically important feature of this ground-breaking piece of legislation, the (admittedly exceptionally unwieldy 469 sections over 424 pages and in places heavily process-driven) ’89 Act, which governs almost all that CYF does and its initial implementation, also needs to be seen as a product of Rogernomics and Ruthanasia; i.e. small government, low public expenditure and deregulation. Neither Rogernomics nor Ruthanasia would have had any truck with having high levels of aspiration for vulnerable children.
The potentially ugly
So what of family/whanau, hapu, iwi, and biculturism? While the report calls for the need for “more work to be done on supporting the connection of children to their cultures and communities” (p. 14), is this merely about engaging with Maori and other cultures with greater cultural competence, for example when taking their children into care? Or perhaps just helping children already in care to make cultural links by understanding their whakapapa at an abstract level?
Is the panel signalling a more selective use of Family Group Conferences for some children? Or less use of, rather than better engagement with, kin care? What parts of the ’89 Act are the panel actually proposing to retain and build upon? Or will it be recommending that it be repealed altogether?
The interim report clearly demonstrates some understanding of long-term foster care. However, a similar degree of understanding is not shown in relation to residential care, youth justice or, in particular, child protection.
While the report makes no explicit proposals in relation to adoption, a return to New Zealand’s US/UK-style wide scale use of adoption of the 1950s and 1960s, or quasi-adoption, seems to be hinted at? Is increasing the supply of babies and children for adoption one of the means by which the interim panel proposes to provide children with the “earliest opportunities for a loving and stable family” (p.12) and “help New Zealanders to make a difference for vulnerable children” (p. 9)?
The panel’s first stated objective is “ensuring that children have the earliest opportunity for a loving and stable family” (p. 9). What is meant by earliest opportunity? Is this simply achieving permanency for children without unnecessary system delays? Or, given that the interim panel’s second principle to “support families to care for their children” (p. 9) explicitly includes new families, is this actually suggesting less, rather than more, support for biological families; i.e. taking more children, and particularly young children into care for placement with a loving and stable family elsewhere?
As highlighted by the panel, the absence of an advocacy service for children in care is a major gap in our system. However, the suggestion that the advocacy service for children in residential and foster care (and hopefully care leavers too) would be funded by a philanthropic organisation on the face of it appears to be just bizarre, as is the proposal that the panel themselves would undertake discussions with potential organisations ahead of the presentation of their main report (I would have thought that they already had enough to do working on the new operating model). Why is the development of an advocacy service not seen as part of their investment approach, and as such as something that would be largely funded by government? Is it because, despite the language in the report, the term ‘investment’ is being used in its more literal sense of offering a financial return through longer term savings? That is, is it being suggested that the overhaul is more about eventually saving money than improving services and outcomes for children and their families?
And finally, while the interim report demonises CYF, it does not demonise professional social work practice. Should we therefore take solace from that? No. Worse than demonising professional social work practice, the report largely ignores it.
Barth, R. P., & Lee, B. R. (2014). Common elements and common factors approaches to evidence-informed children’s services. In A. Shlonsky, & R. Benbenishty (Eds.), From evidence to outcomes in child welfare: An international reader (pp. 59-82). NY: Oxford University.
Biehal, N., Parry, E., Sinclair, I., Green, J., Roberts, C., Kay, C. Roby, A. (2012). The Care Placements Evaluation (CaPE) evaluation of Multidimensional Treatment Foster Care for adolescents (MTFC-A) [English Department for Education Research Report, RR 194]. Retrieved from http://www.york.ac.uk/inst/spru/research/pdf/cape.pdf
Biehal, N., Ellison, S. & Sinclair, I. (2011). Intensive fostering: An independent evaluation of MTFC in an English setting. Children and Youth Services Review, 33, 2043-2049.
Dixon, J., Biehal, N., Green, J., Sinclair, I., Kay, C. & Parry, E. (2014).Trials and tribulations: Challenges and prospects for randomised controlled trials of social work with children. British Journal of Social Work, 44, 1563-1581.
Domanski, D. (2012). Participation and the CREATE Foundation: Creating a better life for children and young people in care. Social Work Now, 49, 21-30.
Gillingham P. (2015, October 6). Why the PRM will not work [Blog post]. Retrieved from https://www.reimaginingsocialwork.nz/2015/10/why-the-prm-will-not-work/
MacDonald, G., & Turner, W. (2008). Treatment foster care for improving outcomes in children and young people. Cochrane Database of Systematic Reviews 2008(1), 1-53. doi:10.1002/14651858.CD005649.pub2.
Ministerial Advisory Committee on a Māori Perspective for the Department of Social Welfare. (1986). Puao-Te-Ata-Tu (Day Break). Report of the Ministerial Advisory Committee on a Māori Perspective for the Department of Social Welfare. Wellington, New Zealand: Department of Social Welfare.
Modernising Child, Youth and Family Expert Panel. (2015). Modernising Child, Youth and Family interim report. Retrieved from https://www.msd.govt.nz/documents/about-msd-and-our-work/work-programmes/cyf-modernisation/interim-report-expert-panel.pdf
Munro, E. R., Pinkerton, J., Mendes, P., Hyde-Dryden, G., &. Herczog, M. (2011). The contribution of the United Nations Convention on the Rights of the Child to understanding and promoting the interests of young people making the transition from care to adulthood. Child and Youth Services Review, 33(12), 2417-2423. doi:10.1016/j.childyouth.2011.08.025
Office of the Children’s Commissioner. (2015). State of Care 2015: What we learnt from monitoring Child, Youth and Family. Retrieved from http://www.occ.org.nz/assets/Publications/OCC-State-of-Care-2015.pdf
Office of the High Commissioner for Human Rights. (1989). United Nations Convention on the Rights of the Child. Retrieved from http://www.ohchr.org/Documents/ProfessionalInterest/crc.pdf
Shlonsky, A., & Benbenishty, R. (2014). From evidence to outcomes in child welfare: An international reader. NY: Oxford University.
State Services Commission, the Treasury, & the Department of Prime Minister and Cabinet.(2015). Performance Improvement Framework: Formal review of the Ministry of Social Development (MSD). Retrieved from https://www.ssc.govt.nz/sites/all/files/pif-msd-review-may11.PDF
Sundell, K., & Ferrer-Wreder, L. (2014). The transportability of empirically supported interventions. In A. Shlonsky, & R. Benbenishty (Eds.), From evidence to outcomes in child welfare: An international reader (pp. 41-58). NY: Oxford University.
Treasury. (2015). Understanding integrated administrative data to understand children at risk of poor outcomes as young adults [Analytical paper 15/01]. Retrieved from http://www.treasury.govt.nz/publications/research-policy/ap/2015/15-01
United Nations General Assembly. (2010). Guidelines for the alternative care of children. Retrieved from http://www.iss-ssi.org/2009/index.php?id=25