ChildandYouthPopulationStandardsPilotTestEvaluationReport

Child and Youth Population Standards Pilot Test Evaluation Report

Article

Introduction

Accreditation Canada (formerly the Canadian Council on Health Services Accreditation (CCHSA)) has had a strong interest in the quality of health services for children and youth for many years. Accreditation Canada has established a partnership with Canadian Association for Pediatric Health Centres (CAPHC) to support standards and accreditation for children's health services to improve health care for Canadian children and youth. Accreditation Canada and CAPHC have also engaged with strategic partners, Child Health BC, to support this work.

Accreditation Canada has expanded the new accreditation program to offer standards of care, both for child and youth services and for populations. Service based standards include Obstetrics and Perinatal Care, while population based standards are available for both the Maternal and Child and Child and Youth populations. Population standards focus on ensuring a population-based approach to child and youth health among and between sectors (e.g. health, social, education), with particular emphasis on continuity of care and how services are organized, coordinated, and delivered to this population across the continuum of care. This approach will permit organizations to review their services from complementary perspectives of direct service delivery and how their services are positioned within the broader care continuum.

In collaboration with CAPHC and Child Health BC, Accreditation Canada worked with three British Columbia Health Authorities (HA) to pilot test the new Child and Youth Population standards. VIHA and Fraser HA completed the standards self-assessment questionnaire, an on-site survey visit, and provided feedback on the standards. Vancouver Coastal HA used the standards Self-Assessment questionnaire and participated in a conference call to provide feedback about the Self-Assessment questionnaire and the Population standards.

Feedback and opportunities for improvement were gathered from pilot organizations during the on-site survey visit, through face-to-face meetings, by teleconference, and by evaluation survey forms. 

Self-Assessment On-line Questionnaire

The majority of feedback indicated that the self-assessment questionnaire touched on topics that were important to the organization. Nevertheless, several areas were identified that needed enhancement, including:

• Improved education and information at beginning of questionnaire to provide more guidance to respondents
• More targeted scope for audience; because content was perceived to be broad, staff at different levels of the organization answered questions differently, but results do not include information about respondent so unable to differentiate results
• Fewer topics (i.e. criteria) per question so more clear what area of standards needs to be improved
• Narrower descriptors between categories
• Availability of location data so regional differences can be examined and analyzed
• Increased clarity regarding the "priority" setting, and impact on action plan, e.g. is it priority for completion, or a priority for importance regardless of completion?
• Further explanation of different coloured flags, including information on how the thresholds are defined
• Adding space for a comment or free text option at questions

Time to complete the questionnaire was thought to be reasonable, though there was wide variation, ranging from 10 to 30 minutes.

On-Site Survey

There was consensus feedback that the on-site survey visit was successful and valuable. Staff and organizations were very positive about the Surveyor team, found them to be flexible, and appreciated that team's expertise and in-depth understanding of child and youth services. The feedback and suggestions from Surveyors during the organization debrief were useful. There was agreement that future onsite surveys for the Child and Youth Population standards should include a Surveyor with pediatric background, and experience in how child health is organized and delivered across large regional settings.

Pilot testing the Population standards identified several areas that may require a modified approach to activities and timelines to fully evaluate the standards, particularly in the context of a large regional health authority. Enhancements suggested by pilot organizations include:

• Finalizing survey schedule further in advance to ensure input from full range of:
  • Non-health partners, e.g. education, child protection (keeping mind that child protection programs must be appropriate for tracer.)
  • Health care partners across the care continuum, e.g. mental health, public health, community and tertiary hospitals

• Establishing linkages and gathering feedback from primary care providers; particularly since it is an important aspect of child and youth health, and may present scheduling challenges

• Visits to full range of child and youth health services, programs, and locations (e.g. urban, rural, remote) that make up range of child health services in a region

• Input from range of services to aboriginal or First Nations populations

• Enhanced focus on contracts and contract management

• A community activity where organization is participant rather than driver

• Sufficient meetings with leadership team to gather understanding of how child and youth:
  • population needs are identified (particularly for hard to reach populations , e.g. refugees)
  • services are developed, achieved, and measured o services are resourced, given competing priorities of health authority

• Comprehensive understanding of health records, e.g. links between variety of health programs, use of electronic health records and strategies to reduce use of paper records

Pilot organizations had several questions about the tracer process, particularly whether a focused process was able to evaluate a large, comprehensive, and diverse program. In addition, some health care services children receive may be less amenable to tracer evaluation, e.g. children are in treatment for short periods of time or may never enter a facility.

Individual staff felt apprehensive about tracer process, i.e. individual meetings with surveyors, but had positive feedback after the experience.

Pilot organizations provided feedback that the advance selection of charts was time consuming. A suggested alternative was for organizations to include child/family participation in accreditation in standard consent forms. Input could then be obtained from children, youth and families by requesting records at time of survey visit, which would facilitate gathering patient information.

Standards

The majority of feedback regarding the standards was very positive. Most agreed that standards covered the key quality issues for Child and Youth health at the population level and did not contain redundant criteria. Overall the standards were well-received and staff felt the standards were reflective of the work being done. Some language revisions suggestions were made and many of these suggestions will be incorporated into subsequent version of the standards. Organizations define child and youth differently (e.g. 0 - 18, 0 - 19, 0 - 17), and Maternal / Child (also called perinatal, maternal / infant, maternal / newborn) often includes pregnancy and first 30 days post-partum. These varied definitions may lead to different standards evaluating multiple aspects of single health care services, e.g. public health.

Summary and Next Steps

Overall, there was consensus that the pilot testing the Child and Youth Population standards was a valuable project and that the new standards an important and useful addition to accreditation program.

Pilot organizations agreed that the new accreditation process provided relevant feedback, and that participating in the pilot allowed them to be an in a better position to be part of a full accreditation process. Pilot organizations also stressed the importance of sufficient preparation and planning time to prepare for the on-site survey visit, and the value of education so there is a broad understanding of the unique focus of Population standards and how they integrate with the services offered by the organization.
Accreditation Canada will use the input and feedback identified during pilot testing to improve how Child and Youth Population standards are evaluated. Next steps for Accreditation Canada include further enhancements to version 3 of the standards and associated Self- Assessment questionnaire, adjustments to the on-site survey scheduling process, internal education sessions, and development of templates to facilitate planning and preparation for Accreditation Canada and organizations using the Population standards.

Topic

Other

Sub Topic

Guidelines and Practices 

Description

Canadian Association of Paediatric Health Centres
dmaynard@caphc.org