Quality Times Volume 1 Issue 1 01012023

QUALITY TIMES January 3, 2023 Volume 1 Issue 1 A publication of the Children and Family Services Section Continuous Quality Improvement Program INSIDE THIS ISSUE CRUISING ALTITUDE Cruising Altitude 1 Cory Pedersen, Director Children and Family Services Section The Whiteboard 2 Welcome to the inaugural edition of Quality Times: The quarterly newsletter News Ticker 8 for the Continuous Quality Improvement program of North Dakota’s public child welfare system! Spotlight 9 The goal of this publication is to provide stakeholders with brief information The Dashboard 10 about the quality improvement process, child welfare data, system issues, and change projects in North Dakota. “Quality is everyone’s responsibility.” Quality Times is divided into 5 sections: ~ W. Edwards Deming • Cruising Altitude delivers the “30,000 feet view” of quality improvement in North Dakota. • The Whiteboard contains educational information for the reader’s quality toolbox. • News Ticker presents news, announcements, and tidbits of information from the quality improvement world. • Spotlight highlights CQI champions and their commitment to improving outcomes for children and families. • The Dashboard displays an at-a-glance view of the system’s key performance indicators. Quality Times will be published in January, April, July, and October of each year. If you have an idea for an article to be featured in the publications, please contact CQI staff at [email protected].

QUALITY TIMES Page 2 The Whiteboard • CQI is holistic. It continually integrates and incorporates information using multiple aspects of data from across the Lauren Sauer, Assistant Director child welfare system to make sure we can establish Children and Family Services Section linkages and bring about lasting improvements. CQI is here! Isn’t CQI just another name for QA? Some of us will respond to that statement with, “It’s about The answer to that question is simply, “Nope!” In child time!” Others are probably questioning, “What is this mystical welfare, quality assurance refers to conducting case record CQI that he’s talking about?” Whatever your questions, CQI reviews and other activities to assess compliance with State, isn’t something to be feared or dreaded. CQI is here to help Federal, and/or local requirements. While QA case review is improve outcome for those we serve…and maybe make an important component of CQI, CQI is a much broader things easier for us along the way. process. QA is an activity to assess compliance whereas CQI is an effort to foster continual improvement. You can think of it What is CQI? this way: Quality Assurance is a component of CQI. CQI, on CQI stands for Continuous Quality Improvement. In the the other hand, encompasses Quality Assurance and other field of child welfare, CQI is generally seen as a process for tools. evaluating the quality of work with children, youth, and families. It’s a process for measuring outcome of the work Some of the characteristics that often distinguish straight up with children and families around the areas of safety, QA from CQI include: permanency, and wellbeing. It is also seen as a process for gathering information to guide program improvement efforts • (QA) Retrospective vs. (CQI) Proactive: QA tends to be and reinforce best practices. a separate activity that is implemented in reaction to something that has happened. CQI is designed to be According to the federal Administration for Children and proactive – activities designed around being ahead of Families, CQI is a comprehensive continuous learning things – using data to look at what is happening before process implemented by an organization to identify, describe, something bad happens but can quickly react as needed. and analyze strengths and problems and then test, implement, learn from, and revise solutions. CQI is: • Focused on (QA) WHO vs. (CQI) WHY: QA often is • A Philosophy, as it places a high value on focused on who did or did not do something (again, reactionary). Something went wrong – please come up Improvement. with a plan to fix it. CQI is about taking the time to fully • A System, as it is a coherent set of components that understand the why before jumping into fix-it (solution) mode. supports the continuous improvement process. • A Process, as it seeks to identify, describe, and •(QA) Top down vs. (CQI) multidirectional: CQI is focused on getting multiple perspectives, so you really analyze strengths and problems and then test, implement, understand the problem. learn from, and revise solutions. • (QA) Reactive) vs. (CQI) Proactive) Key features of a CQI system are: • (QA) Select staff involved vs. (CQI) all staff engaged. • CQI is data driven. In fact, data is at the very heart of Everyone has a role in CQI. continuous quality improvement. • (QA) Management driven (directive) vs. (CQI) being • CQI is intended to be proactive rather than reactive. employee, stakeholder, and consumer-driven. Everyone Data is used to identify problems as they are coming drives towards improvement outcomes. rather than waiting for someone else to tell us something is wrong. It’s just a new fad…right? • CQI is inclusive. Everyone has a place in continuous CQI is not the latest fad, and it certainly isn’t something new. quality improvement, from those with lived-experience to Some argue that the origins of today’s CQI processes can go system stakeholders to agency staff at all levels. back to 1789 and Eli Whitney and his mass-producing of muskets by fitting the interchangeable parts in an assembly

Page 3 line. Out of the work of Ohno came Kaizen, which is Japanese for continual What we know of CQI today improvement. Masaaki Imai Imai really goes back a bit over a popularized the concepts, systems hundred years. In the late Shewhart and tools of Kaizen in the mid 80s. 1910s and early 1920, Walter Shewhart, working for Western James Womack took the Toyota Electric Company, created the Production System and expanded and process control chart. popularized it into what we know as the Lean method of improvement. A student of Shewhart, W. Edwards Deming took Shewhart’s work and expanded upon it. He moved toward the Bill Smith, while working for Motorola in the mid 80s, application of statistical methods to introduced Six Sigma, a statistical process control industrial production and management. technique that is tied directly Womack back to the works of Walter He took what was referred to as the Shewhart. And Eli Goldratt’s Theory of Constraints shares Shewhart Cycle, which was Plan-Do-See commonalities with Plan-Do-Study-Act, which – like Six and created the Deming Cycle of Plan- Sigma -- takes us back to the very beginning. Do-Study-Act…which is the basis of the So, we can see how the works of one Deming CQI Cycle North Dakota’s child welfare influenced so many and led to a solid system uses. foundation for continuous quality improvement. Deming and Joseph Juran were both instrumental in the revitalization of post-war Japanese industry, meeting with industry leaders and universities to teach their quality Juran management principles. They are considered by many as the reason for the Japanese economic miracle. This refers to Japan's record period of For the child welfare system, our work economic growth between the post-World War II era and began in earnest in 2012 when the Goldratt the end of the Cold War. During the economic boom, Japan Administration for Children and Families released the rapidly became the world's second-largest economy (after Information Memorandum on establishing and maintaining the United States). continuous quality improvement systems in state child welfare agencies. ACF noted in the memorandum that the foundation Around the same time, Taiichi Ohno – working at Toyota – and much of the work in continuous quality improvement has created the Toyota Production System, been focused on the manufacturing industry. They also noted which was used to enhance quality and that the concepts proven there can successfully be used in productivity and reduce waste. child welfare to improve the outcomes for children and TPS resulted in the Just-in-time' (JIT) families. management philosophy, which This is what brought us to today. We have taken what has come before us, expanded it, and customized it to the needs supplies, and goods are produced of North Dakota. to meet customer demand exactly Ohno at the time they are needed. He is also credited with creating the Kanban board, to manage Just-in-Time processes.

QUALITY TIMES Page 4 Working the Wheel There are several CQI models out there. You may have heard of Lean or Six Sigma. We follow a simple and straightforward six step approach. The first step involves identifying and understanding the problem. We ask things like, “What is the problem?” “Who is affected by the Problem?” and “Why is it occurring?” Is it an unmet need or maybe a need for improvement in a current practice or process? What do our data tell us about the population that’s most affected and why that is the case? For instance, let’s say we are seeing that many children in the system move frequently from one foster placement to another. We know from the data that children in care move on average once every 10 months. You are aware of the problem, but the reasons for the problem are less clear. It might be that foster parents are not given accurate information about the children placed in their homes, that foster parents don’t receive adequate training to parent the children placed with them, that foster parents aren’t accessing sufficient supports when a crisis occurs, or that there are no mechanisms in place to prevent a placement change. It might be a completely different reason or a combination of one or more of these or other reasons. Each of these reasons, or combination of reasons, will require a different solution. The point is, if we don’t know the underlying cause or causes of the problem, we risk implementing a solution that will not work. For this example, let’s say we identified the problem you want to address as a lack of placement stability. Our data has told us that the lack of placement stability is primarily affecting older children who are experiencing mental health symptoms and we’ve identified the fact that their foster parents are not receiving sufficient support and services as a possible reason for the problem. Step 2 is to search for solutions. Questions to ask during this step are: “What do we already know about how to solve this problem?” “What are the experiences of other agencies with this problem?” “What more can we learn about solutions from our own data?” “What are the root causes of the problem?” and “What are the needs of the target population?” Returning to our example, a review of literature on placement stability, targeted case reviews that include interviews with foster parents and youth, and consultation with experts are pointing you toward addressing the unmet need for support and services of foster parents of children older than 12 with mental health symptoms. Step 3 is to develop the theory of change. A theory of change describes the change needed in the current system to achieve the intended outcome. Developing it is a collaborative process involving internal and external stakeholders and is based on extensive research, data collection, and analysis. Your theory of change should incorporate your definition of the problem and your solution. Back to our example, working with a team of our colleagues and external stakeholders, the theory of change we’ve articulated for the target population is that children in the system who are 12 or older with mental health symptoms will experience increased placement stability if the following supports are provided to their foster parents: regular access to respite care services and a hotline to call to receive immediate services and supports if they experience a crisis in the placement.

Page 5 The 4th step, also done in collaboration with your partners, is North Dakota designed the CQI teaming structure to consist to prepare to implement the solution in your organization by of two primary team areas: adapting or developing the innovation. And existing intervention can be implemented as is or adapted to your The first is the State CQI Council. The Council is the primary agency’s specific circumstances, while a new intervention driver the statewide CQI process and is comprised of a wide must be developed from the ground up. In adapting the range of state-level agency staff and stakeholders. Members solution in our example, we will also have to find more foster of the State CQI Council come together on a quarterly basis, parents to be respite care providers and set up a hotline for using data and other sources of information, to bring to light foster parents in addition to developing new policies and and gain a better understanding as to what is working well procedures. and what is not in relation to our core strategic priorities. We know that the work doesn’t occur once every quarter, though. Step 5 is implementing the solution. This is often done in Short or long-term sub-groups are formed as needed to stages with initial implementation occurring on a smaller scale target or support specific areas in need of improvement and to allow for adjustments before full implementation occurs. It’s facilitate activities. One good example is the already-formed important to ensure that implementation is occurring as Data Analytics Team. This team is instrumental in identifying intended. Fidelity to an intervention is extremely important and analyzing the data we need because without fidelity you will not know if your results are to make informed decisions. the result of your intervention or not. To go back to our example, we were surprised to find during initial The second implementation that foster parents were not calling the hotline team area as much as expected. Upon examination, it was discovered consists of a total that case workers were not consistently providing information of four Cross-Zonal about the new hotline or the phone number to foster families. CQI Teams Adjustments were made to the training of caseworkers – so representing each of that they would regularly disseminate the hotline number – the state’s Human Service before full implementation. Zones, Division of Juvenile Services, and Tribal partners from across the state. The last step in the process is to monitor and assess the Cross-Zonal CQI Teams are the drivers for the solution to determine if improvements are being observed. local CQI process (see Page 6). Like the Council, the Cross Based on your assessment, you may need to make changes Zonal CQI Teams will meet on a quarterly basis with or reject the innovation entirely if it has been implemented subgroup work during the interim. with fidelity, but improvements are not being realized. In our example, early and continued assessments have indicated Because the teams cannot do everything on their own nor do that the intervention has resulted in more placement stability they have all the resources they need to accomplish their for the target population. work, there is a CQI Support Team. The team will provide or arrange for technical support and needed resources to the We’ve just gone through the CQI Cycle…we just worked the State and Cross-Zonal CQI teams. The CQI Support Team wheel! will chaired by the CQI Administrator. The team will include members of the CQI Council including, at a minimum, the It’s Not a One-Person Show! Director of the UND Children and Family Services Training CQI is a team sport! We’ve all heard, “It takes a village to Center, the QA Unit Manager, the Foster Care/In-Home raise a child.” The same can be said about implementing Case Management Administrator, a Data Science & and sustaining CQI. Analytics Data Scientist, designated CQI staff, representatives from DJS such as field training officers, and representative Continuous quality improvement is not a one-person show. from Tribal agencies. To make this successful, support is needed from many.

QUALITY TIMES Page 6

Page 7 What Role Can I Possibly Have? from the systems. Some may be thinking that, with three core CQI teams, there are no roles for them to play in continuous quality The consequences of poor-quality data can include things improvement. like: Au contraire, mon frère! •Being unable to “tell our story.” Without quality data we cannot tell others what is happening in our system. Or we Remember CQI is a team sport. There is a role for everyone may come up with an inaccurate story because the data is to play on the team. For instance: wrong. • Volunteer to be a member of your Cross Zonal CQI Team. •The loss of credibility if we continually display inaccurate The Cross Zonal Team is the driver of CQI at the local data. level. There’s plenty of opportunity for people to be involved in the work of analyzing local data, identifying •An inability to make informed decisions. We will be back issues and root causes, and developing solutions. Get to making our decisions based on anecdotal reports or involved! solutions that were tried by other agencies – without any evidence that they’ll work for us. • Learn all you can about continuous quality improvement and be a champion of the process. Create a CQI- •Financial penalties to the federal funding that North receptive culture and climate throughout your agency. The Dakota receives, funding that goes directly to services for more you know…! children and families. • Bring issues you encounter to the attention of leadership Data is the heart of Continuous Quality Improvement. We and your Cross Zonal Team. Shining the light on what need to do whatever we can to ensure that it is high-quality. works and doesn’t will help us improve outcomes for those we serve…and may make things easier for all of us in the We are in this Together! long run. Continuous Quality Improvement in North Dakota’s child welfare system has been a longtime coming. Though at times One of the most important activities you can participate in is it may seem like a daunting process, CQI holds the promise ensuring that we have access to accurate information to make of better outcomes for those we serve while, perhaps, making informed decisions that lead to improved outcomes. things easier for us along the way. Our future together is full of promise. Perhaps Neil Armstrong said it best, “And what Key to improving outcomes is having timely, accurate, and will that future bring? I don’t know, but it will be exciting.” complete data. We’ve all heard the adage: Garbage In, Garbage Out. There are any number of ways for an For additional information on CQ!: agency to develop issues with their data quality. Some of the primary issues include: • Capacity Building Center for States https://capacity.childwelfare.gov/states/topics/cqi •Management Information System (MIS) data that is either • HHS Insider https://ndgov.sharepoint.com/sites/DHS- missing completely or incomplete: Caseworkers, ChildrenAndFamilyServices/SitePages/Children-and- Family-Services-Division.aspx supervisors, providers, or other staff not entering critical data • into the system. Even a small percentage of missing case information can skew results in performance data reports. •Delays in entering data into MIS: Staff may be entering the data into the system, but they are entering the data late. •Data entry errors by staff: Staff are entering the data, and doing it on time, however they are doing it wrong. •Staff and stakeholders having differing interpretation of data. •Restrictions inherent in your MIS – i.e. technical glitches h df b ld d

QUALITY TIMES Page 8 News Ticker Just for Fun CQI Support Team CQI Support Team Using Story Mapping to Inform Child and Family Services Reviews Planning Efforts (From the US Dept. of Health & Human Services) The Children’s Bureau has developed story maps on several topics, which can help visualize and understand child welfare data by race, age, and location. View the story maps: Child Abuse and Neglect: https://storymaps.arcgis.com/stories/334e7c28200f44 00888b6ecae2b5447f Victimization and Child Abuse and Neglect: https://storymaps.arcgis.com/stories/97d1e1c66ef74d a09ead05cc03bb023d Foster Care Entries: https://storymaps.arcgis.com/stories/226aedbc497f48 d69ff5e2d5a21cb727 Upcoming Events: • Cross Zonal CQI Team 1 Training January 17, 29, 24, 2023 (9am – 12:30pm) • CQI Implementation Team January 26, 2023 (9:30 – Noon) • Cross Zonal CQI Team 1 Quarterly Meeting February 2023 (TBD) • Cross Zonal CQI Team 4 Quarterly Meeting February 2, 2023 (1pm-3pm) • State CQI Council Quarterly Meeting March 16, 2023 (9am – Noon)

Page 9 SPOTLIGHT Lauren J. Sauer Position: Assistant Director and CQI Administrator, Children and Family Services Section, Education: North Dakota Department of Health and Human Services Master of Public Administration: Health Administration Concentration, University of North Dakota Master of Education: Clinical Counseling, North Dakota State University Introduction When I joined the Children and Family Services Division, I was charged with to quality: implementing a continuous quality improvement program for child welfare. Teacher who Tom Hesford, my English teacher when I was a junior in high school. He encouraged influenced you the most: me to develop my writing skills and believed in my ability. Best career advice: Don’t be afraid to make mistakes. God made erasers for a reason. Learn from what happened and move on. You’ll be stronger for it. When others make mistakes, Previous respond with grace. It’s better to solve a problem together than to have to pick up the noteworthy jobs: pieces alone. Personal: I worked the behavioral health field from being a psychiatric technician on an inpatient Currently Reading: psychiatric unit to providing counseling to being a program administrator running Favorite ways to relax: statewide programs. The job that was the most fun, though, was when I delivered printing products for a local printer. I was 19 years old, drove around in a beat-up Quality quote: orange delivery van that belched black smoke and had a hole in the floor through which the delivery book routinely fell, and got to meet new people every day. Each day was a new adventure. Wife, Tammie; 3 sons – Alex is a police officer and twins, Elijah and Gabriel, are in third grade; 2 mutts – Gracie and Nevada. Me: Elton John Doing crafts like resin art or outdoor decorations from pallets, mystery dinners, gaming with my kids, zoning out with a good movie or book. Quality is not an act; it is a habit. ~ Aristotle. Another way to look at it: Quality is not a function; it’s a mindset.

QUALITY TIMES Page 10 The Dashboard CQI Support Team The Dashboard displays an at-a-glance view of the system’s key performance indicators. This quarter’s dashboard includes information from North Dakota’s Child and Family Services Review Data Profile from the AFCARS and NCANDS submissions as of 6-28-22. This report is received in August and February of each year. Visualizations are presented as Risk-Standardized Performance (RSP). This is derived from a multi-level statistical model and reflects the state’s performance relative to states with similar children and takes into account the number of children the state served, the age distribution of these children, and, for one indicator, the state’s entry rate. It uses risk adjustment to minimize differences in outcomes due to factors over which the state has little control and provides a more fair comparison of state performance against the national performance. The vertical bars in the line graph represent the lower RSP and upper RSP of the 95% RSP (confidence) interval, and national performance (NP) is the dotted black line.

Page 11

QUALITY TIMES Page 12

Page 13

QUALITY TIMES Page 14

Page 15

QUALITY TIMES Page 16

For Additional Information: Continuous Quality Improvement Program Children and Family Services Division 600 East Boulevard Avenue, Dept. 325 Bismarck, ND 58505-0250 Phone: (701) 328-2316 Fax: (701) 328-3538 Email: [email protected]