2022 UNPSA_AI Chest 4 ALL (DMS-TU) For Thai People_E book_Timeline and Evaluation Report Result

2022 UNPSA

“AI Chest 4 All (DMS–T U)” For Thai People

1. Objective of the initiative

Tuberculosis (TB) and lung cancer are two major health problems worldwide. In 2019, pulmonary TB resulted in 11 million cases and 1.3 million deaths globally. Thailand is one of only 14 countries on all three lists of “high burden countries” for TB. The TB incidence rate in Thailand is 137 per 100,000. Globally, lung cancer resulted in 1.8 million deaths in 2020, according to the WHO. In Thailand, the incidence of lung cancer was 22.8 and 11.5 per 100,000 among males and females, respectively, in 2018. Early screening with chest X–ray (radiography) prevents the spread of TB and increases lung cancer survival. However, X–ray image interpretation requires trained radiologists, of which there are only 1,400 in Thailand. Furthermore, most of these specialists work in urban areas. The wait time for results is at least 1 hour. Some patients must leave from work to receive results. This difficulty has resulted in high costs and also excess mental health strain. We have developed an advanced technology named “AI (Artificial Intelligence) Chest 4All (DMS–TU) For Thai People.” This highly efficient innovation achieves rapidity, equality, and comprehensive coverage of high–quality health care services able to consistently reach segments of the Thai population who live in remote areas. These impacts are especially notable in TB and lung cancer patients, as well as high–risk groups, including the elderly, low mobility and bedridden patients, and inmates. Many in these populations still lack sufficient access to medical care. Furthermore, this project was a technological development according to this concept of “by Thai people for Thai people,” which resulted in lower costs compared to imported technologies from abroad. We aimed to inspire the next generation to create something useful for the Thai people and to establish good health and sustainable well–being for all.

2. Alignment with the Selected Category

The “AI Chest 4All (DMS-TU) For Thai People” innovation is the result of automated deep–learning technology that provides accurate and rapid diagnosis of chest– related diseases, crucial for timely detection and early medical treatment. This innovation has been implemented in all regions of Thailand. It has enabled equitable access to high efficiency health services for vulnerable populations in Thailand, including TB and lung cancer patients, the elderly, low mobility patients, and inmates. Therefore, it has been categorized as “Fostering innovation to deliver inclusive and equitable services” of UNPSA2022.

3. Alignment with the 2030 Agenda

a. Please specify which SDGs and target(s) the initiative supports and describe concretely how the initiative has contributed to their implementation. Our initiative supports SDG Target 3.3 (end the epidemics of AIDS, tuberculosis, malaria and neglected tropical diseases and combat hepatitis, water–borne diseases and other communicable diseases) and Target 3.4 (reduce by one third premature mortality from non–communicable diseases through prevention and treatment and promote mental health and well–being). Our initiative contributed to these goals by quickly and accurately detecting lung abnormalities. It helped overcome the lack of specialized radiologists in remote areas and increased overall efficiency in diagnosis. TB is still a major problem in Thailand. Therefore, this innovation will greatly assist in reducing the burden of TB and reduce premature deaths from lung cancer. This cutting–edge technology shortens treatment time, reducing complications in both TB and lung cancer patients.

b. Please describe what makes the initiative sustainable in social, economic and environmental terms. Our “AI Chest 4All (DMS–TU) For Thai People” initiative was provided free of charge by governmental decree from the Director General of the Department of Medical Services (DMS), Ministry of Public Health. In addition, the Faculty of Engineering at Thammasat University provided the operating system. Our main objective is maximizing benefits to all people. By reaching vulnerable populations and those who live in remote areas, it is socially sustainable and beneficial for all Thai people.

4. Relevance to leaving no one behind

a. Please explain how the initiative has addressed a significant shortfall in governance, public administration or public service within the context of a given country or region. Pulmonary TB and lung cancer remain major health problems in Thailand. Accurate and rapid diagnosis of chest–related diseases is crucial for providing timely detection and early medical treatment for infected individuals. Our main objective is to aid radiologists in Thailand (especially in rural areas where there is shortage of staff) in formulating faster and more precise diagnosis for patients. Patients in remote areas, vulnerable populations, and high–risk groups (i.e., the elderly, low mobility and bedridden patients, and inmates) have limited and inequitable access to the necessary tools for rapid diagnosis and treatment due to distance and limited resources. Thus, the “AI Chest 4All (DMS–TU) For Thai People” program addresses those shortfalls. Currently, there are three portions of the program: 1. AI as a standalone program for small hospitals (45 hospitals) 2. AI connected with PACS (Picture Archiving and Communication System) for medium to large hospitals (77 hospitals) 3. AI installed in mobile chest X–ray vehicles for remote areas and inmates (5 service units)

b. Please describe how your initiative addresses gender inequality in the country context. A service evaluation at Udonthani Cancer Hospital from January 1st to December 31st, 2020, totaled 12,671 uses (9,168 females, 3,503 males), including 3,630 uses among the elderly. The evaluation found that all chest X–ray images were interpreted by the innovation regardless of gender, age, or status in the order received, resulting in 100% equal services. The use of AI provided unbiased gender equality and coverage.

c. Please describe who the target group(s) were, and explain how the initiative improved outcomes for these target groups. Government agencies have implemented this initiative in 122 hospitals, spread across all regions of Thailand, and in 5 mobile chest X–ray service vehicles, thus targeting populations who live in remote areas of Thailand. The targeted groups included TB and lung cancer patients, with a particular focus on high–risk groups, including the elderly, low mobility and bedridden patients, and inmates. Many in these populations still lack sufficient access to medical care. An evaluation of the program’s implementation at Lampang Hospital, located in the north of Thailand, found encouraging improvements in diagnosis outcomes. The resulting sensitivity, specificity, and accuracy rates were as high as 98.3%, 95.2%, and 97.2% for the categories of suspected active pulmonary TB, suspected lung malignancy, and no finding. These results demonstrate the initiative is an effective tool for chest radiograph screening, especially among pulmonary TB patients. Increased diagnosis will reduce the spread of TB and increase the speed of screening. Therefore, we can say that “AI chest for all (DMS–TU) for Thai people” is a smart technology for pulmonary TB elimination in Thailand.

5. Implementation

a. Please describe how the initiative was implemented including key developments and steps, monitoring and evaluation activities, and the chronology. “AI Chest 4 All (DMS–TU) For Thai People” Timeline 1. Development (Jul.–Sep. 2018): The use of deep learning has become a fundamental and established discipline that uses AI to help analyze, critique, and recognize patterns from a given dataset as a way to help diagnose diseases via radiological image scanning. Deep learning, a subcategory of machine learning, uses an established neural network system to enable the creation of back–to–back models with pre–set programs capable of achieving the model’s desired results via the use of AI. These foundational networks and algorithms were established. 2. Performance Testing (Sep. 2018–Sep. 2019): The AI models were trained through data simulation to interpret chest radiographs. 3. Research (Sep. 2018–Sep. 2019): Further development of the technology was achieved via research using chest images of willing participants approved under the Ministry of Public Health’s Human Research Ethics Committee (No. 6/2019). 4. Implementation (Dec. 2019–Apr. 2020): Government agencies implemented the initiative in 122 hospitals, covering all regions of Thailand, and in 5 mobile chest X–ray service vehicles. The program targeted patients in remote areas, the elderly, those with limited mobility, and prison inmates. 5. Assessment (Apr. 2020–Dec. 2020): The initiative was evaluated for accuracy, speed, and overall satisfaction. 6. Improvement (May 2020–May 2021): The program was improved based on the obtained results. Furthermore, we plan on inventing a deep learning model framework for lung segmentation and localization of COVID–19 from radiograph images in 2022.

“AI Chest 4 All (DMS–TU) For Thai People” Journey Nov, 2019 Dec, 2019 “SMART PUBLIC HEALTH” “Finished AI “AI Chest free accessing Chest project ” announcement by Director-General Department of Medical Services” Nov, 2020 1. AI Chest 4 All for Pneumonia screening project 2. AI Chest 4 All in Lung CA Multisite project (Cancer 7+1, DMS) *Development* 2022 Nov, 2019 – Oct, 2020 “COVID–19 Model” 1. AI stand alone May, 2021 2. Edge software “AI Chest 4 All 3. PACs & Cloud Re–training” connection Dec, 2018 “AI Chest project Oct, 2020 beginning” “AI Chest 4 All completed implementation for 45 government hospitals”

b. Please clearly explain the obstacles encountered and how they were overcome. In the early phase, the biggest obstacle was data management. At first, substantial manpower was required to physically transport data between departments and several adjustments were necessary to account for incompatible systems. As a solution, a cooperation agreement was signed between the government and the private sector to connect the AI initiative to the PACS system. This method has since been implemented in 77 hospitals across all regions of Thailand. The increased connectivity and computing power greatly improve access to accurate radiological interpretations for vulnerable population in remote areas.

6. Innovation

a. Please explain in what way the initiative is innovative in the context of your country or region. A significant innovation of the “AI Chest 4 All (DMS–TU) For Thai People” model is its unique pre–processing approach that uses different pre–processing techniques for each layer of the 3–channel radiography image. Furthermore, the massive image database can utilize the power of “Big Data” to provide novel insights into TB and lung cancer diagnosis. Machine learning allows the program to continuously improve. Outside of the program itself, it can lead to other changes. Policy makers can leverage this database to formulate disease prevention strategies, and it can provide a framework for implementing AI in other health care services.

b. Please describe, if relevant, how the initiative drew inspiration from successful initiative in other regions, countries and localities. The development group used the concept of “by Thai people for Thai people,” which promotes the belief that Thai people are capable of producing and owning technology used to solve health problems and these solutions will be better adapted to Thai society. This mantra promotes domestic development of technologies without reliance on high–priced technologies imported from abroad. Thus, “AI Chest 4 All (DMS–TU) For Thai People” is the first innovation of chest X–ray image interpretation by Thai and for Thai people.

c. If emerging and frontier technologies were used, please state how these were integrated into the initiative and/or how the initiative embraced digital government. “AI Chest 4 All (DMS–TU) For Thai People” connected to several systems including: 1. Picture Archiving and Communication System, or PACS, which improved efficiency and speed of the chest X–ray interpretation process. 2. Mobile radiography vehicles, which provided access to the service for those located in remote areas, the elderly, those with limited mobility, and prison inmates. 3. Online services, including a cloud system and telemedicine, which allowed for convenient access.

7. Adaptability

a. Has the initiative been transferred and/or adapted to other contexts (e.g. other cities, countries or regions) to your organization’s knowledge? If yes, please explain where and how. The “AI Chest 4 All (DMS–TU) For Thai People” program has been implemented in 122 hospitals across all regions of Thailand. It was also installed in 5 mobile chest radiograph service vehicles to screen for pulmonary TB among people in remote areas, including migrant workers at the border, and among prison inmates. These groups still have insufficient access to medical care. In the future we plan to extend these services to neighbouring countries (Myanmar, Laos, Vietnam, Cambodia), which have health problems similar to Thailand. Together, we hope to make progress towards the 2030 SDG goals in terms of TB elimination.

b. If not yet transferred/adapted to other contexts, please describe the potential for transferability. The plan to expand the use of the “AI Chest 4 All (DMS–TU) For Thai People” program will include the following services: 1. Coverage AI service implement for Thai people in all areas. 2. Screening travelers and migrant workers crossing Thailand's borders for infectious diseases, especially pulmonary TB, in order to prevent disease importation. 3. Screening prison inmates prior to incarceration for pulmonary TB and other pulmonary infections to prevent spread in the close confines of jails. Future plans include extending the program to neighbouring countries (Myanmar, Laos, Vietnam, Cambodia), which have similar health problems to Thailand.

8. Resources

a. What resources (i.e. financial, human or others) were used to implement the initiative? Personnel: 1. Udonthani Cancer Hospital and Central Chest Institute of Thailand, DMS, MOPH, Thailand 2. Centre of Excellence in Intelligence Informatics, Speech, Language Technology and Service Innovation (CILS), Faculty of Engineering, Thammasat University, Thailand Budget: 1. Office of Academics Fund, Department of Medical Services, Ministry of Public Health (2.3 million Baht) 2. Academic Fund Udonthani Cancer Hospital Foundation (5 million Baht) Data: (200,000 chest radiography images) 1. Udonthani Cancer Hospital, DMS, MOPH. 2. Central Chest Institute of Thailand, DMS, MOPH. Technology: 1. Udonthani Cancer Hospital, DMS, MOPH. 2. CILS, Faculty of Engineering, Thammasat University.

b. Please explain what makes the initiative sustainable over time, in financial and institutional terms. This initiative is consistent with the 20–year National Strategic Plan’s (2018– 2037) third strategy on human resource development by enhancing good health and with the plan’s fourth strategy on creating opportunities and social equality by reducing inequality and distributing economic, social, and technological prosperity. Furthermore, the Director General of the 8th Regional Health Service has encouraged the use of this innovation for pulmonary TB screening in 7 provinces in the northeastern area. It has also received support from the private sector.

9. Evaluation

a. Was the initiative formally evaluated either internally orexternally?  Yes  No

b. Please describe how it was evaluated and by whom? 1. The program was evaluated as part of the Public Service Award 2021 application (the highest national award available in recognition of outstanding public sector development) for service innovation by the Office of the Public Sector Development Commission, Office of the Prime Minister, Thailand. 2. The program’s performance was evaluated by Lampang Hospital, Lampang province, Office of the Permanent Secretary, Ministry of Public Health, Thailand (the results are shown in the figure).

c. Please describe the indicators and tools used. 1. The Public Service Award 2021 evaluated the following criteria: 1.1 The goal of the initiative and its specific objectives; 1.2 Process and implementation, including key developments and steps, monitoring and evaluation activities, and the timeline; 1.3 Evidence of output and outcomes, including quality of implementation, challenges faced, main outcomes, and impacts. 1.4 Sustainable Development Goals and alignment with the 2030 Agenda The resulting evaluation score was more than 90 %. 2. The initiative’s performance was evaluated by Lampang Hospital on the criteria of speed, accuracy, and overall satisfaction.

d. What were the main findings of the evaluation (e.g. adequacy of resources mobilized for the initiative, quality of implementation and challenges faced, main outcomes, sustainability of the initiative, impacts) and how this information is being used to inform the initiative’s implementation. This innovation was evaluated by two government agencies: 1.Office of the Public Sector Development Commission, Office of the Prime Minister, as part of the Public Service Award 2021 2.Lampang Hospital, Lampang province, Thailand as part of the performance evaluation (results are shown in the figure below)***

Implementation Outcome Disease Sens* Spec* ACC** No finding 95.16 95.05 95.25 Suspected active pulmonary TB Suspected lung malignancy 98.28 88.87 97.21 Abnormal heart and great vessels 85.83 Intra thoracic abnormality 97.20 95.15 96.56 Extra thoracic abnormality 94.08 80.05 94.20 87.66 83.42 85.41 82.99 83.07 Remark: *=Percentage, Sens; sensitivity, Spec; specificity, ACC; Accuracy, Credited by: Lampang Hospital, Office of the Permanent secretary, Ministry of Public Health, Thailand

These results provided assurance to healthcare facilities that this program is beneficial to healthcare consumers, in that it provided equitable health services via a novel, efficient, and high-speed technology. The awards recognized the program’s innovative approach to increase health care access to populations with inequitable access, including rural, elderly, immobile, and incarcerated populations. The awards also recognized the program’s contributions to good health and sustainable well–being for all Thai people.

10. Institutional Setup

Please describe how the initiative is inscribed in the relevant institutional landscape (for example, how it was situated with respect to relevant government agencies, and how the institutional relationships with those have been operating). Government sector 1.The medical team developed and evaluated the “AI Chest 4 All (DMS–TU) For Thai People” program, conducted among: 1.1 Udonthani Cancer Hospital, DMS, MOPH, Thailand 1.2 Central Chest Institute of Thailand, DMS, MOPH, Thailand 2. The software development and subsequent evaluation was completed by: 2.1 Centre of Excellence in Intelligence Informatics, Speech, Language Technology and Service Innovation (CILS), Faculty of Engineering, Thammasat University, Thailand 3. A support team was comprised of member from: 3.1 Udonthani Cancer Hospital, DMS, MOPH, Thailand 3.2 Centre of Excellence in Intelligence Informatics, Speech, Language Technology and Service Innovation (CILS), Faculty of Engineering, Thammasat University, Thailand 3.3 Technology Assessment and Research Department, DMS, MOPH, Thailand Private sector 1.Contribution from the private sector including marketing and software installation, with partnerships with: 1.1 Thai GL Co, Ltd. 1.2 FINE MED Company Limited

11. Stakeholder Engagement

The 2030 Agenda for Sustainable Development puts emphasis on collaboration, engagement, coordination, partnerships, and inclusion. Please describe which stakeholders were engaged in designing, implementing and evaluating the initiative and how this engagement took place. Key support was received from 1. The announcement that “AI Chest 4All (DMS-TU) For Thai People” would be offered free of charge by the Director General of the Department of Medical Services (DMS), Ministry of Public Health. 2. The Faculty of Engineering at Thammasat University, which supported and maintained the database and also solved technological problems. A key strength of this program was its ability to foster collaboration and coordination across various sectors of the government and the private sector. Thailand’s public healthcare delivery system is recognized globally as an efficient model for universal health coverage. This program has built on this strength. Being flexible and innovative in a large public healthcare delivery network is difficult, but this program leveraged relationships to utilize the strengths of each stakeholder. These outcomes will contribute to SDG Targets 3.3 and 3.4 and are in line with the goals of Thailand’s Ministry of Public Health for “healthy people, happy staff, and a sustainable health system.”

12. Lessons Learned

Please describe the key lessons learned, and how your organization plans to improve the initiative A key takeaway was the importance of having a clear goal. Long–term planning by leadership is very important. It was key to have a system–wide view to recognize the need for increased capacity of health services and develop a solution that could be widely adopted. Another key takeaway was the challenges encountered with trying to integrate multiple health information systems. Data management was a major hurdle early on. The process initially required workers to transmit data across departments and systems. The solution included a cooperation agreement signed with the private sector to create a unified image database and transmission system (PACS), which greatly improved the efficiency and expandability of the program. Next Steps (2021–2023) 1.Expanding the number of hospitals integrated with PACS and increasing the number of mobile X–ray vehicles to increase the throughput of the system and broaden access to the technology. 2.Connecting the system to include telemedicine consultations, which will lessen the burden on consumers and bring more equitable healthcare quality. 3.Developing a large database on a cloud platform that can be used for problem analysis and proper strategic policy planning. It also supports health research and other innovations to respond to disruptive changes.

“AI Chest 4 All (DMS–TU) for Thai people” is the precious treasure of Thai people