HTAi 2022 Annual Meeting Oral Presentation Abstract Book

["Oral Presenta\ufffdon Number: OP57 Abstract Title: The Identification Of Technological Innovations To Address The Challenge Of Antimicrobial Resistance Using Horizon Scanning Approaches Author Names: Lucy Barrass ([email protected]), Abigail Roberts, Oluwatomi Arisa, Emily Robertson, Aoife Oliver, Oshin Sharma, Janet Kinnersley, Dawn Craig, Anne Oyewole Introduction Inappropriate prescribing of antibiotics is a significant driver of antimicrobial resistance (AMR) which is a global health challenge. Technological innovations present an opportunity to reduce demand for antimicrobials through infection prevention, detection, and management. The National Institute for Health and Care Research (NIHR) Innovation Observatory (IO) has developed horizon scanning methods to identify promising innovations (devices\/diagnostics\/digital) and anticipate technological trends. Together these insights build a comprehensive landscape and presents a significant opportunity for decision-makers and HTAs to consider the clinical, financial, infrastructural, and logistical provisions to improve preparedness for the potential adoption of these future innovations. Methods The IO developed a detailed dataset of technologies by formulating search strategies for AMR, based on a comprehensive list of terms and input from expert panels. Primary and secondary sources were systematically scanned using a combination of traditional scanning methods, automated and novel artificial intelligence (AI)\/machine learning techniques. Sources included clinical trial registries, MedTech news, academic sources, funding agencies, commercial sites, and regulatory authorities. Results Our global dataset identified over 3000 innovative preventative, detection, and monitoring technologies mapped across AMR clinical pathways (including sepsis, respiratory tract infections). Development activity largely concentrated in the United States of America and United Kingdom. Emerging trends included the application of novel materials to prevent infections (e.g., catheter coatings) and novel analytical techniques (e.g., biosensors, microfluidics, breath analysis) to support optimal patient treatment. Data analysis revealed a high proportion of technologies were diagnostic innovations addressing unmet needs such as rapid and accurate detection (including drug-resistant infections). Conclusions The rapid development and application of technological interventions presents an opportunity to strengthen national AMR strategies worldwide, through the adoption of new innovations. Improvements in exiting technologies, along with technological advancements have the potential to support appropriate prescribing of antimicrobials and thus address the rise in AMR.","Oral Presentation Number: OP59 Abstract Title: The Tunisian Guidelines For Pharmacoeconomic Analysis: What We Need To Know Author Names: Nabil Harzallah ([email protected]), Jaafar Chemli, Hela Grati, Marie Jebali, Mouna Jameleddine, Chokri Hamouda Introduction As a new milestone in health technology assessment (HTA) implementation in Tunisia, L\u2019Instance Nationale de l'Evaluation et de l'Accr\u00e9ditation en Sant\u00e9 (INEAS)\u2014the Tunisian HTA body\u2014published a set of methodological guidelines to support HTA dossier submission by the pharmaceutical industry. Including, \u2018guide for submitting clinical data for an HTA at INEAS\u2019, 'methodological choices guide for pharmacoeconomic analysis at INEAS\u2019, and \u2018methodological choices guide for budget impact analysis at INEAS\u2019. We aim to report the major methodological recommendations of the pharmacoeconomic analysis guideline. Methods The \u2018methodological choices for pharmacoeconomic analysis at INEAS\u2019 guideline was reviewed and the major recommendations were retrieved and reported. Results The reference analysis required by INEAS is the cost-utility analysis systematically combined with a cost- effectiveness analysis (cost per life-year gained) from the public payers\u2019 perspective. The choice of any other type of analysis must be duly justified. Comparators should include alternative treatments which are considered to be \u2018the standard of care\u2019 (i.e., interventions routinely used in Tunisia for the same indication) and in which public resources are invested. The time horizon should be sufficiently long to reflect all differences in costs and outcomes. Additionally, a discount rate of 5 percent per year is recommended. The best available evidence for efficacy, safety and quality of life is required. An indirect measure of patient preference, through a validated measurement instrument is preferred for utility calculation. Cost inputs should be identified from Tunisian sources. Health resource utilization should reflect the care pathway in Tunisia. INEAS favors the use of a recognized model. Uncertainty and impact of the input parameters on the results should be assessed and reported through probabilistic and deterministic sensitivity analyses. Model validation tests to assess face validity and internal validity should be performed, and a discussion of the methods used provided. Demonstration of external validity is required. Results should be presented in incremental cost-utility and cost-effectiveness ratios. Conclusions The recommenda\ufffdons of \u2018methodological choices for pharmacoeconomic evalua\ufffdon at INEAS\u2019 is an important step to facilitate and harmonize pharmaceu\ufffdcal companies\u2019 submissions and to enhance the use of these analyses in decision-making.","Oral Presentation Number: OP60 Abstract Title: Methodological Guidance And Doctrine Of The French National Authority For Health For Economic Evaluation Author Names: V\u00e9ronique Raimond ([email protected]), Commission for Economic and Public Health Evaluation Introduction The French National Authority for Health (HAS) \\\"defines and issues guidelines and medico-economic opinions on prevention, healthcare, prescription, and best care strategies, and contributes to their comparison or ranking to support public health and optimize health insurance spending.\\\" Based on a decade of producing cost-effectiveness evaluations, the Economic Evaluation and Public Health Committee (CEESP) issued two documents to frame its activity related to the economic evaluation of health products: (i) the new guidance highlights the expectations of the CEESP regarding cost-effectiveness evaluations; (ii) the doctrine elucidates the grading of methodological reservations expressed during the technical appraisal of manufacturers' submissions, the CEESP's statements regarding its findings, and the key messages it wishes to convey to public decision-makers, especially to negotiate healthcare product prices. Methods We aim at sharing the content of these documents and describing the willingness of the CEESP to support decision-makers in implementing evidence-based pricing policies. Results The new guidance provided an opportunity for HAS to stress the importance of interpreting the evaluations, which are often perceived as highly technical. In this perspective, several guidelines call for more reasoned reflection on the objectives of the evaluation upon its conception, along with a constant effort to justify the methodological choices made and an extensive interpretation of the results produced. The doctrine highlights two steps taken by the CEESP, mainly built on analyzing the cost-effectiveness evaluation's uncertainty. First, the ability to characterize the level of the ICER in a context where no thresholds for willingness-to-pay exist in France; second, the suggestion of specific regulation schemes to increase the cost-effectiveness of the products. Conclusions The CEESP developed the new guidance and its doctrine as conditions to ensure the usefulness of the economic evaluation for decision-making.","Oral Presentation Number: OP62 Abstract Title: Patients\u2019 Opinion On Health Technology Assessment Reports: An Analysis Of Brazilian Health Technology Assessment In 2021 Author Names: Denis Satoshi Komoda ([email protected]), Carlos Roberto Correa, Marilia Berlofa Visacri, Daniela Santos, Flavia Maia Introduction The Brazilian health technology assessment (HTA) process includes calls for public consultations, in which society can give its opinion on reports emitted by the National Committee for Health Technology Incorporation (CONITEC). Open and closed queries for public consultation are performed by official formularies and can be accessed online at CONITEC webpage. Queries are divided into two categories of reports: clinical protocols and guidelines, and incorporation\/exclusion demands. Incorporation\/exclusion queries are subdivided in two additional categories: opinion and experience, or technical. In this study we analyze the weight of patients\u2019 participation in opinion and experience queries and their opinion (pro or con) on inclusion\/exclusion of health technologies. Methods Formularies concerning concluded public consultations on health technology incorporation\/exclusion reports were extracted from CONITEC website from 1 January to 26 November 2021. Entries on the opinion and experience formularies included amongst others, a close-ended question about the opinion of participants on health technology incorporation\/exclusion reports (\u201cfavorable\u201d\/\u201cagainst\u201d\/\u201cneither\u201d). In this study, we analyzed patients\u2019 opinion contained within concluded public consultations on incorporation\/exclusion of health technologies. Results A total of 63 health technology incorporation\/exclusion queries were performed in the analyzed period, of which there were only four exclusions. A total of 32,209 contributions were registered. \u201cPatients\u201d, \u201cHealth professionals\u201d, \u201cFamily or caregivers\u201d, \u201cInterest on the theme\u201d, accounted for 99.4 percent (13.5, 16.7, 32.3, 36.7%, respectively). Patient participation accounted for 4,367 (13.5%) entries. The total number of opinions in favor of the presented documents by the \u201cPatients\u201d was 4,268 (97.7%), 59 (1.4%) disagreed and 40 (0.9%) had no opinion. Conclusions Public consultation of official HTA reports is a very useful tool to legitimize decisions through social participation. Although patient participation is not numerically the most important category to contribute on public consultation queries, patients are, if not the most influential stakeholder, the main recipient of decisions concerning health technologies incorporations. Further analyzes shall investigate experience narratives included in public consultation queries.","Oral Presentation Number: OP63 Abstract Title: Patient and Public Perspectives On The Scottish Medicines Consortium Detailed Advice Document Author Names: Miranda Pierre ([email protected]), Jennifer Dickson, Lindsay Lockhart, Jackie McCormack Introduction The Scottish Medicines Consortium (SMC) conducts early health technology assessment (HTA) of new medicines, the primary output of which is a document referred to as the Detailed Advice Document (DAD). This comprises an overview of all data considered on the clinical and cost-effectiveness of the medicine, as well as the input from patient groups (PGs), patients, and carers. In 2020, SMC commenced a stakeholder evaluation of the DAD including a workshop with PGs and public partners (members of the public who volunteer with SMC) to explore the potential for using the DAD more widely. Methods PGs and public partners, all having significant experience of engaging with SMC, participated in the workshop. Feedback was gathered using virtual post-it notes, collated and analysed for key themes. We also gathered oral feedback from participants. Sample DADs were distributed for two medicines recently appraised, one of which included a Patient and Clinician Engagement (PACE) meeting. These were chosen because they reflect different aspects of public and patient involvement at SMC, including how this is presented in the DAD. Results Overall, the workshop participants (n=7) recognised the DAD was a useful document for the clinicians who are its primary audience. Its language was perceived to be challenging, including complex information that is not accessible to a wide audience and may only be fully understood by those with a good understanding and knowledge of HTA. This was a key barrier to using the DAD more widely, in particular the health economics information. Suggestions for broadening the audience of the DAD included summaries of key points and an introductory section clarifying the purpose of the DAD and its intended audience, along with signposting to the plain language summary produced by SMC. These will be implemented where possible. Conclusions Improving how SMC communicates decisions to patients and the public, by working in partnership with these stakeholders, will help strengthen public involvement throughout the HTA process.","Oral Presenta\ufffdon Number: OP64 Abstract Title: NICE Listens: Engaging The Public On How To Address Health Inequalities In Health Technology Assessment And Guideline Development Author Names: Katharine Cresswell ([email protected]), Koonal Shah Introduction Involving and engaging the public is an essential step to engender trust and confidence in HTA organizations. In 2021 the National Institute for Health and Care Excellence (NICE) launched NICE Listens, a new programme of deliberative public engagement seeking to address topics that have complex social, moral, or ethical dimensions. Health inequalities (HI), defined as unfair and avoidable differences in health across populations, was the first topic. The aim was to understand how the public would like NICE to act in regard to HI. Despite repeated attempts to tackle HI in England, the gaps in life expectancy between the most and least deprived continue to widen. NICE has committed to addressing HI in its five-year strategy and NICE Listens forms part of a comprehensive engagement strategy to understand how best to do this. Methods Twenty-eight (28) members of the public from across England took part in four iterative two-hour online workshops, held fortnightly in late 2021. They consisted of both plenary and breakout sessions and incorporated a range of stimuli including trade-off exercises and interview clips with HI experts. Results The findings show clear public support for HI being a high priority for NICE, albeit with limits on how and when HI should be addressed. Actions towards reducing HI should focus on supporting a preventative and systemic response. Importantly, there is a need for a transparent process for incorporating HI within NICE guidance as well as rigorous staff training in understanding and addressing HI. Recommending technologies that benefit the majority even when not accessible for all is acceptable if there are clear plans to manage access gaps. Conclusions Reducing health inequalities should be a high priority for NICE and other HTA organizations. Organizations should seek to have clear processes for embedding HI in decision-making. Priority should be given to actions that focus on prevention of ill health and those that have wider system impacts.","Oral Presentation Number: OP65 Abstract Title: An Overview Of Participatory Approaches, Stakeholders, Methods, Topics and Challenges In Medical Device Development: A Scoping Review Author Names: Kas Woudstra ([email protected]), Rob Reuzel, Maroeska Rovers, Marcia Tummers Introduction Stakeholder involvement in medical device development draws much attention. To make well-considered methodological choices while involving stakeholders, it is essential to know what approaches are available and what challenges they bring in practice. Therefore, the aim of this review was to study which participatory approaches are used in the early stages of the lifecycle of medical device development, and to describe the most important characteristics of these approaches. Methods We conducted a scoping review and searched PubMed, Embase and Web of Science for articles published between July 2014 \u2013 July 2019. Papers were included if they presented original research featuring any form of stakeholder participation in the development of medical devices. We used The Spectrum of Public Participation to categorise the approach of each paper. We describe four characteristics of each approach: the stakeholders involved, data-collection methods, topics addressed, and the challenges associated with the approaches as perceived by the researchers. Results From the 14,838 papers from the initial search, 278 were included. All papers could be categorized into three levels of participation: collaboration, involvement, and consultation. The results show that patients and healthcare professionals are most frequently engaged in all approaches, besides stakeholders like citizens, relatives, and experts. The most often used data-collection methods are workshops in the collaboration approach, and interviews in the involvement and consultation approach. Topics addressed in all approaches are: the initial problem, requirements of devices, design choices, testing of devices, and procedural aspects of the involvement. Challenges in the approaches are related to sampling, analysis, social dynamics, feasibility, and closure. Conclusions This review shows that despite the abundance of methods mentioned in literature, there are three main approaches to involving stakeholders in device development: collaboration, involvement, and consultation. These mainly differ in the degree of power that is granted to stakeholders, but are comparable in terms of data-collection methods, stakeholders, topics, and challenges.","Oral Presentation Number: OP67 Abstract Title: Considerations Of Treatment Novelty In Health Technology Assessment Author Names: Patricia Synnott ([email protected]), Daniel Enright, Dominic Voehler, Stacey Kowal, Daniel Ollendorf Introduction A recent proliferation of value frameworks, as well as the emergence of innovative approaches to treating disease (e.g., cell\/gene therapies) have been accompanied by an increased focus on nontraditional elements of value. We sought to understand whether and how health technology assessment (HTA) agencies consider novel aspects of treatment in value assessments. Methods We defined treatment novelty as follows: (i) a new mechanism of action or administration; (ii) addresses an unmet need; or (iii) confers a distinct benefit that transforms clinical practice or that is difficult to quantify. We reviewed technical guidance and peer-reviewed literature to investigate how organizations in eight countries (Australia, Canada, England, France, Norway, the Netherlands, Sweden, and the United States) consider aspects of this definition. Results All (n = 8) organizations give special consideration to interventions that address an unmet need, particularly in cancer, rare diseases, and other severe conditions. Nearly all (n = 5) organizations consider whether an intervention produces benefits that may not be adequately quantified. Organizations in England, Norway, and France sometimes recommend drugs with less favorable cost-effectiveness estimates than traditionally considered if the drug addresses rare or severe conditions, or if its quality-of-life benefit is thought to be inadequately quantified. The Institute for Clinical and Economic Review in the United States models cost- effectiveness in rare diseases using both a modified societal and health care system perspective. Importantly, the benefits of novel treatments are frequently considered uncertain, particularly treatments with a new mechanism of action. When uncertainty is high, organizations in Canada, England, France, the Netherlands, and Sweden sometimes issue conditional recommendations until additional evidence is submitted. England and Australia have used risk sharing agreements for drugs determined to be novel but uncertain. Conclusions The most widely considered aspects of treatment novelty in HTA are unmet needs and potential benefits that are not easily measured. The willingness to pay for novel treatments is often greater, despite inherent uncertainties about benefit and cost-effectiveness.","Oral Presentation Number: OP68 Abstract Title: Real-World Experience With The National Institute For Health And Care Excellence\u2019s Willingness To Accept Less Costly, Less Effective Healthcare Technologies Author Names: Suzanne Caverly ([email protected]), Edel Falla, Jennifer Gaultney, Dwayne Boyers, Elisabet Jacobsen Introduction Research suggests a different willingness to pay for more effective interventions than willingness to accept (WTA) for less effective interventions which has prompted debate as to whether the threshold in the southwest (SW) quadrant should be kinked to reflect this disparity. Acceptance of less costly, less effective interventions with incremental cost-effectiveness ratios (ICERs) in the SW-quadrant presents an opportunity for resource-constrained healthcare systems by releasing resources for other purposes, which is of particular importance during a pandemic. The National Institute for Health and Care Excellence (NICE)\u2019s methods guide suggests the threshold for decision-making for SW-quadrant interventions be the same as for more expensive, more effective interventions. To assess NICE\u2019s WTA less effective treatments, the objective was to review the outcomes and decision drivers for interventions presenting SW-quadrant ICERs. Methods A review of NICE health technology appraisals (HTAs) containing SW-quadrant ICERs identified from 2015- 2021 was conducted. Appraisal details were extracted and analyzed to identify trends in the WTA and decision drivers. Results The HTA review identified twenty-one submissions containing SW-quadrant ICERs in the base-case\/scenario analysis. 81 percent received a positive recommendation, with ICERs ranging from GBP 30,000-GBP 4.2m (EUR 35,264-EUR 5m) compared to a range of GBP 789-GBP 50,905(EUR 927-EUR 59,837) for negative recommendations. The HTAs covered a wide range of therapeutic areas including psoriasis, multiple sclerosis and multiple oncology indications. Decision drivers identified that may have had a positive influence on final outcomes included a high net monetary benefit, a small QALY difference, clinical unmet need, poor tolerability of existing treatments, and oral administration route. Conclusions The analysis suggests that there is a high rate of acceptance of interventions with ICERs in the SW-quadrant, however, the threshold for acceptance is unclear. The high frequency of HTAs with SW-quadrant ICERs identified in this review indicates the need for further guidance on such interventions in the NICE reference case.","Oral Presentation Number: OP70 Abstract Title: Gaps In The Evaluation Of Clinical Decision Support Software (CDSS): Interviews With Australian Policymakers Author Names: Mah Laka ([email protected]), Drew Carter, Tracy Merlin Introduction Clinical Decision Support Software (CDSS) can improve the quality and safety of care by providing patient- specific diagnostic and treatment recommendations. However, robust evaluation is required to ensure that the recommendations provided are clinically valid, up-to-date, and relevant to a specific clinical context. Most evaluation studies assess CDSS performance from the perspective of end-user requirements. But only occasionally is CDSS subject to stringent pre- and post-market evaluation, making it difficult to determine the safety and quality in practice. This study aimed to assess CDSS evaluation in Australia to identify gaps in evaluation approaches. Methods We conducted 11 semi-structured interviews with different policymakers from committees involved in digital health activities in Australia. Data were thematically analyzed using both theory-based (deductive) and data- driven (inductive) approaches. Results Our findings indicated that evaluating CDSS as a purely technical intervention has overly narrowed the assessment of benefits and risks by inadequately capturing the sociotechnical environment. Existing evaluation methods, adopting a static view of the implemented system, cannot discern the impact of the dynamic clinical environment and rapidly evolving technology on CDSS performance. The timeframe of evaluation studies are also incongruent with fast software upgrade cycles, with clinical practices and software potentially changing by the time evaluation is complete. The regulation of software as a medical device depends on the intended use. CDSS are exempt from regulation because they only \u2018produce advice\u2019; however, this ignores the fact that they can transition to specifying a diagnosis and treatment after a software update. There is no framework for continuous post-market monitoring, and this is especially important when a CDSS algorithm can change and impact on patient management. Conclusions The sociotechnical environment is a significant factor influencing the impact of CDSS on clinical practice, therefore evaluation approaches must acknowledge the dynamic nature of clinical and organizational contexts. Pragmatic and data-driven methodologies are required for CDSS evaluation that acknowledge the evolving landscape of clinical practice and its relationship to technology.","Oral Presentation Number: OP71 Abstract Title: PriTec Tool 2: Adaptation For Selection Of Technologies To Be Assessed For Inclusion Into The Health Care System Author Names: Leonor Varela Lema ([email protected]), Maria Maceira-Rozas, Janet Pu\u00f1al-Riob\u00f3o, Maria Jos\u00e9 Faraldo-Vall\u00e9s Introduction The PriTec Tool is an automatically executable multicriteria web application developed in 2009 by the Galician Health Knowledge Agency (avalia-t; ACIS) for the prioritization of technologies susceptible of post- introduction observation or obsolescence. Currently, the tool has been updated and improved to support the selection process of technologies to be assessed for inclusion into the National Public Health Care Portfolio. The aim of this work is to present the updated version of the tool (PriTec Tool 2) with the new functionalities. Methods The development of the tool was based on a mixed-method approach, comprising a systematic review, followed by a five-step process: (i) development of the preliminary proposal of prioritization criteria and domains; (ii) contextualization and validation of the criteria and domains by a multidisciplinary group of key stakeholders; (iii) assessment of validity, reliability and suitability of criteria; (iv) weighting of domains; and (v) evaluation of applicability, reliability and reproducibility of the tool Results The tool consists of 15 criteria categorized in 5 domains. The web application ranks the technologies through automatic computation of the weighted average of the different criteria and generates a comparative analysis of the individual or working group results. The application allows access to different options: working groups, case studies or technology comparison. It allows for individual prioritizations or managing working groups. When applied to prioritize the Spanish Network of Health Technology Assessment (HTA) Agencies yearly workplan it achieved an intraclass correlation coefficient of 0.71 (95% confidence interval 0.62, 0.88). Conclusions The updated PriTec Tool-2 can be very useful to guide decision-making regarding the assessments that would be mostly needed to ensure health, equity and sustainability. The tool stands out for its simplicity and ease of application. It is acknowledged that the tool could be of great interest to policy makers, HTA bodies and other health decision-makers worldwide.","Oral Presentation Number: OP72 Abstract Title: Software Tools For Systematic Literature Review In Medicine: A Review And Feature Analysis Author Names: Kevin Kallmes ([email protected]), Kathryn Cowie, Nicole Hardy, Karl Holub Introduction Systematic reviews (SRs) are central to evaluating therapies but have high costs in time and money. Many software tools exist to assist with SRs, but most tools do not support the full process, and transparency and replicability of SR depends on performing and presenting evidence according to established best practices. In order to provide a basis for comparing between software tools that support SR, we performed a feature-by- feature comparison of SR tools. Methods We searched for SR tools by reviewing any such tool listed the Systematic Review Toolbox, previous reviews of SR tools, and qualitative Google searching. We included all SR tools that were currently functional, and required no coding and excluded reference managers, desktop applications, and statistical software. The list of features to assess was populated by combining all features assessed in four previous reviews of SR tools; we also added five features (manual addition, screening automation, dual extraction, living review, and public outputs) that were independently noted as best practices or enhancements of transparency\/replicability. Then, two reviewers assigned binary \u2018present\/absent\u2019 assessments to all SR tools with respect to all features, and a third reviewer adjudicated all disagreements. Results Of 53 SR tools found, 29 were excluded, leaving 24 for assessment. Thirty features were assessed across six classes, and the inter-observer agreement was 86 percent. DistillerSR (Evidence Partners; n = 26\/30, 87%), Nested Knowledge (Nested Knowledge; n = 25\/30, 83%), and EPPI-Reviewer Web (EPPI-Centre; n = 24\/30, 80%) support the most features followed by Giotto Compliance (Giotto Compliance; n = 23\/30, 77%), LitStream (ICF; n = 22\/30, 73%), and SRDB.PRO (VTS Software; n = 21\/30, 70%). Seven tools support fewer than half of all features assessed: RobotAnalyst, SyRF, Data Abstraction Assistant, SWIFT-Review, SR- Accelerator, RobotReviewer, and COVID-NMA. Notably, only 10 tools (42%) support direct search, 7 (29%) offer dual extraction, and 13 (54%) offer living\/updatable reviews. Conclusions DistillerSR, EPPI-Reviewer Web, and Nested Knowledge each offer a high density of SR-focused web-based tools. By transparent comparison and discussion regarding SR tool functionality, the medical community can choose among existing software offerings and note the areas of growth needed, most notably in the support of living reviews.","Oral Presentation Number: OP73 Abstract Title: Tools That Can Aid Adaptive HTA To Ensure Rapid, Efficient, And Pragmatic Priority Setting: A Scoping Review Author Names: Lieke Fleur Heupink ([email protected]), Elizabeth Peacocke, Ingvil S\u00e6terdal, Lumbwe Chola, Katrine Fr\u00f8nsdal Introduction Producing new health technology assessments (HTA) can be a time-consuming process. With finite resources in HTA agencies, limited capacities in countries without formalized HTA processes, and growing interest for lifecycles approaches valuing health technologies; innovative and efficient HTA processes are needed. \u201cAdaptive HTA\u201d, referring to the pragmatic use of HTA methods and existing (HTA) evidence, might offer solutions. We will present the results from a scoping review that mapped existing tools, methods, practices to transfer existing HTAs; and reflect on these findings given our own experiences of adaptation processes in LMICs. Methods We undertook a scoping review and systematically searched five electronic databases. Inclusion of articles followed strict in- and exclusion criteria. Data extraction focused on information regarding tools, methods, and practices that could aid the transferability of HTA analysis. Here, HTAs referred to full-HTAs and other HTA products, as partial HTAs, economic evaluations, or systematic reviews. Lastly, we mapped the possible overarching factors that can affect transferability. Results The search (November 2020) identified 2030 hits, of which 19 were included. Most HTA transfers followed five steps that closely resemble a de novo HTA process. The identified transferability tools, often checklists, were merely aids or a \u201ccatalyst\u201d for the transfer and provided limited guidance for the whole transfer process. Contrastingly, we identified three frameworks that can support the whole process: European Network for HTA (EUnetHTA) Adaptation Toolkit, TRANSFER framework for systematic reviews, and paper series on systematic reviews for economic evaluations. Lastly, our findings pointed to various challenges and knowledge gaps; especially for transfers in low and middle income countries evidence is limited. Conclusions The re-use of existing evidence in HTA reports is not new; and readily part of de novo and adaptive processes. The innovative nature of adaptive HTA comes from its ability to unpack the process of adaptation and transferability. Simultaneously, this scoping review highlighted gaps in existing adaptive methods, and could aid future adaptive HTA process for experienced and new HTA-doers.","Oral Presentation Number: OP74 Abstract Title: Assessing Public Confidence Towards COVID-19 Vaccines Through Social Media Insights Leveraged Using Artificial Intelligence Techniques Author Names: Rhiannon Green, Kate Lanyi ([email protected]), Christopher Marshall, Dawn Craig Introduction In areas where public confidence is low and there is a lack of understanding around behaviors, such as COVID-19 vaccine hesitancy, there is a need to explore novel sources of evidence. When leveraged using artificial intelligence (AI) techniques, social media data may offer rich insights into public concerns around vaccination. Currently, sources of \u2018soft-intelligence\u2019 are underutilized by policy makers, health technology assessment (HTA) and other public health research agencies. In this work, we used an AI platform to rapidly detect and analyze key barriers to vaccine uptake from a sample of geo-located tweets. Methods An AI-based tool was deployed using a robust search strategy to capture tweets associated with COVID-19 vaccination, posted from users in London, United Kingdom. The tool\u2019s algorithm automatically clustered tweets based on key topics of discussion and sentiment. Tweets contained within the 12 most populated topics with negative sentiment were extracted. The extracted tweets were mapped to one of six pre- determined themes (safety, mistrust, under-representation, complacency, ineffectiveness, and access) informed using the World Health Organization\u2019s 3Cs vaccine hesitancy model. All collated tweets were anonymized. Results We identified 91,473 tweets posted between 30 November 2020 and 15 August 2021. A sample of 913 tweets were extracted from the twelve negative topic clusters. Of these, 302 tweets were coded to a vaccine hesitancy theme. \u2018Safety\u2019 (29%) and 'mistrust\u2019 (23%) were the most commonly coded themes; the least commonly coded was \u2018under-representation\u2019 (3%). Within the main themes, adverse reactions, inadequate assessment, and rushed development of the vaccines as key findings. Our analysis also revealed widespread sharing of misinformation. Conclusions Using an AI-based text analytics tool, we were able to rapidly assess public confidence in COVID-19 vaccination and identify key barriers to uptake from a corpus of geo-located tweets. Our findings support a growing body of evidence and confidence surrounding the use of AI tools to efficiently analyze early sources of soft-intelligence evidence in public health research.","Oral Presentation Number: OP76 Abstract Title: \u201cThunderbirds Are Go!\u201d Rapid Response HTA Outputs For COVID-19 Author Names: Lirije Hyseni ([email protected]), Elizabeth Islam, Dionne Bowie, Paul Dimmock Introduction The COVID-19 pandemic has highlighted the need for rapid assessment of potential health technologies that can improve health outcomes in COVID-19 patients, as well as helping pressurised health service provision. Medical technologies play a key role in the COVID-19 pandemic, especially diagnostic tests and respiratory technologies. This study evaluates the rapid response work that the medical technology evaluation programme (MTEP) at the National Institute for Health and Care Excellence (NICE) has done in response to the COVID-19 pandemic. Methods Companies routinely submit medical technologies for evaluation by NICE through HealthTech Connect, which is an online portal for devices, diagnostics and digital technologies intended for use in the NHS or wider United Kingdom health and care system. During the COVID-19 pandemic, companies were able to use a designated email address if they perceived their technology may benefit the healthcare system regarding the COVID-19 pandemic. This new system bypassed the usual full registration and data submission. All technologies were reviewed that were submitted via HealthTech connect and email between March 2020 and June 2021. Results During this period, 20 technologies were submitted to MTEP. Most of these technologies were submitted via email. These technologies consisted of a mix of digital, diagnostic, and respiratory technologies. Seven technologies were selected for a rapid COVID-19 MedTech innovation briefing (MIB), with one specifically addressing issues around waiting lists because of knock-on effects of COVID-19 restricting normal clinical work. A further six technologies were not selected because of limited evidence, while one was not selected because it was not perceived as innovative. The other five technologies were progressed as normal MIBs as there was not enough evidence of potential benefits related to COVID-19 to expedite to a rapid COVID-19 MIB. In total, two technologies were selected for medical technology guidance (myCOPD and Anaconda) and are currently in development. Conclusions MTEP has responded to the COVID-19 pandemic by prioritising and producing rapid COVID-19 MIBs on technologies to improve health and social care.","Oral Presentation Number: OP78 Abstract Title: Taking A Societal Perspective In Health Technology Assessment: Is Environmental Impact A Special Case? Author Names: Juliet Kenny ([email protected]), Koonal Shah Introduction A source of debate among the health technology assessment (HTA) community is what perspective should be taken in health economic evaluations. Many stakeholders advocate that a societal perspective is taken in order to include a comprehensive range of costs and outcomes and (in theory) make societally optimal decisions. The Second Panel on Cost-Effectiveness in Health and Medicine recommended that a societal perspective be presented alongside a health sector one. The Second Panel included environment as one item on its impact inventory\u2014alongside productivity, education, and others\u2014intended to support the use of a societal perspective. However, many HTA agencies, including the National Institute for Health and Care Excellence (NICE), have continued to use health sector-specific evaluations to inform decision-making. The presentation seeks to examine whether consideration of the environmental impact of healthcare requires\/implies the formal adoption of a societal perspective in health economic analyses. Methods The presentation will provide an overview of the societal perspective, explaining how it differs from a health sector perspective and describing its main strengths and weaknesses. We then present policy analysis undertaken by NICE\u2019s Science Policy and Research team to identify reasons for measuring environmental impact in HTAs and examine whether these align with the broader arguments for or against adopting a societal perspective in economic analyses. Results Three reasons for considering environmental impact are iden\ufffd\ufb01ed: (i) to support parallel policies which demand healthcare system transforma\ufffdon against emissions targets; (ii) to ensure planetary and human health, in the future as well as the present; and (iii) to o\ufb00set future healthcare resource use. We show that only the third reason aligns with arguments related to the choice of perspec\ufffdve for economic analyses. Moreover, this reason is arguably beter aligned to maintaining a (poten\ufffdally modi\ufb01ed) health sector perspec\ufffdve. The implica\ufffdons of the results will be discussed with reference to upda\ufffdng reimbursement decision-making frameworks, such as those used by NICE, to account for the environmental consequences of healthcare.","Oral Presentation Number: OP79 Abstract Title: Incorporating Environmental Impacts Into Health Technology Assessment: An Examination Of Potential Approaches And Challenges Author Names: Juliet Kenny ([email protected]), Koonal Shah, Michael Toolan, Nick Crabb, Felix Greaves, P\u00e1ll J\u00f3nsson, Judith Richardson, Sarah Walpole Introduction In light of government and healthcare system commitments to reducing the carbon footprint of healthcare, health technology assessment (HTA) agencies are increasingly motivated to investigate how to consider environmental sustainability in their assessments and guidance. This constitutes a major departure from the existing remits and objectives of most agencies, which typically focus on improving population health outcomes. This presentation seeks to identify options for incorporating environmental impact data into HTA and to examine the main challenges, focusing on the National Institute for Health and Care Excellence (NICE) as a case study. Methods We present four broad approaches that could be pursued, informed by policy analysis undertaken by NICE. The strengths, weaknesses and implications of each approach are assessed. Results The first option is to act as an \u2018information conduit\u2019, aggregating and distributing in a standardised format environmental impact information that is provided voluntarily by health technology manufacturers. The second is to present complementary analyses of environmental impact data, separately but alongside results from established health economic analyses (\u2018parallel evaluation\u2019 model). The third is to incorporate environmental impact data into health economic analyses, for example by monetizing environmental outcomes, so that quantitative estimates of treatment value are directly affected by environmental benefits and costs (\u2018integrated evaluation\u2019 model). The fourth is to create new decision-making frameworks for evaluating healthcare interventions that are not expected to improve health-related outcomes, but claim to have relative environmental benefits. Conclusions We conclude that these approaches are not mutually exclusive, and all involve some degree of benefit and risk. We explain why the parallel evaluation model may be the most appropriate approach for NICE as a first response to the increased demand for guidance on the environmental impact of health technologies. We also outline activities being undertaken by NICE and other agencies such as the Canadian Agency for Drugs and Technologies in Health to develop new methodologies for incorporating environmental impact data into their HTAs.","Oral Presentation Number: OP80 Abstract Title: \u2018Green Metrics\u2019 - Incorporating Environmental Dimensions In Health Technology Assessment Author Names: Tjerk Jan Schuitmaker-Warnaar ([email protected]), Sevgi Fruytier, Callum Gunn Introduction Climate change as the result of human action and the risks it poses to human health are well established. In healthcare there is increasing attention to climate and environmental impacts of the use of medical and health technologies. As part of a lifecycle approach, health technology assessment (HTA) needs to take climate and environmental impacts into account. In 2020, the new definition of HTA added the dimension 'environmental aspects', with which the value of health technologies can be determined and assessed in terms of their impact on the environment. This led several HTA organizations to explore opportunities for including environmental impacts in HTA procedures. It is, however, yet unclear how many researchers and HTA organizations are already working on this sustainability dimension, in what way, with which (international) partners, and what they have achieved as of now. Furthermore, the complex relations between the climate crisis, environmental pollution, health and care are difficult to trace, and methods are scarce. In HTA, there is an increasing need for outcome measures that, in addition to clinical utility, effectiveness, efficiency or satisfaction, also quantify the environmental impact of medical interventions (i.e., green metrics). Methods We report on (i) a scoping of international (research) groups and (HTA) organizations that are working on green metrics; (ii) a literature review into the state of affairs with regard to metrics and methods; and (iii) an impact analysis of possible future inclusion of green metrics in HTA procedures. We supplemented a review of (grey) literature with interviews with HTA organizations pioneering with green metrics, and we have conducted a review of available scientific literature, yielding examples of incorporation of environmental aspects into HTA and reports on practical implications. Results Carbon dioxide emissions and pollution by the health sector are currently being explored as green metrics. Differences between direct and indirect environmental impacts complicate the evaluation. Conclusions Green metrics should eventually make it possible to assess sustainability in healthcare as part of a lifecycle approach.","Oral Presentation Number: OP81 Abstract Title: Do Sustainable Healthcare Principles Inform Guidance Development? An Exemplary Case Study In Respiratory Care Author Names: Sarah Walpole ([email protected]), Naba Khan, James Smith, Olga Rostkowska, Fiona Miller Introduction At the United Nations Climate Change Conference (COP26) in 2021, over 40 countries made commitments to low carbon, sustainable health care. Respiratory care provides a case study to explore how existing evidenced-based guidance can inform progress towards more sustainable care pathways and technologies. Our aim is to identify whether environmental aspects of health technology assessment (HTA) are referenced in guidance and the extent to which the four principles of sustainable health care (prevention, self-care, streamlining, and low carbon technology) are promoted in guidance. Methods Internet searches enabled identification of current national guidance on management of respiratory diseases in English, French or Polish. Guidances were reviewed to identify references to environmental aspects of HTA and recommendations that align with each of the four sustainable healthcare principles. Results Guidance on respiratory care is produced by varied stakeholders globally. Some principles of sustainable health care are frequently reflected in guidance to improve quality of care, but others are missed where environment sustainability is not considered. Reference to HTAs incorporating environmental impacts is lacking. There is limited engagement with the environmental impacts of inhalers in guidance. Guidance created by clinician groups (e.g., Greener Practice) and research networks (e.g., Centre for Sustainable Health Systems) has responded more quickly to the need to address sustainability concerns compared to guidance produced by national public bodies. Conclusions HTA organisations may need to take a broader perspective, incorporating environmental impacts in assessments. This could have an influential role in enabling evidence-informed guidance and development of sustainable care pathways and technologies. Limitations of our study were lack of evaluation of local guidance due to limited capacity, language restrictions, and subjectivity in assessing whether each sustainable healthcare principle was addressed in guidance. There may be limited transferability of our results to other specialties or settings. Further research on the sustainability impacts and relative merits of different health technologies and care pathways is required to inform HTA and guidance.","Oral Presentation Number: OP82 Abstract Title: Patient Involvement In Health Technology Assessment Author Names: Lyazzat Kulembekova, Lyazzat Kosherbayeva ([email protected]), David Hailey, Adlet Tabarov, Anar Imanbekova Introduction Patients are increasingly involved in the decision-making process for health technology assessment (HTA), but the question of at what stage they can be involved is still controversial. In Kazakhstan, the HTA process began in 2010. Over the past 2 years, implementation of a project to develop a priority-setting tool based on evidence-informed deliberative processes has made it possible to discuss the participation of patients in HTA. We explored the possibilities of participation of patients or a patient-oriented group in the HTA process. Methods Structured interviews were held with eight people with interests in HTA. Two were representatives of universities, two from a patient-oriented group, two experts in HTA, and two physicians. Interviews were held online by mobile phone or Zoom for 25\u201330 minutes. Question structures were formed based on the report \u2018Patient Involvement in Health Technology Assessment in Europe 2010\u2019. Seven stages were considered. Results All participants partially or completely agreed with the involvement of patients at the HTA stages of identification and prioritization. One or two did not agree with their involvement at the HTA assessment, information production, internal and external review, and diffusion and dissemination stages. Challenges for patient involvement in HTA can be related to other commitments for patients and their carers, lack of financial affordability, conflict of interest, and lack of capacity of the HTA agency to involve them. Five participants agreed on challenges for patients to being meaningfully involved in decision-making on health technologies. These included understanding which institution makes the decisions, finding an interlocutor within the decision-making body, and understanding the decision-making process. Other issues were technical and language difficulties, lack of commitment from decision-makers and the legal or policy framework for patient involvement in HTA decision-making. Conclusions Patients can participate in HTA, but the HTA agency must first prepare and agree on the level of patient participation, and develop measures to reduce barriers such as language difficulties, and patient obligations.","Oral Presentation Number: OP83 Abstract Title: Joining Efforts To Improve Patient Involvement In Health Technology Assessment: The Case Of The RedETS Patient Interest Group Author Names: Yolanda Tri\u00f1anes ([email protected]), Eva Reviriego-Rodrigo, Luc\u00eda Prieto- Rem\u00f3n, Blanca Novellas-Arriba, Edurne Gallastegui, M\u00e1ximo Molina-Linde, Matilde Palma-Ruiz, Ana Toledo- Chavarri Introduction Patient involvement (PI) has become a key priority to the Spanish Network of Agencies for Assessing National Health System Technologies and Performance (RedETS). As part of the national strategy to promote PI, an interest group was created in 2017 to share knowledge, develop methodologies and standardise PI processes. The aim of this work is to analyse the main activities of the Patient Interest Group 5 years after its launch and to reflect on possible needs and challenges. Methods Narrative description and an in-depth analysis of the main activities of the Interest Group from 2017 to the present. Results The group is composed of HTA researchers from the 8 regional agencies in Spain and is supported by the Ministry of Health and the RedETS council. It currently has the participation of 26 researchers, organized into different working subgroups. The initial lines of work were the analysis of the situation, the development of procedures, and the initiation of training materials for patients on HTA. At present, the main projects are the development of metrics for evaluating the impact of patient participation, the development of procedural materials to promote methodological process standardisation (e.g., a flowchart with the main process steps, checklists, templates), and the design and piloting of virtual training for patients in HTA. New lines include the analysis of the ethical challenges of PI and the feasibility of setting up an HTA patient registration system and a patient forum to facilitate participation. In addition, the interest group has promoted the exchange of relevant information for PI and the organisation of capacity building activities. Conclusions The RedETS Patient Interest Group is encouraging the development of activities, reflection on collective experiences, and tools that facilitate PI in Spain. Among the main challenges are the need to ensure the quality and applicability of PI and to analyse the views of patients who have actively participated in HTAs.","Oral Presenta\ufffdon Number: OP84 Abstract Title: Cost Consequence Analysis: A Potential Framework To Incorporate Patient Preferences Into Health Technology Assessment And Reimbursement Decisions Author Names: Jennifer Whitty ([email protected]), Kevin Marsh, Eric Low, Koonal Shah, Mendwas Dzingina Introduction Patient preferences (PPs) are an important source of evidence in health technology assessment (HTA). However, a methodological framework to achieve their integration in decision-making is lacking. We aim to investigate the potential role of evaluative frameworks to integrate PP evidence into HTA and decision- making. Methods We undertook a scoping review to identify potential methodological frameworks to consider PP evidence in HTA and evidence of the acceptability of these frameworks for decision-makers. We searched PubMed, Cochrane, and the grey literature to identify relevant studies, reports, or guidance documents. We restricted our search to the use of PP rather than patient experience data and excluded articles solely relating to deliberative approaches. Results Frameworks identified as having the potential to integrate PP evidence included cost-utility analysis, cost- consequence analysis (CCA), the efficiency-frontier approach, and multi-criteria decision analysis. All have been used in various HTA contexts, but not necessarily for inclusion of PP evidence. Distinct benefits and challenges of integrating PP data were identified for each framework. These included the theoretical basis of the frameworks, their ability to consider non-health as well as health outcomes, and their ability to separate outcomes based on PPs from outcomes based on population preferences. There is limited evidence and no consensus on the application of these frameworks to consider PPs in HTA or on their acceptability for decision-makers. However, CCA has the advantage that it is both based on economic decision theory and it leaves patient preferences disaggregated from population preferences in an HTA. Conclusions The frameworks identified in this review offer potential approaches to systematically and transparently integrate PPs into HTA and decision-making. Based on the review findings, we propose a research agenda to explore the potential of CCA in particular. We anticipate that our findings will augment the recommendations of the Innovative Medicines Initiative PREFER project, which are expected to report in 2022.","Oral Presenta\ufffdon Number: OP86 Abstract Title: Chatbot-Based Symptom-Checkers: A Systematic Review Author Names: Reinhard Jeindl ([email protected]), Gregor Goetz Introduction Symptom-checkers are digital health applications (DHA) with diagnostic algorithms. These symptom-checkers claim to improve the diagnostic process and patient guidance. After asking the user to describe the symptoms using a chatbot interface, the symptom-checkers offer a list of potential diagnoses, and\/or give recommendations for appropriate action (self-care, doctor\u2019s visit, or emergency care). Because of the growing number and increasing use of these diagnostic DHA, there is a need to evaluate the evidence. Methods We updated a British evidence synthesis on symptom-checkers from the National Institute for Health Research (NIHR, 2019). For the systematic update search, we selected four databases. The following endpoints were selected: effectiveness, safety, diagnostic accuracy, triage accuracy, organizational and patient-relevant endpoints. For accuracy studies included from the update search, we assessed the risk of bias (RoB) using the quality assessment tool of diagnostic accuracy studies (QUADAS-2). Results The NIHR-report included 27 studies. We added 14 additional studies via update search. One randomized- controlled-trial (RCT) reported a prolonged illness duration when using symptom-checkers (statistically non- significant). No harms when using symptom-checkers were identified (six observational studies). The diagnostic accuracy ranged from 14-84.3 percent (ten observational studies), the triage accuracy ranged from 33-100 percent (eleven observational studies). For organizational endpoints, the results were inconsistent (one RCT, six observational studies). The patient perspective indicates a high usability for symptom-checkers, but the limited description of symptoms and the missing verbal interaction with health personnel were mentioned as hindering factors (nine survey-studies). The QUADAS-2 assessment for RoB was low in one, and high in seven studies. Conclusions The studies were often conducted using fictitious case-vignettes, limiting the validity of the evidence. Therefore, the results for the diagnostic and triage accuracy are insufficient to demonstrate a benefit in real- world settings. Additionally, there is a concern for misdiagnosis and overdiagnosis. We recommend a continuous monitoring of these diagnostic DHA, using high-quality studies.","Oral Presenta\ufffdon Number: OP87 Abstract Title: Value From A Multi-stakeholder Perspective: A Framework To Assess Digital Health Solutions For Improving Chronic Disease Management Author Names: Madeleine Haig ([email protected]), Caitlin Main, Danitza Chavez-Montoya, Panos Kanavos Introduction Innovative digital health technologies (DHTs) may present new aspects of value that are not appropriately accounted for in current health technology assessments. In discovering what value means in the context of DHTs, multi-stakeholder collaboration is essential. Methods A scoping literature review was conducted to identify current value assessment criteria and proposed methodologies across three health systems: United States of America, United Kingdom, and Germany. A Delphi exercise was conducted with stakeholders from the following groups: users, healthcare practitioners, decision-makers, supply-side actors, and influencers. Based on a review of assessment frameworks in the study countries and consultations with experts from each stakeholder group, researchers proposed value constructs in five domains: health inequalities, data rights and governance, technical and security, economic characteristics, clinical characteristics, and user preferences. In Delphi round one, participants commented on the proposed constructs and submitted their own. A thematic analysis identifying key concepts and themes of the participant proposed constructs and comments was used to incorporate this information for round two. Then, participants rated each value construct on an \u2018importance\u2019 Likert scale in two decision contexts: user-facing DHTs and system-facing DHTs. In round three, participants were presented with the consensus judgement for each construct, with the opportunity to change their answer. Value constructs with equal to or greater than 70 percent consensus were included in the final framework. Rounds four and five were, respectively, value judgements on a Likert scale and a presentation of consensus for a therapeutic area to test the final framework. Results Initially 32 value constructs were proposed by researchers, 20 of which were changed or removed based on round one feedback. Additional constructs were added based on participant suggestions resulting in forty- five value constructs in round two. The final framework will be available after round three closes on 20 December 2022. Conclusions The multi-stakeholder Delphi approach ensures that all suggestions and value judgements are weighted equally across stakeholder groups. The resultant value framework can be used to inform policymaking around health technology assessment of DHTs.","Oral Presentation Number: OP88 Abstract Title: Drawing Lines In The Sand: How Do We Define The Scope Of Analysis In HTA And Economic Evaluation? Author Names: Marina Richardson ([email protected]), Fiona Miller, Nick Daneman, Beate Sander Introduction We explore how the scope of analysis is defined in health technology assessment (HTA) and economic evaluation and consider the potential implications of these decisions. Methods The scope of analysis, including decisions about which methods and domains of HTA to include in the assessment, which costs, and health outcomes are most meaningful, and which comparators are the most relevant are typically informed by the needs of the decision-maker. We undertook two systematic scoping reviews to assess: (i) to what extent systems thinking is considered in literature-based technology assessments; and (ii) how the scope of the analysis is defined in economic evaluation using Clostridioides difficile infection as an exemplar. We synthesized the findings from these reviews and offer three key observations for future research and exploration in the field of HTA. Results Our scoping reviews found that the scope of analysis in economic evaluations typically focus on single interventions, often ignoring upstream and downstream interventions. Similarly, published technology assessments have narrowly defined and inconsistent scopes of analysis, with limited consideration of indirect health and non-health impacts. Three key observations for the field of HTA include: (i) economic evaluations focus on the value of single heath interventions. A focus on a single health intervention may simplify the analysis; however, will this siloed decision-making lead to optimal health resource allocation? (ii) published assessments have inconsistently defined scopes of analysis. A decision problem that focuses on the needs of the decision-maker is practical; however, will inconsistencies in perspectives across assessments create unfair conceptualizations of value? (iii) HTA is technology-focused, not patient-focused. A technology- focused HTA system aligns with the technology diffusion process; however, does this move us away from the patient-centered mandate of HTA? Conclusions The dynamic nature of HTA leads to many conceptualizations of value. Considering the potential implications of narrowly defined, inconsistent, and technology-focused scopes of analyses may have consequences on achieving a patient-centered high-quality health system.","Oral Presenta\ufffdon Number: OP89 Abstract Title: The CE-Signal, A New Simplified Health Technology Assessment Method To Determine Whether Interventions Are Cost-Effective Author Names: Atse Huisman, Saskia Knies,Lonneke Timmers, Joost Enzing ([email protected]), Rick Vreman, Terry Vrinzen, Ly Tran Introduction Conducting a cost-effectiveness analysis (CEA) is resource consuming, and therefore the Dutch National Health Care Institute (ZIN) only performs those for interventions with a high budget impact. Sometimes, cost- effectiveness (CE) estimates are clearly below or far above reference values, which makes full cost- effectiveness assessments less vital. The objective of this study was to develop an efficient and simplified method to identify interventions that are clearly cost-(in)effective. Methods The method makes use of headroom analysis. Several HTA experts and other relevant stakeholders have been asked to provide feedback on a preliminary version of the CE signal. Results The method consists of five steps. In the first step (i) the relevant willingness-to-pay threshold is determined. Reference values are used by ZIN for the maximum willingness-to-pay per incremental quality-adjusted life year (QALY), depending on burden of disease. In next step (ii) the health gain that can realistically be obtained with the new treatment is estimated. Hereby the effect of the intervention on the clinical outcomes, quality of life and gained life years is determined to estimate the number of QALYs gained, including uncertainty. Then (iii) the societal cost maximum (SCM) of the new treatment is calculated by multiplying step 2 with step 1. In step four (iv) the incremental treatment costs are estimated looking at both the costs and savings for both treatments options for the average patient. In the final step (v) the incremental treatment costs are compared to the SCM to determine if the intervention is probably cost effective, probably not cost-effective or if a conclusion cannot be drawn. Conclusions This method has proven to be feasible and could be a valuable addition to the current cost-effectiveness assessment toolbox. The CE-signal is being validated against performing a full cost-effectiveness analysis.","Oral Presentation Number: OP90 Abstract Title: Optimizing Health Technology Assessment And Appraisal For Orphan Drug Reimbursement: Experiences And Tools For Improvement Author Names: Alessandra Blonda ([email protected]), Yvonne Denier, Isabelle Huys, Steven Simoens Introduction The very high costs of orphan drugs, together with the uncertainties regarding their (cost-)effectiveness raise questions regarding the efficiency and legitimacy of their health technology assessment (HTA) and appraisal process. The aim of the present, qualitative study was to investigate how experts on the reimbursement of these treatments perceive the HTA and appraisal process in their country. Moreover, it aimed to provide specific conditions and practical recommendations for their improvement. Methods 22 European experts from 19 different countries were included in a qualitative survey and semi-structured interviews. Transcripts were analysed using the qualitative data analysis software Nvivo. A grounded theory approach was adopted to develop a set of well-defined concepts from the cyclic analysis of the empirical data. Results First, analysis of the expert interviews yielded five good practices for an efficient HTA and appraisal of orphan drugs: a high level of transparency, patient involvement, a clear decision-making structure with room for flexibility, mechanisms to minimize bias and an explicit consideration of the opportunity cost. Meanwhile, participants highlighted several barriers to the overall process, such as a lack of trust between the different stakeholders and imbalances in negotiation power. In addition, the results allowed to identify a number of \u2018contextual\u2019 determinants that may undermine the legitimacy of the final decision, such as bias and the perverse effects of the orphan drug legislation. Drawing from the experts\u2019 experiences, a toolkit was developed that includes an extensive number of specific recommendations (and conditions) for decision- makers to improve the legitimacy and efficiency of their HTA and appraisal of orphan drugs. Conclusions Overall, the results showed that decision-makers should focus on limiting the impact of the contextual determinants rather than improving the methods included in the HTA. This will contribute to further legitimize reimbursement choices for orphan drugs towards the wider public.","Oral Presentation Number: OP91 Abstract Title: The Current State Of Disease-Specific Registries For The Monitoring Of Expensive Pharmaceuticals In The Netherlands Author Names: Wim Goettsch ([email protected]), Lonneke Timmers, Dorien Lobbezoo, Marielle Hagen Introduction On a regular basis, new and innovative medicines come to the market; with uncertainties surrounding their exact effect in patients. To address these uncertainties, disease-specific registries are commonly being used. The Dutch National Health Care Institute (Zorginstituut Netherland [ZIN]) started, in collaboration with stakeholders, a national program on the management and coordination of disease-specific registries. The main goal is to improve the quality and consistency of these registries and thereby increase the value of the data in monitoring innovative, expensive medicines. As a starting point for this program, we performed a study on the current state of these registries in the Netherlands. Methods Using an initial list of 114 registries, we sent out questionnaires to 58 disease-specific registries that collected information on medicine use. Thirty registries responded and their registries were used for the analysis. Results Of 30 registries that responded, 15 registries collect information on orphan medicinal products. Most registries are in the field of internal medicine (n = 8), oncology (n = 6), and rheumatology (n = 5). The size of the registries ranged between 250 patients (orphan diseases) and more than 10,000 patients (oncology). Only 13 registries collect information on patient reported outcomes. Data collection is mostly performed manually and standard coding systems such as Systemized Nomenclature of Medicine (SNOMED), Logical Observation Identifiers, Names, and Codes (LOINC), and The International Classification of Diseases 10th Revision (ICD-10) are not routinely used. Finally, our results show that most registries are (partly) dependent on the funding of pharmaceutical companies. Conclusions Our assessment shows varia\ufffdon in the type, goal of, governance, and funding of the disease-speci\ufb01c registries inves\ufffdgated. We believe that these results show the importance of further na\ufffdonal coordina\ufffdon of the disease-speci\ufb01c registries to increase the usability of their data to address the uncertain\ufffdes surrounding these innova\ufffdve, expensive medicines.","Oral Presentation Number: OP92 Abstract Title: Impact Of Real-World Evidence On Health Technology Assessment And National Guidance For Interventional Procedures: A UK Perspective Author Names: Lakshmi Mandava ([email protected]), John Powell, Kevin Harris, Anastasia Chalkidou Introduction National Institute for Health and Care Excellence (NICE) interventional procedures programme (IPP) issues guidance on the safety and efficacy of new interventional procedures (IPS). The IPP considers a range of evidence from randomized controlled trials (RCTs) to case series to make final recommendations. Real-world evidence (RWE) can provide additional information on long-term outcomes and patient population characteristics that are not easily captured by RCTs. This study explores the impact of RWE in complementing RCTs on long-term safety and efficacy for national guidance development. Methods We review the impact of RWE in IPS guidance (IPG) updates and change in guidance recommendations in the last 5 years. This is done by analyzing NICE guidance updates and supporting evidence. A range of RWE was considered in the supporting evidence, for example, registries and clinical audits. Results The addition of RWE evidence to IPP guidance update has led to significant changes in the recommendations. For example, in one recent IPG, standard arrangements recommendation based on short-term RCT evidence was changed to a research recommendation when RWE showed an increase in long-term adverse effects. In another recent IPG update, special arrangements recommendation based on short-term RCT evidence was changed to a research recommendation when RWE reported long-term inferior efficacy and safety for the new intervention compared to current standard of care. A complete overview of results of the last 5 years will also be presented at the meeting. Conclusions These findings indicate that increased availability of RWE in HTA has the potential to impact national guidance recommendations. In addition, it shows how RWE can fill the evidence base gaps created by RCTs. Such data can confirm or contradict the findings of RCTs, or generate questions needing further research, or support disinvestment in non-effective technologies.","Oral Presentation Number: OP93 Abstract Title: Informing Efficient Diagnostic Monitoring Pathways Using Prospective Cohort Data: A Case Study In Neovascular Age-Related Macular Degeneration Author Names: Graham Scotland ([email protected]), Rodolfo Hern\u00e1ndez, Charlotte Kennedy, Katie Banister, Beatriz Goulao, Jonathan Cook, Sobha Sivaprasad, Ruth Hogg, Augusto Azuara-Blanco, Heinrich Heimann, Maria Dimitrova, Richard Gale, Mia Porteous, Craig Ramsay, Usha Chakravarthy Introduction Several diagnostic tests are often adopted into diagnostic pathways for specific indications without strong evidence to support their use. In this context, real-world prospective cohort studies in combination with decision modelling can generate evidence to support decision-making. The Early Detection of neovascular Age-Related Macular Degeneration (EDNA) study was a prospective cohort designed to assess the diagnostic accuracy and cost-effectiveness of several diagnostic monitoring tests used in routine practice for the detection of neovascular age-related macular degeneration (nAMD) in the second eye of patients being treated for unilateral disease. Methods Five-hundred and fifty-two participants with newly diagnosed unilateral nAMD were monitored for up to 3 years in 24 UK eye clinics. The diagnostic monitoring performance of five index tests was compared: self- reported change in visual function, Amsler test, clinic measured change in visual acuity, fundus assessment by clinical examination or colour photography, and spectral-domain optical coherence tomography (SD-OCT). The reference standard was fundus fluorescein angiography (FFA). A patient-level state transition model was used to simulate the onset of nAMD in the second eye, and assess the impact of different tests on the timing of detection and treatment, and associated costs and quality adjusted life years (QALYs) over a 25-year time- horizon. Results One hundred and forty-five (26.3%) patients developed active nAMD in the study eye, of whom 120 had an FFA at detection. SD-OCT had the highest sensitivity (91.7 percent (95% CI: 85.2-95.6) and provided high specificity (87.8% (95% CI: 83.8-90.9)). It generated more QALYs and lower health and personal social care costs compared to all other monitoring tests. The combination of SD-OCT with fundus-examination provided a marginal increase in sensitivity over OCT alone, but the associated incremental cost-effectiveness ratios was >GBP 100,000 per QALY. Conclusions The efficiency of diagnostic pathways for nAMD may be improved by using SD-OCT alone to monitor the second eye of people being treated for unilateral disease. Prospective cohort studies embedded into routine practice offer value for informing decisions surrounding the use of technologies already in routine use.","Oral Presentation Number: OP94 Abstract Title: Online Elicitation Of Personal Utility Functions (OPUF): A New Tool For Eliciting EQ-5D-5L Value Sets On The Societal-, Group-, Subgroup-, And Individual-Level Author Names: Paul Schneider ([email protected]), John Brazier, Nancy Devlin Introduction The \u2018Online elicitation of Personal Utility Functions\u2019 (OPUF) is a new method for valuing health states. It is based on compositional preference elicitation techniques. In contrast to established, decompositional techniques, such as time trade-off or discrete choice experiment (DCE), the OPUF approach does not require hundreds or thousands of respondents, but allows estimating utility functions for small (patient) groups and even on the individual level. The objective of this study was to generate and compare EQ-5D-5L value sets on the societal-, group-, subgroup-, and individual-level. Methods A demonstration version of the EQ-5D-5L OPUF Tool is available at: https:\/\/eq5d5l.me. It broadly consists of three valuation steps: dimension weighting, level rating, and anchoring. Responses were combined on the individual level to construct personal utility functions. Every respondent also completed three conventional DCEs. Preferences were aggregated across individuals to estimate a societal and various group-level preference functions. We then assessed the heterogeneity of preferences between groups using descriptive statistics and k-means cluster analysis. Results A representative sample (n = 1,000) of the United Kingdom (UK) population was recruited through the prolific online platform. On average, it took participants about 7 minutes to complete the survey. Data of 874 respondents were included in the analysis. For each respondent, we constructed a personal EQ-5D-5L value set. The derived utility functions predicted respondents\u2019 choices in DCE with an accuracy of 78 percent. On the societal level, the predicted values for the EQ-5D-5L health states ranged from -0.376 to 1. Health state preference varied greatly between groups. This was largely due to differences in the anchoring, while there was near consensus on the relative importance of the five EQ-5D dimensions between groups. Demographic characteristics explained only a small proportion of the variability. Conclusions Using the OPUF approach, we were not only able to estimate a new EQ-5D-5L value set for the UK, but also to examine the underlying individual preferences in an unprecedented level of detail.","Oral Presentation Number: OP95 Abstract Title: Examining The Feasibility And Acceptability Of Valuing The Lebanese Version Of The SF-6D Using Standard Gamble Author Names: Samer Kharroubi ([email protected]), Yara Beyh, Marwa El Harake, Dalia Dawoud, Donna Rowen, John Brazier Introduction The SF-6D is a preference-based measure of health, derived from the SF-36 for economic evaluation. No value set exists for the SF-6D in Lebanon and other Arabic speaking countries in the Middle East. The aim of this study was to examine the feasibility and acceptability of using the standard gamble (SG) technique to generate preference\u2010based values for the Arabic version of SF\u20106D in a Lebanese population. Methods The SF\u20106D was translated into Arabic using forward and backward translations. Forty\u2010nine states defined by the SF\u20106D were selected using an orthogonal design and grouped into seven sets. A gender\u2010occupation stratified sample of 126 Lebanese adults from the American University of Beirut were recruited to value seven states and the pits (worst) SF-6D health states using SG. Mean and individual level multivariate regression models were fitted to estimate preference weights for all SF\u20106D states. The quality of data and the predictive power of the models were compared with results from the United Kingdom (UK). Results All respondents completed the interviews with 25 percent reporting that the SG task was difficult and 21 percent felt some degree of irritation or boredom. A total of 992 (98% out of 1,008 observations) SG valuations were useable for econometric modeling. There was no significant change in the test\u2013retest values of 21 subjects. The mean absolute errors in the mean and individual level models were 0.036 and 0.050, respectively, both of which were lower than the UK results. The random effects model adequately predicts the SG values, with the worst state having a value of 0.322 compared to 0.271 in the UK. Conclusions This study confirmed that it was feasible and acceptable to generate preference values with the SG method for the Arabic SF\u20106D in a Lebanese population. This would be the first step towards developing SF-6D value set for Lebanon to be used in economic evaluation studies and to support resources allocation decisions.","Oral Presenta\ufffdon Number: Abstract Title: Patient Reported Outcome Measure-Select App Author Names: Elise Quik ([email protected]) Introduction A patient\/person-reported outcome measure (PROM) is a questionnaire that measures patient's health self- assessment (PROs), such as pain\/wellbeing. HTx is a Horizon 2020 project supported by the European Union lasting for 5 years from January 2019. The main aim of HTx is to create a framework for the Next Generation Health Technology Assessment (HTA) to support patient-centred, societally oriented, real-time decision- making on access to and reimbursement for health technologies (HTs) throughout Europe. Individual values, needs, and preferences should be taken into account when appraising HTs and in clinical decision-making. Better guidance is needed about how to develop and select generic and\/or more specific outcome measures that are relevant to patients and fit-for-purpose. Our goal is to provide that guidance via the PROM toolbox and enable users to select PROMs by using this. The HTx project has received funding from the European Union\u2019s Horizon 2020 research and innovation programme under grant agreement N\u00ba 825162. Methods A framework for the assessment of PROMs was discussed with experts, tested for a subset of PROMs (Linnean menu) and resulted in the PROM-cycle. Multiple literature studies were performed on PROMs, their properties, and characteristics. The general population was studied as well as cancer, diabetes, multiple sclerosis, myelodysplastic syndrome, and (long-lasting) COVID-19. This resulted in an Excel database (PROM- overview) made available via a user-friendly web-application helping users to select PROMs: the PROM select app via https:\/\/www.prom-select.eu\/proms. Results Multiple tools were developed and are available in the PROM toolbox starting with the PROM-guide, Literature review on the use of PROMs, PROM-cycle accompanied by the Linnean menu. In addition, PROM- links, The making of the PROM-overview & PROM-select app, PROM-overview, PROM-select app, its usability test report and PROM toolbox presentation can be found on https:\/\/www.zorginzicht.nl\/ondersteuning\/prom-toolbox-summary-in-english#what-is-the-prom-cycle. Conclusions The PROM toolbox guides users and helps them to select PROMs and trustworthy information and finding relevant scientific articles.","Oral Presentation Number: OP97 Abstract Title: Stakeholders\u2019 Involvement In Brazilian Health Technology Assessment Incorporation In 2021: A Statistical Description Of Demands To The Brazilian National Committee For Health Technology Incorporation Author Names: Denis Satoshi Komoda ([email protected]), Carlos Roberto Silveira Correa, Marilia Berlofa Visacri, Daniela Santos Introduction The demands for incorporating or excluding health technologies within the Brazilian Public Health System (SUS) can be requested by different stakeholders, including public administration, pharmaceutical companies, specialists\u2019 and patients\u2019 organizations\/associations. The Brazilian National Committee for Health Technology Incorporation (CONITEC), part of the Ministry of Health, is the responsible organ to evaluate these demands and emit recommendations. The aim of this study is to show an overview of stakeholders according to the type of technology under request. Methods On 26 November 2021, a search was performed at CONITEC website. Health Technology Assessment (HTA) incorporation reports from 1 January to 26 November 2021 were extracted. From these reports, data regarding the demanded technology (e.g., medications, diagnostic tests, etc.), demand (technology incorporation, exclusion, or alteration), pharmacological classification according with the Anatomical Therapeutic Chemical (ATC), and requester\u2019 categorization (e.g., pharmaceutical companies, official administration organs, etc.). Results Preliminary results showed a total of 77 health technologies, in 63 HTA incorporation submissions, of which 87 percent (n = 67) were medications, 9 percent (n = 6) were new medical procedures. Only one medical device, one vaccine and two diagnostic tests were requested. Technology incorporation accounted for 94.8 percent of the demands (73), and 4 demands of exclusion. 71 percent of the requested medications were classified, according to first ATC coding level, within groups L (Antineoplastic and immunomodulating agents), N group (Nervous system), A group (Alimentary tract and metabolism), and R group (Respiratory system), accounting 28.3, 17.9, 8.9 and 8.9 percent, respectively. Regarding stakeholders, the Brazilian Ministry of Health and its associated departments were responsible for 57.1 percent of the demands, while pharmaceutical companies requested 37.6 percent of incorporations. Other requesters included the Federal Justice Department, patients\u2019, and specialists\u2019 organizations, summing up with only four demands. Conclusions These results present the numerical weight of stakeholders in the Brazilian HTA incorporation system, with special attention to Brazilian bureaucracy and pharmaceutical companies. Further analysis regarding association between demand and other variables such as budgetary impact, costs, and ICD-10, shall deepen our understanding of different stakeholders\u2019 role in Brazilian HTAi.","Oral Presentation Number: OP99 Abstract Title: Participatory Approaches Used In Research And Development Of Medical Devices: A Comparison Of Focus groups, Interviews And Surveys Author Names: Kas Woudstra ([email protected]), Marcia Tummers, Catharina Klijn, Lotte Sondag, Floris Schreuder, Rob Reuzel, Maroeska Rovers Introduction Multi-stakeholder engagement in the development and evaluation of medical devices is crucial for aligning devices with stakeholders\u2019 views, needs, and values. Focus groups, interviews, and surveys are often used to involve stakeholders, but these methods have rarely been compared to analyse their relative merits. Therefore, we systematically compared these three methods in terms of themes, interaction, and feasibility. Methods The methods were compared in a case-study on surgery with a new endoscopic device for patients with intracerebral haemorrhage. We asked patients, relatives, healthcare professionals and decision-makers about their perspective on this device, and about their perceived quality of hospital care. We conducted the focus groups and interviews in one explorative and one interactive round. The comparison was made in terms of number and content of themes, who and how participants interact, and in terms of hours that needed to be worked by researchers to apply a method. Results We enrolled 18 participants in the focus groups, 17 in the interviews and 43 in the survey. Focus groups generated 31 and 19 themes, and interviews 58 and 40 themes in the explorative and interactive round. Surveys generated 42 themes. Interviews produced various themes about the device that did not occur in the other methods. In the two rounds of the focus group, 13 and 42 percent of the interactions were directly between participants. In interview round one, 98 percent of the interactions were between the interviewer and participant, whereas 80 percent of the interactions in round two were discussions of other participants\u2019 opinions. In focus groups participants were inclined to emphasise agreement, whereas the interviews generated more in-depth discussions. Interviews took three times as many hours as the focus groups and survey. Conclusions Methods for multi-stakeholder involvement in device development vary considerably. These methodological differences should be taken into account when selecting a method for engaging stakeholders in the early stages of the lifecycle of a medical device.","Oral Presentation Number: OP103 Abstract Title: Enhancing Legitimacy And Coherent Value Appraisal Across Interventions In Healthcare And Social Services: Strategy Of The Qu\u00e9bec Agency Author Names: Mireille Goetghebeur ([email protected]), Monika Wagner, Isabelle Ganache, Olivier Demers-Payette, Mich\u00e8le De Guise Introduction The Ins\ufffdtut na\ufffdonal d'excellence en sant\u00e9 et en services sociaux (INESSS) makes recommenda\ufffdons regarding introduc\ufffdon, coverage, op\ufffdmal use or withdrawal of interven\ufffdons in physical and mental health and social services to support fair and reasonable decisions. The objec\ufffdves of this work were to develop a statement of principles and ethical founda\ufffdons for a common appraisal framework across diverse interven\ufffdons, to enhance legi\ufffdmacy and coherence of evalua\ufffdon prac\ufffdces, and develop re\ufb02ec\ufffdve approaches throughout the ins\ufffdtu\ufffdon to opera\ufffdonalize the principles. Methods To develop this statement, INESSS reviewed its practices across different units, surveyed the literature on innovative practices and the evolution of HTA, and undertook an extensive internal and external consultative process. The principles are used to develop reflective activities as part of a continuous improvement strategy. Results The adopted approach to value appraisal considers the contributions of interventions to the Triple Aim of health and social services systems as well as their organizational and sociocultural feasibility and impacts (clinical, populational, economic, organizational, sociocultural dimensions). This approach is articulated around five principles including: (i) evaluating the most relevant objects and adapting evaluation modalities; (ii) mobilizing and integrating diverse types of knowledge; (iii) supporting multidimensional deliberation including diverse perspectives; (iv) developing fair and reasonable recommendations; (v) promoting value creation by supporting the implementation of recommendations and re-evaluation. Although all principles contribute to the legitimacy and credibility of the recommendations, which we aim to implement and consolidate through a set of activities, deliberation is an important part of the process that we are striving to improve. A first set of reflective activities are planned to support its operationalization, including: materials to promote a common understanding of the diverse aspects of the deliberation, reflective workshops on selected past projects, and sharing emerging reflections across INESSS units to further continuous improvement in operationalizing the principles. Conclusions Moving forward, INESSS's strategic intention is to mobilize its staff and collaborators to facilitate the rigorous, agile and coherent application of these principles.","Oral Presenta\ufffdon Number: OP105 Abstract Title: Gaps, Challenges And Priorities For Neurodegenerative Disease Research And Health Technology Assessment Author Names: Fatima Salih, Angela Bradshaw, Lewis Killin, Nina Coll, Carlos D\u00edaz, Ana Diaz-Ponce, Dianne Gove, Dalia Dawoud ([email protected]) Introduction There are many challenges encountered in the health technology assessment (HTA) of treatments for the management of neurodegenerative diseases like Alzheimer\u2019s disease. A recent example where such challenges were visible was the assessment of aducanumab, a medication developed for Alzheimer\u2019s disease. As part of the Innovative Medicines Initiative (IMI) Neuronet project, a series of workshops were held to understand the gaps in neurodegenerative disease research and identify the key research priorities, from the perspective of different stakeholders, that can inform decision-making and enable timely access to effective and safe treatments. Methods Three workshops were organised by Neuronet to consult with HTA and regulatory experts, people with dementia and their supporters, and academic and industry researchers. The first consultation was with the European Working Group of People with Dementia and their supporters, and their feedback was fed into the following two workshops which were conducted with HTA and regulatory experts, and with researchers. Discussions during the workshops were used to develop a summary of the gaps in research, and priorities from the perspective of each stakeholder group to be submitted to IMI to inform future research funding allocation. Results Key priorities that were highlighted from an HTA and regulatory perspective were the collection of longitudinal data to understand the natural history of the disease, the generation of data on long-term clinical outcomes and the use of trial endpoints that are relevant to clinical practice. There is also a need for disease-modifying treatments that is yet to be met in this disease area. Furthermore, the involvement of the public, particularly people with dementia and relevant organisations, in setting research priorities and informing study designs was highlighted as a priority by all stakeholders. Conclusions There are key priorities for future neurodegenerative research from an HTA and regulatory perspective to address current gaps and challenges. It is important to involve different stakeholders in priority setting and decision-making to produce relevant and satisfactory research outputs.",""]