["246 Y. Lee et al. 2022) and also increases the self-ef\ufb01cacy (increased familiarity, increased con\ufb01dence, and reduced anxiety) and satisfaction of in-hospital healthcare providers (Chiang et al. 2022). Recently, the use of virtual patients in nursing education was introduced, and it was found that students can improve their learning achievement, self-ef\ufb01cacy, and com- munication skills. Additionally, learning with virtual patients was enjoyable for nursing students (Kabir et al. 2022). Interestingly, students\u2019 satisfaction with virtual education is highly correlated with IT technology environment availability and administrative and educational service quality. It was found that an environment that provides stable online access and active teaching styles was important (Moghadam et al. 2022). 3 Research Method 3.1 Research Design This study adopted an experimental study using a non-equivalent control group and a non-synchronized post-test design to understand the effect of VR applied education on performance ability, con\ufb01dence and practice satisfaction of nursing students (Noh and Kim 2019). Table 1 shows the details of experimental design used for this study. Table 1. Experimental design Target group Pre-survey Apply Post-skill Post-survey assessment experiment performance assessment evaluation Experimental A E1 X1 E2 E3 Group Experimental B E1 X2 E2 E3 Group Experimental C E1 X3 E2 E3 Group X1: Self-learning using realistic practical contexts using HMD. X2: Self-learning using web-based practical contents contexts. X3: Self-learning using video practice contents contexts. E1: Evaluation of con\ufb01dence of performers before arbitration. E2: Evaluation of skills after self-learning by group. E3: Evaluation of con\ufb01dence and practical satisfaction of performers after self-study by group. The experimental group is a group using HMD-based VR practice content, a group using web-based VR practice content, and a group using video practice content. The group using HMD-based VR practice contents (X1) wore goggles with VR practice contents, operated by controller, and wore Oculus QEUST2, which moved in virtual space according to the movement of the controller. The group using web-based VR practice contents accessed links and used laptops to study. Before training, a survey was conducted to asses skill performer\u2019s con\ufb01dence. Learning was conducted freely","Effect of Virtual Reality Based Nursing Skills 247 in different spaces and research assistants were established to check whether learners had completed their learning. After learning experience by each group, four professors majoring in nursing conducted practical tests based on the \u201cperformance items\u201d of the Nursing Education Evaluation Institute, and after the practical test, a post-mortem survey on skill performance con\ufb01dence and practice satisfaction was conducted. 3.2 Research Participants The target population of this study is second-year nursing college students, who have completed basic nursing practice and are about to start their third year of nursing school. The sample is composed of students enrolled in their second-year of nursing school at one University in South Korea. The recruitment announcement stated the purpose of the study as well as the expected time commitment required of participants. After evaluat- ing applicants of the following criteria, researchers recruited 30 eligible participants to participate in the study. Criteria for selecting study participants: (1) Individual who has completed basic nursing practice (2) Has no prior experience learning skills through virtual reality technology (3) Understands the purpose of the research and voluntarily signs the consent form for participation in the research Criteria for excluding study participants (1) A person who withdraws voluntary participation in the study after complaints of dizziness, from wearing googles (2) Any individual who is unable to continue participating in the research due to health reasons while conducting the research; 3.3 Research Tool Skill Performance Ability Skill performance is evaluated using the key skills checklist for each student, presented by the Korea Nursing Education Evaluation Institute (2017) and prior knowledge is not measured. The researcher should arbitrarily present the patient status and prescription for each scenario to the students. Then, students are evaluated based on the knowledge they exhibit through the three areas of testing. Higher scores for each subject translate into higher skill performance ability. Self-con\ufb01dence in Skill The researcher modi\ufb01ed the tool developed by Ko & Kim (2017) to create a skill perfor- mance evaluation. This tool consists of 10 questions, and focuses on the essential items of the manual checklist of intradermal injections and intravenous injections presented by the Korea Nursing Education Evaluation Institute, which explain the purpose of admin- istration, principle of administration, injection site selection, accurate technique, and waste separation. Each question is measured using a Likert scale from 1\u20135, with 1 being","248 Y. Lee et al. \u201cvery low \u201cand 5 being \u201cvery high,\u201d measuring the con\ufb01dence the students have using these techniques. Satisfaction with the Practice In this study, Yoo (2000)\u2019s tool, which measured learning satisfaction in practice using standardized patients, was modi\ufb01ed and used by Kim & Jeong (2021)\u2019s tool, excluding the instructor\u2019s satisfaction item. Each question is measured using a 5-point Likert scale, with 1 being \u201cstrongly disagree\u201d and 5 being \u201cstrongly agree\u201d to indicate satisfaction with the practice. The higher the total score, the higher satisfaction students exhibited with the practice. VR Practice Context Researchers developed virtual reality practice scenarios for student nurses, to give stu- dents the opportunity to utilize interactive practice scenarios and further build their skill set. These contexts include preparation items at each stage of the scenario, recorded educational content for nursing students through voice recognition software, and the ability for students to check their choices and the correct answers during each scenario. Data Analysis Method The collected data were recorded using the IBM SPSS 25.0 program, and speci\ufb01c analysis methods adopted in this study are as follows. First, the participants\u2019 demographic and sociological characteristics, skill perfor- mance ability, skill performance con\ufb01dence, and practice satisfaction are calculated as a frequency, percentage, average and standard deviation through descriptive statistics. Second, the prior homogeneity of demographic and sociological characteristics, skill performance con\ufb01dence, skill performance ability, and practice satisfaction among the experimental groups is veri\ufb01ed. Third, the reliability of the skill performance ability, skill performance con\ufb01dence, and practice satisfaction measurement tool is measured by Cronbach\u2019s \u03b1 coef\ufb01cient. Finally, the ANOVA is used to test the hypotheses by com- paring the effects of each type of learning context. The signi\ufb01cance level used in all tests conducted is .05. (p < .05). 4 Findings 4.1 General Characteristics of Participants. The general characteristics of the participants are as follows. As previously mentioned, participants are 2nd year of nursing college students in South Korea. The majority of participants for this study are female students (90%, n = 27) and few male students (10%, n = 3). Most of students (93%) aged 21 to 22 (93%, n = 28) and small number of students are aged 23 to 25 (7%, n = 2). 4.2 Skill Performance, Con\ufb01dence, and Learning Satisfaction According to the Skill Education Method With regard to intravenous injection skills, the average score of skills performance by group X1 (HMD-based VR) was 21.4 points, 20.6 points for X2 group (Web-based VR),","Effect of Virtual Reality Based Nursing Skills 249 and 20.4 points for X3 group (video). The highest score for intravenous injection skills performance was X1 group which used HMD-based VR practice content. In the case of Intradermal injection skills, the X1 group scored 25.3 points, the X2 group scored 21.8 points, and the X3 group scored 18.6 points, and X1 group achieved highest score among all three groups. Interestingly, the results showed regarding both intravenous injection and Intradermal injection skills, X1 group achieved the highest scores. The difference between the groups with the highest average score and lowest average score was 1.0 points for intravenous injection and 6.7 points for intravenous injection respectively. The ANOVA test was used to investigate the intergroup program application effect (see Table 2). It was found that the difference in con\ufb01dence in performing skills before and after learning by group did not show a statistically signi\ufb01cant difference in intravenous injection (F = .010, p = .456). Interestingly, the X2 group showed the highest difference, and in the case of intravenous injection, there was no statistically signi\ufb01cant difference (F = .149, p = .793) among groups. Although there was no statistically signi\ufb01cant difference in practice satisfaction by group (F = 2.565, p = .096), the X2 group showed the highest score. Table 2. Difference in skills performance and con\ufb01dence before and after training Characteristic Difference in skill performance and con\ufb01dence Training satisfaction before and after training Intradermal injection Intravenous injection M(SD) Fp M(SD) F p X1 .267 \u00b1 1.04 .010 .456 .35 \u00b1 1.01 .149 .793 4.17 \u00b1 .54 2.565 .096 X2 .322 \u00b1 .93 .39 \u00b1 .79 4.34 \u00b1 .44 X3 .277 \u00b1 .88 .18 \u00b1 .91 3.86 \u00b1 .45 4.3 Comparison of HMD-based VR Content Application Group and Web-based VR Content Application Group This study provided autonomous learning using VR contents to improve skill competency of nursing college students and evaluated the effect on skill performance ability, skill con\ufb01dence, and practice satisfaction. Due to the lack of instructors, spatial, and \ufb01nancial limitations have emerged as problems in nursing education following the COVID-19 pandemic. Learning to use using VR content suggests\/ highlights this approach as a positive alternative to conventional teaching and learning methods. The skill performance showed the highest score in the group to which HMD-based VR content was applied, and the difference was greater in intravenous injection than in vitro injection (Kilmon et al. 2010). This highlighted that more VR-based education was needed for complex nursing skills, and it was proven that learning with VR content was effective in complex nursing skills education as intravenous injections induced more questions from patients because it is more complicated than in vitro injections. The difference in con\ufb01dence and","250 Y. Lee et al. practice satisfaction of the performers before and after learning showed higher scores in the group applying VR content than those using general video learning. In the case of nursing skills, the use of interactive practice content, such as selecting preparations and situations, and voice recognition functions, was similar between the two groups. The difference in con\ufb01dence and practice satisfaction of the performers before and after learning showed higher scores in the group applying VR content than those using general video learning. In the case of nursing skills, the use of interactive practice content, such as selecting preparations and situations, and voice recognition functions, was similar between the two groups. In addition, a small number of studies have been conducted on the development and application of education programs using augmented reality or HMD-based VR content in nursing skills education, but there are no studies comparing HMD-based VR content and web-based VR content. While the method of using HMD has the advantage of maximizing user immersion, it has raised some disadvantages, such as dizziness caused by the user\u2019s view screen movement, collision caused by invisible external environment, and the high cost of purchasing HMD (Chen et al. 2020; Zaybak 2018). 5 Conclusion The aim of this study was to examine the effects of using virtual reality to learn core nursing techniques measured via ratings of performance ability, con\ufb01dence, and practice satisfaction. This study provides important empirical evidence on the effects of virtual reality-based nursing skills in nursing education, and it enhances nursing students\u2019 per- formance ability, con\ufb01dence and satisfaction. Furthermore, this study indicated that the appropriate use of virtual reality nursing education content can be considered as open- ing new windows for future research on the use of immersive virtual reality in dynamic clinical situations. This study shows that virtual reality-based nursing skills is an effec- tive teaching and learning method to enhance various positive educational outcomes for nursing students. Specially, web-based VR content can minimize these problems and allow learners to have effective learning experiences without time and space constraints in familiar PC and mobile environments. However, there is a limitation in generalizing the effect by applying a limited number of programs in universities. To supplement this, future research intends to propose a follow-up study. First, no separate knowledge pre- test was performed on the assumption that all students would have similar knowledge. Therefore, in future studies, homogeneity should be secured by con\ufb01rming not only the experience of nursing technology but also the background know-how. Second, it cre- ates a foundation for self-directed learning through the development of integrated VR education content that includes both theory and practice. References Aukstakalnis, S., Blatner, D.: Silicon Mirage: The Art and Science of Virtual Reality. Peachpit Press, Berkeley (1992) Bailey, L., Emory, J.: High-\ufb01delity simulation improves con\ufb01dence in nursing students. Teach. Learn. Nurs. 17, 191\u2013194 (2022)","Effect of Virtual Reality Based Nursing Skills 251 Bang, S.S., Kim, I.O.: Relationship among essentials of fundamental nursing skills performance, stress from work and work capability of new clinical nurses. J. Korean acad. Soc. Nurs. Educ. 20(4), 628\u2013638 (2014) Behmadi, S., Asadi, F., Okhovati, M., Sarabi, R.E.: Virtual reality-based medical education versus lecture-based method in teaching start triage lessons in emergency medical students: virtual reality in medical education. J. Adv. Med. Educ. Prof. 10(1), 48 (2022) Chen, F.Q., et al.: Effectiveness of virtual reality in nursing education: meta-analysis. J. Med. Internet Res. 22(9), e18290 (2020) Chiang, D.H., et al.: Immersive virtual reality (VR) training increases the self-ef\ufb01cacy of in- hospital healthcare providers and patient families regarding tracheostomy-related knowledge and care skills: a prospective pre\u2013post study. Medicine 101(2), e28570 (2022) Fuglsang, S., Bloch, C.W., Selberg, H.: Simulation training and professional self-con\ufb01dence: a large-scale study of third year nursing students. Nurse Educ. Today 108, 105175 (2022) Guerrero, J.G., Ali, S.A.A., Attallah, D.M.: The acquired critical thinking skills, satisfaction, and self con\ufb01dence of nursing students and staff nurses through high-\ufb01delity simulation experience. Clin. Simul. Nurs. 64, 24\u201330 (2022) Kabir, H., et al.: Association between preference and e-learning readiness among the Bangladeshi female nursing students in the COVID-19 pandemic: a cross-sectional study. Bull. Natl. Res. Centre 46(1), 1 (2022). https:\/\/doi.org\/10.1186\/s42269-022-00697-0 Kilmon, C.A., Brown, L., Ghosh, S., Mikitiuk, A.: Immersive virtual reality simulations in nursing education. Nurs. Educ. Perspect. 31(5), 314\u2013317 (2010) Kim, K.S., Jeong, H.C.: Effect of self-directed practice using augmented reality simulation on nursing students\u2019 performance con\ufb01dence ability, and practice satisfaction. J. Korean Soc. Simul. Nurs. 9(1), 57\u201368 (2021) Ko, E., Kim, H.Y.: Effects of simulation-based education combined team-based learning on self- directed learning, communication skills, nursing performance con\ufb01dence and team ef\ufb01cacy in nursing students. J. Korean Acad. Fund. Nurs. 24(1), 39\u201350 (2017) Medley, C.F., Horne, C.: Using simulation technology for undergraduate nursing education. J. Nurs. Educ. 44(1), 31\u201334 (2005) Mesquita, H.C.T., Santana, B.D.S., Magro, M.C.D.S.: Effect of realistic simulation combined to theory on self-con\ufb01dence and satisfaction of nursing professionals. Esc. Anna Nery 23 (2019) Moghadam, Z.E., Saeedi, M., Bahreini, A.: Worldwide Students\u2019 Satisfaction with Virtual Education during the COVID-19 Pandemic: A Systematic Review (2022) Noh, G.O., Kim, D.H.: Effectiveness of a self-directed learning program using blended coaching among nursing students in clinical practice: a quasi-experimental research design. BMC Med. Educ. 19(1), 1\u20138 (2019) Nurse-Clarke, N., Acevedo, B.H., Tesoro, M., Lundy, T., Clifton, M., Mathew, M.: Active learning strategies to enhance student success. J. Nurs. Educ. 61(3), 167 (2022) Oh, M.Y.: A Study on the clinical competence according to experiences of core basic nursing skills, self-con\ufb01dence during clinical practice in nursing students. Unpublished Master\u2019s dissertation, Kongju National University, Kongju (2017) Padilha, J.M., Machado, P.P., Ribeiro, A.L., Ramos, J.L.: Clinical virtual simulation in nursing education. Clin. Simul. Nurs. 15, 13\u201318 (2018) Paik, H.J.: Educational evaluation of competency in nursing skills through open laboratory self- directed practice. J. Korean Acad. Fundam. Nurs. 11(1), 13\u201320 (2004) Rosli, N.A., Choo, T.Y., Idris, D.R.: Impact of preceptorship models for undergraduate nursing students and its implementation: systematic review. Edit. Board 14(1), 111 (2022) Rim, D., Shin, H.: Development and assessment of a multi-user virtual environment nursing simulation program: a mixed methods research Study. Clin. Simul. Nurs. 62, 31\u201341 (2022)","252 Y. Lee et al. Song, S.R. Kim, Y.J.: Effect of a self-evaluation method using video recording on competency in nursing skills, self-directed learning ability, and academic self-ef\ufb01cacy. J. Korean Acad. Fundam. Nurs. 22(4), 416\u2013423 (2015) Verkuyl, M., Hughes, M.: Virtual gaming simulation in nursing education: a mixed-methods study. Clin. Simul. Nurs. 29, 9\u201314 (2019) Zaybak, A.: Comparison of the effectiveness of a virtual simulator with a plastic arm model in teaching intravenous catheter insertion skills. Comput. Inform. Nurs. CIN 36(2), 98\u2013105 (2018) Yoo, M.S.: Development of standardized patient managed instruction for a fundamentals of nursing course, Unpulished doctoral dissertation, Yonsei University, Seoul, Korea (2000)","VR and EEG Combined Self-monitoring Platform of Cognitive Care Bohee Lee(B) and Tae-Heon Lee Research Scientist of Looxid Labs, Seoul, Republic of Korea [email protected] Abstract. VR has the potential as a digital healthcare solution as much as its entertainment function. Continuous monitoring and early diagnosis of cognitive decline is the only way to avoid cognitive deterioration where the risk increases with chronological aging. Backed up by real-time neural signals of EEG, the combination of VR and EEG could be a useful method to manage one\u2019s cognitive status. Here, we provide supportive results to the usability of the all-in-one VR- EEG cognitive care platform in generations including a growing population of older adults. Keywords: Virtual reality \u00b7 Electroencephalography \u00b7 Self-monitoring platform \u00b7 Cognitive decline 1 Introduction Until recently, virtual reality (VR) was considered an entertainment tool, which led to the majority consumption of the young generation. However, the covid-19 pandemic has brought changes in many aspects of our life, including the perspective toward VR (Singh et al. 2020). Beyond the entertaining aspect, the VR metaverse is drawing attention from various industries, including virtual conferences, work-from-home, distance education, and digital therapeutics. Furthermore, the number of older persons worldwide is about 13.5% of the global population in 2021 (Michel et al. 2021). The speed of older popula- tion growth is faster than ever before but the current healthcare system is not ready for healthy aging. As a result of expanding VR technology into the health-related industry, the need for effective and reliable digital health technology applicable to all generations has grown. 2 Literature Review VR has the potential to be a powerful tool in health-related \ufb01elds and medical inter- ventions. Immersion in a realistic virtual scene diverts patients\u2019 attention away from pain and lets them manage their pain while getting injections (A. Li et al. 2011) or burn treatment (Sharar et al. 2008). A virtually reconstructed environment also provides an \u00a9 The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 T. Jung et al. (Eds.): XR 2022, SPBE, pp. 253\u2013262, 2023. https:\/\/doi.org\/10.1007\/978-3-031-25390-4_22","254 B. Lee and T.-H. Lee ecologically valid scenario useful to researchers and patients with mental health prob- lems. For patients with post-traumatic stress disorder and speci\ufb01c phobia, a controllable virtual space is very suitable for treatment (Kothgassner et al. 2019; Maples-Keller et al. 2017). Finally, anatomy study and surgery training became available thanks to the inter- active interface and realistic simulations (Samadbeik et al. 2018; Seymour et al. 2002). However, only a few VR healthcare platforms target older adults and their medical care purposes, despite the growth of the older population. The severity of neurocognitive disorders (NCDs), caused by Alzheimer\u2019s disease, vascular disease, or Parkinson\u2019s disease, is an emerging health issue due to the increase in the older population (American Psychiatric Association 2013; Moreno et al. 2019). It is known that both incidence and prevalence increase with age (Fratiglioni et al. 1999), and NCDs affect the autonomy of life (Li et al. 2017). Nevertheless, there is no cure yet despite its high incidence. The only way to deal with NCDs and maintain a healthy life is to discover one\u2019s cognitive decline early and keep monitoring the cognitive state. Mobile and wearable VR is a useful method for self-monitoring and self-management of cognitive status. The \ufb02exibility of VR content also allows customizing the level of cognitive assessment and training for individuals (Bauer & Andringa 2020). This study aims to show the effectiveness of VR in home-based healthcare services and its scienti\ufb01c reliability in autonomous cognitive care of all ages by using elec- troencephalography (EEG). Thus, we developed simple gami\ufb01ed tasks for cognitive assessment. Also, to support scienti\ufb01c reliability with biofeedback data, we introduced EEG and combined the technology with VR. Head-mounted display (HMD) attachable EEG records a real-time brain activity of users in a non-invasive way, without any help of medical experts by simply wearing an all-in-one headset. Through a survey and test- ing of the VR-EEG platform by users of ages from the 20s to 90s, we demonstrate a feasible self-monitoring and self-managing platform of cognitive care applicable to all generations. 3 Methodology 3.1 Participants A total number of hundred sixty-\ufb01ve adults (120 females, 45 males) in their 20s to 90s participated in the study. Most participants had no history of being diagnosed with neurological diseases, except 29 participants claimed mild cognitive impairment. For further analysis, we regrouped our subjects into four age groups: a group of 20s to 40s, 50s, 60s, and 70 years and older. We recruited various age ranges to see how people adapt to using VR as a self-monitoring tool for one\u2019s cognitive performance. Accordingly, randomly selected participants from representative age groups responded to the usability evaluation questions after the experiment.","VR and EEG Combined Self-monitoring Platform of Cognitive Care 255 3.2 Self-monitoring Platform of Cognitive Performance, LUCY Fig. 1. LUCY hardware setting To investigate the feasibility of VR and EEG in self-monitoring healthcare services, we developed a cognitive care solution named LUCY. LUCY is a complete solution with both the hardware (VR and HMD attachable EEG; Fig. 1) and software (VR game- based cognitive contents; Fig. 2) provided. We used Pico Neo 2 Eye (Pico Interactive, San Francisco, CA, United States) for playing VR cognitive games and recorded brain signals from headset-attached EEG (Looxid Labs, Seoul, Republic of Korea). Game contents were displayed on the VR screen and users played games using haptic controllers. First, participants were asked to use a hardware setup consisting of a frontal six- channel EEG (AF3, AF4, AF7, AF8, Fp1, and Fp2 of the International System 10\u201310, located in the Prefrontal cortex) mounted on a VR headset. The reference electrode was placed on the right earlobe, and the ground electrode was placed beside the reference electrode. Unlike conventional wet-EEG with wires and voluminous ampli\ufb01ers, wireless dry-EEG allowed autonomous healthcare to be truly available. After a few minutes of the instruction session, participants performed tasks designed to evaluate six cognitive functions described as below. 1. Focused attention: based on the Choice Reaction Time (CRT) task (Levinoff et al. 2005), it was designed to assess focused and sustained attention while waiting to respond to an unexpected target. White cubes on the 4 by 4 grid appear on the black screen as stimuli and certain cubes enlighten in blue and yellow. The subject should remember the order and color of highlighted cubes and beam the light with controllers of each matching hand (blue: left hand, yellow: right hand) in the correct order. 2. Selective attention: based on the Tricky ball task (Belchior et al. 2013), it was designed to assess selective attention on moving targets. Among seven dispersed","256 B. Lee and T.-H. Lee Fig. 2. LUCY game-based cognitive contents blue balls, one to three target balls are highlighted in yellow (target increases as the player succeed). Then all balls turn back into the blue and move within the screen to interfere with the subject\u2019s attention on the target ball. 3. Depth perception: the task was designed to assess the correct perception of depth (Armbr\u00fcster et al. 2008). In the three-dimensional virtual space, the subject should estimate the depth of presented balls by selecting them in order from the one near the screen to the farthest one. 4. Working memory (also called \u2018Recall\u2019 in LUCY): based on the Corsi block-tapping task (Kessels et al. 2000), it was designed to encode stimuli, maintain the memory during brief-term and recall correctly. White cubes in the 3 by 3 grid appear on the black screen, then turns three to seven cubes into blue consecutively as the dif\ufb01culty level increases. The subject should correctly remember and select highlighted cubes in the right order. 5. Object location: the task was designed to assess short-term spatial memory (Hamp- stead et al. 2018). The subject \ufb01rst observes the scene with several objects placed here and there. After the scene transition, objects disappear, and the subject is supposed to place the objects in the same location as seen previously. 6. Text reading: a short passage is given to the subject to read out loud. Since the reading \ufb02uency and accuracy were evaluated subjectively by the operator, the task was not incorporated in further analysis. By playing simple games, the players can self-evaluate the current status of one\u2019s cognitive function in terms of scores and response time. Then the behavioral response pro\ufb01les and EEG features were extracted during the tasks and analyzed to evaluate cognitive status in their age groups. 3.3 EEG recording and Preprocessing In the absence of an external stimulus, EEG was recorded in the resting-awake state with the eyes closed to measure background brain activity. Then, task-induced EEG","VR and EEG Combined Self-monitoring Platform of Cognitive Care 257 was recorded while the subjects performed VR games. EEG signals were recorded at a sampling rate of 500 Hz and event-related time stamps were extracted from one single recorded data point. We then pre-processed the EEG recordings as follows: (1) high- pass \ufb01ltering with a cut-off frequency of 0.5 Hz, (2) band-stop \ufb01ltering with a cut-off frequency of 60 Hz, and (3) band-pass \ufb01ltering with cut-off frequencies of 1 Hz and 50 Hz. All \ufb01ltering was implemented using a third-order Butterworth \ufb01lter. Bandwidths range of alpha (8\u201313 Hz) for further analysis was de\ufb01ned based on previous studies (Lee et al. 2022; Markand 1990; Schumacher et al. 2020). 3.4 Statistical Analysis We used the Wilcoxon rank-sum test to determine whether there was a signi\ufb01cant difference between age groups. The signi\ufb01cance of observed values collected during experiments was estimated by considering a P-value of 0.95 as the threshold. 4 Findings 4.1 Usability Evaluation of VR Healthcare Platform in All Ages In the survey of 33 representative samples from their 20s to 70s, we asked four questions of evaluating the general questions and three speci\ufb01c questions on using LUCY as follows: 1. Was the length and frequency of instruction adequate? 2. Was the administration program simple and easy to follow? 3. Were the VR contents simple and easy to play? 4. Was the operator\u2019s instruction comprehensible? 5. How was your experience with LUCY? a. Was LUCY interesting for you? b. Will you continue using LUCY? c. Would you recommend LUCY to others? Representatives answered on a scale of 1 to 5, an average of all seven questions, and three speci\ufb01c LUCY questions were calculated for further analysis. The average score of the overall usability of VR per se was 3.98, which was quite positive based on the 5-point scale. The usability and personal interest in using LUCY scored 4.20, even higher than the overall average (Fig. 3). With previous research from AARP1 that demonstrated increasing interest of older adults in using VR and mobile technology (Nelson Kakulla 2019), we con\ufb01rmed that all ages, including the older gener- ation, are positive towards VR applications in healthcare services and capable of self-care management.","258 B. Lee and T.-H. Lee Fig. 3. VR and LUCY usability evaluation Fig. 4. EEG alpha reactivity (left), VR game score (right-top), and VR game response time (right-bottom) of participants 4.2 Scienti\ufb01c Reliability of Using VR-EEG Combined Platform for Self-cognitive Management EEG data were recorded during the eyes-closed resting state and the VR cognitive tasks. Alpha reactivity was calculated as the relative difference in alpha power over prefrontal electrodes when playing cognitive games to the resting state (Schumacher et al. 2020). Cognitive decline is considered to be associated with chronological age (Legdeur et al. 2018; Salthouse 2009). Figure 4 on the left shows gradually decreasing alpha reactivity from the 20s\u201340s to 50s and 60s. Behavioral responses from six cognitive tasks were analyzed in terms of game score based on 100 points, and the response time was measured based on seconds (Fig. 4). Among six cognitive domains, focused attention, working memory, and depth perception showed signi\ufb01cant differences in the game score between the age groups under the 70s 1 American Association of Retired Persons.","VR and EEG Combined Self-monitoring Platform of Cognitive Care 259 and the group of 70s and older (Wilcoxon rank-sum test, focused attention p = 0.0058; working memory p = 0.00006; depth perception p = 0.0057). Also, a similar trend was observed between the focused attention, object location, and depth perception tasks that as the age group gets older, the response time becomes slower. 5 Discussion The current study demonstrated the effectiveness of VR in home-based healthcare ser- vices and its scienti\ufb01c reliability in autonomous cognitive care of all ages by using VR- attached EEG. As we introduced our newly developed cognitive care platform, LUCY, to the population of all ages, users showed positive reactions and some expectations of using it further. The all-in-one solution of EEG and VR games provided scienti\ufb01cally reliable data re\ufb02ecting the real-time biofeedback signals of users. The results imply that a complete VR solution has the potential of an effective method for all generations to monitor their cognitive status. Until recently, because of its entertainment-focused usage, VR has been consumed mainly by the younger generations in the 10s to 30s who like playing video games. However, as metaverse emerges as a promising replacement for real life in a diverse industry, the VR needed a change in its impression. In this aspect, what we observed from the usability survey in all generations including 50-plus age groups, is important in demonstrating the openness and acceptance of new technology, which implies their need for advanced technology. Accelerated by COVID-19, older adults relied on technology to maintain some sense of normalcy and social connection. Accordingly, a signi\ufb01cant portion of adults 50-plus expresses interest in new technology, especially health-related innovations such as diagnostic tools and daily objects that automatically track health measures (Kakulla 2021). Therefore, the results of this study are signi\ufb01cant that VR was accepted by the elderly as a tool for health care, to whom are the most unwilling to accept new technologies despite their vulnerability to cognitive decline. Scienti\ufb01c evidence is a must factor for supporting the credibility and quality of the home healthcare apparatus. Through gami\ufb01ed cognitive assessment and training contents from LUCY, we con\ufb01rmed that chronological age and reaction time are proportional (Bellis 1933). Many physical related factors, such as muscular and sensory aging can explain slowing response and lower performance (Welford 1984), which in the end lead to cognitive decline in older adults. One step further, LUCY applied neural dynamics which provides more prudent data by completing the results with real-time biofeedback data. Conventional EEG is wired and constructed with wet electrodes which necessarily requires an expert\u2019s help. Wireless and HMD compatible EEG allowed easier access to delicate data for the public and elevated scienti\ufb01c reliability. As we measure frontal lobe brain activity during the resting state and also while playing VR games, alpha reactivity declined as users got older. This corresponds with previous studies that the degree of alpha wave change is small as cognitive function deteriorates (Lee et al. 2022; Schumacher et al. 2020). Finally, we provided the obtained data analysis report in a comprehensible form to users incorporating all age groups. From the results, we con\ufb01rmed that the combination of VR gami\ufb01ed contents and EEG has the potential of good indicators to examine the tendency of cognitive decline.","260 B. Lee and T.-H. Lee Although the usage of VR has inde\ufb01nite potential in many practical domains, it still has some hurdles. Symptoms including eye strain, headache, disorientation, nausea, and vomiting occurred by cybersickness are critical factors that reduce the usability of VR (LaViola 2000). A number of theories have been tried to discover where this comes from, although its associated symptoms remain unsolved. Recent studies attempted several techniques that control the range of \ufb01eld of view and navigation speed (Kim et al. 2008), and camera rotation and jumping speed (Farmani & Teather 2020) to overcome the problem, which is quite encouraging. Another barrier is the digital divide that implies the gap between people who can bene\ufb01t from advanced technology and those who are unable to. The phenomenon is based on multiple factors like socioeconomic status, education, geography, age, ability, and language (Jaeger et al. 2012). Bridging the gap by simplifying the access, tailoring the security, and enabling affordability would lower the barrier of access to VR utilization (Datta et al. 2019), which ultimately enables the realization of genuine digital mental healthcare. 6 Conclusion What we can learn from this study is that 1) the risk of cognitive decline increases with age, and 2) older persons are quite positive about using new technologies. In conclusion, VR can be a new solution for future health care services which applies to all generations. VR has the potential to decrease the dependence on care professionals, which allows users to maintain autonomy. By gathering rich real-time biofeedback data from controlled virtual environments, monitoring of cognitive care is possible regardless of time and place. Although there are remaining limitations, such as cybersickness and the digital divide, what is clear is that the potentiality of VR as a digital health care solution is limitless in the future. References American Psychiatric Association: Diagnostic and Statistical Mental Disorders (Dsm 5). In: American Psychiatric Association (2013) Armbr\u00fcster, C., Wolter, M., Kuhlen, T., Spijkers, W., Fimm, B.: Depth perception in virtual reality: distance estimations in peri- and extrapersonal space. Cycberpsychol. Behav. 11(1), 9\u201315 (2008). https:\/\/doi.org\/10.1089\/cpb.2007.9935 Bauer, A.C.M., Andringa, G.: The potential of immersive virtual reality for cognitive training in elderly. Gerontology 66(6) (2020). https:\/\/doi.org\/10.1159\/000509830 Belchior, P., et al.: Video game training to improve selective visual attention in older adults. Comput. Hum. Behav. 29(4), 1318\u20131324 (2013). https:\/\/doi.org\/10.1016\/j.chb.2013.01.034 Bellis, C.J.: Reaction time and chronological age. Proc. Soc. Exp. Biol. Med. 30(6), 801\u2013803 (1933). https:\/\/doi.org\/10.3181\/00379727-30-6682 Datta, A., Bhatia, V., Noll, J., Dixit, S.: Bridging the digital divide: challenges in opening the digital world to the elderly, poor, and digitally illiterate. IEEE Consumer Electron. Magaz. 8(1), 78\u201381 (2019). https:\/\/doi.org\/10.1109\/MCE.2018.2867985 Farmani, Y., Teather, R.J.: Evaluating discrete viewpoint control to reduce cybersickness in virtual reality. Virtual Reality 24(4), 645\u2013664 (2020). https:\/\/doi.org\/10.1007\/s10055-020-00425-x","VR and EEG Combined Self-monitoring Platform of Cognitive Care 261 Fratiglioni, L., De Ronchi, D., Ag\u00fcero-Torres, H.: Worldwide prevalence and incidence of dementia. Drugs Aging 15(5) (1999). https:\/\/doi.org\/10.2165\/00002512-199915050-00004 Hampstead, B.M., Towler, S., Stringer, A.Y., Sathian, K.: Continuous measurement of object location memory is sensitive to effects of age and mild cognitive impairment and related to medial temporal lobe volume. Alzheimer\u2019s Dementia (Amsterdam, Netherlands) 10, 76\u201385 (2018). https:\/\/doi.org\/10.1016\/j.dadm.2017.10.007 Jaeger, P.T., Bertot, J.C., Thompson, K.M., Katz, S.M., DeCoster, E.J.: The intersection of public policy and public access: digital divides, digital literacy, digital inclusion, and public libraries. Public Lib. Quar. 31(1), 1\u201320 (2012). https:\/\/doi.org\/10.1080\/01616846.2012.654728 Kakulla, B.: 2022 Tech Trends and Adults 50-Plus (2021). https:\/\/doi.org\/10.26419\/res.00493.001 Kessels, R.P., van Zandvoort, M.J., Postma, A., Kappelle, L.J., de Haan, E.H.: The Corsi Block- Tapping Task: standardization and normative data. Appl. Neuropsychol. 7(4), 252\u2013258 (2000). https:\/\/doi.org\/10.1207\/S15324826AN0704_8 Kim, Y.Y., Kim, E.N., Park, M.J., Park, K.S., Ko, H.D., Kim, H.T.: The application of biosignal feedback for reducing cybersickness from exposure to a virtual environment. Pres. Teleoper. Virtual Environ. 17(1), 1\u201316 (2008). https:\/\/doi.org\/10.1162\/pres.17.1.1 Kothgassner, O.D., Goreis, A., Kafka, J.X., Van Eickels, R.L., Plener, P.L., Felnhofer, A.: Virtual reality exposure therapy for posttraumatic stress disorder (PTSD): a meta-analysis. Eur. J. Psychotraumatol. 10(1) (2019). https:\/\/doi.org\/10.1080\/20008198.2019.1654782 LaViola, J.J.: A discussion of cybersickness in virtual environments. SIGCHI Bull. 32(1), 47\u201356 (2000). https:\/\/doi.org\/10.1145\/333329.333344 Lee, B., et al.: Synergy through integration of wearable EEG and virtual reality for mild cognitive impairment and mild dementia screening. IEEE J. Biomed. Health Inform. 1 (2022). https:\/\/ doi.org\/10.1109\/JBHI.2022.3147847 Legdeur, N., Heymans, M.W., Comijs, H.C., Huisman, M., Maier, A.B., Visser, P.J.: Age depen- dency of risk factors for cognitive decline. BMC Geriatrics 18(1) (2018). https:\/\/doi.org\/10. 1186\/s12877-018-0876-2 Levinoff, E.J., Saumier, D., Chertkow, H.: Focused attention de\ufb01cits in patients with Alzheimer\u2019s disease and mild cognitive impairment. Brain Cogn. 57(2), 127\u2013130 (2005). https:\/\/doi.org\/10. 1016\/j.bandc.2004.08.058 Li, A., Monta\u00f1o, Z., Chen, V.J., Gold, J.I.: Virtual reality and pain management: current trends and future directions. Pain Manage. 1(2) (2011). https:\/\/doi.org\/10.2217\/pmt.10.15 Li, G., et al.: cognitive trajectory changes over 20 years before dementia diagnosis: a large cohort study. J. Am. Geriatrics Soc. 65(12) (2017). https:\/\/doi.org\/10.1111\/jgs.15077 Maples-Keller, J.L., Yasinski, C., Manjin, N., Rothbaum, B.O.: Virtual reality-enhanced extinction of phobias and post-traumatic stress. Neurotherapeutics 14(3), 554\u2013563 (2017). https:\/\/doi.org\/ 10.1007\/s13311-017-0534-y Markand, O.N.: Alpha rhythms. J. Clin. Neurophysiol. 7(2) (1990). https:\/\/journals.lww.com\/cli nicalneurophys\/Fulltext\/1990\/04000\/Alpha_Rhythms.3.aspx Michel, J.-P., Leonardi, M., Martin, M., Prina, M.: WHO\u2019s report for the decade of healthy age- ing 2021\u201330 sets the stage for globally comparable data on healthy ageing. Lancet Healthy Longevity 2(3) (2021). https:\/\/doi.org\/10.1016\/s2666-7568(21)00002-7 Moreno, A., Wall, K.J., Thangavelu, K., Craven, L., Ward, E., Dissanayaka, N.N.: A systematic review of the use of virtual reality and its effects on cognition in individuals with neurocognitive disorders. Alzheimer\u2019s Dementia Transl. Res. Clin. Intervent. 5 (2019). https:\/\/doi.org\/10.1016\/ j.trci.2019.09.016 Nelson Kakulla, B.: Gaming Attitudes and Habits of Adults Ages 50-Plus (2019). https:\/\/doi.org\/ 10.26419\/res.00328.001 Salthouse, T.A.: When does age-related cognitive decline begin? Neurobiol. Aging 30(4) (2009). https:\/\/doi.org\/10.1016\/j.neurobiolaging.2008.09.023","262 B. Lee and T.-H. Lee Samadbeik, M., Yaaghobi, D., Bastani, P., Abhari, S., Rezaee, R., Garavand, A.: The applications of virtual reality technology in medical groups teaching. J. Adv. Med. Educ. Profession. 6(3) (2018) Schumacher, J., et al.: EEG alpha reactivity and cholinergic system integrity in Lewy body demen- tia and Alzheimer\u2019s disease. Alzheimer\u2019s Res.Therapy 12(1) (2020). https:\/\/doi.org\/10.1186\/ s13195-020-00613-6 Seymour, N.E., et al.: Virtual reality training improves operating room performance results of a randomized, double-blinded study. Ann. Surg. 236(4) (2002). https:\/\/doi.org\/10.1097\/000 00658-200210000-00008 Sharar, S.R., et al.: Applications of virtual reality for pain management in burn-injured patients. Expert Rev. Neurotherapeuti. 8(11) (2008). https:\/\/doi.org\/10.1586\/14737175.8.11.1667 Singh, R.P., Javaid, M., Kataria, R., Tyagi, M., Haleem, A., Suman, R.: Signi\ufb01cant applications of virtual reality for COVID-19 pandemic. Diabet. Metabol. Synd. Clin. Res. Rev. 14(4) (2020). https:\/\/doi.org\/10.1016\/j.dsx.2020.05.011 Welford, A.T.: Between bodily changes and performance: some possible reasons for slowing with age. Exp. Aging Res. 10(2), 73\u201388 (1984). https:\/\/doi.org\/10.1080\/03610738408258548","Art, Design and Theatre Experiences","Cinematic Virtual Reality: How Emotional Responses Vary Across Movie Genre and Technological Format Mariana Berga Rodrigues1(B), Sandra Maria Correia Loureiro2, and Miguel Sales Dias3 1 FCT Doctoral Grant UI\/BD\/151513\/202, ISCTE \u2013 Instituto Universit\u00e1rio de Lisboa and Business Research Unit (BRU-IUL), Lisboa, Portugal [email protected] 2 ISCTE \u2013 Instituto Universit\u00e1rio de Lisboa and Business Research Unit (BRU-IUL), Lisboa, Portugal [email protected] 3 ISCTE \u2013 Instituto Universit\u00e1rio de Lisboa, ISTAR, Lisboa, Portugal [email protected] Abstract. The current study seeks to explore whether virtual reality (VR) can be more effective that traditional 2D in in\ufb02uencing emotional responses (namely, empathy and sympathy) considering different movie genres. A factorial design 2 (movie genre: animation; documentary) \u00d7 2 (VR; 2D) between-subjects experi- ment was employed. A total of 145 participants were recruited. Findings suggest that sympathy and empathy responses to movies are affected by the technological format used to experience a movie, being higher in VR than in 2D. Regarding sym- pathy, the conducted analysis suggests that the movie genre has a higher impact than the format. Additionally, the present study concludes that the interaction between technological format and movie genre has a signi\ufb01cant impact on WOM. Keywords: Virtual reality \u00b7 Cinematic virtual reality \u00b7 Empathy \u00b7 Sympathy \u00b7 WOM \u00b7 Motion picture 1 Introduction Movies have a narrative potential that allows individuals to understand personal and societal issues. Considering the high-risk investment on movie production, the narrative and marketing efforts are decisive factors to the success in the box of\ufb01ce. As Kerrigan (2018) argues, movies\u2019 importance lay in their ability to tell myths, despite the technol- ogy used to do so. However, even though the narrative is determinant, technology has always played a major role in the motion picture industry since the beginning of cinema (Eliashberg et al. 2006). Virtual reality (VR) represents the construction or use of a virtual world where users feel immersed as if they were part of that environment (Loureiro et al. 2019; 2020, 2022). At the core of a VR immersive experience is the empathy factor (Shin 2018; Rodrigues & \u00a9 The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 T. Jung et al. (Eds.): XR 2022, SPBE, pp. 265\u2013270, 2023. https:\/\/doi.org\/10.1007\/978-3-031-25390-4_23","266 M. B. Rodrigues et al. Loureiro 2022). Empathetic responses are often considered by screenwriters and \ufb01lm critics the ultimate experience a spectator can have from a movie. Therefore, considering the multisensory stimuli the technology can provide, there has been a high investment in VR in the motion picture industry. Yet cinematic virtual reality is still at an early stage and the literature is scarce on how consumers will perceive it. Our study gives a contribution to the literature by exploring how emotional responses vary across technological mediums (2D and VR), considering different movie gen- res (documentary and adventure). Additionally, we seek to understand how emotional responses in\ufb02uence word-of-mouth (WOM), a determinant brand attitude regarding con- sumers consumption decisions of movies (Angelino et al. 2021; Loureiro et al. 2021; Yoon et al. 2017). 2 Literature Review The \ufb01rst forms of cinema were more concerned with the display itself than with the narrative due to the curiosity for the device (Kerrigan 2018). However, as \ufb01lmmak- ers progressed, the attention turned to narrative engagement. The way humans process narratives is similar to how they process and interpret their experiences. Narratives can strengthen emotional connections, stimulate imagination, memories, passion, and contributes to confer sense to the world (Laer et al. 2018; Loureiro 2020). Ross (2020) explains that VR can provide processes that create a more direct expe- rience of the movie to the spectator and can even heighten empathy in the audience. According to Rooij (2019), and regarding the motion picture industry, sympathy requires some sort of supportive emotion for a character and comprehension for the events in the narrative. However, sympathy is a more distanced feeling than empathy. Empathy is a stronger emotional response and, therefore, a key element when examining the emo- tional relationship between the spectator and the \ufb01ctional characters (Escalas & Stern 2003; Loureiro et al. 2020). Different mediums can enhance transportation differently (Anaza et al. 2019). According to Shin and Biocca (2017), VR viewing, and interaction improve users\u2019 engagement and are positively associated with consumers\u2019 experience of empathy. Therefore, it would be relevant to understand how the technology affect con- sumers\u2019 empathy and sympathy comparing with 2D traditional screens. On this basis and considering the existent literature the following hypotheses are proposed: H1: There is an interaction effect between technological format (2D or VR) and movie (documentary and animation) on sympathy. H2: There is an interaction effect between technological format (2D or VR) and movie (documentary and animation) on empathy. The motion picture industry is a risky venue (Eliashberg et al. 2007). The failure of a movie regarding box of\ufb01ce performance can have a high negative \ufb01nancial impact on movie studios, even major ones (Eliashberg et al. 2007). Movie genre is an important determinant of movie consumption decisions by the audience. Movie genre preferences is also positively related to consumer personality and lifestyles (Desai & Basuroy 2005). Further, as literature contends, both positive and negative WOM plays a crucial role in determining moviegoers\u2019 consumption decisions as it allows consumers to evaluate the quality of a movie even before watching it. In fact, WOM can be decisive of a movie\u2019s","Cinematic Virtual Reality: How Emotional Responses Vary Across Movie Genre 267 success, considering that positive WOM can lead to a greater audience watching the movie and, therefore, greater box of\ufb01ce performance (Yoon et al. 2017; Craig et al. 2015). Therefore, considering the expectations but also the limitations and challenges CVR has, the following hypothesis is suggested: H3: There is an interaction effect between technological format (2D or VR) and movie (documentary and animation) on WOM (Fig. 1). Fig. 1. Conceptual model 3 Methodology This study followed a factorial design 2 (movie story: documentary, animation) \u00d7 2 (technology format: 2D, VR) between-subjects experiment. The dependent variables are sympathy, empathy and WOM. We decided to use two existing movies to increase this study\u2019s ecological validity. The movies chosen for this research have VR and 2D counterparts\u2019 available. Two different movie genres were considered: animation and documentary. As research contends (Wehrmann & Barros 2017), a single movie can be labelled with multiple genres. In this case, the two movies selected for this study (Is Anna Ok? and Crow: The Legend) are also short movies that were computer generated. The 2D counterparts are available on YouTube, and the VR version was downloaded to be experienced with the Oculus Rift headset device. The experiment consisted of two groups of participants: the control group (experienced the movies in 2D) and the experimental group (experienced the movies in VR). A total of 145 participants were recruited (78 in the control group). 4 Results The two-way ANOVA was applied to test the hypotheses H1, H2 and H3 through the test of Between-Subjects effect (see Table 1). Regarding H1, we accept the null hypothesis and conclude that the independent variables and their interaction do not have a signi\ufb01cant effect (F(1, 141) = 2.366, p = .126) on sympathy. Thus, H1 is not supported. However - considering the independent Format - there is a statistically signi\ufb01cant difference (p = 0.02), meaning that Format has a signi\ufb01cant effect on the dependent sympathy. The same","268 M. B. Rodrigues et al. occurs with the independent Movie. A similar situation is observed to the interaction effect of technological format and movie genre on empathy (F(1, 141) = 2.656, p < .05) is signi\ufb01cative and so H2 is not supported (only the single effects are signi\ufb01cant). Regarding the variable WOM, the interaction effect of technological format and movie genre is signi\ufb01cant (F(1, 141) = 16.623, p < .05) supporting H3. According to the Partial Eta column, 10.5% of the variance in WOM can be attributed to the interaction between Format and Movie Genre. Table 1. Tests of between-subjects effects for sympathy, empathy and WOM Source Df Mean square F Sig Partial eta square Sympathy Corrected Model 3 6.667 8.305 .000 .150 Format 1 8.242 10.267 .002 .068 Movies 1 10.804 13.459 .000 .087 Format*Movies 1 1.899 2.366 .126 .017 Error 113.187 141 .803 Total 5453.667 145 Corrected Total 133.188 144 Empathy Corrected Model 3 66.757 34.333 .000 .422 Format 1 182.550 93.885 .000 .400 Movies 1 20.561 10.574 .001 .070 Format*Movies 1 5.164 2.656 .105 .018 Error 274.161 141 1.944 Total 3043.444 145 Corrected Total 474.432 144 WOM Corrected Model 3 82.150 43.122 .000 .478 Format 1 180.739 94.874 .000 .402 Movies 1 52.742 27.686 .000 .164 Format*Movies 1 31.668 16.623 .000 .105 Error 268.612 141 1.905 Total 3648.000 145 Corrected Total 515.062 144","Cinematic Virtual Reality: How Emotional Responses Vary Across Movie Genre 269 5 Conclusions Sympathy and empathy responses to movies are affected by the technological format used to experience a movie, being higher in VR than in 2D. However, the effect of the technological format was greater in empathy responses than in sympathy responses. In fact, regarding sympathy, the conducted analysis suggests that the movie story has a higher impact than the format. By the way of contrast, the format in which a movie is presented affects empathy more than the movie story. Thus, VR affects humans\u2019 perceptions and cognitive levels (Shin 2018). Additionally, this study concludes that the interaction between technological format and movie genre has a signi\ufb01cant impact on WOM. Considering both independents individually, it can be stated that even though both movie genre and technological format have a signi\ufb01cant effect on WOM, the variable is more in\ufb02uenced by the technological format than by the movie genre. Moreover, we can also observe that both movies had higher scores for WOM in VR, thus, investing in VR techniques can contribute to positive WOM of a movie. This research contributes to theory by exploring how VR differs from traditional 2D screens regarding constructs that re\ufb02ect consumer behavior aspects (namely, empathy and sympathy). In addition, regarding the managerial implications, although movies are a highly pro\ufb01table industry, it is also a risky venue due to the high costs associated. WOM can be a determinant factor movie consumption. Therefore, studios and movie producers should invest in movies that motivate positive WOM by carefully selecting and preparing the story and gradually prefer the VR over the 2D. The story should \ufb01t with the personality and identity of the target audience. As our \ufb01ndings suggest, the technologic format can also create a difference and VR tends to be more appealing than 2D. References Anaza, N.A., Kemp, E., Briggs, E., Borders, A.L.: Tell me a story: the role of narrative transporta- tion and the C-suite in B2B advertising. Ind. Mark. Manage. 89, 1\u201314 (2019). https:\/\/doi.org\/ 10.1016\/j.indmarman.2019.02.002 Angelino, F., Loureiro, S.M.C., Bilro, R.G.: Exploring tourism students\u2019 engagement through telepresence, pleasantness of the experience and memory: a virtual reality approach. J. Promot. Manag. 28(5), 669\u2013685 (2021). https:\/\/doi.org\/10.1080\/10496491.2021.2009614 Craig, C.S., Greene, W.H., Versaci, A.: E-Word of mouth: early predictor of audience engagement. J. Advert. Res. 55(1), 62\u201372 (2015). https:\/\/doi.org\/10.2501\/JAR-55-1-062-072 Desai, K.K., Basuroy, S.: Interactive in\ufb02uence of genre familiarity, star power, and critics\u2019 reviews in the cultural goods industry: the case of motion pictures. Psychol. Mark. 22(3), 203\u2013233 (2005). https:\/\/doi.org\/10.1002\/mar.20055 Eliashberg, J., Hui, S.K., Zhang, J.: From Story line to box of\ufb01ce: a new approach for green-lighting movie scripts. Manage. Sci. 53(6), 881\u2013893 (2007). https:\/\/doi.org\/10.1287\/mnsc.1060.0668 Eliashberg, J., Elberse, A., Leenders, M.: The motion picture industry: critical issues in practice, current research, and new research directions. Mark. Sci. 25(6), 638\u2013661 (2006). https:\/\/doi. org\/10.1287\/mksc.1050.0177 Escalas, J.E., Stern, B.B.: Sympathy and empathy: emotional responses to advertising dramas. J. Consum. Res. 29, 566\u2013578 (2003). https:\/\/doi.org\/10.1086\/346251","270 M. B. Rodrigues et al. Laer, T., Visconti, L.M., Feiereisen, S.: Need for narrative. J. Mark. Manag. 34(5), 484\u2013496 (2018). https:\/\/doi.org\/10.1016\/j.jbusres.2018.10.053 Loureiro, S.M.C.: Managerial Challenges and Social Impacts of Virtual and Augmented Reality, pp. 1\u2013280. IGI Global, Hershey, PA (2020). https:\/\/doi.org\/10.4018\/978-1-7998-2874-7 Loureiro, S.M.C, Bilro, R.G., Angelino, F.J.A.: Virtual reality and gami\ufb01cation in marketing higher education: a review and research agenda. Spanish J. Mark. (2020). https:\/\/doi.org\/10. 1108\/SJME-01-2020-0013 Loureiro, S.M.C., Correia, C., Guerreiro, J.: Mental imagery, product involvement and presence at virtual reality supermarket. J. Creat. Commun. (published online 27April 2022) (2022). https:\/\/ doi.org\/10.1177\/09732586221086655 Loureiro, S.M.C., Guerreiro, J., Ali, F.: 20 years of research on virtual reality and augmented reality in tourism context: a text-mining approach. Tour. Manage. 77, 104028 (2020). https:\/\/ doi.org\/10.1016\/j.tourman.2019.104028 Loureiro, S.M.C., Guerreiro, J., Eloy, S., Langaro, D., Panchapakesan, P.: Understanding the use of Virtual Reality in marketing: a text mining-based review. J. Bus. Res. 100, 514\u2013530 (2019). https:\/\/doi.org\/10.1016\/j.jbusres.2018.10.055 Loureiro, S.M.C., Guerreiro, J., Japutra, A.: How escapism leads to behavioral intention in a virtual reality store with background music? J. Bus. Res. 134, 288\u2013300 (2021). https:\/\/doi.org\/ 10.1016\/j.jbusres.2021.05.035 Rodrigues, M., Loureiro, S.M.C.: Virtual Reality in the motion picture industry: the relationship among movie coolness, sympathy, empathy, and word-of-mouth. J. Promot. Manag. 28(2), 144\u2013159 (2022). https:\/\/doi.org\/10.1080\/10496491.2021.1987964 Rooij, M.: Carefully constructed yet curiously real: how major american animation studios gen- erate empathy through a shared style of character design. Anim. Interdiscipl. J. 14(3), 191\u2013206 (2019). https:\/\/doi.org\/10.1177\/1746847719875071 Ross, M.: Virtual Reality\u2019s new synesthetic possibilities. Television New Media 21(3), 297\u2013314 (2020). https:\/\/doi.org\/10.1177\/1527476418805240 Shin, D.: Empathy and embodied experience in virtual environment: to what extent can virtual reality stimulate empathy and embodied experience? Comput. Hum. Behav. 78, 64\u201373 (2018). https:\/\/doi.org\/10.1016\/j.chb.2017.09.012 Shin, D., Biocca, F.: Exploring immersive experience in journalism. New Media Soc. 20, 2800\u2013 2823 (2017). https:\/\/doi.org\/10.1177\/1461444817733133 Kerrigan, F.: Movies. Consump. Mark. Cult. 21(5), 503\u2013509 (2018). https:\/\/doi.org\/10.1080\/102 53866.2017.1306207 Yoon, Y., Polpanumas, C., Park, Y.J.: The impact of word of mouth via Twitter on moviegoers\u2019 decisions and \ufb01lm revenues. J. Advert. Res. 57(2), 144\u2013158 (2017). https:\/\/doi.org\/10.2501\/ JAR-2017-022 Wehrmann, J., Barros, R.C.: Movie genre classi\ufb01cation: A multi-label approach based on convo- lutions through time. Appl. Soft Comput. 61, 973\u2013982 (2017). https:\/\/doi.org\/10.1016\/j.asoc. 2017.08.029","On the In\ufb02uence of Virtual Reality Technology (VR) on Animation Art Design Ying Wu(B) Graduate School of Advanced Imaging Science, Multimedia and Film, Chung-Ang University, Seoul, South Korea [email protected] Abstract. A virtual reality animation is a new form of animation after the perfect combination of technology and art. This animation is a fusion of arti\ufb01cial com- puter intelligence, simulation technology, display technology, sensor technology, and other technologies. With the development of science and technology and the advancement of technology, virtual reality technology has gradually become an essential form of technology in the contemporary era. Keywords: Virtual reality \u00b7 Animation design \u00b7 Animation character \u00b7 Animation script \u00b7 Interactive experience 1 Introduction In this new era, virtual animation fully integrates computer multimedia technology with image, sound, animation, video, and other elements, making the creative forms of ani- mation more abundant, and making animation expression more intuitive and ef\ufb01cient (Burdea & Coiffet 2003). With the development of science and technology, interdisci- plinary research has allowed the boundaries of animation design to continue to expand. In the era of media blending and rapid technological development, virtual reality immer- sive content, as an essential direction of future art creation, is being paid more and more attention by society. Since the industry has just started, the output of content works is uneven. Many virtual realities animations work needs to be theoretically combed and new laws discovered. How to give play to the interactive narrative characteristics of virtual reality animation within the scope of existing technology and analyze its nar- rative rules and narrative strategies has become the focus of this article\u2019s research and discussion and is intended to provide theoretical references for future creative practice. As people gradually understand and pay attention to virtual reality animation, its appli- cation and development have become more widespread and enthusiastic. Although this kind of virtual reality technology has not been entirely successfully used in animation creation, the future development direction of animation can be glimpsed through the existing virtual reality content (Deerin 1995). \u00a9 The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 T. Jung et al. (Eds.): XR 2022, SPBE, pp. 271\u2013281, 2023. https:\/\/doi.org\/10.1007\/978-3-031-25390-4_24","272 Y. Wu 2 Literature Review From the literature research on virtual reality animation, the foreign research on virtual reality animation is primarily concentrated in medical and industrial production. In the Google Scholar search platform, there are 2270 related documents searched for \u201cVirtual reality animation.\u201d The main research content is medical care, arti\ufb01cial intelligence (AI: Arti\ufb01cial Intelligence), industrial production. Throughout the development process of animation, the artistic development of ani- mation and animation technology are closely linked and inseparable. Especially in today\u2019s information age, animation has entered paperless, digital, and three-dimensional, deeply imprinted by technology. Today\u2019s animations all have a common problem: the singleness of information transmission. The audience can only blindly accept the infor- mation instilled by the animation, and the animation lacks interaction with the audi- ence. Virtual reality animation combines current animation and advanced virtual reality technology to depend on virtual reality technology (Deerin 1995). The development of virtual reality technology was \ufb01rst produced in response to the demand for simulation technology. In 1961, Morton Heilig put forward the idea of building an environment that simulates the environment of the block, trying to make people feel the environment of the fundamental block through this system, including sound, picture, smell, touch, so he is also called Pioneer of the concept of \u201cimmersive virtual environment.\u201d By 1986, Michael McCreevey completed a relatively uniform virtual reality display system. This system is also a current system model, including a display system, sound system, external Setup sensory system, computer processing system, Etc (Anthes et al. 2016). At the beginning of the 20th century, McLuhan said in Understanding Media: The Extensions of Man (Understanding Media: The Extensions of Man) that virtual reality will become the ultimate medium for people to perceive and communicate freely. Ed Catmull also started from the experience of video games and believed that the best way of telling virtual reality is not a linear narrative. However, it should include linear narrative in the narrative method to use linear narrative to construct a more complex interactive and non-linear narrative. In 2001, Marie-Laure Ryan explored the narrative of virtual reality in his book \u201cNar- ration as Virtual Reality.\u201d He believes that virtual reality produces narrative materials with characters, scenes, and behaviors but does not have a narrator, a virtual reality sys- tem. It is not just a story but a matrix of double stories: stories that can be experienced and stories that can be told. 3 Methodology This thesis is mainly divided into three parts: historical status and market demand analysis research, theoretical research, case analysis research. The main research methods are as follows: The literature research method is to obtain information by collecting relevant docu- ments at home and abroad, searching for keywords such as \u201cvirtual reality animation\u201d and \u201cinteractive narrative design\u201d involved in the subject, sorting out and analyzing the","On the In\ufb02uence of Virtual Reality Technology (VR) 273 content of relevant documents to fully and correctly understand the research history and current situation of the interactive narrative design of virtual reality animation. Interdisciplinary research methods-This article will try to apply multidisciplinary knowledge and theoretical methods to the research of virtual reality interactive narrative design through the research of art, design, interactive design, and other interdisciplinary research, and absorb research experience and methods in different \ufb01elds to explore the dynamics of virtual reality in the era of digital media communication. Draw the theoretical basis of interactive narrative design. The case study method analyzes and investigates the interactive narrative-related works of virtual reality animation at home and abroad to \ufb01nd out what elements and characteristics of virtual reality animation\u2019s current interactive narrative method. Explore these cases\u2019 design process and design experience to master their design rules and com- bine their own practical experience to summarize the design strategy of virtual reality interactive narrative design. This article conducts a detailed and in-depth analysis of the entire animation creation activities from virtual reality. It analyzes the impact of virtual reality on the main body of animation art creation, the impact on animation art creation methods, and animation artworks and audience groups. I hope that through this analysis, can \ufb01nd a perfect intersection of virtual reality and animation, and stimulate animation creators to think about a new mode of animation itself, and work hard to analyze the current situation of virtual reality animation, the characteristics of virtual reality animation, communication application media, and applications. Discuss the difference between virtual reality and traditional animation in terms of mouth, production process, and aesthetics. 4 Findings This article discusses the interactive narrative language of virtual reality animation by comparing the interactive narrative of virtual reality animation and traditional animation narrative. After gaining a clearer understanding of narratology and interactive narration, this article focuses on how to integrate interactive narrative methods into the content of the story in the virtual reality media environment and summarizes the following points through speci\ufb01c case analysis: 4.1 The In\ufb02uence of the Script (1) Virtual reality technology can broaden creators\u2019 ideas and allow creators to maximize their imagination. The cartoon is a collection of imaginative stories. The famous Japanese animation master Hayao Miyazaki also said: \u201cAnimation is an art that is so pure and simple and allows us to express through imagination\u201d. The most signi\ufb01cant difference between traditional story scripts and animation scripts is the degree of imaginative expression (Zeng 2021). The writing of animation scripts requires the creators to observe the details of daily life and needs to be good at summarizing the insights in life. It also requires the creators to present their wild imaginations through words and use these ideas as the \u201c\ufb01rst step\u201d. They are displayed in a visual form of text. If animation creators don\u2019t","274 Y. Wu have such imagination, they can\u2019t create a naive image like \u201cChinchilla\u201d, which is a product of combining ordinary cats in a simple life with bizarre ideas. Totoro\u2019s honest and straightforward expression and cute actions have attracted many audiences. In the movie, the bus and the cat are one. However, the protagonist travels happily on such a bus in the comics. Without such a rich imagination, it would be impossible to create a touching story and a gripping storyline without careful observation and insight into life. From this point of view, creating animation requires \u201cbizarre\u201d imagination and unrestrained creativity. Only these accurate details in life and the ideas of animation creators can make animation stories more exciting and childlike. However, there are many creative limitations in the animation creation process; the main reason is technical reasons. For example, a scene often appears in animation scenes: \u201cUnder a clear sky, the sun shines warmly on the earth. A beautiful princess walks in the garden with her Prince Charming, the birds sing cheerfully, and the two People are loving and loving, enjoying the beautiful world of two people. They passed through the \ufb02owers and came to the fountain pool. At this time, the music sounded, the prince gently held the princess\u2019s hand and danced gracefully, and the fountain followed the rhythm of the music. Regularly jumping\u2026\u201d The story is like this, which is also our most common. Still, if we add the audience factor to this storyline, the development of this story will be more attractive to the audience and increase the interaction factor of the audience. It is still under the clear sky, and the sun is shining on the earth. A beautiful princess walks in the garden with the prince charming in her heart, the birds sing cheerfully, and the two are in love. They were enjoying the beautiful world of two people. They passed through the \ufb02owers and came to the side of the fountain pool. At this time, the music sounded, the prince gently held the princess\u2019s hand and danced, and the fountain also danced regularly with the rhythm of the music. At this time, the audience can adjust the type o music in the play according to their mood. If the audience wants to add some comedy colour and playful plot, the audience will stretch out a hand and pat the prince secretly on the shoulder, just like an invisible person participating in the play. \u201cOr, extend a hand to change the direction of the fountain.\u201d If in traditional animation creation, these are simply impossible to achieve, but with the help of virtual reality technology, the plot is entirely achievable in theory and can even be done better. In this way, the role of virtual reality in animation creation is the support of technology, the broadening and enlightenment of the creator\u2019s thinking. (2) Virtual reality technology can make the character of animated characters more clear; virtual reality technology makes the character modelling more realistic and realistic, magical and fantasy. Whether the animation is vivid or not directly relates to the character modelling and character in the animation script design process. The language and actions are the expressions of a character\u2019s personality. Different characters in the animation speak other languages, and the tone of speech is also different. For example, in the cartoon \u201cZootopia\u201d, the host Nick Wilde, a fox who lived by cheating in Zootopia, was hurt by discrimination and prejudice as a child and gave up his ideals. In the early stage of the story, Nick Wilde\u2019s words were full of local ruf\ufb01ans, like a living commoner. The character design in the animation script needs to be based on the character, cos- tume, living environment, and each character in the play. The most signi\ufb01cant difference","On the In\ufb02uence of Virtual Reality Technology (VR) 275 between characterization and live performance in animation is that it is more random, not limited by character modelling. It is easier to create a shape that \ufb01ts the character\u2019s per- sonality and can even be designed without traditional character modelling requirements. Because of the emphasis on the immersion of animation in virtual reality animation, the audience can observe from multiple angles, not only from a single perspective. There- fore, animation character design can be carried out more abundantly. Multi-curve and multi-angle planning can be carried out, instead of being limited to traditional forms of expression, only focusing on a picture displayed to the audience. Characters outside the image do not need to be considered. Virtual reality animation greatly extends the form of energy and requires careful design outside the screen because the audience determines the screen. The character design in virtual reality animation is suf\ufb01cient, re\ufb02ecting the characteristics of animation characters and increasing the dimension space of animation performance. (3) Virtual reality technology can improve audience participation and enable audiences to have a better viewing experience (Fig. 1). Fig. 1. Pirates of the Caribbean \u201cPirates of the Caribbean\u201d in Disneyland is such a typical interactive creative work, adjusted according to participating audiences\u2019 actions. This interactive sculpture is made of devices that emit sound and light alone. Some of the nearest devices will emit sound and light when the audience approaches. After a long time, the whole sculpture will be quiet as if these devices have already been installed. Participants get to know each other and adapt to each other. Although \u201cPirates of the Caribbean\u201d is only a typical representative of interactive installation art, this artwork can fully re\ufb02ect the interactivity of virtual reality technology. They emphasize the audience\u2019s participation and that it is an indispensable part of this artwork. In virtual reality, the audience has long been not just passively receiving, nor just watching as cramming. He emphasized that the audience is a part of this art. The same is valid for creating virtual reality animations, emphasizing the audience\u2019s participation. Of course, suppose there is a roller coaster scene in the animation plot. In that case, if the audience can experience the same experience as getting on the car, the audience can feel the left and proper shaking, up and down vibration, and even feel the wind caused by the sudden drop. These can be realized in the design of virtual reality animation. Creators must take these interactive factors into account when","276 Y. Wu creating animation scripts, and they need to focus on how to mobilize the audience\u2019s participation when designing. 4.2 In\ufb02uence of Animated Shots Animation is a way of audio-visual language art. The primary artistic technique is the use of lens language. Lens language directly expresses the creator\u2019s thoughts and inner ideas. What kind of story the shot shows, what kind of content the picture is, and what kind of scene is used all directly have a signi\ufb01cant impact on the emotional expression of the animation? The production cost of animation is higher than that of traditional audio- visual language. When shooting animation, it is necessary to choose the lens language carefully. The same story, the same plot, but when we use different scenes and different shots, the emotion of the story changes completely, even far from it. For example, the same person is standing on the ground, his eyes are looking forward, and one \ufb01nger is pointing into the distance. Here we use one looking up the lens and the other looking down the lens, and the \ufb01nal effect is quite different. One is a stalwart body, and the other is a despicable villain. Therefore, the re-animation of the shots is a crucial part. The author believes that there are many factors in the lens language of animation. As a new technology, the impact of virtual reality technology on the language of animation lens is mainly manifested in: \u2022 The in\ufb02uence of time expression. \u2022 The in\ufb02uence of space. \u2022 The change of lens performance way. \u2022 Holographic and large-screen reality test the lens design (Reynolds 1982). (1) The in\ufb02uence of time. Virtual reality technology When the audience participates in the animation story, the audience\u2019s time also becomes a part of the animation time. The virtual world becomes the participant\u2019s time, and the participant\u2019s time is also integrated into the animation\u2019s time in the timeline. The time in the shots of traditional animation is directly related to the director\u2019s creation, expressing the director\u2019s ideas. The director can use the shots to express the sequence of time, \ufb02ashbacks, interludes, etc., but these require post-production. The editing and processing of the \ufb01lm can only be realized. A documentary \ufb01lm starts shoot- ing when the machine is turned on and does not turn the machine in the middle. It looks like a tape is shot without any processing. It is also a way of recording time with the lens. However, under the in\ufb02uence of virtual reality technology, time is not limited to such post-production operations. There are two ways of animation time here: The adult birds are densely covered with clouds and thunderstorms. Age in virtual reality animations can also be controlled by parameters, changing from a child to an old lady. (The change of the two characters requires the design of the two-character shapes in the early stage, and the number of faces and points of the two characters needs to be controlled to be the same in front of the mouth, and then the two characters can be merged and changed.) It is also possible to set up multiple cameras in a large scene, as the scene in a virtual reality simulation environment is often limited. This way","On the In\ufb02uence of Virtual Reality Technology (VR) 277 of changing is similar to switching between camera changes and scenes in traditional animation. Therefore, speci\ufb01c to the animation production of virtual reality, someone needs to control the change of time. The other is the participant\u2019s sense of time. The virtual reality in the mouth is insep- arable from wearing the data helmet. From wearing data helmet. It can only rely on the data helmet for stereoscopic reality and positioning tracking. In the data helmet, by simulating the imaging of two eyes and how to position the device on the helmet, the position of the helmet is captured by the signal transmitter and signal receiver placed in the room, 3D simulation is performed, and the position of the helmet is this position judges good Bad. When the audience participates in the storyline of the animation, the audience\u2019s time also becomes a part of the animation time so that the virtual world becomes the participant\u2019s time, and the participant\u2019s time is also integrated into the animation timeline. (2) The in\ufb02uence of space. Virtual reality technology can directly simulate the colour, light and shadow, sound, lens, action and so on of three-dimensional things in three-dimensional space. Virtual reality technology can allow the audience to actively hang on to the picture they want to see, breaking the space limitation of traditional animation. In virtual reality, all the space is simulated, but it is accurate. However, the spatial impact of virtual reality on animation is not only visual and physical spatial impact but also manifested in psychological, aesthetic space, perceptual space, and other aspects. The traditional way of expressing animation space mainly focuses on the simulation of three-dimensional things in two-dimensional space, including colour, light and shadow, sound, lens, action, etc. Although these make the audience feel that the three-dimensional space exists on the surface, the audience encounters dif\ufb01culties exploring further. The audience can only touch the side that the director shows to the audience, and the content of other perspectives and other perspectives does not seem to be the same as that of the director. Relationships even shut out the audience. However, virtual reality technology completely solves this problem; This technology allows us to communicate and contact the displayed objects more closely. (3) The lens performance mainly adopts the long lens method. Virtual reality technology can fully display the characters\u2019 time, place, development process, and environment. Since the birth of automatic animation, animation has forged an indissoluble bond with long shots. The animation itself breaks through the limitations of traditional camera technology. Animators can express almost everything they can imagine with pictures. The gorgeous shots attract audiences, and the beautiful pictures have touched many fans. The use of long shots in animation has completely formed one of its iconic features, whether it is Nezha running in the waves in \u201cNezha Naohai\u201d; or \u201cThe Periwig-Maker\u201d (The Periwig-Maker), grey colours, soft music, walking in the streets, many animations with strong artistic forms now also use long shots, and even very few animations have a whole short \ufb01lm in one shot. Long takes are already an iconic element of animation. The theory of long shots in \ufb01lm and television and the representative \ufb01gures include Marshall Mardan and Bazin. In fact, in the earliest days of \ufb01lm and television art, people set up cameras to shoot a \ufb01lm without editing and no special effects; it was a one-time","278 Y. Wu shooting because there was not much theoretical guidance. At most, it stopped in the middle to switch scenes. After the appearance of the montage, people gradually began to pay attention to the combination of shots, and then the shots became short, and some shots were just like pictures. Now MTV art is the ultimate art type of montage performance. In MTV, it is even possible to switch between several shots in one second, equivalent to one picture per frame. With the continuous improvement of the theory, the long shot has been continuously known by people again, and it has caused an upsurge in academic discussions. Everyone can express their opinions and let a hundred \ufb02owers bloom. The use of long takes in animation can be very attractive. In the long shot of the \ufb01erce \ufb01ght between Nezha and the evil Dragon King at the end of \u201cNezha Raising the Sea\u201d, the panning shot explains the dragon palace scene of the entire duel. The panning lens adopts non-horizontal tilting and rotating panning, which destroys the psychological effect of the viewers when they usually watch the picture, causing a sense of instability. Then use the overhead lens to explain the two sides of the game, and then zoom in to make a partial close-up of the subject, which directly affects the viewer. Emotions. Drive up and integrate into the environment of this competition. This set of shots uses panning and panning to follow the character\u2019s movement, especially after a round of engagement. The camera is lowered to follow the dragon king being knocked to the ground, re\ufb02ecting the tremendous force of the protagonist Nezha. The camera moves correspond to the characters in the play. The screen shows what the characters in the play have seen and heard, which can express a subjective solid colour. The parties involved in the situation make the appreciators emotionally resonate. This long shot perfectly combines the movement of the camera and the joint movement of the camera and the subject, and at the same time, with the fast-paced scene changes, it vividly creates a tense atmosphere during a \ufb01erce duel (Reynolds 1982) (Fig. 2). Fig. 2. The long shot in the movie \u201cNezha\u201d","On the In\ufb02uence of Virtual Reality Technology (VR) 279 Virtual reality technology has changed the concept of the lens. In traditional anima- tion or \ufb01lm and television art, the director\u2019s shots are directed and arranged. In the art of \ufb01lm and television, a videographer is even more needed to shoot, and even more, than ten cameras are required for shooting in a \ufb01lm, and then montage editing and synthesis are performed. However, in virtual reality animation, people don\u2019t need to watch the shots arranged by the director. In fact, from another perspective: \u201cThe camera is dead in virtual reality.\u201d For each audience, from the moment they walk into this virtual reality animation story, they have already started watching the story, and the camera will start recording until the participating audience walks out of the virtual reality animation. The- oretically, their machines are all. It is not closed. The entire animation is a big long-shot theory if it is explained according to the current long-shot theory. It is not an accident that the emergence of virtual reality animation increases the use of long shots because people have been trying to get closer to the \u201creal\u201d, and many people are striving for this goal. 4.3 Impact on Animated Artworks and Audiences Virtual reality technology deepens the immersion and \u201cinteraction between the work and the audience\u201d of animation works and brings a brand new visual experience to the audience. (1) Immersion of animation works. With the support of virtual reality technology, animation works are more immersive. Virtual reality has begun to be accepted by people in modern thought and new aesthetic concepts. People\u2019s ability to accept new things is more vital than any other era. People never want to experience the creator\u2019s world for real. Virtual reality animation fully satis\ufb01es the audience\u2019s experience requirements. The current animated \ufb01lms have begun to explore technology and content to create a more realistic and natural environment (Kim et al. 2012). The immersion experience is still a passive experience that stays on the basics. For example, the 4D and SD animation movie experience we have seen uses passive interactive methods to immerse our audience here. Most of the 4D and SD animation movie experience items use some beautiful clips in the animation and then program these movie clips to increase the special effects settings. The commonly used 4D and SD animation movies have the function of shaking and vibrating. Pre-programmed data control the chair, and the adjustment of the chair depends on the air pressure below. When an animated shot has a similar shake or a unique atmosphere needs to be added, interactive facilities need to be added. Under the action of this kind of equipment, you can thoroughly experience the shaking brought by the earthquake or simulate the surprise of a mouse passing through your back. However, this immersive experience still only stays on the surface and is still in its infancy. In theory, the immersive experience of virtual reality animation has dramatically surpassed the immersive feeling of the existing 4D movie experience. (2) Interaction of animation works. Another prominent feature of virtual reality works is an interactive relationship between the works and the audience. Traditional animation creation is divided into animation works and animation audiences, and there is a clear dividing line between the two. The audience before the screen is the audience, and the animation works are","280 Y. Wu on the screen. When the animation is played, the audience and the animation work are simply a relationship of acceptance and acceptance. At most, the audience has a brief and vague aftertaste after watching. It is not interactive, let alone mobilizing the audience\u2019s enthusiasm to participate in the interaction. Virtual reality animation breaks through this barrier, dilutes the dividing line between the work and the audience, and even combines the two organically, shocking the audience visually and psychologically. \u201cSelders writes of the joy of \ufb01nding the pinnacle of expe- rience in his mediocre work, speaking of the German expressionist \ufb01lm Dr. Caligari\u2019s Cabin: \u2018When a certain moment comes, everything is just right, and you happen to be a part of it. It\u2019s either re\ufb01ned or unimaginably gross; the ecstasy in it keeps it away from everything else.\u201d This is what Selders describes as a theory of \u201cmemorable moments.\u201d The interactivity of virtual reality has changed the previous mode that audiences sit quietly and watch animated movies. Audiences can walk into the scenes in the animated stories to experience. For example, the picture below shows one of the applications of virtual reality animation - interactive ground projection. In this entry, it is simulated that people are walking in a pond. The projection projects the picture on the ground. When people pass by, it will cause the water in the picture to \ufb02uctuate. Stepping on the lotus leaf will also produce ripples. When the \ufb01sh encounters people, they will \ufb02ee in fright. Although this is only a primary interactive design, it is different from the previous pure video mode, adding fun and interactivity. 5 Conclusion The main conclusions of this thesis are: (1) Virtual reality technology is marching into the art \ufb01eld in enormous strides, and it has become a new media art form, subversively narrowing the distance between the audience and the artwork. (2) The intervention of virtual reality technology presents immaterial, super-reality, and non-linear aesthetic characteristics, which will signi\ufb01cantly enhance the audience\u2019s aesthetic experience and broaden the artist\u2019s artistic, creative thinking. (3) The intervention of virtual reality technology has played a positive role in the consumption, communication, and aesthetic value of new media art. However, it is also necessary to see its shortcomings and possible adverse effects. As one of the most popular pronouns today, animation has become a post-dinner talk. The overheated animation discussion seems to have caused the development of animation to lose its direction. The essence of animation is a form of artistic expression. With the continuous enrichment and development of technology, the artistic expression of animation is also keeping up with the pace of the times, exploring and breaking new forms and methods. Throughout the historical development process of animation, every step of animation progress is accompanied by the footsteps of science and technology. Now virtual reality technology has also brought tremendous transformative energy to animation development. The effects presented by the new vision will have trans-age signi\ufb01cance for the development of animation. In the Internet age, modern humans can quickly learn and accept new things and are not blindly excluded from external things. It will inevitably add to virtual reality technology\u2019s technical promotion and practice in various industries, and animation is, of course, no exception. In the future, in creating","On the In\ufb02uence of Virtual Reality Technology (VR) 281 virtual reality animation, the audience will be more immersed in the director\u2019s idea, and the director will have more creative space to swim in the creative ocean. Shortly, the animation will be a fairy tale kingdom of \u201conly unexpected, nothing impossible.\u201d The main conclusions of this thesis are: (1) Virtual reality technology is marching into the art \ufb01eld in enormous strides, and it has become a new media art form, subversively narrowing the distance between the audience and the artwork. (2) The intervention of virtual reality technology presents immaterial, super-reality, and non-linear aesthetic characteristics, which will signi\ufb01cantly enhance the audience\u2019s aesthetic experience and broaden the artist\u2019s artistic, creative thinking (Ma et al. 2010). The intervention of virtual reality technology has played a positive role in the consumption, communication, and aesthetic value of new media art. However, it is also necessary to see its shortcomings and possible adverse effects. As one of the most popular pronouns today, animation has become a post-dinner talk. The overheated animation discussion seems to have caused the development of animation to lose its direction. The essence of animation is a form of artistic expression. With the continuous enrichment and development of technology, the artistic expression of animation is also keeping up with the pace of the times, exploring and breaking new forms and methods. Throughout the historical development process of animation, every step of animation progress is accompanied by the footsteps of science and technology. Now virtual reality technology has also brought tremendous transformative energy to animation development. The effects presented by the new vision will have trans-age signi\ufb01cance for the development of animation. In the Internet age, modern humans can quickly learn and accept new things and are not blindly excluded from external things. It will inevitably add to virtual reality technology\u2019s technical promotion and practice in various industries, and animation is, of course, no exception. In the future, in creating virtual reality animation, the audience will be more immersed in the director\u2019s idea, and the director will have more creative space to swim in the creative ocean. Shortly, the animation will be a fairy tale kingdom of \u201conly unexpected, nothing impossible.\u201d References Anthes, C. et al.: State of the art of virtual reality technology. In: 2016 IEEE Aerospace Conference. IEEE (2016) Burdea, G.C., Coiffet, P.: Virtual Reality Technology. Wiley (2003) Deering, M.F.: HoloSketch: a virtual reality sketching\/animation tool. ACM Trans. Comput. Hum. Interact. 2(3), 220\u2013238 (1995) Kim, Y.A., Shin, S.Y., Kim, J.H.: Identi\ufb01cation patterns for immersion of visual storytelling-based on disney animation. J. Korea Contents Assoc. 12(7), 54\u201364 (2012) Ma, M., Zheng, H., Lallie, H.: Virtual reality and 3D animation in forensic visualization. J. Forensic Sci. 55(5), 1227\u20131231 (2010) Reynolds, C.W.: Computer animation with scripts and actors. ACM SIGGRAPH Comput. Graph. 16(3), 289\u2013296 (1982) Zeng, R.: Research on the application of computer digital animation technology in \ufb01lm and television. J. Phys. Conf. Ser. 1915(3), 032047 (2021)","An Industry in Crisis: Virtual Mediums for Theatre and Live Performance James Simpson1,2,3(B) 1 Copper Candle Ltd., London, UK [email protected] 2 Rose Bruford College, Sidcup, UK 3 University of East London, London, UK Abstract. The pandemic created an opportunity within a crisis for theatres and live performances. It took away the ability to experience live events in person but encouraged new experimentation and discovery of virtual mediums to replace them. This paper looks at three case studies from work undertaken by Copper Candle and Rose Bruford College during the pandemic which were developed as alternatives to physical theatre but allowed audiences to continue to experience live performances from remote locations. The research considers the work of Auslander and Phelan (Auslander 1999; Phelan 2003) and their discourse on Liveness in performances. It expands on the association of the need for liveness with theatre and experiential events by challenging the expectation that events should be live because the performer, not the audience require it to be live. This paper introduces the idea of the performer centric bias, a consideration of how existing research and professional practice is guiding the industry to make incorrect assumptions about the necessity of liveness in virtual performances. Keywords: Metaverse \u00b7 Theatre \u00b7 Motion capture \u00b7 Liveness \u00b7 Performer centric bias \u00b7 Virtual 1 Introduction Digital transformation is a term that many industries have had to engage with in the last decade and 2020 has provided a means for businesses to force rapid change and create environments that support remote work and virtual presence. But how does this translate for a creative sector whose business is to provide live, in-person entertainment? Before the pandemic of 2020 that saw the mass closure of entertainment venues around the world, the concept of delivering live theatre virtually was reserved for the experimental theatre companies, usually who had the funding and capacity to undertake R&D that would provide low returns in terms of \ufb01nancial income and business growth. In the two years that followed, theatre and live events have had to adapt in a number of ways to think of new ways of presenting live performances in virtual mediums which has opened debate on the types of mediums that work and don\u2019t work for audiences. \u00a9 The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 T. Jung et al. (Eds.): XR 2022, SPBE, pp. 282\u2013293, 2023. https:\/\/doi.org\/10.1007\/978-3-031-25390-4_25","An Industry in Crisis: Virtual Mediums for Theatre and Live Performance 283 This paper examines the nature of these virtual mediums as the theatre industry realigns itself for virtual presentation of its artform. It looks at case studies from the authors own practice throughout 2020 and 2021 which tested a variety of mediums with different successes and failures to help identify key verticals in virtual creative practice. As such, it reviews the situation with a combination of academic and real-world industrial experiences that pulls apart these new ideas. 2 Challenges: Meeting the Expectations of Audiences The argument presented is that the theatre and live events industry has a natural home in the virtual medium and that, once the formula has been identi\ufb01ed, this new sector will live beyond the necessity that brought about it\u2019s transformation in 2020. Understanding the nature of \u201cliveness\u201d and \u201ctogetherness\u201d as key aspects of the theatrical experience will contextualise the issues of digitisation which can be dis- cussed against an equivalent medium; the live streaming or cinema playback of recorded theatrical productions. Whilst only notional, it might help to theorise the origins of theatrical storytelling and seek to explain how spatial content via VR and MR is more relevant to the origins of theatre than the commercial theatre industry of the 20th and 21st century is to many. For the largest part of humankinds history, storytelling has been imaginative and told through narration around camp\ufb01res. Whilst this served for millennia, storytelling started to become shared to larger gatherings with gradually increasing complexity of visual material. Since the Greeks introduced amphitheatres as a way of managing the acoustics of a single speaker to address crowds, there has been an evolution of theatrical spaces that have slowly changed the viewing area from being spatial to being in front of us. This has seen the move from open spaces to proscenium arch theatre in the last 200 years and now to stories being viewed through a 7 phone screen. XR allows us to put our stories back around us spatially, as originally experienced, but with enhanced digital aesthetics that technology can now support. The necessity to develop new formats for live theatre have modi\ufb01ed the expectations of audiences who, for a period in time when live theatre was closed, was able to make do with watching live streamed or even pre-recorded theatre experiences via their TVs or phones. Our evolution as humans has prioritised the spatial experience, which is possibly why we prefer the physical attendance of live events over the digital, which was proven when viewing \ufb01gures for Londons more popular streamed theatres (The Young Vic, The Old Vic, The ROH and NT Live) haven\u2019t performed as well since live theatre has resumed. If this means we prefer to physically attend live events over digital, then how do we account for the 27 million people who watched Travis Scott playing a 10 min set in the game Fortnite? If we assume that there is a viable market for live events in the metaverse (this paper doesn\u2019t presume to know the answer or seek to measure this) then the formula for making live events work for audiences who seem to prefer to engage with physical over digital needs discovering. Is it the type of audience? Or is it the type of works in the medium? Or is it something else? In the research undertaken between Copper Candle and Rose Bruford College, several variants have been identi\ufb01ed which help to solve this challenge.","284 J. Simpson 3 The Performer Centric Bias The \ufb01rst discovery is a possible limitation to the producing culture of live events which \ufb01lters what is necessary and important to audiences from what is the experiences of the makers of theatre. 3.1 In Research Much of these ideas aren\u2019t formally documents, but come from an understanding of the culture of theatre and live events from the author and fellow researchers experiences based on a combined 75 years experience in the live events industry. What they noticed is a disparity of support and appreciation within the live entertainment sector between performers, artists and technologists. There has been literature written on bias within performers, particularly around the casting of parts biasing attractiveness, race and age, but the idea of the role of the performer being biased over other roles in the theatrical production process is not strongly represented. This in itself is evidence of the performer bias and that it exists amongst researchers themselves. A possible explanation for this is that due to the nature of the challenges getting work as a performer, many have turned to education and research in the latter part of their careers, whereas the opposite may be true for designers and technicians who tend to \ufb01nd more work and opportunities as their experience develops. This is a sad situation for industry performers, but is also one of the possible reasons for bias towards performers as so many seek new careers. 3.2 In Practice The reason this is important and relevant is that it helps us understand better the impor- tance of liveness, and the reason why in both industry and academia we have a skewed appreciation for the need of liveness. The issue is the way we look at, assess and qualify the success of our productions. We rely overly on critical review, peer review and the opinion of experts in our judging processes. When productions are discussed in review, the focus is predominantly on the quality of the performers, their portrayal of the characters they represent and their ability to carry over the narrative. At awards such as the Oscars and Oliviers, whether intentionally or not, the highest regarded award is always for the leading actor\/actress. Directors focus overly on the performers in their creative process from the earliest conception to the technical and dress rehearsal which is evidenced when they share their notes which shows performers blocking1 and positions before the scenery is entirely thought out and the technical and dress rehearsals usually involving a directors note session that is dominated by performance notes. An example of this is in a publication by Katie Mitchell (Mitchell 2009) which is supposed to be a handbook of directors craft but fails to acknowledge her relationship with design roles. 1 Blocking is a term used in theatre for planning the location and movements of the performer on the stage and usually begins as notes the director makes and is one of the \ufb01rst parts of the creative process as the actors are blocked into their positions.","An Industry in Crisis: Virtual Mediums for Theatre and Live Performance 285 In Mitchells book, she suggests that lighting is subservient to the performers because it relies on their work being completed in the \ufb01rst instance \u201cThe \ufb01rst function of theatre lighting \u2013 communicating time and place \u2013 can be prepared before rehearsals begin. The other three functions can only be ful\ufb01lled after the moves and actions that the actors will perform are created in the rehearsal room.\u201d Given the inherent lack of \ufb02exibility of lights, supported by structure and cabling, a lighting design will value highly the opportunity to iterate the performance and the lighting around each others constraints and not by designing via a waterfall method where lighting has to respond to the performers in the \ufb01rst instance. Mitchell also indicates the subservience of the lighting designer in the process \u201cin- volve the lighting designer in all your decisions about time, place and immediate circum- stance, as these will determine the lighting for the scenes.\u201d This can be interpreted as an acknowledgement of a hierarchy in the design process where the choices of lighting are dictated by the director instead of a collaborative creative process. This is in opposition to the rest of her book which describes how creatively collaborative her work with the performers is through many days and sometimes weeks of creative workshops. The \ufb01nal evidence of this bias is in the quantity of time devoted to lighting design role, which amounts to 4 paragraphs (less than 1 page) of her 256 page book. Either this is because she doesn\u2019t value the role highly enough or because she has made an assumption, or is aware, of her intended audiences bias towards the performer. Evidence for this bias is recorded in other ways which makes performance the most coveted role in theatre and the people that make the important decisions, reviews and opinions in our industry are largely performers themselves or ex-performers or failed performers, for instance in the biggest award of an Oscar or Olivier Award being the best actor\/actress, or the amount of lines of text dedicated to the quality of the performances in a review and little acknowledgement of other elements of the performance experience. It was a notable achievement that the technical director of the ROH left to become General and Artistic Director of the Opera Royal de Wallonie-Liege. It was signi\ufb01cant enough that the CEO of the ROH at the time described it as the \ufb01rst time he had seen it happen and he was fully engaged in the opera community of CEOS. Whilst it is not a rule everywhere, you are far more likely to \ufb01nd artistic directors, reviewers and awards panels made up of people who had a background or desire to be a performer than from any technical or design discipline. The result of this is that when we consider the reason why we do things the way we do in theatre and live events, we have to be wary of the institutional bias towards the performers point of view which, as you will see in Sect. 4 on liveness, might be creating an undesirable and expensive method for creating a product that is wholly unnecessary. To provide balance on this however, one of the reasons we are drawn to performers more generally is that we are focused on the human being because we are all humans ourselves and are biologically programmed to read depths of information from them in ways that are subliminal and covert for conveying the story. Whilst this might explain the bias, storytelling has examples that don\u2019t rely on human beings and dialogue, such as Pixar Animations Wall-E which starts with 45 min of sto- rytelling using no dialogue and no humanoid performance \u2013 even facial expressions were limited to a low resolution pixel display and 2 \ufb02appy eyebrows. It is possible to","286 J. Simpson convey storytelling and performance without human performance and this is also proven through events such as Son et Lumiere2 which rely on nothing other than visual specta- cles to create entertainment. Even though they still use human beings in the performance, orchestral recitals such as Classical Spectacular at the Albert Hall are examples of per- formances which don\u2019t rely on the human conveying a story with their own characterised performance. 4 Liveness To understand the commercial reality of liveness, we need to understand a well- documented but not particularly appreciated process that is applied in particular to commercial theatre and many large-scale events and concerts \u2013 the timecode generator. At the heart of every production is a timecode generator which is essentially a clock that cleverly keeps all elements of a production in time with each other. It sends a click to the conductor to control the speed of their baton movements which in turn can control the entire cast and orchestra to keep them in time with each other and the other technology the timecode system is connected to. Lighting, sound, automation, pyrotechnics and video effects are all programmed to respond on command of the time code and rarely require human interaction to trigger something in response to what is happening in the rest of the production. It is effectively a tape recording that, if paused, would stop an entire production in the same way as pressing pause on Net\ufb02ix would stop a \ufb01lm from playing. Some commercial theatres back the dancers with additional singers voices and backup voices are often recorded for the principals in case of illness. Audiences are often unaware of a live band being part of a show when they are contained in a room at the back of the stage instead of in an orchestra pit, particularly as the authenticity of the sound is being reproduced through electronic ampli\ufb01cation and the natural timbres of the instruments used isn\u2019t reaching the audiences ears directly. Auslander is one of the principle researchers on this subject (Auslander 1999) as is Peggy Phelans work (Phelan 1993) who often contradicts Auslander. We are drawn back to the idea of the performer centric bias being involved in their discourse too. In a published interview with Marquard Smith for the journal of visual culture (Phelan 2003) Phelan says \u201cI\u2019m not against technology. But I think when one is showing a video one is showing a video\u2026\u201d and she goes on to discuss the importance of performance over and above this technology. \u201cwhat distinguishes performance, ontologically, from the photographic and recording arts. Performance\u2019s ephemeral nature, I was arguing, is absolutely powerful and can serve as a rejoinder to the \u2018preserve everything\u2019\u201d. These texts are incredibly successful at providing new intuition and ideas that may help ideate new solutions to the challenges of creating a \u201clive\u201d experience for audiences via pre-recorded mediums, but the apparent bias towards performers is contrary to the technological discussion that we need. This doesn\u2019t mean that it isn\u2019t relevant research 2 Son et Lumiere is a story told entirely with light and sound, using architectural back drops or effects equipment to support the light as a medium. It is popular for entertainment displays in hotels (The Bellagio Las Vegas), Immersive experience (Disney Worlds night time displays) or for historical sites (The Great Pyramids of Giza).","An Industry in Crisis: Virtual Mediums for Theatre and Live Performance 287 but the magnetism of this subject by these very popular researchers has left a hole in the discussion around liveness in other areas of the performative experience, such as technology, onboarding mechanisms and the requirements of an audience amongst themselves. Going back to the mechanical construction of commercial theatres performance system, when performers were asked their opinion on the value of liveness in theatre, their responses often repeated; \u201cit\u2019s in the moment\u201d to \u201canything could happen\u201d to \u201cyou feel connected to the show\u201d. None of these can really be true if the show is running on a pre-recorded track where even the performers themselves are unwittingly the puppets of a timecode generator. Anything could happen only if there was a mistake, so are we saying that live entertainment is only acceptable if it isn\u2019t perfect? The irony of this issue is that whilst, at least in casual conversation, everyone seems to agree that liveness is essential to de\ufb01ne what theatre is, no-one wants to watch a production with mistakes. The inconvenient truth to creating a perfect live performance that is repeatable is that the mechanical and technical layers introduced to the production are designed to speci\ufb01cally avoid mistakes from happening. This is not least because of practical concerns such as safety, but also due to the expectations of the audience that are paramout from a business context but not in the design process. The most important response from a participant has always been that from anyone who exhibits a performer centric bias who would make comments such as \u201cnothing is quite like standing on a stage in front of a live audience\u201d or \u201cyou feed off of the energy of the audience to support your performance\u201d. None of these are viable reasons for needing a production, with all of the expense and risk that comes with making a production live, to actually be live. These responses are more about the performers personal feelings as an actor instead of what the production needs. Theatre is live essentially because it has to be. The physical nature of it prevents much more from being pre-recorded than their currently is, with control systems unwit- tingly providing time and pace and continuity for the entire production. With the advent of virtual metaverse theatre, we have the opportunity to assess whether we need live performance in our productions, or if in fact we just need experiences that support live audiences? Our research has generated interesting conclusions which suggests that the shared audience experience is more important than the live performance and that this can be supported by live performance if it services this goal, but live performance for the sake of itself is not solely responsible for creating the sensation of a live experience for an audience. 5 Practical Studies The study will cover three experiments which attempted to discover the formula for a successful virtual theatre medium, which were developed individually and in combina- tion between Rose Bruford College and Copper Candle. It will discuss the role of XR in combination with digital performance capture, such as Motion Capture and Volumetric Capture, as well as the challenges for distributing content worldwide to remote audiences and the solutions found for these.","288 J. Simpson 5.1 Constellations The \ufb01rst study is a virtual re-imagining of Constellations by Nick Payne and directed by Patrick O\u2019Kane as part of a piece of work at Rose Bruford College with MFA Collaborative Theatre Making students (Fig. 1). Fig. 1. Constellations performed within the social VR platform, Mozilla Hubs The piece was created to solve a solution where several students located in 4 time zones were able to come together to create an alternative theatre experience which used ubiquitous technology solutions such as Mozilla Hubs to allow them to perform remotely via an avatar (Simpson 2020). The audience were asked to participate as avatars in a digital theatre piece that resem- bled the model used by immersive theatre practitioners such as Punchdrunk, Complicit\u00e9 and The Builders Association (Biggin and Biggin 2017; Harvie and Lavender 2010). They watched a production whose narrative was based on the idea of the multiverse whilst themselves experiencing a multiverse of themselves and their performance as they engaged with the story as another persona. They became digital humans who, under the protection of their relative anonymity as avatars, engaged with the narrative more proactively, incorporating their own actions into the narrative (Broadhurst and Price 2017; Hayles 1999; Parker-Starbuck 2011). This agency of audiences in theatre is common, particularly in the immersive theatre sector and the behaviour of audiences in this production created a notion that audiences engage in digital mediums in the same way as physical once you overcome the interface you use to access it (Breel 2015).","An Industry in Crisis: Virtual Mediums for Theatre and Live Performance 289 The same was discovered by the design team who learned that when a solution in the digital system wasn\u2019t clear, either creatively or technically, using existing theatrical practice usually led to a digital solution to the problem. This builds on existing research by the author on the subject of immersive digital visualisation through the use of AR and VR solutions where design teams would investigate solutions to physical problems using digital tools to understand them (Carver and Beardon 2004; Lopes Ramos et al. 2020). Although this project was designed to solve a problem caused by the pandemic, the work inspired new ways of thinking about virtual performance through avatars with limited ranges of movements, as well as a better understanding of behaviours of typical theatre audiences in virtual mediums. These were: \u2022 Work with the performers early on in the medium and spend time \ufb01nding ways to perform, such as measuring how to get eye contact, how to time movements to relate to the dialogue and what level of voice performance supports the range of movement the avatar can create. \u2022 Assume the audience doesn\u2019t know how to use technology and therefore keep the experience as simple as possible with clear explanations and lots of guides built in. \u2022 Onboard the audience to the experience with instructions that are built into the story but clearly able to explain the systems. \u2022 Explore the range of opportunities that virtual mediums provide, such as pre-recorded content, 360 videos and renderings, and interaction points for the audience. This helps balance the losses audiences experience when they compare the experience to a physical theatre experience. \u2022 This medium is closer to a computer game than it is to physical theatre, so don\u2019t try to build a piece of live theatre and assume it will translate. 5.2 Break A Leg Break A Leg is a virtual platform that allows amateur and community theatre groups to continue to engage in social theatre making using a web-app. Users upload videos of themselves singing to a backing track which is then synced with other singers and presented on a real time engine scene for at home consumption through a browser. The research in this piece focuses on the market, audience expectations and critically, the size and state of the amateur dramatics industry (Simpson 2021) (Fig. 2). The UK amateur dramatics industry, if measured by turnover and ticket sales, exceeds the professional industry. There are 2,500 amateur theatre groups registered in the UK, producing more than 30,000 productions a year and 40,000 choirs with over 2,000,000 members. The UKs choral singers exceeds the number of amateur and social football players, yet it doesn\u2019t receive the funding or support of it\u2019s professional industry in the way the football community does (Chorus America 2021; Noda 2020).","290 J. Simpson Fig. 2. Hairspray performed within Unreal Engine on the platform, Break A Leg An informal survey of 1,000 professional technicians and designers in the theatre and events industry found that over 600 of them claim to have found their interest in their career by engaging with amateur dramatics when they were younger. For performers this number is almost 100%. Break A Leg was created to address the lack of support to the grass roots industry and provide a way for this market to digitally transform into a new vertical that could persist beyond the pandemic. This project ended up being a solution to a pandemic situation which, fortunately, was not needed in the long run. However, it did create new discoveries which informed best practice when developing virtual theatre systems. These included: \u2022 2D \ufb02at videos perform well in a spatial 3D environment and are a suitable, low cost solution to high quality performance content over motion capture avatars and low-budget volumetric capture solutions. \u2022 Performers engage with the medium better when performing with each other and under the guidance of a digital dramaturg3. \u2022 Creative teams need to design for the medium and consider the audience will be watching a piece of theatre through a device that doesn\u2019t behave the same way as a theatre venue. \u2022 The business model for consumer, virtual, live entertainment doesn\u2019t exist yet but is growing steadily in the freemium4 market. 3 A dramaturg in theatre is a role that ensures continuity of the performers, the narrative and the design, often representing the authors intentions. A digital dramaturg is a term invented at Copper Candle to describe someone who ensures these elements are represented appropriately in the virtual medium. 4 Freemium offers free content with optional premium content available under a paywall or subscription model.","An Industry in Crisis: Virtual Mediums for Theatre and Live Performance 291 5.3 The Round The \ufb01nal piece of research is for a solution to the \u201clive\u201d problem by live-streaming motion capture data from a studio or theatre space directly to users phones at home for an AR experience. This project called The Round uses innovative solutions to manipulate and control real time content to any device in the world from a single remote location using traditional theatre lighting control systems. This allows \u201clive\u201d theatre to be achieved with a single studio and existing practitioners who don\u2019t need to upgrade their practice to create theatre experiences (Winsor and Brignall 2021) (Fig. 3). Fig. 3. Motion Capture performance rehearsal at Rose Bruford College as part of Innovate UK funded project, The Round This research will allow this paper to explore the nature of \u201cliveness\u201d, it\u2019s relevance and visibility to audiences who have grown used to on-demand entertainment at home as well as exploring the necessity of upskilling traditional theatre professionals to work in real-time and virtual content creation tools (Bolter and Grusin 2000; Hann 2018; Hansen 2006). As context to these approaches of virtual theatre making, research has been under- taken by looking, and working on, other formats of virtual theatre created before and after the pandemic including the RSCs production of \u201cThe Dream\u201d, Marshmellos con- cert in Fortnite, The Tempest presented in \u201cThe Under Presents\u201d and many of the digital streams provided via Zoom, Youtube and subscriber platforms such as Disney+ used by mainstream and commercial theatre producers. Although the Round was developed for a speci\ufb01c medium (mobile Augmented Real- ity), it allowed discoveries to be made that led to new research which has been discussed in this paper, such as the performer centric bias and liveness in virtual theatre. In addition the project developed these new ideas: \u2022 Performance continuity is hard to achieve technically until there is a solution to synchronising physical performance with the digital streams.","292 J. Simpson \u2022 Designing for the medium is essential as a theatrical production doesn\u2019t translate naturally into XR technology, particularly mobile AR. \u2022 Having an existing theatrical creative team develop the piece is possible providing they are supported by a digital dramaturg to help the transition into virtual worlds. \u2022 Even with high quality digital avatars and advanced facial motion capture, the uncanny valley disrupts the audiences relationship with the character. \u2022 All development work, including early character development and design ideas should be introduced to the \ufb01nal medium as soon as possible in the process to ensure they are viable. 6 Conclusion This paper discusses the social and professional implications of a new live entertainment sector. Arguably, it can\u2019t be called theatre or games and instead needs to adopt a new term that appropriately represents its place in the new metaverse and sits alongside live streaming services and digital theatre alike. There are challenges that need to be addresses to support this sectors emergence and whilst the uptake during the pandemic was high due to the need for virtual to replace physical mediums, there are still issues that need to be considered before it can emerge as a creative sector in its own right. The performer centric bias and the considerations around liveness are a big part of this as it encourages mis-guided development of creative works to assume that the rhetoric of live and performer \ufb01rst are the correct ones. In a medium that is highly technical and requires creativity from a range of disciplines, it is important that collaboration in this new sector becomes \ufb02atter and more democratic than live performance has allowed in the past. A common theme in all three of the case studies presented is to design for the medium and not assume that work created in alternative live mediums will translate effectively. The role of a digital dramaturg will be an important part of assisting existing artists to translate their work across sectors and could become a dominant role in the future of live, virtual performances. Acknowledgements. The Innovate UK consortium partners; Rose Bruford College, Copper Candle, The Round and Condense Reality. Rose Bruford Colleges students in Collaborative Theatre Making (MFA). References Auslander, P.: Liveness: Performance in a Mediatized Culture, p. 179 (1999) Biggin, R.: Interactivity and immersion: theoretical approaches. In: Immersive Theatre and Audi- ence Experience, pp. 59\u201377. Springer, Cham (2017). https:\/\/doi.org\/10.1007\/978-3-319-620 39-8_2 Bolter, J.D., Grusin, R.A.: Remediation: Understanding New Media. MIT Press, Cambridge (2000) Breel, A.: Audience agency in participatory performance. Participations: J. Audience Reception Stud. 12(1), 368\u2013387 (2015)","An Industry in Crisis: Virtual Mediums for Theatre and Live Performance 293 Broadhurst, S., Price, S.: Digital bodies: creativity and technology in the arts and humanities (2017) Carver, G., Beardon, C.: New Visions in Performance: The Impact of Digital Technologies. Swets & Zeitlinger Publishers, Leiden (2004) Chorus America: Articles\u2014Chorus America. Chorus America (2021). https:\/\/www.chorusame rica.org\/explore\/articles Hann, R.: Beyond scenography (2018) Hansen, M.B.N.: Bodies in code: interfaces with digital media. In: Bodies in Code: Interfaces with Digital Media. Routledge, Taylor & Francis Group (2006). https:\/\/doi.org\/10.4324\/978020394 2390 Harvie, J., Lavender, A.: Making contemporary theatre. In: Theatre Theory Practice Performance (2010) Hayles, K.: How We Became Posthuman: Virtual Bodies in Cybernetics, Literature, and Informatics. University of Chicago Press, Chicago (1999) Lopes Ramos, J., Dunne-Howrie, J., Maravala, P.J., Simon, B.: The post-immersive manifesto. Int. J. Perform. Arts Digit. Med. 16, 1\u201317 (2020). https:\/\/doi.org\/10.1080\/14794713.2020.176 6282 Mitchell, K.: The Director\u2019s Craft: A Handbook for the Theatre. Routledge, Abingdon (2009) Noda: NODA. Noda (2020). https:\/\/www.noda.org.uk\/ Parker-Starbuck, J.: Cyborg theatre. In: Cyborg Theatre. Palgrave Macmillan, UK (2011). https:\/\/ doi.org\/10.1057\/9780230306523 Phelan, P.: Unmarked: the politics of performance, p. 207 (1993) Phelan, P.: Performance, live culture and things of the heart. J. Vis. Cult. 2(3), 291\u2013302 (2003). https:\/\/doi.org\/10.1177\/1470412903002003002 Simpson, J.: Immersive technologies: digital transformation of theatre. Protocol 25(4), 22\u201327 (2020) Simpson, J.: Break a leg (2021). https:\/\/www.breakaleg.club\/ Winsor, T., Brignall, P.: Reality Check Productions (2021). https:\/\/www.realitycheckproductions. co.uk\/the-round","AR Development and Design","Integration of Smart Glasses for Knowledge Transfer in Industrial Remote Maintenance: Learnings from Practice Anuja Hariharan(B), Monika Risling, Artur Felic, and Tobias Ostertag Future Labs, CAS Software AG, Karlsruhe, Germany {anuja.hariharan,artur.felic,Tobias.ostertag}@cas.de, [email protected] Abstract. The use of remote augmented reality (AR) based guidance is seen as an opportunity to address the growing complexity of industrial tasks. In this paper, we look into the question of how remote AR support using smart glasses reduces the maintenance time of complex machinery, and factors that in\ufb02uence their accep- tance for productive use. A design science approach was adopted to develop a prototype for a state-of-the-art smart glass for a German small and medium Enter- prise (SME) with complex machinery. An evaluation was conducted with 2 indus- tries (one with low and the other with high requirement of AR technology). First, we found that using a remote maintenance application, a time reduction of up to 46% was expected to be achieved. Second, despite its novelty, guidance via AR is accepted better when data privacy concerns are addressed, and when the enterprise has a higher requirement for AR maintenance. Keywords: Human computer interaction \u00b7 Remote maintenance \u00b7 Augmented Reality glasses \u00b7 Suitability of AR 1 Introduction Knowledge transfer is de\ufb01ned as \u201cthe conveyance of knowledge from one place, person, system or ownership to another\u201d (Liyanage et al. 2009). Service and maintenance pro- cesses in particular require ef\ufb01cient knowledge transfer methods, where knowledge is rarely documented and important skills are limited to a few people (Webel et al. 2013; Zhou et al. 2008). The current worldwide pandemic has necessitated remote digital ser- vice processes & collaborative digital tools. These tools should enable information and knowledge exchange between employees of different levels of expertise (Wang et al. 2014) as well as achieve remote services seamlessly with available personnel on the customer-end, where conducting service procedures with own maintenance personnel is time-consuming and expensive. Augmented Reality (AR) smart glasses have been identi\ufb01ed as a powerful supporting technology in various tasks such as product design, assembly, maintenance, quality con- trol and material handling (Syberfeldt et al. 2017). In the area of maintenance, the crucial \u00a9 The Author(s), under exclusive license to Springer Nature Switzerland AG 2023 T. Jung et al. (Eds.): XR 2022, SPBE, pp. 297\u2013308, 2023. https:\/\/doi.org\/10.1007\/978-3-031-25390-4_26","298 A. Hariharan et al. advantages of smart glasses lie in moving away from hands-held devices to head-mounted devices and an alternative control system by voice (Quint et al. 2017; Schulzrinne 2018). Del Amo et al. (2018) present a systematic review to identify the relations between AR visualization techniques in authoring, context-awareness and interaction-analysis in the context of maintenance applications, to identify which knowledge transfer goals can be achieved by AR. According to Fu et al. (2010), remote maintenance allows a time- saving to the traditional process of 60% to 90%. Despite these exceeding advantages reported in research, adoption of AR in practice by manufacturing companies has been slow (Syberfeldt et al. 2017). In this project we investigate the usability and acceptance aspects of Augmented Reality glasses for maintenance processes of a German manufacturing SME. A voice- controlled augmented reality application was designed, developed, and evaluated. Eval- uation results of employees of the manufacturing SME with real-time maintenance use cases are presented. The paper concludes with an outlook and implications for SME\u2019s in adopting smart glass technology for maintenance in practice. 2 State of the Art This work outlines a new attempt to develop an application to support untrained workers in maintenance and repair tasks on complex machines and machining centres. Current applications that are similar to the goal of this work, and the research gap in remote maintenance with AR devices are presented in the following. 2.1 Overview of Existing AR Maintenance Applications Early developments of AR technologies, can be seen in the 1960s where computer graphics were displayed through a head-mounted device (Sutherland 1968). Since then, AR has gained a great amount of popularity from theoretical scienti\ufb01c work to everyday use in smartphones (Schulzrinne 2018). Particularly, AR smart glasses are increasing in popularity as an important technology in the smart factories of the future (Syberfeldt et al. 2017). AR smart glasses have been identi\ufb01ed as a powerful supporting technology in various tasks such as assembly, maintenance, quality control and material handling (Syberfeldt et al. 2017). Studies reveal general proof of concept and successful results on productivity and quality because of smart glasses (Paelke 2014; Syberfeldt et al. 2016). A great number of AR devices are marketed as general customer products or as design of\ufb01ce products (Syberfeldt et al. 2017). In the area of maintenance, several studies elaborated approaches to improve maintenance tasks (Quint et al. 2017; Schulzrinne 2018). According to Fu et al. (2010), remote maintenance allows time-saving to the traditional process of 60% to 90%. Despite these advantages, development of smart glass applications seldom consider the maintenance processes in companies into account and hence only a few AR solutions are being used in manufacturing processes (Syberfeldt et al. 2016; Syberfeldt et al. 2017). One example of a technology used is the marker-based AR application by the German enterprise Schaef\ufb02er Technologies AG & Co. KG (Schaef\ufb02er1). It illustrates 1 https:\/\/www.schaef\ufb02er.com\/."]
Search
Read the Text Version
- 1
- 2
- 3
- 4
- 5
- 6
- 7
- 8
- 9
- 10
- 11
- 12
- 13
- 14
- 15
- 16
- 17
- 18
- 19
- 20
- 21
- 22
- 23
- 24
- 25
- 26
- 27
- 28
- 29
- 30
- 31
- 32
- 33
- 34
- 35
- 36
- 37
- 38
- 39
- 40
- 41
- 42
- 43
- 44
- 45
- 46
- 47
- 48
- 49
- 50
- 51
- 52
- 53
- 54
- 55
- 56
- 57
- 58
- 59
- 60
- 61
- 62
- 63
- 64
- 65
- 66
- 67
- 68
- 69
- 70
- 71
- 72
- 73
- 74
- 75
- 76
- 77
- 78
- 79
- 80
- 81
- 82
- 83
- 84
- 85
- 86
- 87
- 88
- 89
- 90
- 91
- 92
- 93
- 94
- 95
- 96
- 97
- 98
- 99
- 100
- 101
- 102
- 103
- 104
- 105
- 106
- 107
- 108
- 109
- 110
- 111
- 112
- 113
- 114
- 115
- 116
- 117
- 118
- 119
- 120
- 121
- 122
- 123
- 124
- 125
- 126
- 127
- 128
- 129
- 130
- 131
- 132
- 133
- 134
- 135
- 136
- 137
- 138
- 139
- 140
- 141
- 142
- 143
- 144
- 145
- 146
- 147
- 148
- 149
- 150
- 151
- 152
- 153
- 154
- 155
- 156
- 157
- 158
- 159
- 160
- 161
- 162
- 163
- 164
- 165
- 166
- 167
- 168
- 169
- 170
- 171
- 172
- 173
- 174
- 175
- 176
- 177
- 178
- 179
- 180
- 181
- 182
- 183
- 184
- 185
- 186
- 187
- 188
- 189
- 190
- 191
- 192
- 193
- 194
- 195
- 196
- 197
- 198
- 199
- 200
- 201
- 202
- 203
- 204
- 205
- 206
- 207
- 208
- 209
- 210
- 211
- 212
- 213
- 214
- 215
- 216
- 217
- 218
- 219
- 220
- 221
- 222
- 223
- 224
- 225
- 226
- 227
- 228
- 229
- 230
- 231
- 232
- 233
- 234
- 235
- 236
- 237
- 238
- 239
- 240
- 241
- 242
- 243
- 244
- 245
- 246
- 247
- 248
- 249
- 250
- 251
- 252
- 253
- 254
- 255
- 256
- 257
- 258
- 259
- 260
- 261
- 262
- 263
- 264
- 265
- 266
- 267
- 268
- 269
- 270
- 271
- 272
- 273
- 274
- 275
- 276
- 277
- 278
- 279
- 280
- 281
- 282
- 283
- 284
- 285
- 286
- 287
- 288
- 289
- 290
- 291
- 292
- 293
- 294
- 295
- 296
- 297
- 298
- 299
- 300
- 301
- 302
- 303
- 304
- 305
- 306
- 307
- 308
- 309
- 310
- 311
- 312
- 313
- 314
- 315
- 316
- 317
- 318
- 319
- 320
- 321
- 322
- 323
- 324
- 325
- 326
- 327
- 328
- 329
- 330
- 331
- 332
- 333
- 334
- 335
- 336
- 337
- 338
- 339
- 340
- 341
- 342
- 343
- 344
- 345
- 346
- 347
