The Evolution and Maturation of Team in Organizations

Harvey et al. A Systems View of Team Learning Climate business schools. In general, this should lead managers to present in TLC. We hope that our efforts in this paper offers the better appreciate the complexity of their impact and reduce the opportunity for scholars to take more of a systems view in their impression of direct connectedness between their actions and the research on TLC, and for leaders to embrace the complex, yet desired outcomes. crucial, role they play in continuously shaping team members’ beliefs. This is all very challenging, but the rewards are well worth Thinking of TLC as an equilibrium that needs balance also it, as teams continue to flourish in science and in the field. brings the notion of time to the fore. It moves away from the perception of TLC as a starting point or a definite state DATA AVAILABILITY represented as an intrinsic dialectical quality (learning vs. non- learning climate). Managers can then better understand why TLC No datasets were generated or analyzed for this study. is never a fait accompli and rather an enduring accomplishment that revolves around managing several emergent states over time. AUTHOR CONTRIBUTIONS Going back to Senge (1990), this is at the foundation of the reflexivity and inquiry skills necessary for organizations to thrive over the long haul. CONCLUSION JFH and MC developed the research idea and wrote most of the manuscript. PML assisted them on parts of the manuscript, Team scholarship has primarily focused on emergent states in particularly the literature review. isolation, limiting our understanding of the proper “milieu” among them or our insights into how they operate jointly. FUNDING Therefore, it is not immediately apparent how the various emergent states differ from each other, or where they overlap This research was supported by funding from the Social Sciences (Bell et al., 2012). This has led to scholarship that does not always and Humanities Research Council of Canada (Grant No. 430- take into account the complexity of the bundle of emergent states 2017-00527). REFERENCES Phenomenology of Groups and Group Membership, eds H. Sondak, M. Neale, and E. Mannix (Bingley: Emerald Group Publishing), 49–84. Alexander, L., and Van Knippenberg, D. (2014). Teams in pursuit of radical Bunderson, J. S., and Sutcliffe, K. M. (2003). Management team learning innovation: a goal orientation perspective. Acad. Manag. Rev. 39, 423–438. orientation and business unit performance. J. Appl. Psychol. 88, 552–560. Button, S. B., Mathieu, J. E., and Zajac, D. M. (1996). Goal orientation in Ancona, D. G., and Caldwell, D. F. (1992). Bridging the boundary: external activity organizational research: a conceptual and empirical foundation. Organ. Behav. and performance in organizational teams. Adm. Sci. Q. 37, 634–665. Hum. Decis. Process. 67, 26–48. Carmeli, A., and Gittell, J. H. (2009). High-quality relationships, psychological Argote, L., Aven, B. L., and Kush, J. (2018). The effects of communication networks safety, and learning from failures in work organizations. J. Organ. Behav. 30, and turnover on transactive memory and group performance. Organ. Sci. 29, 709–729. 191–206. Chen, G., and Tjosvold, D. (2012). Shared rewards and goal interdependence for psychological safety among departments in China. Asia Pac. J. Manag. 29, Argyris, C. (1990). Overcoming Organizational Defenses: Facilitating 433–452. Organizational Learning. Boston, MA: Allyn and Bacon. Chen, G., Tjosvold, D., Zhao, H., Ning, N., and Fu, Y. (2011). Constructive controversy for learning and team effectiveness in China. Asia Pac. J. Hum. Bandura, A. (1997). Self-Efficacy: The Exercise of Control. New York, NY: W. H. Resou. 49, 88–104. doi: 10.1177/1038411110391708 Freeman. Chiniara, M., and Bentein, K. (2018). The servant leadership advantage: when perceiving low differentiation in leader-member relationship quality influences Beal, D. J., Cohen, R. R., Burke, M. J., and McLendon, C. L. (2003). Cohesion team cohesion, team task performance and service OCB. Leadersh. Q. 29, and performance in groups: a meta-analytic clarification of construct relations. 333–345. J. Appl. Psychol. 88, 989–1004. Cronin, M. A., Gonzalez, C., and Sterman, J. D. (2009). Why don’t well-educated adults understand accumulation? A challenge to researchers, educators, and Bell, B. S., Kozlowski, S. W., and Blawath, S. (2012). “Team learning: a citizens. Organ. Behav. Hum. Decis. Process. 108, 116–130. theoretical integration and review,” in The Oxford Handbook of Organizational Cronin, M. A., and Vancouver, J. B. (2018). “The only constant is change: Psychology, Vol. 1, ed. S. W. Kozlowski (Oxford: Oxford University Press), expanding theory by incorporating dynamic properties into one’s models,” in 859–909. The Handbook of Multilevel Theory, Measurement, and Analysis, eds S. E. Humphrey and J. M. LeBreton (Washington, DC: American Psychological Bergman, J. Z., Rentsch, J. R., Small, E. E., Davenport, S. W., and Bergman, S. M. Association), 89–114. (2012). The shared leadership process in decision-making teams. J. Soc. Psychol. Cronin, M. A., Weingart, L. R., and Todorova, G. (2011). Dynamics in groups: are 152, 17–42. we there yet? Acad. Manag. Ann. 5, 571–612. De Jong, B. A., and Elfring, T. (2010). How does trust affect the performance of Bowers, C., Kreutzer, C., Cannon-Bowers, J., and Lamb, J. (2017). Team resilience ongoing teams? The mediating role of reflexivity, monitoring, and effort. Acad. as a second-order emergent state: a theoretical model and research directions. Manag. J. 53, 535–549. Front. Psychol. 8:1360. doi: 10.3389/fpsyg.2017.01360 DeShon, R. P., Kozlowski, S. W. J., Schmidt, A. M., Milner, K. R., and Wiechmann, D. (2004). A multiple-goal, multilevel model of feedback effects on the Bresman, H. (2010). External learning activities and team performance: a regulation of individual and team performance. J. Appl. Psychol. 89, 1035–1056. multimethod field study. Organ. Sci. 21, 81–96. Bresman, H., and Zellmer-Bruhn, M. (2013). The structural context of team learning: effects of organizational and team structure on internal and external learning. Organ. Sci. 24, 1120–1139. Bunderson, J. S., and Boumgarden, P. (2010). Structure and learning in self- managed teams: why “bureaucratic” teams can be better learners. Organ. Sci. 21, 609–624. Bunderson, J. S., and Sutcliffe, K. M. (2002). “Why some teams emphasize learning more than others: evidence from business unit management teams,” in Toward Frontiers in Psychology | www.frontiersin.org 11908 July 2019 | Volume 10 | Article 1441

Harvey et al. A Systems View of Team Learning Climate Dragoni, L., and Kuenzi, M. (2012). Better understanding work unit goal Harvey, J. F., Bresman, H., and Edmondson, A. C. (2018). Team Learning orientation: its emergence and impact under different types of work unit Capabilities: A Meso Model of Sustained Innovation and Superior Firm structure. J. Appl. Psychol. 97, 1032–1048. doi: 10.1037/a0028405 Performance. Harvard Business School Technology & Operations Mgt. Unit Working Paper No. 19–059. Duhigg, C. (2016). What Google Learned from its Quest to Build the Perfect Team, Vol. 26. New York, NY: The New York Times. Harvey, J. F., Johnson, K. J., Roloff, K. S., and Edmondson, A. C. (in press). From orientation to behavior: the interplay between learning orientation, Dweck, C. S. (1986). Motivational processes affecting learning. Am. Psychol. 41, open-mindedness, and psychological safety in team learning. Hum. Relat. 1040–1048. Hinsz, V. B., Tindale, R. S., and Vollrath, D. A. (1997). The emerging Edmondson, A. C. (1999). Psychological safety and learning behavior in work conceptualization of groups as information processors. Psychol. Bull. 121, teams. Adm. Sci. Q. 44, 350–383. doi: 10.1186/1748-5908-7-77 43–64. Edmondson, A. C. (2002). The local and variegated nature of Huber, G. P. (1991). Organizational learning: the contributing processes and the learning in organizations: a group-level perspective. Organ. Sci. 13, literatures. Organ. Sci. 2, 88–115. 128–146. Ilgen, D. R., Hollenbeck, J. R., Johnson, M., and Jundt, D. (2005). Teams in Edmondson, A. C. (2003). Speaking up in the operating room: how team leaders organizations: from input-process-output models to IMOI models. Annu. Rev. promote learning in interdisciplinary action teams. J. Manag. Stud. 40, 1419– Psychol. 56, 517–543. 1452. Janis, I. L. (1972). Victims of Groupthink: A Psychological Study of Foreign-Policy Edmondson, A. C. (2018). The Fearless Organization: Creating Psychological Safety Decisions and Fiascoes. Boston, MA: Houghton Mifflin. in the Workplace for Learning, Innovation, and Growth. Hoboken, NJ: Wiley. Kahn, W. A. (1990). Psychological conditions of personal engagement and Edmondson, A. C., Bohmer, R. M., and Pisano, G. P. (2001). Disrupted routines: disengagement at work. Acad. Manag. J. 33, 692–724. team learning and new technology implementation in hospitals. Adm. Sci. Q. 46, 685–716. Kane, T. D., Zaccaro, S. J., Tremble, T. R. Jr., and Masuda, A. D. (2002). An examination of the leader’s regulation of groups. Small Group Res. 33, 65–120. Edmondson, A. C., Dillon, J. R., and Roloff, K. S. (2007). 6 three perspectives on team learning: outcome improvement, task Mastery, and group process. Acad. Klein, K. J., and Kozlowski, S. W. (2000). Multilevel Theory, Research, and Methods Manag. Ann. 1, 269–314. in Organizations: Foundations, Extensions, and New Directions. San Francisco, CA: Jossey-Bass. Edmondson, A. C., and Harvey, J. F. (2017). Extreme Teaming: Lessons in Complex, Cross-Sector Leadership. Bingley: Emerald Publishing Limited. Koeslag-Kreunen, M., Van den Bossche, P., Hoven, M., Van der Klink, M., and Gijselaers, W. (2018). When leadership powers team learning: a meta-analysis. Edmondson, A. C., and Harvey, J. F. (2018). Cross-boundary teaming for Small Group Res. 49, 475–513. doi: 10.1177/1046496418764824 innovation: integrating research on teams and knowledge in organizations. Hum. Resour. Manag. Rev. 28, 347–360. Kozlowski, S. W. J. (2015). Advancing research on team process dynamics: theoretical, methodological, and measurement considerations. Organ. Psychol. Edmondson, A. C., and Lei, Z. (2014). Psychological safety: the history, renaissance, Rev. 5, 270–299. and future of an interpersonal construct. Annu. Rev. Organ. Psychol. Organ. Behav. 1, 23–43. Kozlowski, S. W. J., and Chao, G. T. (2012). The dynamics of emergence: cognition and cohesion in work teams. Manage. Decis. Econ. 33, 335–354. Edmondson, A. C., Winslow, A. B., Bohmer, R. M., and Pisano, G. P. (2003). Learning how and learning what: effects of tacit and codified knowledge Kozlowski, S. W. J., and Chao, G. T. (2018). Unpacking team process dynamics on performance improvement following technology adoption. Decis. Sci. 34, and emergent phenomena: challenges, conceptual advances, and innovative 197–224. methods. Am. Psychol. 73, 576–592. doi: 10.1037/amp0000245 Ellis, A. P., Hollenbeck, J. R., Ilgen, D. R., Porter, C. O., West, B. J., and Moon, Lee, F., Edmondson, A. C., Thomke, S., and Worline, M. (2004). The mixed effects H. (2003). Team learning: collectively connecting the dots. J. Appl. Psychol. 88, of inconsistency on experimentation in organizations. Organ. Sci. 15, 310–326. 821–835. LePine, J. A. (2005). Adaptation of teams in response to unforeseen change: effects Ericksen, J., and Dyer, L. (2004). Right from the start: exploring the effects of early of goal difficulty and team composition in terms of cognitive ability and goal team events on subsequent project team development and performance. Adm. orientation. J. Appl. Psychol. 90, 1153–1167. Sci. Q. 49, 438–471. LePine, J. A., Piccolo, R. F., Jackson, C. L., Mathieu, J. E., and Saul, J. R. (2008). Forrester, J. W. (1968). Principles of System. Cambridge, MA: Productivity Press. A meta-analysis of teamwork processes: tests of a multidimensional model and Gibson, C. B., and Earley, P. C. (2007). Collective cognition in action: relationships with team effectiveness criteria. Pers. Psychol. 61, 273–307. accumulation, interaction, examination, and accommodation in the Lester, S. W., Meglino, B. M., and Korsgaard, M. A. (2002). The antecedents and development and operation of group efficacy beliefs in the workplace. consequences of group potency: a longitudinal investigation of newly formed Acad. Manag. Rev. 32, 438–458. work groups. Acad. Manag. J. 45, 352–368. Gist, M. E. (1987). Self-efficacy: implications for organizational behavior and human resource management. Acad. Manag. Rev. 12, 472–485. Lewis, K. (2004). Knowledge and performance in knowledge-worker teams: Gong, Y., Kim, T. Y., Lee, D. R., and Zhu, J. (2013). A multilevel model of team goal a longitudinal study of transactive memory systems. Manag. Sci. 50, orientation, information exchange, and creativity. Acad. Manag. J. 56, 827–851. 1519–1533. Gully, S. M., Incalcaterra, K. A., Joshi, A., and Beaubien, J. M. (2002). A meta- analysis of team-efficacy, potency, and performance: interdependence and Liang, D. W., Moreland, R., and Argote, L. (1995). Group versus individual training level of analysis as moderators of observed relationships. J. Appl. Psychol. 87, and group performance: the mediating role of transactive memory. Pers. Soc. 819–832. Psychol. Bull. 21, 384–393. Hackman, J. R. (1976). “Group influence on individuals,” in Handbook of Industrial and Organizational Psychology, ed. M. D. Dunnette (Chicago, IL: Rand- Lott, A. J., and Lott, B. E. (1965). Group cohesiveness as interpersonal attraction: a McNally), 1455–1525. review of relationships with antecedent and consequent variables. Psychol. Bull. Hackman, J. R. (1990). Groups That Work (and Those That Don’t): Creating 64, 259–309. Conditions for Effective Teamwork. San Francisco, CA: Jossey-Bass. Hackman, J. R., and Morris, C. G. (1975). Group tasks, group interaction process, Mach, M., Dolan, S., and Tzafrir, S. (2010). The differential effect of team members’ and group performance effectiveness: a review and proposed integration. Adv. trust on team performance: the mediation role of team cohesion. J. Occup. Exp. Soc. Psychol. 8, 45–99. Organ. Psychol. 83, 771–794. Hackman, J. R., and Walton, R. E. (1986). “Leading groups in organizations,” in Designing Effective Work Groups, ed. P. S. Goodman (San Francisco, CA: Marks, M. A., Mathieu, J. E., and Zaccaro, S. J. (2001). A temporally based Jossey-Bass), 72–119. framework and taxonomy of team processes. Acad. Manag. Rev. 26, 356–376. Hammedi, W., van Riel, A. C., and Sasovova, Z. (2013). Improving screening decision making through transactive memory systems: a field study. J. Prod. Mathieu, J. E., Kukenberger, M. R., D’innocenzo, L., and Reilly, G. (2015). Modeling Innov. Manag. 30, 316–330. reciprocal team cohesion–performance relationships, as impacted by shared leadership and members’ competence. J. Appl. Psychol. 100, 713–734. Mathieu, J. E., Luciano, M. M., D’Innocenzo, L., Klock, E. A., and LePine, J. A. (in press). The development and construct validity of a team processes survey measure. Organ. Res. Methods McGrath, J. E. (1964). Social Psychology: A Brief Introduction. New York, NY: Holt Rinehart & Winston. Frontiers in Psychology | www.frontiersin.org 11919 July 2019 | Volume 10 | Article 1441

Harvey et al. A Systems View of Team Learning Climate Mehta, A., Feild, H., Armenakis, A., and Mehta, N. (2009). Team goal orientation Staw, B. M., Sandelands, L. E., and Dutton, J. E. (1981). Threat rigidity and team performance: the mediating role of team planning. J. Manag. 35, effects in organizational behavior: a multilevel analysis. Adm. Sci. Q. 26, 1026–1046. 501–524. Mehta, A., and Mehta, N. (2018). Knowledge integration and team effectiveness: a Sterman, J. D. (2000). Business Dynamics: Systems Thinking and Modeling for a team goal orientation approach. Decis. Sci. 49, 445–486. Complex World. New York, NY: Irwin. Moreland, R. L., Argote, L., and Krishnan, R. (1996). “Socially shared cognition Tasa, K., and Whyte, G. (2005). Collective efficacy and vigilant problem solving in at work: transactive memory and group performance,” in What’s Social About group decision making: a non-linear model. Organ. Behav. Hum. Decis. Process. Social Cognition? Research on Socially Shared Cognition in Small Groups, eds 96, 119–129. J. L. Nye and A. M. Brower (Thousand Oaks, CA: Sage), 57–84. van Ginkel, W. P., and van Knippenberg, D. (2008). Group information elaboration Morgeson, F. P., DeRue, D. S., and Karam, E. P. (2010). Leadership in teams: and group decision making: the role of shared task representations. Organ. a functional approach to understanding leadership structures and processes. Behav. Hum. Decis. Process. 105, 82–97. J. Manag. 36, 5–39. Vancouver, J. B. (2005). The depth of history and explanation as benefit and bane Nembhard, I. M., and Edmondson, A. C. (2006). Making it safe: the effects of leader for psychological control theories. J. Appl. Psychol. 90, 38–52. inclusiveness and professional status on psychological safety and improvement efforts in health care teams. J. Organ. Behav. 27, 941–966. Vancouver, J. B., Thompson, C. M., Tischner, E. C., and Putka, D. J. (2002). Two studies examining the negative effect of self-efficacy on performance. J. Appl. Newman, A., Donohue, R., and Eva, N. (2017). Psychological safety: a systematic Psychol. 87, 506–516. review of the literature. Hum. Resour. Manag. Rev. 27, 521–535. VandeWalle, D. (1997). Development and validation of a work domain goal Otte, K. P., Konradt, U., Garbers, Y., and Schippers, M. C. (2017). Development orientation instrument. Educ. Psychol. Meas. 57, 995–1015. and validation of the REMINT: a reflection measure for individuals and teams. Eur. J. Work Organ. Psychol. 26, 299–313. Waller, M. J., Okhuysen, G. A., and Saghafian, M. (2016). Conceptualizing emergent states: a strategy to advance the study of group dynamics. Acad. Pescosolido, A. T. (2001). Informal leaders and the development of group efficacy. Manag. Ann. 10, 561–598. Small Group Res. 32, 74–93. West, M. A. (1996). “Reflexivity and work group effectiveness: a conceptual Porter, C. O. (2005). Goal orientation: effects on backing up behavior, performance, integration,” in Handbook of Work Group Psychology, ed. M. A. West efficacy, and commitment in teams. J. Appl. Psychol. 90, 811–818. (Chichester: Wiley), 555–579. Rapp, T. L., Bachrach, D. G., Rapp, A. A., and Mullins, R. (2014). The role of team Whyte, G. (1998). Recasting Janis’s groupthink model: the key role of collective goal monitoring in the curvilinear relationship between team efficacy and team efficacy in decision fiascoes. Organ. Behav. Hum. Decis. Process. 73, 185–209. performance. J. Appl. Psychol. 99, 976–987. doi: 10.1037/a0036978 Wilson, J. M., Goodman, P. S., and Cronin, M. A. (2007). Group learning. Acad. Reagans, R., Argote, L., and Brooks, D. (2005). Individual experience and Manag. Rev. 32, 1041–1059. experience working together: predicting learning rates from knowing who knows what and knowing how to work together. Manag. Sci. 51, 869–881. Wong, S. S. (2004). Distal and local group learning: performance trade-offs and tensions. Organ. Sci. 15, 645–656. Sanner, B., and Bunderson, J. S. (2015). When feeling safe isn’t enough: contextualizing models of safety and learning in teams. Organ. Psychol. Rev. 5, Zhang, Y., Olenick, J., Chang, C. H., Kozlowski, S. W. J., and Hung, H. (2018). 224–243. TeamSense: assessing personal affect and group cohesion in small teams through dyadic interaction and behavior analysis with wearable sensors. Proc. Schippers, M. C., Den Hartog, D. N., Koopman, P. L., and van Knippenberg, D. ACM Interact. Mob. Wearable Ubiquitous Technol. 2, 1–22. (2008). The role of transformational leadership in enhancing team reflexivity. Hum. Relat. 61, 1593–1616. Conflict of Interest Statement: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could Senge, P. (1990). The Fifth Discipline: The Art and Practice of the Learning be construed as a potential conflict of interest. Organization. New York, NY: Doubleday. Copyright © 2019 Harvey, Leblanc and Cronin. This is an open-access article Shuffler, M. L., Diazgranados, D., Maynard, M. T., and Salas, E. (2018). distributed under the terms of the Creative Commons Attribution License (CC BY). Developing, sustaining, and maximizing team effectiveness: an integrative, The use, distribution or reproduction in other forums is permitted, provided the dynamic perspective of team development interventions. Acad. Manag. Ann. original author(s) and the copyright owner(s) are credited and that the original 12, 688–724. doi: 10.5465/annals.2016.0045 publication in this journal is cited, in accordance with accepted academic practice. No use, distribution or reproduction is permitted which does not comply with these terms. Smith, G. T., McCarthy, D. M., and Anderson, K. G. (2000). On the sins of short-form development. Psychol. Assess. 12, 102–111. doi: 10.1037//1040-3590. 12.1.102 Frontiers in Psychology | www.frontiersin.org 21020 July 2019 | Volume 10 | Article 1441

ORIGINAL RESEARCH published: 12 July 2019 doi: 10.3389/fpsyg.2019.01633 Learning From the Past to Advance the Future: The Adaptation and Resilience of NASA’s Spaceflight Multiteam Systems Across Four Eras of Spaceflight Edited by: Jacob G. Pendergraft1*, Dorothy R. Carter1, Sarena Tseng1, Lauren B. Landon2, Igor Portoghese, Kelley J. Slack3 and Marissa L. Shuffler4 University of Cagliari, Italy 1 Department of Psychology, University of Georgia, Athens, GA, United States, 2 KBRwyle, Houston, TX, United States, Reviewed by: 3 National Aeronautics and Space Administration, Washington, DC, United States, 4 Department of Psychology, Clemson Jared B. Kenworthy, University, Clemson, SC, United States The University of Texas at Arlington, Many important “grand” challenges—such as sending a team of humans on a voyage to United States Mars—present superordinate goals that require coordinated efforts across “multiteam Shane Connelly, systems” comprised of multiple uniquely specialized and interdependent component The University of Oklahoma, teams. Given their flexibility and resource capacity, multiteam system structures have great potential to perform adaptively in dynamic contexts. However, these systems may United States fail to achieve their superordinate goals if constituent members or teams do not adapt their collaboration processes to meet the needs of the changing environment. In this *Correspondence: case study of the National Aeronautics and Space Administration (NASA)’s Spaceflight Jacob G. Pendergraft Multiteam Systems (SFMTSs), we aim to support the next era of human spaceflight [email protected] by considering how the history of manned spaceflight might impact a SFMTS’s ability to respond adaptively to future challenges. We leverage archival documents, including Specialty section: Oral History interviews with NASA personnel, in order to uncover the key attributes This article was submitted to and structural features of NASA’s SFMTSs as well as the major goals, critical events, and challenges they have faced over 60 years of operation. The documents reveal Organizational Psychology, three distinct “eras” of spaceflight: (1) Early Exploration, (2) Experimentation, and (3) a section of the journal Habitation, each of which reflected distinct goals, critical events, and challenges. Frontiers in Psychology Moreover, we find that within each era, SFMTSs addressed new challenges adaptively by modifying their: (1) technical capabilities; (2) internal collaborative relationships; Received: 13 December 2018 and/or (3) external partnerships. However, the systems were sometimes slow to Accepted: 27 June 2019 implement needed adaptations, and changes were often spurred by initial performance Published: 12 July 2019 failures. Implications for supporting future SFMTS performance and future directions for MTS theory and research are discussed. Citation: Pendergraft JG, Carter DR, Keywords: teams, multiteam systems, spaceflight, adaptive performance, organizational practices, evolution and Tseng S, Landon LB, Slack KJ and adaptability Shuffler ML (2019) Learning From the Past to Advance the Future: The Adaptation and Resilience of NASA’s Spaceflight Multiteam Systems Across Four Eras of Spaceflight. Front. Psychol. 10:1633. doi: 10.3389/fpsyg.2019.01633 Frontiers in Psychology | www.frontiersin.org 2101 July 2019 | Volume 10 | Article 1633

Pendergraft et al. SFMTS Adaptations INTRODUCTION collaboration within and across teams and develop strategies for mitigating those barriers. The United States’ National Aeronautics and Space Administration (NASA) and directives from the President This case study aims to lay a foundation for supporting have set an ambitious goal: send manned Long-Duration SFMTS performance in the future by analyzing the history of Exploration Missions (LDEMs) to deep-space destinations like SFMTS performance over the past 60 years of NASA’s spaceflight Mars within the next two decades (National Aeronautics and program. We argue that considering the collaboration practices Space Administration [NASA], 2014; Trump, 2017). LDEMs and procedures that have been established previously within represent a new frontier for humanity, and could be one of a MTS or its embedding environment is an important first the greatest achievements in human history. However, these step when attempting to facilitate future adaptive performance. missions will also present immense difficulties and test the Indeed, scholars have long argued that teams’ histories can capabilities of all involved. Factoring prominently among substantially impact their futures (McGrath et al., 2000; the anticipated difficulties of LDEMs is the team risk or the Hollenbeck et al., 2014). Through a review of archival documents, “risk of performance and behavioral health decrements due we uncover the key features of SFMTSs and the major focuses, to inadequate cooperation, coordination, communication, critical events, and challenges SFMTSs have contended with in and psychosocial adaptation within a team” (Landon et al., the past. Further, we consider the ways in which SFMTSs have 2016, p. 5). The “team risk” in a LDEM is not limited to the adapted to meet the challenges of previous eras of spaceflight. risks of collaboration failures within the spaceflight crew. In doing so, we align with previous research on teams that LDEMs will require unprecedented levels of collaboration acknowledges “adaptation lies at the heart of team effectiveness” across complex “spaceflight multiteam systems” (i.e., “SFMTSs”) (Burke et al., 2006, p. 1189) and identify aspects of prior comprised of the space flight crew and numerous teams on Earth adaptations within the spaceflight context that must shift or (Mesmer-Magnus et al., 2016). advance further in order to achieve the goals of LDEM. In fact, many of the most important problems facing CASE STUDY APPROACH today’s organizations and societies —including responding to natural disasters (DeChurch et al., 2011), uncovering The purpose of this research is to better understand how NASA’s major scientific discoveries (Falk-Krzesinski et al., 2010), and SFMTSs have learned from and adapted in response to pivotal translating medical breakthroughs to practice (Asencio et al., events and transitions in the space program over the past 60 years 2012)—represent “grand challenges” (George et al., 2016) that of space exploration. Toward these ends, we reviewed publicly require intensive collaboration across interdependent systems available archival documents that provide first-hand information comprised of multiple uniquely specialized groups or teams. regarding how NASA’s SFMTSs responded to critical events. Our These “teams of teams” or “multiteam systems” (i.e., “MTSs”; case study was guided by three research questions which were Mathieu et al., 2001) are increasingly prevalent in today’s world grounded in extant theory and research on MTSs (Zaccaro et al., because these structures offer greater resource capacity than 2012; Shuffler et al., 2015). These research questions, our data single teams but more flexibility than traditional organizations collection, and analysis procedures are described below. and thus, are expected to respond adaptively to complex and evolving task demands (Marks et al., 2005; Porck et al., Research Questions 2018). Research Question 1 Despite their potential to achieve important goals, extant Our first research question How are NASA’s SFMTSs structured? research suggests that MTSs often fail due to breakdowns in (e.g., What teams are involved? What interteam relationships collaboration and coordination within and/or across component are relevant?) is based in prior theoretical work which has teams (Zaccaro et al., 2012). For example, MTS theory argues identified the key definitional features of MTSs (Mathieu, 2012) that interteam collaboration breakdowns are particularly likely and delineated the attributes of these systems that might in systems comprised of teams with very different areas of impact performance (Zaccaro et al., 2012). Defined formally, expertise, backgrounds, norms, priorities, or organizational MTSs are: “two or more teams that interface directly and memberships (Luciano et al., 2018). Furthermore, MTSs interdependently in response to environmental contingencies often appear in contexts that are ambiguous, dynamic, multi- toward the accomplishment of collective goals” (Mathieu et al., faceted, and require rapid responses (Shuffler and Carter, 2001, p. 289). All MTSs have in common two features: two 2018). Yet, research on dynamic task contexts suggests that or more component teams, and a hierarchical goal structure dynamism and uncertainty can present added problems whereby component team pursue separate team-level goals in for collaboration (Luciano et al., 2018) and members and addition to one or more shared “superordinate” goal. teams may fail to shift their processes and procedures adaptively to meet evolving task demands (Moon et al., However, as Zaccaro et al. (2012) argue MTSs can vary 2004; Hollenbeck et al., 2011). Therefore, when MTSs face an widely with regard to the types of “compositional,” “linkage,” important grand challenge, like a LDEM, which has critical and “developmental” attributes affecting MTS functioning. consequences for failure, it is often necessary to understand Compositional attributes are descriptive aspects of the individuals the specific features of the system (e.g., team characteristics, and teams comprising the system and can include demographic evolving task demands) that might present barriers to effective features of the MTS, the size of the system (e.g., number Frontiers in Psychology | www.frontiersin.org 2202 July 2019 | Volume 10 | Article 1633

Pendergraft et al. SFMTS Adaptations of teams), the relative characteristics of the component performance (LePine, 2005; Burke et al., 2006; Baard et al., teams (e.g., the functional specialization of component teams), 2014). Therefore, the second two research questions guiding and the degree to which the system crosses organizational our case study of NASA’s SFMTSs acknowledge that teams’ boundaries. Linkage attributes reflect the formal and informal histories (and their prior adaptations) matter to their futures: connections among members and teams and can include (2) What are major goals, critical events, and challenges have patterns of task interdependence driven by the MTS goal NASA’s SFMTSs faced in the past?; and (3) In what ways have hierarchy, communication, trust, and leadership structures. NASA’s SFMTSs adapted over time in response to evolving goals, Finally, developmental attributes are the properties of the system events, and challenges? (e.g., What organizational practices have connected to temporal development such as the system’s genesis been implemented?). (e.g., if the system was appointed or emergent), and the stability of the membership over time. The history of a MTS might facilitate subsequent performance or constrain it. In some instances, when future challenges As a guiding theoretical framework, MTSs researchers share similar features to those encountered in the past, typically leverage classic input-process-output (Steiner, 1972; prior adaptations represent a valuable resource which teams McGrath, 1984; Hackman, 1987) or input-mediator-output-input may draw on to inform their options for future adaptation. (IMOI model; Ilgen et al., 2005) views of team functioning Where anticipated challenges diverge from those encountered and performance to understand multiteam functioning. Within previously, a thorough understanding of past challenges and the these models, inputs reflect factors affecting team functioning adaptations made in response to them may guide subsequent (e.g., personality, knowledge, training, attitudes). The effects of adaptation strategies by allowing team members to identify the inputs are transmitted through mediators, such as teamwork areas where further improvement on existing systems may be processes (e.g., coordination behaviors, information sharing, needed. Conversely, circumstances may require teams to change backup behaviors; Marks et al., 2001) or emergent psychological their behaviors, but reliance on past approaches may prevent states (e.g., trust, shared cognition; Kozlowski and Ilgen, adaptation. For example, research has shown that it is much 2006) to team outputs (e.g., performance, viability). In MTSs, easier for teams to shift from loosely coupled or decentralized inputs (e.g., compositional attributes; Zaccaro et al., 2012) task decision-making structures toward more tightly coupled or residing at the individual, component team, and system level centralized structures than it is to shift in the opposite direction shape the interactions and relationships within and across (Moon et al., 2004; Hollenbeck et al., 2011). teams (e.g., linkage attributes), and MTS outcomes. These performance outcomes then become inputs during subsequent Therefore, we consider the ways in which NASA’s SFMTSs phases of performance. have previously adapted to evolving challenges. We suggest that considering the history of SFMTS adaptations could In summary, extant research argues that MTSs can vary provide a foundation for future LDEMs. First, an awareness widely in their structures and other compositional, linkage, of past adaptations may provide guidelines for the types of and developmental attributes. Moreover, the structures and adaptations that may benefit the system in the future. Second, attributes of MTSs are significant determinants of systems understanding prior challenges may allow for better prediction of performance. For example, drawing from a long history of the performance decrements that may result from the challenges research on intergroup relations (Sherif, 1958; Tajfel et al., of LDEMs if further adaptations are not instituted. Finally, 1979), Luciano et al. (2018) argue that the degree to which an advance awareness of potential performance decrements component teams differ from one another with regard to their may allow NASA and organizational researchers to apply functional capabilities, norms, work processes, and priorities, countermeasures, correcting for these challenges before their can create boundary-enhancing forces between teams that stifle consequences can manifest. Examining the past to inform the interteam collaboration and system performance. Therefore, our future may be particularly important in multiteam settings like first research question is based in the understanding that MTS an SFMTS, which could differ appreciably from less complex structures and other attributes are critical to system performance. stand-alone teams studied in laboratory settings or other types of organizations. Research Questions 2 and 3 Data Collection Approach Although research on organizational teams has often treated team tasks, composition, and environments as though they were We used transcripts from NASA’s JSC Oral History Project (JSC stable over time (Ilgen, 1999; Mathieu et al., 2017), scholars OHP) as the foundation of our archival document search. The have also pointed out that teams and MTSs are complex purpose of the JSC OHP was to “capture the history from the adaptive systems that experience evolving task demands, shifting individuals who first provided the country and the world with group memberships, and feedback loops with their embedding an avenue to space and the moon” (Madison, 2010). The JSC environments (Kozlowski and Klein, 2000; McGrath et al., OHP transcripts represent interviews with individuals spanning a 2000; Mathieu et al., 2014). The prior experiences, outcomes, wide range of roles within NASA, including managers, engineers, memories, and practices that have accumulated within a team technicians, astronauts, and other employees. Our review was or system in response to evolving task demands are likely to conducted entirely using publicly available documents. As such, shape subsequent behaviors and outcomes (e.g., McGrath et al., additional IRB, NASA, or interview participant approval was not 2000; Hollenbeck et al., 2014). Moreover, a team or system’s required for the use of these resources. ability to adapt to major changes is a hallmark of effective Frontiers in Psychology | www.frontiersin.org 2303 July 2019 | Volume 10 | Article 1633

Pendergraft et al. SFMTS Adaptations We used the JSC OHP as the foundation of our archival technique (Wohlin, 2014). In the first step, we began by compiling analysis for three key reasons. First, by virtue of their inclusion all available transcripts from the JSC OHP (n = 374 transcripts). in the JSC OHP, the events described in the transcripts can Then, the first and third authors read through each transcript be assumed to be of importance to the organization, from the and removed all transcripts that did not contain references to one perspective of NASA itself. These events often represented critical or more manned space mission and/or did not make multiteam milestones in NASA’s spaceflight legacy. In many cases, this interactions a central focus of the interview. This resulted in a was because the events described were pivotal in prompting much smaller subset of 30 focal JSC OHP transcripts containing altered patterns of action that were key to later successes, or information relevant to our research questions. These sources marked the surmounting of persistent and lasting problems explicitly discussed SFMTS collaboration during a manned space which would establish a template for future action. Often, mission. The decision to focus on multiteam collaboration the focus of the interviews could be described as “crisis” involving members of NASA’s MCC, along with our restricted events, although significant successes were also frequent topics. focus on manned spaceflight missions, was guided by the Therefore, although the documents largely exclude day-to-day recognition that “crew-ground” relations—between members of functioning of NASA and MCC which is sure to have substantial the spaceflight crew and MCC personnel—will be critical to impacts on the operation of the system as well, the OHP provides the success of future space exploration missions to deep space an ideal basis for identifying pivotal events and transitions within destinations (Landon et al., 2018). the space program. Although the events that are the focus of the JSC OHP represent a small proportion of the totality In many cases, the JSC OHP interviewees referenced events of NASA’s 60-year history, these events continue to exercise and mission details but did not explain the technical details of disproportionate impact on NASA’s operations. the events and/or the longer-term decisions that were made in response to the events thoroughly. For example, the following Second, the JSC OHP documents represented first-hand quote from an oral history interview with NASA flight engineer accounts of pivotal events and NASA transitions from the Christopher Kraft regarding the early stages of the Spacelab perspective of interview subjects who were intimately familiar program demonstrates the type of statement which required with and/or played a prominent role in the events described. more explanation: The selection of oral history project subjects was often guided by the familiarity of the subject with one or more formative “It just was sort of a long arduous task to get anything events or periods in the history of the organization. The interview done. . .You know what the arrangement was.” – Kraft (1991, 28 transcripts are presented with limited revisions to preserve their June), Flight Engineer (underlined emphasis added). conversational tone, and typically range between approximately 30 and 60 pages per interview. Participants were prompted by Therefore, in the second step of our data collection, we a NASA oral historian—whose questions are recorded in the generated a list of all of the manned spaceflight missions transcripts—to recall their personal experiences and perceptions referenced in the 30 focal JSC OHP transcripts. Then, we of prominent events or periods in NASA’s history. gathered official NASA- or government agency-produced documentation (e.g., investigation reports, government Third, the subjects of the oral histories tended to provide announcements, international agreements, etc.) related to a substantial amount of detail in terms of the intrapersonal the focal events in order to supplement our understanding of states (e.g., stress levels, motivation, affect, etc.) and interpersonal these events (n = 18 official documents). In cases where these relationships and behaviors (e.g., trust, shared cognition, documents also lacked sufficient detail, we gathered additional information sharing) acting on the system at the time of the sources (n = 60 additional sources) that provided more detail events in question. Details about internal states and interpersonal about the events in question. These additional sources included relationships and motivational factors are frequently omitted NASA articles (e.g., online blogs), mission archives (i.e., overview from more formal technical records but are highly relevant descriptions of mission goals, technical aspects, and task focus), to the functioning of MTSs (Zaccaro et al., 2012; Rico et al., other NASA documents (e.g., NASA history office gallery 2017; Luciano et al., 2018). The type of unique insights into entries), and articles from external news sources. The additional the internal and interpersonal states gleaned through the JSC NASA documentation was instrumental in helping us establish a OHP documentation are exemplified by the following quote clearer view of the situational facts of many events, particularly from astronaut Michael Foale, regarding the aftermath of the the granular details of individual missions. In total, these first collision of an unmanned Progress resupply spacecraft with the two data collection steps resulted in a total of 108 sources. Mir station: In a third step, two Subject Matter Experts (SMEs), who are “So that was a pretty hard time, because we got very tired. And intimately familiar with the history of NASA, refined the initial that was the hardest time I ever had on the station, was that set sources by eliminating sources which referenced events the period, because we just got so tired. Of course, the commander’s SMEs did not believe had played a significant role in the history morale was pretty – he was just shot, stunned.” – Foale (1998, 16 of the organization and/or any sources that they deemed to be June), astronaut. unreliable or inaccurate. Specifically, the majority of excluded documents were removed due to their irrelevance to central Collection of Archival Documents developments in the history of NASA (n = 22), while a smaller proportion were removed due to inaccuracies or inconsistencies Our collection of archival documents progressed in a series (n = 6). The majority of these six cases were excluded due to of three steps and leveraged an adapted snowballing review Frontiers in Psychology | www.frontiersin.org 2404 July 2019 | Volume 10 | Article 1633

Pendergraft et al. SFMTS Adaptations inconsistencies with other NASA documentation regarding the as well, although these also tended to appear in more explicit chief causes of events, as well as factual inconsistencies identified detail in investigation reports following performance failures. by comparison with other sources in a minority of cases. This For example, the Report of the Presidential Commission on the SME evaluation process resulted in a final set of 80 sources. Space Shuttle Challenger Accident contains sections explicitly Appendix A provides a complete list of these sources. These detailing the actions taken to implement the recommendations of sources discussed events occurring between 1960 and the present the commission (Presidential Commission on the Space Shuttle day, roughly spanning the operational history of NASA’s MCC. Challenger Accident, 1986). Throughout, where quoted material Table 1 and Figure 1 summarize the types of resources identified appears in the text, bracketed material represents sparingly added and their frequencies by year, respectively. text to provide clarity (drawing from statements elsewhere in the interview) and allow for concise quotation. Ellipses represent Analysis of Archival Documents omitted text from the original statement, similarly used to limit the quotation to the required information. Our research team coded each of the events described in the identified sources in order to identify the answers to our three CASE STUDY FINDINGS: SFMTS research questions. To begin, the first three co-authors read each STRUCTURES, CHALLENGES, AND of the sources and generated answers to the research questions ADAPTATIONS independently. Then, the coding team met and came to a group consensus regarding the answers to the three research questions. Research Question 1: How Are NASA’s Lastly, the coding team’s findings were then evaluated and refined SFMTSs Structured? by two SMEs familiar with the functioning and history of NASA. To Research Question 1, we evaluated the MTS structures in Answers to the research questions were primarily derived use during the manned spaceflight missions discussed in the from the oral history interview documents and were extracted JSC OHP transcripts and the relationships within and across for each of the focal events. For example, information about teams that appear to be pivotal to SFMTS success. Prior work the structure of the system and the nature of the component has identified the spaceflight crew and the teams comprising teams was frequently available from the oral histories themselves NASA’s Mission Control Center (MCC) as key component teams as was a great deal of information pertaining to the interteam in a SFMTS and argued that ground-crew relations are critical to relationships within the system. The following quote from spaceflight mission performance (Landon et al., 2018). Located William Reeves exemplifies this: at Johnson Space Center (JSC) in Houston, Texas, United States, NASA’s MCC is the organization primarily responsible for “They assigned me to head up the first consultant group that went directing a space exploration mission and monitoring the vehicle over to Russia, to their Control Center, to support from their during manned space missions. The staff of MCC is chiefly Control Center, real time. At the same time, there was a group of tasked with ensuring the safety of the crew and the completion Russians that came over here, Russian flight controllers, that formed of mission objectives. Indeed, we identified many references to a consultant group that was in our Control Center.” – Reeves (1998, ground-crew relations in the archival documents. For example, 22 June), flight controller. astronaut Bonnie Dunbar discussed communication regarding various systems: Likewise, the goals and challenges of relevant missions were frequently discussed by the interviewees, who were typically “We had a Mission Control Center for the payloads in southern acutely aware of them. For example, Michael Barratt responds to Germany, so that’s where we talked... to their engineers when we a prompt to discuss challenges early in an interview: were operating the payloads, or we would talk to their researchers if they were enabled. If we wanted to talk about Spacelab systems, then “I think some of the most significant challenges, of course, were we’d talk back to Houston... and so I would talk to both Houston working with our international partners. In particular working with and to München.” — Dunbar (2005, 20 January), astronaut. our former Cold War adversaries, our Russian friends.” – Barratt (2015, 30 July), flight surgeon and medical systems designer. Interestingly, we also identified multiple references to ground- ground relations between members of distinct but interdependent When additional information on mission goals was required, component teams on Earth—particularly between front room the supplemental documents (e.g., mission logs) frequently and backroom teams in the MCC. For example, another quote provided sufficient detail through stated mission objectives. from astronaut Bonnie Dunbar illustrates the importance of System adaptations were frequently described in the oral histories ground-ground relations to the success of the Shuttle-Mir program and the subsequent ISS: TABLE 1 | Summary of resources included in archival analyses. Count “I think flight crews are probably the easiest to integrate Source Type 30 across the board—because they share a common goal... But we 11 integrated researchers, we integrated flight controllers, we integrated NASA Oral Histories 39 managers, and it was a necessary thing to do before we actually Official NASA or government reports NASA articles, NASA mission archives, other NASA documents, articles from external news outlets Frontiers in Psychology | www.frontiersin.org 2505 July 2019 | Volume 10 | Article 1633

Pendergraft et al. SFMTS Adaptations FIGURE 1 | Frequency of critical events across decades and source type. started the International Space Station.” – Dunbar (1998, 16 in that system’s backroom team. Given this interdependent June), astronaut. arrangement of teams, NASA’s MCC operates as a smaller MTS embedded in the broader SFMTS involved in a mission. In fact, as the following quote from David C. McGill illustrates, Figure 2 provides a simplified depiction of the MTS structure since the beginning of NASA’s space program, spaceflight within the MCC. missions have involved large and complex systems integrating different areas of expertise: The SFMTS structures and relationships in these systems are governed by the nature of the goals pursued by constituent “Building large systems is very much a team sport. It takes a lot of members and teams. That is, constituent members and teams people to do it that range all the way from the architects at the top complete different proximal (e.g., individual-level, team-level) to the software developers and procurement organizations. There’s goals, which contribute to the overall, superordinate goal of a large number of people involved, and there’s decisions being made the system (Mathieu et al., 2001). The accomplishment of the all up and down this hierarchy.” – McGill (2015, 22 May), MCC superordinate goal (mission success and crew safety, in the Lead System Architect. case of MCC) requires interdependent interactions among the component teams. In pursuit of this superordinate goal, the McGill goes on to further discuss the challenges of component teams within the system will exhibit some form of communicating across a large network of individuals functional process interdependence, meaning that the component collaborating on a project, while communicating ambiguous teams must work interdependently while accomplishing goals. demands to all involved. The challenges of arriving at effective The exact form and nature of this interdependence will vary and flexible solutions, discussed throughout the interview, according to the needs of the system, and may change over characterize much of spaceflight. the course of a given mission. An example of a goal hierarchy within MCC is depicted in Figure 3, using the console positions Originally influenced by military organizations, NASA presently in use with the ISS. organized its early structures using a hierarchical structure of specialized teams reporting to a central authority. Within National Aeronautics and Space Administration’s front room MCC, this structure is comprised primarily of frontroom team serves as a hub for the integration of information from and backroom teams. Specifically, the MCC is organized wide ranging disciplines within the organization. Internally, into several disciplines, each assuming responsibility for a backroom personnel typically communicate with their flight hardware system or a specific aspect of the vehicle and controller on the frontroom team; information passed between mission. Each discipline is represented on the frontroom backroom teams is most often routed through their respective team by a flight controller, who is a discipline specialist. flight controllers, who confer directly. These interactions are The appointed leader of the frontroom team, overseeing and represented in Figure 2 by the dashed lines within the MCC. coordinating all flight systems, is called the flight director. The backroom teams are located in separate rooms from the During a mission, the flight controllers monitor their assigned frontroom team of flight controllers. Communication between system using telemetry data from the vehicle and direct frontroom flight controllers and backroom flight controllers radio communication with the crew. Each flight system’s occurs through audio and computer-based methods including frontroom flight controller is supported by additional personnel email and internal web pages. Frontiers in Psychology | www.frontiersin.org 2606 July 2019 | Volume 10 | Article 1633

Pendergraft et al. SFMTS Adaptations FIGURE 2 | Simplified depiction of NASA’s MCC MTS structure. MCC frontroom team is comprised of the flight director (FD) and flight controllers (FC). Dashed lines indicate supporting relationships between FC and disciplinary backroom teams. Relationships between the MCC MTS and outside teams are depicted as solid double headed arrows. FIGURE 3 | Example goal hierarchy within MCC during an ISS expedition with a need for integration of efforts between frontroom team (Team 4) and backroom teams (Teams 1–3). This SFMTS structure remains the basis for the organization communication in the midst of past crisis events. Effective of MCC, although the composition of the MTS and the communication between the backroom and frontroom team is distribution of tasks within it have shifted in response to the critical, to ensure that information is effectively transmitted from needs of the missions at the time. Under the present SFMTS the backroom teams through to the crew as needed and in organization, crew and frontroom teams must interact efficiently a timely manner. to share information on current and upcoming states of the crew and their taskwork. The discretionary monitoring of In addition to the frontroom and backroom team interactions, this information sharing is largely in the hands of the flight MCC teams interact with the spaceflight crew, with other teams director to determine, a decision role which has notably shaped within the broader organization (e.g., management teams), and in more recent years (see findings related to Research Question Frontiers in Psychology | www.frontiersin.org 2707 July 2019 | Volume 10 | Article 1633

Pendergraft et al. SFMTS Adaptations July 2019 | Volume 10 | Article 1633 2), with teams from international partner (IP) organizations. 2015–2020 Frontroom flight controllers are usually the only members of MCC who communicate directly with IP flight controllers or with Era 3: Habitation 2010–2015 the crew. Information originating within the backroom teams that must be transmitted to the crew is therefore first relayed 2005–2010 through the frontroom team. These patterns of interactions (indicated by solid double-headed arrows in Figure 2) shape and 2000–2005 restrict the coordination actions taking place within the SFMTS. Era 2: Experimentation 1995–2000 Research Questions 2 and 3: What Are the Major Goals, Events, and Challenges Spaceflight Era 1990–1995 and How Have NASA’s SFMTSs Adapted? 1985–1990 In order to address our second research question (i.e., What 1980–1985 major goals, events, and challenges have NASA’s SFMTSs encountered?), our coding team began by identifying the key 1975–1980 features of each of the events and/or missions described in the focal JSC OHP transcripts. We also searched for commonalities TABLE 2 | Spaceflight eras and corresponding NASA programs. Era 1: Exploration 1970–1975 Shaded areas denote the time period in which a program was active. across the events/missions. Through subsequent discussions with Space Program NASA SMEs, our coding team determined that the spaceflight 1965–1970 missions undertaken over the past 60 years of the space program can be organized into three distinct eras: (1) Early 1960–1965 Exploration, (2) Experimentation, and (3) Habitation. These eras are distinguishable by the goals, events, and challenges Project Mercury encountered by SFMTSs during each period. Table 2 identifies Project Gemini the manned spaceflight programs within each era. Table 3 Apollo Program summarizes the major goals, events, and challenges. With regard Skylab to our third research question (i.e., In what ways have NASA’s Space Shuttle (STS) SFMTSs adapted over time in response to evolving goals, events, Spacelab and challenges?), we determined that during each of the three Shuttle-Mir Program eras, the SFMTSs exhibited adaptations which corresponded to International Space Station the major challenges the systems encountered (summarized in Table 4). These adaptations were centered primarily around shifts and/or enhancements in: (1) technical expertise; (2) internal relationships; and/or (3) external partnerships. The following sections provide narrative descriptions of the major goals, events, challenges and adaptations within the three eras. Era 1: Early Exploration Major Goals In the first era, Early Exploration, missions including Projects Mercury, Gemini, and the Apollo Program were focused on early forays into space exploration, and required rapid improvements in technical expertise. Further, an intense environment of international competition with rival states (often referred to as the “Space Race”) during the Cold War factored prominently in the motivations and goals of this era. Beginning with early achievements in flight beyond the Earth’s atmosphere (e.g., Shepard’s, 1961 Mercury flight) and continuing through the lunar landings of the Apollo missions and the early forays into extended space habitation through the Skylab station, the superordinate goals pursued by NASA’s SFMTSs centered on developing and applying a significant corpus of technical expertise in a very short period of time in an environment characterized by uncertainty and competition. William Anders captured this focus on exploration and the development of technical expertise in Frontiers in Psychology | www.frontiersin.org 2808

Pendergraft et al. SFMTS Adaptations TABLE 3 | Major goals, critical events, and key challenges within three eras of spaceflight (Research Question 2). Era 1: Early Exploration (1960–1980) Major Goals/Objectives • Establish the technical competency needed to overcome the fundamental challenges of spaceflight Critical Events/Mission Milestones • Compete effectively with international rivals (“Space Race”) • First manned orbital flights (Project Mercury) Key Challenges • Development of intra-lunar manned spacecraft (Project Gemini) • Moon landings (Apollo Program) Era 2: Experimentation (1980–2005) • Loss of the AS-204 crew (Apollo 1 fire) Major Goals/Objectives • Apollo 11 moon landing Critical Events/Mission Milestones • Apollo 13 “successful failure” • Launch and maintenance of the Skylab station Key Challenges • Rapidly overcoming basic challenges of manned spaceflight while competing internationally Era 3: Habitation (2000-present) • Overall, progression was anticipated (e.g., Mercury and Gemini programs centered primarily around Major Goals/Objectives Critical Events/Mission Milestones development of technical capabilities; Apollo missions were the culmination of that development) • However, unforeseen setbacks occurred (e.g., Apollo 1 fire, Apollo 13 explosion) Key Challenges • Capitalize on the technical advancements of the previous era to engage in a program of scientific experimentation in space (international competition no longer a key issue) • Space shuttle development and missions (STS) • Hubble telescope maintenance in orbit • Loss of Shuttle Challenger • Loss of Shuttle Columbia • Shuttle-Mir Program/Phase I • Highly complex and technically challenging missions • Notable performance decrements occur as the result of rigid, unclear, and inefficient communication structures; these decrements presented an unanticipated area of challenge • Create and maintain an orbital platform to support continuous human occupation. • Collaborate with an array of international partners to accomplish this shared superordinate goal. • The establishment of the International Space Station (ISS) program • The component launches and orbital assembly of the International Space Station • Multiple missions executed in support and supply of the station • Retiring of the space shuttle program • Increased integration of private partnerships for the supply and maintenance of the station • Expedition missions of unprecedented duration (approximately a year in the longest cases) • Much longer duration missions (presents both technical and interpersonal challenges) • Work successfully with international partners with different norms and work processes • Most of the challenges during this period were not unexpected, but were persistent and critical (e.g., relations between international partners must be maintained continually) his oral history, and conveyed the extremely uncertain nature of challenges and successes described during this era related spaceflight at this time: to discovering a need to build and, subsequently, master an expanding body of technical expertise in the realm of “I didn’t think it was risk free but I thought that the [national] spaceflight. In addition, this era was marked by unexpected reasons for doing it were important, [as well as] the patriotic events that prompted significant adjustments within the and... exploration... [This] all made me decide that... there was system, notably the Apollo 1 fire and the “successful failure” [probably] one chance in three that [we] wouldn’t make it back, during Apollo 13. that there was probably two chances in three that we wouldn’t go there either because we didn’t make it back or [we had to abort] The severe physical and technical challenges inherent to early and one chance in three we’d have a successful mission, [that this exploration strained NASA’s capabilities throughout the first era. was a risk worth taking].” – Anders (1997, 8 October), Apollo 8 Tasked with operating in an unfamiliar environment, NASA Lunar Module Pilot. personnel needed to collaborate intensively to arrive at novel solutions, often in response to problems that were unforeseen at Critical Events, Challenges, and Adaptations the outset of the mission. In many cases, these challenges were Era 1 was marked by a number of prominent events, addressed successfully. Nonetheless, this era was also marked by including the first manned orbital flights (the focus of Project significant failures and tragedies aboard American space vehicles. Mercury), the development of the first effective intra-lunar In many cases, the failures engendered significant changes, manned spacecraft (the chief goal of Project Gemini), and improvements, and/or adaptations during subsequent missions. the six successful moon landings (the focus of the Apollo Program). These events represent a planned progression Prominent among the tragedies driving change within this from early orbital flight to manned lunar landings. The period is the on-board fire and subsequent total loss of the Apollo 1 (AS-204) crew. During a preflight rehearsal on January 27, 1967, Frontiers in Psychology | www.frontiersin.org 2909 July 2019 | Volume 10 | Article 1633

Pendergraft et al. SFMTS Adaptations TABLE 4 | Key SFMTS adaptations across three eras of spaceflight. Era 1: Early Exploration (1960–1980) Summary of Adaptations: NASA’s SFMTSs met the technical competency and external competitiveness demands of Era 1 by establishing and emphasizing formal hierarchies and formalized communication, technical training, and planning procedures Examples: • Established MTS structures based on military organizations. • Established communication processes leveraging technology (e.g., vacuum tube messages; headsets). • Established new training procedures – focused particularly on taskwork (e.g., high-fidelity simulation training for both crew and the ground control teams). • Established contingency planning procedures – by the time of the Apollo missions there was an emphasis on planning for all eventualities and rehearsing/training these scenarios. Era 2: Experimentation (1980–2005) Summary of Adaptations: NASA’s SFMTSs evolved to meet the added complexity of Era 2 task demands by shifting their internal communication, collaboration, and oversight structures and practices. Examples: • Communication processes and structures (particularly internally) changed substantially, in response to unexpected failures. • Center directors were empowered to make more direct contact with NASA management. • An Independent Technical Authority was established to make impartial judgements of launch readiness. • The responsibility of all component teams and contractors to raise concerns related to crew safety or launch readiness was reaffirmed, and reporting practices were articulated. • Training practices now included additional information about communication and coordination processes – ground control teams received updated training on reporting practices based on the recommendations of the Challenger and CAIB reports. • Initial steps toward greater collaboration with the Russian space agency made during Shuttle-Mir program; the number of personnel trained to speak Russian and coordinate with international partners began to increase toward the end of this era. • Technical practices (taskwork training, contingency planning) established during the previous era were refined and expanded. Era 3: Habitation (2000-present) Summary of Adaptations: NASA’s SFMTSs evolved to meet the challenges of multinational collaboration and long-term habituation within Era 3 by enhancing external communication and collaboration structures and practices. Examples: • Frontroom team elements comprised of international partner flight controllers were integrated directly into the NASA and ROSCOSMOS frontroom teams. • NASA crew members learn to speak Russian prior to transport to the station to aid in communication with crewmembers. • Substantial improvements to interagency communication practices/procedures. • Enhanced teamwork training procedures to facilitate shared understanding, collaboration, etc. a fire broke out in the cabin of the Apollo 1 Command Module, teamwork, and technical mastery that would continue to mark resulting in the death of all three crew members (astronauts MCC throughout NASA’s subsequent history. The first adaptation Grissom, White, and Chaffee). Failures in basic protocol as the made by MCC, in response to this episode, was a clear delineation disaster unfolded revealed critical weaknesses in the planning of of component team responsibilities and accountability. As missions and tests. Kranz’s quote emphasizes, teams and individuals within the system were to be directly accountable for the systems under In response to the AS-204 fire, NASA conducted a formal their control. Combined with the functional specialization of inquiry into the incident, under the Apollo 204 Review Board. The frontroom and backroom teams established early in MCC’s report of the board concluded that among other major causes of history, this responsibility directed individual component teams the accident, emergency preparedness during the test had been to work collectively to support the overall success of the inadequate because of the unfueled condition of the rocket and mission, while directing their own internal efforts toward perceived low risk of the test. the disaster instigated a change in the success of their respective systems. The central issue of the behavioral procedures of NASA. On the day following the accountability and control over launch progress would continue disaster, flight control operations branch chief Gene Kranz issued to be a point of struggle for MCC during future missions, what is now known as the “Kranz Dictum,” which would come to as the later loss of the Challenger and Columbia Shuttles exemplify the future identity of MCC. Kranz is quoted in part as would show. Nonetheless, the incorporation of this lesson having delivered the following words in response to the disaster: following the AS-204 fire represents a critical turning point in the history of MCC. “From this day forward, Flight Control will be known by two words: ‘Tough’ and ‘Competent.’ Tough means we are forever accountable In contrast to the Apollo 1 fire, the Apollo 13 emergency for what we do or what we fail to do... Competent means we will represented a successful response to an unforeseen technical never take anything for granted. We will never be found short in challenge that required MCC teams to collaborate extensively our knowledge and in our skills.” – Gene Kranz, Flight Director, with a spaceflight crew to arrive at a novel solution. Dubbed 28 January, 1967. a “successful failure” by NASA, the retrieval of the Apollo 13 crew following this severe failure evidences MCC’s growing Kranz’s specified focus on Flight Control as being “tough technical competency. On April 14, 1970, an oxygen tank and competent” directed a continuing tradition of accountability, Frontiers in Psychology | www.frontiersin.org 21100 July 2019 | Volume 10 | Article 1633

Pendergraft et al. SFMTS Adaptations aboard the Apollo 13 spacecraft exploded. The chaotic continued to place similar demands on knowledge integration atmosphere following the explosion is captured by flight and coordination of efforts among diverse personnel that director Glynn Lunney: prompted the organization of MCC as an MTS initially. “I [returned to the frontroom] and plugged in at the flight director Summary of Era 1 Adaptations console to hear a confusing array of multiple indications of As Table 4 summarizes, during Era 1, NASA adapted primarily problems... The fact of a really serious condition began to dawn to meet the technical competency and external competitiveness on the team as the crew reported the spacecraft venting particles demands of the period by establishing and emphasizing formal as seen out the window... EECOM was concluding that this was not hierarchies, communication, training, and planning procedures. an instrumentation problem and two fuel cells were indeed lost.” – Early in this era, NASA adopted rigid, hierarchical organizational Lunney (2010), Flight Director. structures—and the initial use of the MTS structure—to remain decisive and ensure new information would be rapidly actionable The subsequent days required substantial innovation on the in this uncertain and highly competitive environment. The part of both the crew and ground teams, perhaps shown most basic organization of a frontroom team tasked with integrating memorably in the construction of the “mailbox” device to aid in information among functionally diverse backroom support teams removing carbon dioxide from the Lunar Module (LM). In spite was established early in this era, in response to the technical of significant technical challenges in even voice communication demands of spaceflight itself. Further, including the role of a with the crew, MCC frontroom teams were able to collaborate flight director as a formalized leadership role within this MTS with both the spaceflight crew and backroom support teams to was recognized as critical to accomplishing the system’s goal develop and implement this solution. of integrating knowledge and coordinating efforts among the various component teams and teams outside MCC. The contrast between the AS-204 disaster and the “successful failure” of Apollo 13 highlights a second adaptation instituted Additionally, NASA implemented rapid communication within MCC and NASA more broadly. In the years prior practices facilitated by technology (during this era aided by to Apollo 13, NASA and MCC had engaged in significant radio headsets and vacuum message tubes), and the extensive contingency planning and simulation training. The crew’s use documentation of process which is still observable within of the LM as a “lifeboat” represents an observable outcome of MCC finds its origins during this first era. Exemplified by the increased planning and preparation, as it had been rehearsed crew’s rapid response to the explosion aboard the Apollo 13 during a training simulation despite the perceived unlikelihood spacecraft described above, MCC personnel acknowledged a need of the plan’s implementation. This contingency planning and for extensive rehearsal of even unlikely scenarios, given the simulation reduced the demands on interteam coordination uncertain nature of spaceflight. Thus, MCC developed extensive within the system, allowing teams to respond to unfolding training programs which emphasized technical competencies and events quickly and effectively, without the need to rely on contingency planning to prepare for the uncertain demands of a time-consuming direction from central leadership. This freed complex and evolving mission environment. up communication channels between teams to focus on the transmission of new information, a critical factor in the Era 2: Experimentation Overview system’s success. Major Goals Representing a third adaptation during this era, rapid During the second era, Experimentation, which included communication between component teams and reliance on endeavors such as the Space Shuttle missions and the Shuttle- the largely independent operations of MCC backroom teams Mir Program (i.e., a collaboration between NASA and the Russian allowed MCC personnel to rapidly develop solutions to complex space agency ROSCOSMOS), the tasks conducted aboard the unfolding problems over the course of Apollo 13’s return to Earth. spacecrafts became more complex. During this period NASA’s Glynn Lunney captures this developing ability to rapidly respond SFMTSs’ efforts centered around capitalizing on the technical to new information: advancements of the previous era and conducting research in the unique environment of space. Moreover, following the successes “The MCC pipeline was regularly delivering a number of new and of the Apollo Program (and the end of the “space race”), non-standard checklists for required activities. There were some international competition declined as a central focus of the space very effective leaders of specific areas and probably hundreds of program. As noted by Joseph Allen in his oral history interview, operations and engineering personnel evaluating all options and the transition toward a focus on experimentation in space began astronaut crews testing each procedure in the simulators.” – Lunney prior to the start of the Space Shuttle missions (i.e., during the (2010), Flight Director. later years of Era 1), but was slow to be adopted: As NASA advanced through Era 1, SFMTSs continued to “[Apollo] 14 was Alan Shepard, who wasn’t all that keen on a lot capitalize on accrued technical and behavioral expertise. This of science. But [for Apollo 15, science] really stuck. We had crew leveraging of technical competency resulted in the first successful members [who] liked the science, and we had all kinds of new lunar landing during the Apollo 11 mission in 1969, as well [science] equipment, and it wound up being the first lunar [mission as five subsequent successful lunar landings. In many ways, with geological] traverses that involved some serious distances the base structure of MCC established during this era has not across all kinds of geology in the rover.” – Allen (2003, 28 January), changed until the present day. The missions MCC has been Apollo 15 Support Crew Member. tasked with supporting over the course of NASA’s history have Frontiers in Psychology | www.frontiersin.org 21111 July 2019 | Volume 10 | Article 1633

Pendergraft et al. SFMTS Adaptations Critical Events, Challenges, and Adaptations as ROSCOSMOS objectives aboard the station focused more on The launch and maintenance of the Skylab station, which simply maintaining a manned presence in space (Foale, 1998). was designed to serve as a solar observatory and platform to support scientific experiments, marked a transitional point in However, this era was also characterized by major disasters. NASA’s mission focus and the event which distinguishes Era One of the greatest tragedies to occur during this era of 1 from Era 2. This transition represents the beginning of a spaceflight was the loss of the shuttle Challenger and its entire fusion of both the exploratory focus of the first era and the astronaut crew (STS-51L). A series of aborted launches due to emphasis on experimentation in space, which would come to a range of weather concerns lead to mounting impatience, and dominate the second. an eventual go-ahead for the launch despite concerns over low temperatures. This push to move forward with the launch was Unfortunately, although it was representative of burgeoning exacerbated by plans to widely televise the launch. The conflict confidence in the ability to execute spaceflight successfully, the between caution and the mounting pressure to launch within station was also plagued by technical difficulties beginning with MCC is captured by Steve Nesbitt, a NASA public affairs officer its initial deployment. During launch, a micrometeoroid shield working at MCC at the time: became dislodged, damaging the solar panels intended to supply power to the station. Archival documents revealed that interview “There had been a couple of scrubs in the days before. That was not subjects largely focused on the technical challenges of the station’s unusual. Some of the most conservative people you will ever find construction, deployment, and maintenance. This is notable in an are in Mission Control. If something wasn’t right, they were quite oral history interview conducted with Arnold Aldrich: willing to delay and come back another day. But that mission just went on and on.” – Nesbitt (2016, 28 January), NASA MCC Public “The Skylab 1 first flight had the micrometeoroid protection on Affairs Officer. the outside of the workshop come off during launch, and it took one solar array with it and pinned down the second one, Following the loss of the shuttle Challenger, President Reagan so that the spacecraft got into orbit without thermal protection established a commission to conduct an investigation into and with somewhat limited power... So this temperature was a the disaster and potential ways in which the disaster might big concern. Both Marshall and Johnson immediately moved out have been averted. The commission concluded that “flaws in to figure out how we could quickly ameliorate the overheating (NASA’s) decision making process” were a contributing cause in the workshop.” – Aldrich (2000, 24 June), Deputy Manager of the accident (Presidential Commission on the Space Shuttle (Skylab Program). Challenger Accident, 1986). The report found that failures in communication resulting from incomplete and misleading In spite of these difficulties, maintenance Skylab showcased the information, in conjunction with a NASA management structure increased technical achievement of NASA, with the deployment which permitted known safety issues to bypass shuttle managers, of a sunshield to prevent overheating and two additional led to known risks remaining unaddressed in readiness Extravehicular Activity (EVA) repairs being the focus of the first reviews. In the recommendations provided by the commission, of three manned missions to the station (SL-2). improvements to management and communications factor prominently, with an emphasis on managerial integration Although the loss of the station to orbital decay, in some and improved communication across the organization ways, represented the still-present technical challenges faced by (recommendations II and V; Presidential Commission on NASA, it was also the result of the growing prioritization of the Space Shuttle Challenger Accident, 1986, p. 199–200). the development of the Space Shuttle Program, the centerpiece of the second era. The space shuttle program epitomizes the In response to the commission’s recommendations, the second era. Over the lifetime of the program, the shuttle was hierarchy of organization within the Office of Space Flight was used both as an Earth-to-orbit transportation vehicle as well as an restructured to allow the MCC far more direct access to NASA orbital experimental platform. Similar to the missions comprising administration. Regular, formalized communication between the the first era, shuttle missions were short in duration, lasting for directors of JSC and other organizational components were days to approximately 2 weeks. To facilitate the experimental instituted. Perhaps most notably, the accountability of center mission of the shuttle, a laboratory module called “Spacelab” was directors for the “technical excellence and performance of sometimes incorporated into the shuttle. the project elements assigned to their centers” was reaffirmed (Presidential Commission on the Space Shuttle Challenger NASA’s increasing focus on experimentation was facilitated Accident, 1987, p. 31). These adjustments in the interteam in large part by the technical competencies accrued during collaboration processes of the MCC represent the first integration the previous era. In a revealing passage from a NASA of lessons learned based on the challenges of this era. mission archive on STS-61, maintenance on the Hubble Space Telescope is described as being completed ahead of schedule, Despite the implementation of these recommendations, the with a few unexpected events being handled smoothly. This subsequent loss of the shuttle Columbia would illustrate the characteristically competent mission completion occurs within need for further adaptations in NASA’s internal collaboration. the context of “one of the most challenging and complex On February 1, 2003, the Shuttle Columbia disintegrated while manned missions ever attempted” (Ryba, 2010). Interestingly, reentering the atmosphere, resulting again in a complete crew following the establishment of the shuttle program, NASA’s loss (STS-107). The failure resulted from damage from foam objectives of experimentation often differed from those of IPs, impacting the wing of the spacecraft during launch. In a subsequent investigation, the Columbia Accident Investigation Frontiers in Psychology | www.frontiersin.org 21122 July 2019 | Volume 10 | Article 1633

Pendergraft et al. SFMTS Adaptations Board (CAIB) concluded that NASA engineers had raised Foale was allowed to take part in EVAs to repair the station concerns following the launch that the foam shedding damage following the development of a medical issue by cosmonaut to Columbia may have been more significant than in previous Tsibliev. Accomplishing this goal required MCC personnel to launches. NASA managers did not initiate investigations into this coordinate rapidly with Russian ground control (TSuP) to possibility. Notably, the report concluded that flaws within the secure permission for Foale to conduct the EVAs, as well as organizational structure of NASA were significant contributors to effective coordination among both ground control groups and the disaster, and the loss would likely have occurred irrespective the international members of the crew to quickly familiarize Foale of which individuals were in the managerial roles. with the Russian-made EVA equipment (Foale, 1998). In a second adjustment, following the recommendations Summary of Era 2 Adaptations made by the CAIB, NASA and MCC implemented several changes to the structure and behavior of MCC (Columbia During Era 2, NASA’s SFMTS adapted to meet the Accident Investigation Board [CAIB], 2003). Among these added complexity of task demands by improving internal changes was the establishment of an independent Technical communication, collaboration, and oversight structures and Engineering Authority, “responsible for technical requirements practices. NASA personnel were empowered to raise concerns in and all waivers to them” (Columbia Accident Investigation Board connection with launch readiness directly; the responsibility of [CAIB], 2003, p. 193). In keeping with the recommendations all NASA personnel to raise such concerns as they became aware of the CAIB, the technical authority became the sole authority of them was reaffirmed. Training procedures introduced during for all technical standards, and independently verified launch this era targeted effective internal communication practices readiness with the ability to reject any scheduled launch should an directly. Finally, an Independent Technical Authority was undue risk be found. Critically, the ITA would be funded directly established to make impartial judgments about launch readiness, from NASA headquarters, removing it from any, “connection outside the NASA managerial hierarchy. to or responsibility for schedule or program cost” (Columbia Accident Investigation Board [CAIB], 2003, p. 193). The ability Where failures occurred, they prompted adaptations to of any component team to raise objections about the readiness coordination within MCC and the SFMTS. Where challenges of any system for launch was also reaffirmed. These changes were successfully addressed, the outcomes exemplify critical increased the safety of future shuttle crews by allowing evaluation competencies built during the first era of spaceflight: extensive of launch readiness not subject to constraints or pressures from contingency planning, leveraging of large amounts of training to other elements within the organization. arrive at innovative solutions, and rapid communication among functionally diverse teams. In spite of these successes, structural Despite these two public failures, the program of weaknesses within the MCC resulted in failures during this era, experimentation in space continued largely successfully requiring further changes to be made in order to prevent future throughout the second era. One of the lasting legacies of breakdowns in process. the shuttle program is the ability to launch large payloads into orbit, which would be critical during the following era. As was the case during Era 1, SFMTS adaptations in Era Moreover, beginning in 1995 and continuing through 1998, 2 were often prompted by unexpected external events—in this NASA collaborated with ROSCOSMOS to host American case, often socio-political ones. In particular, the challenges in astronauts aboard the Russian Mir space station (the Shuttle-Mir coordination between teams from NASA and ROSCOSMOS Program). Accordingly, astronauts conducted research aboard demonstrated an increasing need for familiarity both with IP the orbital platform while the space shuttle continued to be equipment and practices, a need which led to the introduction used for resupply and crew transport. During this program, of more extensive SFMTS training within the subsequent era of sometimes called Phase I, NASA MCC personnel learned to form habitation. As a result, during the Shuttle-Mir program, NASA’s conducive working relationships with Russian ground control MCC evolved in their ability to coordinate effectively with IP teams, requiring them to overcome challenges arising from organizations. In fact, the MCC MTS expanded to include remote language and cultural barriers (Reeves, 2009; Hill, 2015). personnel embedded with Russian ground control teams. These international consulting teams represented an early advancement However, international collaboration was undoubtedly in formalizing the relationship between NASA MCC and Russian affected by external socio-political forces. For example, the fall ground control personnel, a challenge which would continue of the USSR in 1991 led to improved relations between the to be addressed during the subsequent era of habitation. Russian Federation and the United States, and a corresponding Subsequently, the success of the Shuttle-Mir program laid the increase in the potential for international collaboration. The groundwork for the International Space Station program—and 1992 agreement between Presidents Bush and Yeltsin solidified the increasingly intense international collaborations that would plans for cooperation in space exploration, leading to the Shuttle- be required by that program. This transition is highlighted in Dr. Mir and subsequent programs, although relations between Michael Barratt’s oral history interview: organizations from the two countries would remain challenging. “Those of us that were heavily involved in the Shuttle-Mir Program A clear demonstration of these challenges can be found in realized two things. How wonderful it would be, because we found astronaut Michael Foale’s time aboard the Mir station. During that we could work with our Russian counterparts quite well, and that period an unmanned Progress spacecraft collided with the how difficult it would be, because they do things very differently station, causing substantial damage and a fire aboard the station. than we do... Without the Shuttle-Mir Program I can’t imagine Despite initial trepidations among the Russian ground teams, starting from scratch and going into such a large program as the Frontiers in Psychology | www.frontiersin.org 21133 July 2019 | Volume 10 | Article 1633

Pendergraft et al. SFMTS Adaptations International Space Station” – Barratt (1998, 14 April), Human context of the ISS, with respect to the ISS’s usage as a platform Research Program Manager. for scientific experimentation: Era 3: Habitation “I think one of the main things is that just looking at the Station as a laboratory, it has grown in capability, and it enables science that Major Goals we could never do before, because it is power-rich, and it has an incredible bandwidth to it... the laboratory that [the ISS has] evolved In the third era, Habitation, which consisted primarily of the into is just incredibly capable.” – Barratt (2015, 30 July). construction of and expeditions aboard the International Space Station (ISS), mission objectives centered on establishing a These competencies were combined with the capabilities for continuous human presence in space in collaboration with IP launching large orbital payloads developed during the era of organizations. The major goal of Era 3 was the construction Experimentation. Leveraging this knowledge and the lessons of and maintenance of an orbital platform to support continuous the Shuttle-Mir program, NASA collaborated closely with a wide human occupation. The primary operational difference between range of IPs to complete the ambitious ISS platform in 2011. As the activities of Era 3 and earlier periods is the extended mission summarized by Michael Suffredini, the legacy of the ISS is to timeframe of ISS expeditions. The ISS has been continuously consciously build and demonstrate capabilities to sustain human inhabited since late 2000, with the longest individual crew habitation in space for extended periods of time. member stays lasting approximately 1 year. Critical Events, Challenges, and Adaptations “The legacy of ISS will be that we created an environment that The challenges facing SFMTSs during Era 3 centered on allowed us to permanently have humans in low-Earth orbit. That, overcoming difficulties related to international collaboration and by its very nature, will mean that the ISS helped us do exploration, the physical challenges of long-duration spaceflight. In Era 3, because we have the capability permanently in low-Earth orbit to do NASA has needed to collaborate intensively with an array of IPs the things we need to do to safely travel beyond low-Earth orbit.” – in pursuit of shared goals. Moreover, whereas previous eras were Suffredini (2015, 29 September), ISS Program Manager. characterized by missions lasting several days, this era is marked notably longer spans of habitation aboard the ISS (e.g., 6 months). Accordingly, NASA SFMTSs have had to develop substantial procedures for coordination among IP ground control teams To support the station, the MCC has engaged in continuous in order to meet the challenges of international collaboration operations for 18 years. This shift from short-duration, high- in spaceflight, as well as building a number of technical intensity missions to a long-term mission timeline requires MCC competencies to facilitate this relationship. Representing a first to operate in fundamentally different ways than they did during adjustment during this era, over the course of the Shuttle-Mir prior missions and eras of spaceflight. New skills relevant to program and subsequent phases of the ISS project a large number the monitoring and maintenance of the crew and station have of NASA engineers learned Russian (Barratt, 1998), and channels become more salient to the present task, shifting the needs of of communication were established which grew more developed the system in important ways. Additionally, extended habitation as communication technologies advanced and communication in space places immense strain on astronauts’ bodies, including between the organizations normalized (Reeves, 2009; Hill, 2015). loss of visual acuity, muscle loss, and loss of bone density. In Among these adaptations were the inclusion of a Russian console turn, these physical challenges can exacerbate the already intense in MCC, as well as a translator loop allowing MCC flight psychological strain on astronauts. Combined with the challenges controllers to listen in on the communications between the of existing for a prolonged period of time in a confined space Russian ground control teams and their crew members aboard alongside a diverse, international crew, the confluence of these the station. Dr. Barratt discusses this finding of common ground psychological strains can be intense. The challenges of intensive in his oral history interview. collaboration with IP organizations are discussed by Dr. Michael Barratt during his 2015 interview for the International Space “Once you get past the language barrier, people understood that Station oral history project: the laws of physics are the same, the laws of orbital mechanics are the same, zero gravity is the same, and it was pretty easy to “I think if anybody had asked us what a good model for find common ground amongst the crewmembers and the supporting making a Space Station would be, the answer would not have engineers. Really language was the only thing in the way there. A lot been to choose a major partner who speaks another language, of United States engineers learned Russian, a lot of Russians learned who uses metric system rather than English system, who has a English, which was quite wonderful. Once we got through that, we totally different engineering philosophy, safety culture, methods of found that we could work together pretty well.” – Barratt (2015, 30 operation, methods of manning. All of that was different.” – Barratt July), Human Research Program Manager. (2015, 30 July), Human Research Program Manager. Lastly, the challenges in terms of interteam relations between The types of challenges described by Michael Barratt in teams in MCC, other NASA teams, and IP teams have resulted the above quote required NASA and their IPs to leverage the in the integration of interpersonal and team skills training into lessons of the previous two eras of spaceflight. As in the era of the training regimen of astronauts and flight controllers. Notably, experimentation, NASA’s SFMTSs in the third era have continued the present iterations of these training practices focus primarily to draw on the technical competencies built during prior eras. on enhancing teamwork within individual teams, rather than Michael Barratt further discusses technical competency in the teamwork processes spanning across multiple teams. Frontiers in Psychology | www.frontiersin.org 21144 July 2019 | Volume 10 | Article 1633

Pendergraft et al. SFMTS Adaptations Summary of Era 3 Adaptations exploration, NASA’s SFMTSs must not lose the gains Adaptations made during this era centered around meeting made in previous eras. The challenges of LDEMs reflect the challenges of multinational collaboration and long- those seen within early exploration, experimentation, and term habitation by developing greatly improved external habitation. However, LDEMs also present new challenges collaboration practices. Altered practices and competencies that will call for new adaptations. Indeed, as shown aided in more rapid and effective communication across in Figure 4, NASA’s SFMTSs will need to significantly organizational and national boundaries, as did dedicated enhance their technical capabilities, internal collaborative training in teamwork practices. Interventions aimed at relationships, and external partnerships in order to achieve teamwork helped ensure that the multinational crew aboard the goals of LDEMs. the station was able to function effectively, and interpersonal conflict resulting from the challenging physical and relational In the following, we discuss the anticipated challenges of the environment was minimized. upcoming era of human spaceflight, and the adaptations that will be required. Given the complex challenges involved in LDEMs, DISCUSSION NASA’s SFMTS will need to adapt substantially across all three domains (i.e., technical expertise, internal coordination, and Drawing from archival sources, this case study identified many external coordination). This need to reconsider existing practices of the collective memories (e.g., mission successes, failures), in the light of new challenges is nothing new to NASA, as our lessons learned, and adaptations or practices implemented review demonstrates. For example, in an oral history interview within NASA’s SFMTSs in the three prior eras of early conducted in May of 2015, MCC lead system architect David exploration, experimentation, and habitation. NASA and their McGill states: IPs are now on the brink of an anticipated fourth era of spaceflight, characterized by LDEMs. The “team risk” will play “Well, how will your design react if suddenly we have a mission that a much larger role than in previous missions, as team and is going to involve three countries to go fly it? How are you going to interteam coordination must be sustained for multiple years tolerate that? How is your system going to respond to all of a sudden as SFMTSs tackle unexpected and even dangerous challenges wide area networking is twice as fast and half as much money as it is (Salas et al., 2015). We expect that whether these systems today? Can you take advantage of that?” – McGill (2015, 22 May), will be able to address the challenges of future missions MCC Lead System Architect. will be impacted by the rich history of the organizational environment, the lessons learned in previous missions, and the First, echoing Era 1, LDEMs will bring demands for adaptation organizational practices related to teamwork that have been in technical expertise. For example, the distances to be traveled implemented within NASA. in LDEMs represent a significant technical challenge. A variety of technical approaches to manned Mars missions and other Synthesizing the Adaptations of Previous LDEMs have been discussed (e.g., the Lunar Gateway platform; Eras to Facilitate Adaptive Performance National Aeronautics and Space Administration [NASA], 2014); in the Next Era of Spaceflight but all will require substantial technical advancements. Further, the distances involved in LDEMs will require extremely long As summarized in Table 4, our analysis of archival documents periods of travel beyond which will place new strains on revealed three broad categories of adaptations used to astronauts. Negative physical effects may become continuously meet the evolving task demands of the previous eras of more severe over the greater mission timeframes of LDEMs. The spaceflight: (1) enhancing technical expertise, (2) enhancing or extended time the crew will be isolated from the rest of the system shifting internal collaborative relationships; and (3) enhancing leads to particularly intense concerns around training retention, external or cross-organizational partnerships. Interestingly, as technical training is known to degrade over time and the we find that NASA’s SFMTSs emphasized these different highly autonomous crew will be less able to rely on support from categories of adaptations in different ways within each era. ground-based teams (Landon et al., 2018). During Era 1, the external competition and the massive demands for improved technical competence meant that the The challenges of LDEMs will also require adaptation with primary focus was on enhancing technical expertise. In Era respect to internal collaboration practices. As an unavoidable 2, NASA complex mission demands continued to require consequence of the massive distances traveled during a LDEM, new technical developments, however, unexpected disasters there will be significant communication delays between the (e.g., the losses of Challenger and Columbia) revealed that spaceflight crew and earthbound teams. At the greatest distance, adaptations were urgently needed with regard to internal communications to or from the crew of a Mars mission collaboration patterns. Lastly, in Era 3, the installation of could take up to 24 min to arrive at their destination. Such the ISS necessitated a focus on external partnerships with communication delays represent a stark contrast with the international agencies. effectively instantaneous communications between MCC and the crew of the ISS. In the third previous eras of spaceflight, crews Figure 4 summarizes the emphasis on different relied heavily on rapid communication with Earthbound teams categories of adaptive behaviors across the previous to arrive at solutions. However, in LDEMs, the crew will need three eras. As we enter into the fourth era of spaceflight to operate far more independently, as reliance on continuous feedback from MCC will not be feasible. Such decentralized authority structures may be necessary for LDEM success but Frontiers in Psychology | www.frontiersin.org 21155 July 2019 | Volume 10 | Article 1633

Pendergraft et al. SFMTS Adaptations FIGURE 4 | Amount of emphasis on different types of adaptations with each spaceflight era. Emphasis varied across eras with regard to (1) enhancing technical competencies (solid black line); enhancing internal collaboration (dashed black line); and (3) enhancing cross-organizational partnerships (gray line). may also present challenges for multiteam coordination and that long-duration spaceflight may place on astronauts and performance (Lanaj et al., 2013). the potential negative effects for interpersonal relations both within the crew and across component teams in Finally, the upcoming era of spaceflight will require continued SFMTS (Palinkas, 2007; Palinkas and Suedfeld, 2008; adaptation in the domain of external coordination and Landon et al., 2018). collaboration. LDEMs will reach further than any prior manned spaceflight mission and will require massive inter- Beyond LDEMs: Theoretical and agency coordination across national and organizational borders. The SFMTSs involved in LDEMs will be comprised of members Practical Contributions from different cultures, backgrounds, nations, and areas of expertise. Such high levels of individual and team differentiation This case study is focused on the specific context of NASA’s are likely to pose challenges for interteam collaboration (Luciano SFMTSs. However, there are at least four ways in which the et al., 2018). Moreover, SFMTSs involved in LDEMs will findings from this research might inform MTS research and experience dynamic environments characterized by expected practices within other contexts. First, our review revealed that (e.g., increased communication delays) and unexpected adaptations with were driven by the focus and challenges of challenges. As a LDEM progresses, different areas of technical the periods in which they were enacted and clustered into expertise will become more or less relevant to the task at one of three general categories: (1) technical competency, (2) hand, resulting in shifts in goal priorities and the relative internal coordination, and (3) external or cross-organizational authority of teams over the course of the mission. As these coordination (see Figure 4). Although the adaptations identified responsibilities may be distributed across IPs teams (as with in archival documents were generally specific to NASA, the the current operation of the ISS) these highly dynamic contexts three-category framework may be useful for conceptualizing may exacerbate tensions surrounding organizational boundaries and advancing MTS adaptations in other contexts. With respect and hinder communication and interteam coordination to MTS research, future empirical work may benefit from the (Luciano et al., 2018). greater specificity of these dimensions, and their relationship with situational and task demands. In practice, organizations Moreover, the consequences of longer-duration mission can target the dimensions of adaptation that have successfully timelines for internal and external collaboration remain in addressed related challenges in the past when preparing for question. Whereas research on team tenure would seem to upcoming challenges. In particular, anticipating the needed suggest that performance of the system will increase over patterns of adaptation may allow for more successful proactive time (Bell, 2007), initial evidence from research conducted intervention–thus avoiding the inefficiencies of adapting after using NASA analog environments has demonstrated that needs are revealed by performance decrements. Strategies when crews are restricted to isolated environments for allowing for more successful proactive adaptation are especially prolonged periods of time, longer team tenure can lead relevant to high-reliability organizations operating in dynamic to collaboration and cohesion decrements as interpersonal environments (HROs) like NASA, the military, and disaster conflicts becomes more severe (Kozlowski et al., 2016). response teams. HROs often operate in unforgiving competitive, Indeed, concerns have been expressed around the strain social, and political environments that are rich in potential Frontiers in Psychology | www.frontiersin.org 21166 July 2019 | Volume 10 | Article 1633

Pendergraft et al. SFMTS Adaptations for error, and where the scale of consequences associated the loss of the unmanned Mars Climate Orbiter (MCO) in with error precludes learning through experimentation 1999 (Mars Climate Orbiter Mishap Investigation Board, 1999). (Weick et al., 1999). Practices like these may be of benefit to even non-HRO organizations, suggesting a wider application of this approach Second, consistent with prior theoretical work on MTSs (e.g., (Weick et al., 1999). Zaccaro et al., 2012), our case study revealed compositional and linkage attributes that factor prominently in the functioning of Lastly, we suggest that our case study approach may be SFMTSs. For example, our review established that component applicable in a range of contexts outside NASA as many teams in the MCC (i.e., frontroom and backroom teams) teams and MTSs have collective performance experience. are highly differentiated along a variety of dimensions (e.g., This work is in keeping with recommendations to conduct areas of expertise, work processes, geographic locations). qualitative ethnographic research prior to and following Although team differentiation is a necessary element of quantitative research within an organization (Ofem et al., MTS collaboration which allows these systems to divide 2012). Given the impact of a MTS’s history on its future complex interdisciplinary tasks into disciplinary subgoals, operations, we expect continued qualitative examinations the extreme levels of differentiation often seen in SFMTSs of this type will serve to better inform LDEMs, and can also incur performance decrements when relationships could serve as the foundation for broader explorations are not managed effectively (Luciano et al., 2018). In fact, of MTS temporal dynamics. These benefits could be whereas the SFMTSs within Era 1 emphasized formal structures further expanded in future research through detailed and separations between teams, in order to tackle new examination of the day-to-day operations of MTSs, with demands in Era 2, the SFMTSs began to permit more direct respect to the enduring effects of these events in the communication channels between people who were otherwise future. Although the need to consider the rich history of disconnected (e.g., occasional guidance from specialists an organization is often acknowledged by practitioners, to crewmembers conducting experiments). These findings there is also a proliferation of “off-the-shelf ” interventions suggest an interesting line of inquiry for MTS researchers– available. This case study may serve as a reminder that MTSs may need to strike the right balance in terms of anchoring organizational interventions in an understanding emphasizing component team separation and integration. of the historical context of the organization may increase However, the optimal balance point may vary based on their effectiveness. evolving task demands. CONCLUSION Third, our analysis of the history of SFMTSs suggests MTS research could benefit from considering MTS performance In conclusion, scholars have argued that a team’s history can and adaptation on a longer time scale than has been significantly impact its future (Marks et al., 2001; Hollenbeck used in previous research. Empirical studies of MTS et al., 2014). Our analysis of the evolution and adaptation functioning have focused primarily on performance as of NASA’s history suggests that the same can be said of a relatively short-term outcome. Although these studies a SFMTS. We find the lessons learned in previous eras of provide valuable contributions to our understanding spaceflight often carry forward into subsequent phases. Our of MTS functioning, our review of NASA archival findings revealed that adaptations typically clustered into one documentation revealed that in several cases, short-term of three general categories and were associated with specific failures in performance led to improved performance in types of task demands and critical events. We suggest that the future (e.g., the structural changes made to NASA’s LDEM SFMTSs will need to capitalize on the gains of the management hierarchy in response to the losses of shuttles past while incorporating additional adaptations in order to Challenger and Columbia). succeed. Thus, this case study demonstrates the value of examining prior patterns of adaptation in preparation for Our findings also provide insight into how adaptation future challenges. might manifest in HRO contexts following a performance failure. Unlike many teams in which creative solutions are AUTHOR CONTRIBUTIONS required (e.g., product development teams), teams and MTSs operating within HROs cannot afford to readily accept short- All authors contributed substantially to the identification, term failures as a means to facilitating learning and adaptation. classification, and analysis of archival documents and to the Nonetheless, errors and failures in performance are a virtual development of the conceptual framing of this manuscript. certainty over the long-term. Our findings indicate that the LL and KS contributed as subject matter experts, and aided key to successful adaptation may lie in maximizing the significantly in the development of a conceptual framework information extracted from the events, and its successful for the classification of archival resources. Finally, all authors integration into future practices. Illustrating this, NASA conducts contributed significant amounts of time and effort to the unflinching internal examinations following critical events to revision of the text and the refining of the conceptual and establish both their immediate and structural causes. Notably historical content. such rigorous investigations do not only occur in cases where human life has been lost or placed at great risk; this dedication to intensive examination in the wake of any failure is exemplified by the rigorous investigation following Frontiers in Psychology | www.frontiersin.org 21177 July 2019 | Volume 10 | Article 1633

Pendergraft et al. SFMTS Adaptations FUNDING Any opinions, findings, and conclusions or recommendations expressed in this study are those of the authors and do not This study is based in part upon work supported by the National necessarily reflect the views of the National Aeronautics and Aeronautics and Space Administration (#80NSSC18K0511). Space Administration. REFERENCES Hill, P. S. (2015). Paul S. Hill (R. Wright, Interviewer). International Space Station Program Oral History Project Edited Oral History Transcript. Aldrich, A. D. (2000). Arnold D. Aldrich (K. Rusnak, Interviewer). NASA Available at: https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/ Johnson Space Center Oral History Project Oral History Transcript. oral_histories/HillPS/HillPS_3-24-15.htm Available at: https://historycollection.jsc.nasa.gov/JSCHistoryPortal/ history/oral_histories/AldrichAD/AldrichAD_6-24-00.htm (accessed October Hollenbeck, J. R., DeRue, D. S., and Nahrgang, J. D. (2014). The opponent process 2, 2018). theory of leadership succession. Organ. Psychol. Rev. 5, 333–363. doi: 10.1177/ 2041386614530606 Allen, J. P. (2003). Joseph P. Allen (J. Ross-Nazzal, Interviewer). NASA Johnson Space Center Oral History Project Oral History Transcript. Hollenbeck, J. R., Ellis, A. P., Humphrey, S. E., Garza, A. S., and Ilgen, D. R. (2011). Available at: https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/ Asymmetry in structural adaptation: the differential impact of centralizing oral_histories/AllenJP/AllenJP_1-28-03.htm versus decentralizing team decision-making structures. Organ. Behav. Hum. Decis. Process. 114, 64–74. doi: 10.1016/j.obhdp.2010.08.003 Anders, W. A. (1997). William A. Anders (P. Rollins, Interviewer). NASA Johnson Space Center Oral History Project Oral History Transcript. Available at: Ilgen, D. R. (1999). Teams embedded in organizations: some implications. Am. https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/ Psychol. 54, 129–139. doi: 10.1037//0003-066x.54.2.129 AndersWA/AndersWA_10-8-97.htm Ilgen, D. R., Hollenbeck, J. R., Johnson, M., and Jundt, D. (2005). Teams Asencio, R., Carter, D. R., DeChurch, L. A., Zaccaro, S. J., and Fiore, S. M. (2012). in organizations: from input-process-output models to IMOI models. Charting a course for collaboration: a multiteam perspective. Transl. Behav. Annu. Rev. Psychol. 56, 517–543. doi: 10.1146/annurev.psych.56.091103. Med. 2, 487–494. doi: 10.1007/s13142-012-0170-3 070250 Baard, S. K., Rench, T. A., and Kozlowski, S. W. (2014). Performance adaptation: Kozlowski, S. W., and Ilgen, D. R. (2006). Enhancing the effectiveness of work a theoretical integration and review. J. Manag. 40, 48–99. doi: 10.1177/ groups and teams. Psychol. Sci. Public Interest 7, 77–124. doi: 10.1111/j.1529- 0149206313488210 1006.2006.00030.x Barratt, M. R. (1998). Michael R. Barratt, M. D. (R. Wright, P. Rollins, Kozlowski, S. W. J., Chao, G. T., Chang, C.-H., and Fernandez, R. (2016). “Team Interviewers). NASA Johnson Space Center Oral History Project. Available dynamics: Using “big data” to advance the science of team effectiveness,” in Big at: https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_ Data at Work: The Data Science Revolution and Organizational Psychology, eds histories/ISS/BarrattMR/BarrattMR_7-30-15.htm S. Tonidandel, E. King, and J. Cortina (New York, NY: Routledge Academic Press). Barratt, M. R. (2015). Michael R. Barratt (R. Wright, Interviewer). International Space Station Program oral History Project Edited Oral History Transcript. Kozlowski, S. W. J., and Klein, K. J. (2000). “A multilevel approach to theory Available at: https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/ and research in organizations: Contextual, temporal, and emergent processes,” oral_histories/Shuttle-Mir/BarrattMR/BarrattMR_4-14-98.htm in Multilevel theory, Research, and Methods in Organizations: Foundations, Extensions, and new Directions, eds K. J. Klein and S. W. J. Kozlowski Bell, S. T. (2007). Deep-level composition variables as predictors of team (San Francisco, CA: Jossey-Bass), 3–90. performance: a meta-analysis. J. Appl. Psychol. 92, 595–615. doi: 10.1037/0021- 9010.92.3.595 Kraft, C. C. (1991). Christopher C. Kraft (A. Dunbar & S. Waring, interviewers). Interview with Christopher C. Kraft. Available at: Burke, C. S., Stagl, K. C., Salas, E., Pierce, L., and Kendall, D. (2006). Understanding https://www.nasa.gov/sites/default/files/atoms/files/19910628_christopher_ team adaptation: a conceptual analysis and model. J. Appl. Psychol. 9, 1189– kraft_oral_history_interview.pdf (accessed October 2, 2018). 1207. doi: 10.1037/0021-9010.91.6.1189 Kranz, E. F. (n.d.). Wikiquote. Available at: https://en.wikiquote.org/wiki/Eugene_ Columbia Accident Investigation Board [CAIB] (2003). Columbia Accident F._Kranz. (accessed November 27, 2018). Investigation Board Report, Vol. I. Washington, DC: Government Printing Office. Lanaj, K., Hollenbeck, J. R., Ilgen, D. R., Barnes, C. M., and Harmon, S. J. (2013). The double-edged sword of decentralized planning in multiteam systems. Acad. DeChurch, L. A., Burke, C. S., Shuffler, M. L., Lyons, R., Doty, D., and Salas, E. Manag. J. 56, 735–757. doi: 10.5465/amj.2011.0350 (2011). A historiometric analysis of leadership in mission critical multiteam environments. Leadersh. Q. 22, 152–169. doi: 10.1016/j.leaqua.2010.12.013 Landon, L. B., Slack, K. J., and Barrett, J. D. (2018). Teamwork and collaboration in long-duration space missions: going to extremes. Am. Psychol. 73, 563–575. Dunbar, B. J. (1998). Bonnie J. Dunbar (R. Wright, interviewer). NASA Johnson doi: 10.1037/amp0000260 Space Center Oral History Project. Available at: https://historycollection.jsc. nasa.gov/JSCHistoryPortal/history/oral_histories/Shuttle-Mir/DunbarBJ/ Landon, L. B., Vessey, W. B., and Barrett, J. D. (2016). Evidence report: Risk of DunbarBJ_6-16-98.htm (accessed October 2, 2018). Performance and Behavioral Health Decrements due to Inadequate Cooperation, Coordination, Communication, and Psychosocial Adaptation within a Team. Dunbar, B. J. (2005). Bonnie J. Dunbar (J. Ross-Nazzal, Interviewer). Houston, TX: NASA. NASA Johnson Space Center Oral History Project. Available at: https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/ LePine, J. A. (2005). Adaptation of teams in response to unforeseen change: DunbarBJ/DunbarBJ_1-20-05.htm (accessed October 2, 2018). effects of goal difficulty and team composition in terms of cognitive ability and goal orientation. J. Appl. Psychol. 90, 1153. doi: 10.1037/0021-9010.90.6. Falk-Krzesinski, H. J., Borner, K., Contractor, N., Fiore, S. M., Hall, K. L., Keyton, 1153 J., et al. (2010). Advancing the science of team science. Clin. Transl. Sci.. 3, 263–266. doi: 10.1111/j.1752-8062.2010.00223.x Luciano, M. M., DeChurch, L. A., and Mathieu, J. E. (2018). Multiteam systems: a structural framework and meso-theory of system functioning. J. Manag. 44, Foale, C. M. (1998). C. Michael Foale (Session 1) (R. Wright, C. Butler, M. Davidson, 1065–1096. doi: 10.1177/0149206315601184 Interviewers). C. Michael Foale (Session 1). Available at: https://history.nasa.gov/ SP-4225/oral-histories/foale.pdf (accessed October 2, 2018). Lunney, G. S. (2010). Apollo XIII. Available at: https://historycollection.jsc.nasa. gov/JSCHistoryPortal/history/oral_histories/LunneyGS/Apollo13.htm George, G., Howard-Grenville, J., Joshi, A., and Tihanyi, L. (2016). Understanding and tackling societal grand challenges through management research. Acad. Madison, L. (2010). JSC Oral History Project. Available at: https://historycollection. Manag. J. 59, 1880–1895. doi: 10.5465/amj.2016.4007 jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/oral_histories.htm (accessed October 2, 2018). Hackman, J. R. (1987). “The design of work teams,” in Handbook of Organizational Behavior, ed. J. W. Lorsch (Englewood Cliffs, NJ: Prentice-Hall), Marks, M. A., DeChurch, L. A., Mathieu, J. E., Panzer, F. J., and Alonso, A. (2005). 315–342. Teamwork in multiteam systems. J. Appl. Psychol. 90, 964–971. doi: 10.1037/ 0021-9010.90.5.964 Frontiers in Psychology | www.frontiersin.org 21188 July 2019 | Volume 10 | Article 1633

Pendergraft et al. SFMTS Adaptations Marks, M. A., Mathieu, J. E., and Zaccaro, S. J. (2001). A temporally based Reeves, W. D. (1998). William D. “Bill” Reeves (M. Davison, R. Wright, C. Butler, framework and taxonomy of team processes. Acad. Manag. Rev. 26, 356–376. interviewers). NASA Johnson Space Center Oral History Project Oral History doi: 10.5465/amr.2001.4845785 Transcript. Available at: https://www.spaceflight.nasa.gov/history/shuttle-mir/ people/oral-histories/reeves.pdf (accessed October 2, 2018). Mars Climate Orbiter Mishap Investigation Board (1999). Mars climate orbiter mishap investigation board phase I report. Houston, TX: NASA. Reeves, W. D. (2009). William D. Reeves (R. Wright, Interviewer). NASA Johnson Space Center Oral History Project Oral History Transcript. Available at: https: Mathieu, J. E. (2012). “Reflections on the evolution of the multiteam systems //www.jsc.nasa.gov/history/oral_histories/ReevesWD/ReevesWD_4-17-09.pdf concept and a look to the future,” in Multiteam Systems: An Organization form for Dynamic and Complex Environments, eds S. J. Zaccaro, M. A. Marks, and Rico, R., Hinsz, V. B., Davison, R. B., and Salas, E. (2017). Structural influences L. A. DeChurch (New York, NY: Routledge), 511–544. upon coordination and performance in multiteam systems. Hum. Resour. Manag. Rev. 28, 332–346. doi: 10.1016/j.hrmr.2017.02.001 Mathieu, J. E., Hollenbeck, J. R., van Knippenberg, D., and Ilgen, D. R. (2017). A century of work teams in the Journal of Applied Psychology. J. Appl. Psychol. Ryba, J. (2010). STS-61. Available at: https://www.nasa.gov/mission_pages/shuttle/ 102, 452–467. doi: 10.1037/apl0000128 shuttlemissions/archives/sts-61.html (accessed October 2, 2018). Mathieu, J. E., Marks, M. A., and Zaccaro, S. J. (2001). “Multi-team systems,” in Salas, E., Tannenbaum, S. I., Kozlowski, S. W., Miller, C. A., Mathieu, J. E., and International Handbook of Work and Organizational Psychology, Vol. 2, eds N. Vessey, W. B. (2015). Teams in space exploration: a new frontier for the Anderson, D. S. Ones, H. K. Sinangil, and C. Viswesvaran (Thousand Oaks, CA: science of team effectiveness. Curr. Dir. Psychol. Sci. 24, 200–207. doi: 10.1177/ Sage), 290. 0963721414566448 Mathieu, J. E., Tannenbaum, S. I., Donsbach, J. S., and Alliger, G. M. (2014). Sherif, M. (1958). Superordinate goals in the reduction of intergroup conflict. Am. A review and integration of team composition models: moving toward a J.Sociol. 63, 349–356. doi: 10.1086/222258 dynamic and temporal framework. J. Manag. 40, 130–160. doi: 10.1177/ 0149206313503014 Shuffler, M. L., and Carter, D. R. (2018). Teamwork situated in multiteam systems: key lessons learned and future opportunities. Am. Psychol. 73, 390–406. doi: McGill, D. C. (2015). David C. McGill (R. Wright, Interviewer). NASA Johnson 10.1037/amp0000322 Space Center Oral History Project Oral History Transcript. Available at: https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/ Shuffler, M. L., Jiménez-Rodríguez, M., and Kramer, W. S. (2015). The science of McGillDC/McGillDC_5-22-15.htm (accessed October 2, 2018). multiteam systems: a review and future research agenda. Small Group Res. 46, 659–699. doi: 10.1177/1046496415603455 McGrath, J. E. (1984). Groups: Interaction and Performance, Vol. 14. Englewood Cliffs, NJ: Prentice-Hall. Steiner, I. D. (1972). Group Process and Productivity. New York, NY: Academic. Suffredini, M. T. (2015). Michael T. Suffredini (R. Wright, interviewer). NASA McGrath, J. E., Arrow, H., and Berdahl, J. L. (2000). The study of groups: past, present, and future. Personal. Soc. Psychol. Rev. 4, 95–105. Johnson Space Center Oral History Project Oral History Transcript. Available at: https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_ Mesmer-Magnus, J. R., Carter, D. R., Asencio, R., and DeChurch, L. A. (2016). histories/ISS/SuffrediniMT/SuffrediniMT_9-29-15.htm (accessed October Space exploration illuminates the next frontier for teams research. Group Organ. 2, 2018). Manag. 41, 595–628. doi: 10.1177/1059601116668763 Tajfel, H., Turner, J. C., Austin, W. G., and Worchel, S. (1979). “An Integrative theory of Intergroup Conflict,” in Organizational Identity: A Moon, H., Hollenbeck, J. R., Humphrey, S. E., Ilgen, D. R., West, B., Ellis, A. P. J., reader, eds W. G. Austin, and S. Worchel (Monterey, CA: Brooks/Cole). et al. (2004). Asymmetric adaptability: dynamic team structures as one-way 56–65. streets. Acad. Manag. J. 47, 681–695. doi: 10.5465/20159611 Trump, D. J. (2017). Presidential Memorandum on Reinvigorating America’s Human space Exploration Program [Memorandum]. Washington, DC: Office National Aeronautics and Space Administration [NASA] (2014). NASA’s Journey to of the President. Mars: Pioneering Next Steps in Space Exploration. Houston, TX: NASA. Weick, K. E., Sutcliffe, K. M., and Obstfeld, D. (1999). Organizing for high reliability: processes of collective mindfulness. Cris. Manag. 3, 81–123. Nesbitt, S. (2016). An Oral History of the Space Shuttle Challenger Wohlin, C. (2014). “Guidelines for snowballing in systematic literature studies and Disaster (M. L. Dean, interviewer). Popular Mechanics. Available at: a replication in software engineering,” in Proceedings of the 18th International https://www.popularmechanics.com/space/a18616/an-oral-history-of-the- Conference on Evaluation and Assessment in Software Engineering. New York, space-shuttle-challenger-disaster/ (accessed October 2, 2018). NY: ACM, 38. Zaccaro, S. J., Marks, M. A., and DeChurch, L. A. (eds). (2012). “Multiteam Ofem, B., Floyd, T. M., and Borgatti, S. P. (2012). “Social Networks and systems: An introduction,” in Multiteam Systems, (New York, NY: Routledge), Organizations,” in A Companion to Organizational Anthropology, eds D. D. 18–47. Caulkins and S. P. Borgatti (Hoboken, NJ: Blackwell Publishing Limited), 147–166. Conflict of Interest Statement: The authors declare that the research was conducted in the absence of any commercial or financial relationships that could Palinkas, L. A. (2007). Psychosocial issues in long-term space flight: overview. be construed as a potential conflict of interest. Gravitat. Space Res. 14, 25–33. Copyright © 2019 Pendergraft, Carter, Tseng, Landon, Slack and Shuffler. This is an Palinkas, L. A., and Suedfeld, P. (2008). Psychological effects of polar expeditions. open-access article distributed under the terms of the Creative Commons Attribution Lancet 371, 153–163. doi: 10.1016/s0140-6736(07)61056-3 License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that the Porck, J. P., Matta, F. K., Hollenbeck, J., Oh, J. K., Lanaj, K., and Lee, S. M. original publication in this journal is cited, in accordance with accepted academic (2018). Social Identification in Multiteam Systems: The Role of Depletion and practice. No use, distribution or reproduction is permitted which does not comply Task Complexity. Briarcliff Manor, NY: Academy of Management. with these terms. Presidential Commission on the Space Shuttle Challenger Accident (1986). Report to the President by the Presidential Commission on the Space Shuttle Challenger Accident. Washington, DC: United States Government Printing. Presidential Commission on the Space Shuttle Challenger Accident (1987). Implementation of the Recommendations of the Presidential Commission on the Space Shuttle Challenger Accident. Washington, DC: United States Government Printing. Frontiers in Psychology | www.frontiersin.org 21199 July 2019 | Volume 10 | Article 1633

Pendergraft et al. SFMTS Adaptations APPENDIX A Retrieved From TABLE A1 | List of sources used in archival analysis. https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/Shuttle- Mir/ ThomasASW/ThomasASW_7- 22- 98.htm Johnson Space Center Oral Histories https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/AldrichAD/AldrichAD_ 6- 24- 00.htm Subject Date of interview https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/DunbarBJ/DunbarBJ_ 1- 20- 05.htm Andrew S.W. Thomas 7/22/1998 https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/Shuttle- Mir/DunbarBJ/ DunbarBJ_6- 16- 98.htm Arnold D. Aldrich 6/24/2000 https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/DunbarBJ/DunbarBJ_ 3- 23- 05.htm Bonnie J. Dunbar 1/20/2005 https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/DunbarBJ/DunbarBJ_ 9- 14- 05.htm Bonnie J. Dunbar 6/16/1998 https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/Shuttle- Mir/FoaleCM/ FoaleCM_6- 16- 98.htm Bonnie J. Dunbar 3/23/2005 https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/Shuttle- Mir/FoaleCM/ FoaleCM_7- 7- 98.htm Bonnie J. Dunbar 9/14/2005 https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/Shuttle- Mir/FoaleCM/ FoaleCM_7- 31- 98.htm C. Michael Foale 6/16/1998 https://www.nasa.gov/sites/default/files/atoms/files/19910628_christopher_kraft_oral_history_ interview.pdf C. Michael Foale 7/7/1998 https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/McGillDC/McGillDC_ 5- 22- 15.htm C. Michael Foale 7/31/1998 https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/ArabianDD/DDA_2- 3- 00- amended.pdf Christopher C. Kraft 6/28/1991 https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/KranzEF/KranzEF_1- 8- 99.htm David C. McGill 5/22/2015 https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/CarrGP/CarrGP_10- 25- 00.htm Donald D. Arabian 2/3/2000 https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/LunneyGS/Apollo13.htm https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/BlufordGS/BlufordGS_ Eugene F. Kranz 1/8/1999 8- 2- 04.htm https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/LousmaJR/ Gerald P. Carr 10/25/2000 LousmaJR_3- 15- 10.htm https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/AaronJW/AaronJW_1- Glynn S. Lunney 7/16/2010 26- 00.htm Guion S. Bluford 8/2/2004 https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/AllenJP/AllenJP_1- 28- 03.htm Jack R. Lousma 3/15/2010 https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/SilverLT/SilverLT_5- 5- 2002.pdf John W. Aaron 1/26/2000 https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/ISS/BarrattMR/ BarrattMR_7- 30- 15.htm Joseph P. Allen 1/28/2003 https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/Shuttle- Mir/BarrattMR/ BarrattMR_4- 14- 98.htm Leon T. Silver 5/5/2002 https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/ISS/SuffrediniMT/ SuffrediniMT_9- 29- 15.htm Michael R. Barratt 7/30/2015 https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/Shuttle- Mir/DyePF/ DyePF_5- 27- 98.htm Michael R. Barratt 4/14/1998 https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/HillPS/HillPS_3-24- 15.htm Michael T. Suffredini 9/29/2015 (Continued) Paul F. Dye 6/16/1998 Paul S. Hill 3/24/2015 Frontiers in Psychology | www.frontiersin.org 22200 July 2019 | Volume 10 | Article 1633

Pendergraft et al. SFMTS Adaptations TABLE A1 | Continued Johnson Space Center Oral Histories Subject Date of interview Retrieved From https://www.spaceflight.nasa.gov/history/shuttle- mir/people/oral- histories/reeves.pdf William D. Reeves 6/22/1998 https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/ReevesWD/ReevesWD_ William D. Reeves 4/17/2009 4-17-09.htm Official NASA or Government Reports Retrieved From Report Name Relevant Mission or https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/Shuttle- Mir/ Program EngelaufPL/EngelaufPL_6- 24- 98.htm https://historycollection.jsc.nasa.gov/JSCHistoryPortal/history/oral_histories/AndersWA/ Phillip L. Engelauf 6/24/1998 AndersWA_10- 8- 97.htm https://history.nasa.gov/rogersrep/actions.pdf William A. Anders 10/8/1997 https://www.hq.nasa.gov/alsj/a13/A13_MissionReport.pdf Actions to Implement the STS-51L https://www.nasa.gov/mission_pages/station/structure/elements/nasa_rsa.html Recommendations of The Presidential Commission on Apollo 13 https://www.nasa.gov/columbia/home/CAIB_Vol1.html the Space Shuttle ISS (http://s3.amazonaws.com/akamai.netstorage/anon.nasa-global/CAIB/CAIB_lowres_full.pdf) Challenger Accident STS-107 STS-51L https://history.nasa.gov/rogersrep/v6index.htm Apollo 13 Mission Report STS-51L https://www.gpo.gov/fdsys/pkg/GPO- CRPT- 99hrpt1016/pdf/GPO- CRPT- 99hrpt1016.pdf Bilateral agreement Space Shuttle Program between NASA and the AS-204 https://history.nasa.gov/stsnixon.htm Russian Space Agency AS-204 https://history.nasa.gov/Apollo204/chro.html Columbia Accident STS-51L https://history.nasa.gov/as204_senate_956.pdf Investigation Board [CAIB] (2003). Columbia accident Human Spaceflight https://history.nasa.gov/rogersrep/genindex.htm investigation board report. Program https://www.nasa.gov/pdf/396093main_HSF_Cmte_FinalReport.pdf Implementation of the Recommendations of the Presidential Commission on the Space Shuttle Challenger Accident Investigation of the Challenger Accident Congressional Report President Nixon’s 1972 Announcement on the Space Shuttle Report of Review Board on Apollo mission AS-204 Report of the Committee on Aeronautical and Space Sciences, United States Senate with Additional Views – Apollo 204 Accident, January 30, 1968 Report of the PRESIDENTIAL COMMISSION on the Space Shuttle Challenger Accident 6/6/1986 Seeking a Human Spaceflight Program Worthy of a Great Nation (Continued) Frontiers in Psychology | www.frontiersin.org 22211 July 2019 | Volume 10 | Article 1633

Pendergraft et al. SFMTS Adaptations TABLE A1 | Continued Retrieved From Johnson Space Center Oral Histories Retrieved From Subject Date of interview https://history.nasa.gov/40thann/define.htm https://www.nasa.gov/image- feature/apollo- 13- lunar- module- mail- box Other Sources https://www.nasa.gov/mission_pages/station/structure/elements/space- station- assembly https://www.nasa.gov/mission_pages/station/cooperation/index.html Source Name Relevant Mission or https://history.nasa.gov/Apollo204/zorn/white.htm Program http://www.esa.int/Our_Activities/Human_Spaceflight/International_Space_Station/Liftoff_ Alexander_Gerst_returns_to_space \"Chronology of Defining Various https://spaceflight.nasa.gov/history/shuttle- mir/Shuttle- Mir_text- only.htm Events in NASA history\" Apollo 13 https://www.nasa.gov/centers/johnson/news/station/1998/iss98- 03.html ISS https://history.nasa.gov/40thann/define.htm Apollo 13 Lunar Module ISS https://www.nasa.gov/mission_pages/shuttle/shuttlemissions/archives/sts- 71.html ’Mail Box’ Gemini 4 https://history.nasa.gov/SP- 4225/nasa4/nasa4.htm Soyuz MS-09 https://history.nasa.gov/SP- 4225/nasa4/nasa4.htm#communications Description of ISS modules Shuttle-Mir program https://history.nasa.gov/SP- 350/ch- 13- 4.html ISS; launch of Zarya https://www.nasa.gov/mission_pages/shuttle/shuttlemissions/archives/sts- 9.html Description of ISS module https://www.nasa.gov/mission_pages/shuttle/shuttlemissions/archives/sts- 27.html participants and roles Apollo-Soyuz https://www.nasa.gov/mission_pages/shuttle/shuttlemissions/archives/sts- 61.html STS-71 https://www.issnationallab.org/about/iss- timeline/ Ed White biography; Shuttle-Mir/NASA-4 https://www.nasa.gov/mission_pages/shuttle/shuttlemissions/archives/sts- 72.html section on Gemini 4 EVA Shuttle-Mir/NASA-4 https://www.nasa.gov/mission_pages/shuttle/shuttlemissions/archives/sts- 74.html Apollo 13 https://www.nasa.gov/mission_pages/shuttle/shuttlemissions/archives/sts- 75.html ESA article on Soyuz STS-9 https://www.nasa.gov/mission_pages/shuttle/shuttlemissions/archives/sts- 114.html MS-09 STS-27 https://www.nasa.gov/mission_pages/shuttle/shuttlemissions/archives/sts- 2.html STS-61 https://www.nasa.gov/mission_pages/shuttle/shuttlemissions/archives/sts- 82.html History of Shuttle-Mir ISS https://www.nasa.gov/mission_pages/shuttle/shuttlemissions/archives/sts- 86.html STS-72 https://www.nasa.gov/pdf/566071main_STS- 135_Press_Kit.pdf International Space Station STS-74 https://nssdc.gsfc.nasa.gov/planetary/lunar/apollo1info.html Status Report – ISS98-03 STS-75 https://www.nytimes.com/1986/01/29/us/shuttle- explosion- mission- control- silence- grief- fill- day- STS-114 horror- long- dreaded.html NASA chronology of STS-2 Apollo-Soyuz missions STS-82 (Continued) STS-86 NASA mission archive on STS-135 STS-71 Apollo 204 (Apollo 1) Challenger STS-51L NASA-4: Fire and Controversy NASA-4: Failures to Communicate NASA history web article on Apollo 13 NASA mission archive on STS-9 NASA mission archive on STS-27 NASA mission archive on STS-61 Timeline of notable ISS events NASA mission archive on STS-72 NASA mission archive on STS-74 NASA mission archive on STS-75 NASA mission archive on STS-114 NASA mission archive on STS-2 NASA mission archive on STS-82 NASA mission archive on STS-86 NASA STS-135 Press Kit NASA web article on Apollo mission AS-204 New York Times Article on STS-51L Frontiers in Psychology | www.frontiersin.org 22222 July 2019 | Volume 10 | Article 1633

Pendergraft et al. SFMTS Adaptations TABLE A1 | Continued Retrieved From https://www.npr.org/templates/story/story.php?storyId=5175151 Johnson Space Center Oral Histories https://www.npr.org/templates/story/story.php?storyId=5174355 Subject Date of interview https://history.nasa.gov/SP- 4208/ch13.htm NPR web article on Challenger STS-51L Challenger mission https://history.nasa.gov/SP- 4208/ch14.htm STS-51L Challenger STS-51L https://www.hq.nasa.gov/pao/History/apollo/apo13hist.html NPR web article on Skylab station https://nssdc.gsfc.nasa.gov/planetary/gemini.html Challenger mission http://www.americaspace.com/2015/02/01/lock- the- doors- columbias- final- flight- part- 4/ STS-51L SL-2 http://www.spacefacts.de/mission/english/gemini- 6.htm http://www.spaceflightinsider.com/missions/iss/space- station- trio- returns- to- earth- after- record- SP-4208 Living and Apollo 13 setting- mission/ Working in Space: A Project Gemini http://www.spaceflightinsider.com/missions/iss/spacewalking- astronauts- finish- canadarm2- work- History of Skylab: Chapter Space Shuttle Columbia at- breakneck- speed/ 13 – Launching Skylab STS 107 http://www.spaceflightinsider.com/missions/iss/soyuz- ms- 07- crew- back- on- earth- after- 168- Gemini 6 days- in- orbit/ SP-4208 Living and ISS Expedition 54 http://www.spaceflightinsider.com/space-flight-history/spaceflight-heritage-sts-9-first-flight- Working in Space: A ISS Expedition 54 spacelab/ History of Skylab: Chapter Soyuz MS-07 14 – Saving Skylab STS-9; Spacelab The Flight of Apollo 13 The Gemini Program (1962-1966) Web article on Columbia STS-107 Web article on Gemini VI Web article on ISS Expedition 54 Web article on ISS spacewalk Web article on Soyuz mission MS-07 Web article on STS-9, Spacelab Frontiers in Psychology | www.frontiersin.org 22233 July 2019 | Volume 10 | Article 1633

ORIGINAL RESEARCH published: 12 August 2019 doi: 10.3389/fpsyg.2019.01660 Advancing Our Understandings of Healthcare Team Dynamics From the Simulation Room to the Operating Room: A Neurodynamic Perspective Ronald Stevens1,2*, Trysha Galloway2 and Ann Willemsen-Dunlap3 1UCLA School of Medicine, Brain Research Institute, Culver City, CA, United States, 2The Learning Chameleon, Inc., Culver City, CA, United States, 3JUMP Simulation and Education Center, The Order of Saint Francis Hospital, Peoria, IL, United States Edited by: The initial models of team and team member dynamics using biometric data in healthcare Michael Rosen, will likely come from simulations. But how confident are we that the simulation-derived Johns Hopkins Medicine, high-resolution dynamics will reflect those of teams working with live patients? We have developed neurodynamic models of a neurosurgery team while they performed a peroneal United States nerve decompression surgery on a patient to approach this question. The models were constructed from EEG-derived measures that provided second-by-second estimates of Reviewed by: the neurodynamic responses of the team and team members to task uncertainty. The M. Teresa Anguera, anesthesiologist and two neurosurgeons developed peaks, often coordinated, of elevated University of Barcelona, Spain neurodynamic organization during the patient preparation and surgery which were similar to those seen during simulation training, and which occurred near important episodes of Sadaf Kazi, the patient preparation and surgery. As the analyses moved down the neurodynamic Johns Hopkins University, hierarchy, and the simulation and live patient neurodynamics occurring during the intubation procedure were compared at progressively smaller time scales, differences emerged United States across scalp locations and EEG frequencies. The most significant was the pronounced suppression of gamma rhythms detected by the frontal scalp sensors during the live *Correspondence: patient intubation which was absent in simulation trials of the intubation procedure. These Ronald Stevens results indicate that while profiles of the second-by-second neurodynamics of teams were similar in both the simulation and live patient environments, a deeper analysis revealed [email protected] differences in the EEG frequencies and scalp locations of the signals responsible for those team dynamics. As measures of individual and team performance become more micro- Specialty section: scale and dynamic, and simulations become extended into virtual environments, these This article was submitted to results argue for the need for parallel studies in live environments to validate the dynamics of cognition being observed. Organizational Psychology, a section of the journal Keywords: teamwork, healthcare, electroencephalography, team neurodynamics, information, operating room, Frontiers in Psychology intubation Received: 31 October 2018 Accepted: 01 July 2019 Published: 12 August 2019 Citation: Stevens R, Galloway T and Willemsen-Dunlap A (2019) Advancing Our Understandings of Healthcare Team Dynamics From the Simulation Room to the Operating Room: A Neurodynamic Perspective. Front. Psychol. 10:1660. doi: 10.3389/fpsyg.2019.01660 Frontiers in Psychology | www.frontiersin.org 2124 August 2019 | Volume 10 | Article 1660

Stevens et al. Team Neurodynamics in Simulation and Live Environments INTRODUCTION are capable of resolving cognitive processes occurring at the milliseconds level using electrical oscillations from different A shift is underway in the ways that we  study the function regions on the scalp (Buzaki, 2006). and evolution of teams. It is being driven by the generation of multimodal biometric dynamic data streams with seconds’ The metric developed, neurodynamic organization (NO), is resolutions, and it is expected that analyses of these data will the tendency of team members to enter into prolonged (>10s) shape our ideas about how teams are assembled, trained, and metastable neurodynamic relationships as they experience supported. (Guastello et  al., 2006; Aebersold, 2018; Guastello disturbances to their rhythms, i.e., periods of heightened and Peressini, 2018; Stevens et  al., 2018b). In healthcare, the uncertainty. This metric is domain neutral and thought to initial understandings of how patterns in dynamic biometric occur when a team’s operating rhythm no longer supports data sets relate to team member interactions and task events the complexity of the task and the team needs to expend will likely come from simulation settings. energy to reorganize into structures that better minimize the “surprise” or uncertainty in the environment (Stevens and High-fidelity simulations provide opportunities for skill Galloway, 2017). Consistent with this hypothesis, the frequency acquisition and maintenance, team training, as well as high- and magnitude of neurodynamic organizations were greater stakes testing, and are widely accepted today as an essential in novice teams compared with experienced submarine navigation educational modality for healthcare professionals (Schmidt et al., teams (Stevens et  al., 2017a). 2013; Thomas et  al., 2015; Staropoli et  al., 2018). Simulation provides a mechanism for standardized clinical education across Measures of NO are grounded in information theory and all learners, allowing exposure to critical events that clinicians based on most biological signals having internal patterns and might never encounter in their career in a live patient. Simulation organizations. Symbolic transformations of discrete data can also provides a mechanism for deliberate practice among learners. be  used to detect and quantitate the fluctuating dynamics of Rare but critical and time-pressured events can be  recreated these patterns (Stevens and Galloway, 2014, 2015, 2017), while in a simulation, so that protocols can be  established and information theory provides the methods for determining when communication problems can be identified and improved upon. and how information is created, stored, shared, and destroyed Finally, simulation provides a safe environment where learners (Shannon, 1948, 1951; James et  al., 2011). can come together as inter-professional teams to practice critical teamwork skills that are often overlooked in clinical teaching. A series of studies spanning high school teams to military These accomplishments have been achieved through continual and healthcare teams (Stevens and Galloway, 2014, 2015, 2017) refinements in simulation technology, performance measurement, has indicated that neurodynamic organizations are likely a and training protocols (Magee, 2003). fundamental property of teamwork. Using information theory metrics, it becomes possible to quantitatively deconstruct the The shift toward more dynamic biometric models of teamwork neurodynamic organization of a team into the contributions provides an opportunity to expand our understanding of the of each team member (Stevens et  al., 2018b). These features spatial and temporal changes in team and team member provide a quantitative platform for comparing the cognitive cognition at a finer granularity than has been previously activities and live patient healthcare environments. possible, and to approach questions that have previously been unapproachable. As these models will most likely be developed The goals of this study were to: from simulation-derived data, it is important to learn how 1. First, determine whether teams that performed a live patient well metrics and models developed from simulated team training reflect those obtained in real-world operating room situations. operation (LPO) developed distinct peaks of neurodynamic Knowing if, and under what conditions, the cognitive responses organization similar to those we have previously observed during for a task deviated between simulated and live patient tasks military and healthcare simulated tasks. We hypothesized that environments would provide ecologic validity for the biometric the anesthesiologist, primary neurosurgeon, and neurosurgery models being developed. resident would develop discrete periods of elevated neurodynamic organization during the patient preparation and surgery, and Where along the biometric time scale of team training that these elevations would occur near episodes of importance (i.e., 10−3 to over 105  s) (Salas et  al., 2015) would differences or uncertainty. be expected? The widespread use of simulations in healthcare 2 . Second, identify whether there were times when the would argue against major differences being seen between neurodynamic/cognitive features of the LPO team member behavioral and biometric measures as these would have likely performances diverged from those expected from similar already been incorporated into simulation developments. events performed during simulations. Differences might be  more expected during the execution of temporally extended episodes of action-control sequences MATERIALS AND METHODS like those found in established surgical procedures or Ethics Statement anesthesia induction. Such episodes contain sub-sequences of actions but are mentally instantiated as one program The study and the informed consent protocols were reviewed unit (Cooper and Shallice, 2000). and approved by the Biomedical IRB, San Diego, CA (Protocol EEG01), and the Order of Saint Francis Healthcare Institutional The approach we  have taken to investigate the detailed Review Board, Peoria IL. All participating subjects gave dynamics of such episodes are EEG-derived measures which Frontiers in Psychology | www.frontiersin.org 225 August 2019 | Volume 10 | Article 1660

Stevens et al. Team Neurodynamics in Simulation and Live Environments written and informed consent to participate in the EEG data of the larynx which was experienced by the AN as a blockage collections and have their data (including images and speech) during an initial intubation (INTB) attempt. When the transient anonymously analyzed per approved applicable protocols. To seizure subsided, a second and successful INTB was performed. maintain confidentiality, each subject was assigned a unique The total scenario time was 800  s. number known only to the investigators of the study, and subject identities were not shared. This design complies with The significant training event in the second simulation was DHHS: protected human subject 45 CFR 46; FDA: informed a fire in the operating room, which required patient and staff consent 21 CFR 50. evacuation. Prior to the fire event, the INTB in this simulation was uncomplicated. The total scenario time was 967  s. Simulations and Live Patient The live patient operation to relieve pressure on the The team members participating in both the simulation and peroneal nerve was performed by a highly experienced surgery were experienced operating room staff at the Order neurosurgeon and a resident neurosurgeon. Succinctly, the of Saint Francis Hospital. It is likely some of them have worked surgery required an incision, an opening of the muscle fascia, together during their professional experiences, but no effort the identification of the nerve, the removal of the pressure, was made to quantify the level of interaction. The simulations and skin closure. The time from the patient entering the performed were part of an integrated curriculum of airway operating room (OR) until the completion of the surgery management that was developed following a clinical needs was 2,891  s. assessment at the Order of Saint Francis Hospital in Peoria, IL. The induction, ventilation, and emergence from anesthesia is Electroencephalography a complicated and uncertain process and one where differences in the cognition used between simulated and live patient Electroencephalography (EEG) data were collected using two ventilations would be  detected if present. EEG 10–20 systems with different sensor options (Figure  1). The 10–20 system permits uniform spacing of electrodes, While we  have reported neurodynamic analyses of over a independent of head circumference, in scalp regions known dozen healthcare team performances (Stevens et al., 2016, 2018b; to correlate with specific areas of cerebral cortex. It is the Stevens and Galloway, 2017), in this paper, we  highlight the standard electrode location method used to collect EEG data dynamics of two, as the same anesthesiologist who performed as well as the standard for most current databases. The the intubation during the live patient surgery performed two simulation-derived EEG signals were acquired using a nine- previous simulations with three intubation events. sensor wet electrode system which provided coverage over the anterior, central, and posterior regions of the scalp (Figure 1A, The first simulation involved the preoperative ventilation open circles). Collecting data for the live patient procedure by the anesthesiologist (AN), assisted by a circulating nurse was constrained by the surgeon requiring a binocular loupe, (CN), and a scrub nurse (SN), where the mannequin exhibited and (possibly) a light source on the top of his head. Additional an adverse response to a relative overdose of aerosolized clearance around the ears was also needed for the stethoscope. lidocaine; this subsequently caused seizure and cardiac The headband-styled 10-sensor dry electrode system used in dysrhythmias. The immune hypersensitivity also caused swelling the live patient data collection was embedded with sensors AB FIGURE 1  |  (A) Schematic of EEG sensor placement (looking down on the scalp) for the simulation tasks (open circles) and live patient (closed circles). (B) Neurodynamic information vs. EEG frequency plot for the average of the two simulation performances (open circles) and the live patient. The live patient data are plotted both for the whole task including patient removal (gray squares), as well as only during the operation (black circles). Frontiers in Psychology | www.frontiersin.org 2326 August 2019 | Volume 10 | Article 1660

Stevens et al. Team Neurodynamics in Simulation and Live Environments primarily in the anterior and posterior scalp regions (Figure 1A, which adaptively estimates and removes sinusoidal artifacts closed circles). from independent components or scalp sensors using a frequency- domain (multi-taper) regression technique with a Thompson A plot of the neurodynamic information at each EEG F-statistic for identifying significant sinusoidal artifacts and frequency bin is shown in Figure 1B. There were no significant independent component analysis. differences in the average NI levels in the 18–40  Hz frequency range. The simulation sensor montage detected higher NI levels Team Neurodynamic Modeling in the theta and alpha/mu frequency bands, due to the relative enrichment of 10-Hz team NI over the central scalp positions. The neurodynamic modeling is a physical to organizational – Unless otherwise noted, subsequent comparisons between the based transformation between what is observed at the team simulation and live patient performances were made using NI level, to the neurodynamic rhythms responsible for those behaviors. levels from the anterior and posterior regions of the scalp In this transformation, the physical units of EEG dynamics and the 18–40  Hz frequency bands. (i.e., microvolts) are transformed into informational units (bits) of organization. The elements of this transformation form a For all studies, the data acquisition began shortly after the hierarchy that spans temporal scales from milliseconds to hours. EEG sensors were adjusted for good contact (<10  Ω). Each person’s EEG data stream were cut into segments of the simulated The EEG power levels of each team member are first separated or live patient performance based on electronic markers inserted each second into high, medium, or low EEG power ranges into the EEG data streams as well as the events observed (Figure 2A). The reporting of team member neurodynamics in videos. The recorded EEG data were preprocessed using at a one-second resolution is in the range (250–500  ms) of functional brain connectivity associated with speech or playing ® ®Matlab -based FieldTrip toolbox (Oostenveld et  al., 2011), guitar in duets (Stephens et  al., 2010; Sanger et  al., 2012), and nonverbal recognitions (Caetano et  al., 2007), or approximately and processed as described previously (Stevens et  al., 2013; a half a second for a two-person action-response round trip. Stevens et  al., 2016). Signals from outside the brain can be  a confounder when interpreting models built from EEG signals, For ease of visualization, the high, average, and low EEG especially signals obtained in complex environments. Commonly power categories are assigned the values 3, 1, and −1. The found artifacts are generated from speech, eye blinks, heartbeats, resulting three-element array, one for each member of a three- breathing rhythms, and other electromyography sources. As person team, is assembled into a three-histogram neurodynamic neurodynamic organizations regularly occur during silence, symbol (NS) that represents the neurodynamic state of the team speech is an unlikely source for most organizations (Stevens at that second. For instance, the symbol in Figure 2B indicates and Galloway, 2014). Regular rhythms associated with eye that at this second, team member 1 had below average, team blinks and heartbeats were identified and removed during data member 2 had above average, and team member 3 had average EEG power levels. The possible combinations of three persons ®preprocessing (Delorme et  al., 2012), and by the interactive Matlab toolbox EEGLAB CleanLine (Mullen, 2012) plugin, BC A D E FIGURE 2  |  Levels of neurodynamic analyses. (A,B) the raw EEG signals from each person are discretized each second into low, average, and high power levels and assembled into a neurodynamic symbol. (C) The symbol matching the three-person power array is determined from the symbol state space lookup table and assembled into a neurodynamic data stream, where, (D) the team symbols are visually mapped and a moving average of entropy calculated each second. (E) Levels of raw EEG and normalized values (i.e., −1, 1, and 3) are calculated from the native EEG data streams. Frontiers in Psychology | www.frontiersin.org 2427 August 2019 | Volume 10 | Article 1660

Stevens et al. Team Neurodynamics in Simulation and Live Environments and three EEG power levels create a 27-symbol neurodynamic information (NI); this procedure makes increased neurodynamic state space (NSS) (Stevens and Galloway, 2014; Stevens et al., information and increased organization both positive values. 2017b). Each NS in the symbolic state space therefore situates the EEG power levels of each team member in the context of RESULTS the levels of the other team members and the context of the Team and Team Member Neurodynamics task. A sequence of these symbols, the neurodynamic data During Simulation Training streams (NDS) contain a neurodynamic history of the team’s performance. The granularity of the analysis can be  increased Tracing the frequency, magnitude, and duration of fluctuations by separating the EEG power into fourths or fifths with the in neurodynamic information provides a quantitative history of computational costs of an exponentially increasing NSS. a team’s neurodynamic responses to events that triggered the team to neurodynamically reorganize. The NI fluctuations of an The temporal expression of NS in all data streams studied experienced anesthesiology team performing a complicated sequence has been dynamic with one subset of symbols being expressed of ventilation procedures during a simulation are shown in for a minute or more, only to be  replaced by another symbol Figure 3. The events in this simulation included an early unsuccessful subset when the task dynamics changed. These NS concentrations INTB attempt (INTB-1), patient seizures requiring a call for a produce local variations in the randomness of the neurodynamic Crash Cart, and a second (successful) INTB attempt (INTB-2) data streams, differences that can be  quantitated by measuring (Figure 3A). This example was chosen from others available the entropy over a 60-s moving window over the symbol stream (Stevens et  al., 2016) as the AN performing this simulation had that is updated each second (Figure 2D). performed a similar procedure during a second simulation, and was also responsible for intubating the patient during the surgery. Entropy is the average surprise of outcomes sampled from a probability distribution or density. A NS density with low The team NI neurodynamic profile was low until 920  s entropy means that, on average, the outcome is relatively predictable and then increased during the first intubation attempt while a system with higher entropy would be  less predictable. (Figure  3B). After decreasing over the next 100  s, the NI In this way, a dynamic and quantitative pattern of organization again increased in response to the patient seizing, and remained (in bits) can be  constructed and reported with a 1-s granularity near the top of the interquartile range (IQR) and then decreased for real-time modeling, or aggregated over a performance for before peaking again during the second intubation attempt. comparisons across teams (Stevens and Galloway, 2017). The heterogeneity underlying the team neurodynamic profile At this point, the entropy-based units of organization was shown by deconstructing the team NI into that of each have become detached from the microvolt meaning of the team member using information theory approaches (Stevens raw EEG signal. For instance, synchronized high-power and et  al., 2018b). There were three NI peaks where the AN and desynchronized low-power alpha EEG rhythms have different CN showed coordinated NI dynamics and these were the first meanings in the context of attention and memory (Klimesch, intubation attempt (r  =  0.75 with AN leading CN at 30 s), 2012), but prolonged periods of either high or low alpha power the episode of seizure (r  =  0.84) and the second intubation would produce elevated neurodynamic organization and would (r  =  0.70 with AN leading CN at 10 s). This coordinated be  viewed as an organized selection of sequential actions behavior decreased during the middle of the task, i.e., between (Cooper and Shallice, 2000). the seizure episodes and the second intubation. The NI of the SN (Figure 3D) showed few defined fluctuations in response In practice, the modeling sequence in Figure 2 first generates to the evolving task, and also little coordination with the the three power categories for individual team members, at dynamics of AN or CN. each sensor channel and at each of forty 1-Hz frequency bins from 1 to 40  Hz (Figure 2A). Entropy calculations across the For each primary event, the AN made comments indicating streams of −1, 1, and 3 symbols of individual data streams uncertainty including: produce team member neurodynamic information profiles across 1. INTB-1: “There is pus or something in the trachea or an regions of their scalp and the EEG frequency spectrum (Figure 2E). obstruction, I  can’t tell which; I  think I  am  going to have The scalp and frequency-wide averages of the team NDS to go through it, do it with the trachea tube… It looks initially pinpoint periods of higher neurodynamic organization like he  is seizing.” which can then be  linked with task events. This initial step 2. Seizure V-tach: “Ok, that’s not unexpected. Let’s go ahead is followed by deconstruction of the team data into each team and take this out if he  is going into tach.” member’s sensor and frequency dynamics around regions of 3. Seizure/INTB-2: “I am  not sure what my other options are. interest (Stevens et al., 2018a). The total number of parallel Because he  has a history of seizures I  think we  are out data streams for a three-person team with every individual of drugs.” wearing a 10-sensor EEG headset, this would be  400 team 4 . INTB-2: “There is something in the trachea… I  am  not NDSs and 1,200 individual team member NDSs, as well as a sure if I  can see if it is a foreign body or…” similar number of parallel entropy data streams. These results suggest that events likely to increase team or individual uncertainty are also those that raise NI levels; in other As increased organization is accompanied by decreased entropy, the individual and team entropy values are subtracted from the maximum entropy for the number of symbols being modeled, i.e., 3.17 bits for 9 symbols or 4.775 bits for 27 symbols, and the resulting values are termed neurodynamic Frontiers in Psychology | www.frontiersin.org 2528 August 2019 | Volume 10 | Article 1660

Stevens et al. Team Neurodynamics in Simulation and Live Environments A B C D FIGURE 3  |  Team and individual neurodynamics during a healthcare simulation. (A) The task event segments. (B) A quantitative neurodynamic information profile is plotted for the team. The NI is a profile of the average bits of information using all sensors and frequencies. The dotted lines indicate the interquartile range (IQR), i.e., 25–75% of the data values, and the gray line indicates the IQR for the randomized data. (C) The NI traces of the AN (dark) and CN (light) during the simulation with selected events labeled. (D) The NI trace of the SN. words, NI may act as a barometer for the uncertainty for each without her actual involvement, both the neurodynamic member, and by extension, for the team (Stevens et  al., 2016). coordination with the AN and CN and the peaks of elevated NI were missing. That is, the task events that will increase The coordinated neurodynamics between the AN and CN NI have to be  meaningful for a person, not just interesting. during events requiring cooperation, yet independent neurodynamics while performing individual tasks, also suggest Team Neurodynamics During a Live the possibility of being able to separately identify periods of Patient Surgery teamwork and taskwork. Lastly, simulation-based neurodynamics may help refine what meaningful information for a team member The surgical team in this example consisted of the AN who might be. While the SN was watching, and likely understood had previously performed ventilation procedures during the details of the different task episodes being performed, simulation training, an experienced neurosurgeon (NS1) and Frontiers in Psychology | www.frontiersin.org 2629 August 2019 | Volume 10 | Article 1660

Stevens et al. Team Neurodynamics in Simulation and Live Environments a neurosurgery resident (NS2), a surgical nurse (SN) and a dynamics, especially during the surgery as shown in the dashed circulating nurse (CN); EEG data were collected and modeled outline (Figure 4D). These are investigated further in Figure 5. for the AN, NS1, and NS2 for this example. The surgical sequence for a peroneal nerve decompression As shown in Figure 4, the operating room setting differed begins with an incision, the spreading of the incision, and from most simulations by lasting three times longer than the opening of the underlying fascia. The nerve is then identified, simulations like that in Figure 3. There were also prolonged isolated, and stimulated if necessary. The tissue source of the periods when team members were outside the room as indicated compression is then identified and removed. by the dotted lines in the Speakers row (Figure 4B). This did not affect EEG collection which was being recorded on a The early surgical segments (until ~2,500 s) were performed headset chip, but it interfered with the ability to link the EEG by NS2 assisted by NS1. During the surgery, there were three with events during those periods. episodes of correlated NI between NS1 and NS2 (r  =  0.79 at a 20-s cross-correlation lag around 1980s), r = 0.43 at ~2,300 s, If the observed simulation neurodynamics were accurate and r  =  0.75 at ~2,400  s), and these occurred while the representations of those occurring during surgery, then with neurosurgeons worked closely together. After the nerve was the operating room team, we  would expect to see: isolated and the source of the nerve compression was identified, 1. The presence of discrete NI peaks near important events. NS1 performed the removal of the compressive block (from 2. The differential responses of team members to these events. 2,460 to 2,709  s); during this final procedure, only the NI of 3. Aligned team member NI fluctuations during NS1 was elevated. coordinated activities. The neurodynamic similarities in the NI profiles derived Consistent with the first goal, the neurodynamics of the from the simulation and live patient-derived conditions indicate surgery team showed discrete peaks of increased NI during that at the level of temporal dynamics, the simulation-acquired the preoperative patient ventilation as well as surgical preparation data provide an accurate representation of the types of and subsequently during the surgery (Figure 4C). The neurodynamics that will be  observed in real-world situations. deconstruction of the team NI into those of the AN, NS1, The coordinated NI dynamics between NS1 and NS2 are and NS2 showed periods of individual and coordinated NI similar to those seen between the AN and CN in Figure 3, therefore substantiating simulations cognitive  - ability to evoke neurodynamic correlates of teamwork. A B C D FIGURE 4  |  Team and individual neurodynamics during a peroneal nerve decompression surgery. (A). Task events. (B). Team member speech. (C). The team NI profile using the average bits of information from all sensors and frequencies. (D). The NI traces of the AN, NS1, and NS2. The dotted rectangle indicates the period of the surgery. Frontiers in Psychology | www.frontiersin.org 2730 August 2019 | Volume 10 | Article 1660

Stevens et al. Team Neurodynamics in Simulation and Live Environments FIGURE 5  |  Event details and team member NI profiles during surgery. B A FIGURE 6  |  (A) The NI values for the different frequency ranges are plotted for the final surgical procedure (2460–2,709 s). The arrow indicates when the surgeon completed his operation. (B) The across-frequency and sensor NI averages for the 10 EEG sensors. The member order in each bar cluster is AN, NS1, NS2. The next analysis examined the degree of neurodynamic until 2,633 s when they abruptly declined (Figure 6A). Coincident heterogeneity present in the extended period of NI associated with this decrease was NS1 completing the removal of the with the removal of the source of nerve compression. The compressive block on the nerve. The beta and gamma frequency analysis during this 4-min period searched for across-frequencies bands predominated after this period and then declined to temporal changes as well as across-the-scalp spatial changes baseline levels over the next minute. in NI dynamics. The NI levels during these 4  min were greatest at sensors The aim of these analyses was to determine if there was O2, F7, P7, and F8 (Figure 6B). The analyses were refined a neurodynamic trajectory from the initiation of the procedure, by generating time x frequency x NI plots for the F7, O2, through the peak period of neurodynamic information, to the and P7 sensors to explore the temporal and spatial sequencing return to a neurodynamic baseline. Neurodynamic information of NI levels across sensors and frequencies (Figure 7). profiles were generated for five EEG frequency bands: delta/ theta (3–7  Hz), alpha (8–11  Hz), mu (12–17  Hz), low beta Early NI increases were detected at the F7, P7, and (18–22  Hz), and high beta/gamma (23–40  Hz). The earliest O2 sensors ~30s into the final surgical procedure and and largest NI levels were in the 3–7  Hz (delta/theta) and were mostly in the 3–11  Hz range. The NI levels at the P7 8–11  Hz (alpha) frequency bands and these remained high sensor were short lived and followed by NI decreases at the F7 sensor. In contrast, the O2 NI levels continued to Frontiers in Psychology | www.frontiersin.org 2831 August 2019 | Volume 10 | Article 1660

Stevens et al. Team Neurodynamics in Simulation and Live Environments increase during the next 2  min and extended toward higher The NI values were binned into the delta/theta (3–7  Hz), frequencies. At epoch 2,633  s, the 3–11  Hz NI abruptly alpha (8–11  Hz), mu (12–17  Hz), low beta (18–22  Hz) high stopped at the O2 sensor, which, as described earlier, occurred beta (23–32  Hz), and gamma (33–40  Hz) bins. These after the alleviation of the nerve compression. During the comparisons were made using only the data from the INTB remaining time before closing the incision, there was an NI windows shown in Figure 8. increase in beta and gamma frequency bands, particularly at the P7 sensor. As previously described, NI is a measure of the organizational patterns in a neurodynamic data stream. As such, they could Neurodynamics of the Anesthesiologist represent persistent patterns of elevated, depressed, or During the Intubation Events intermediate EEG power levels by a team member or a team. Making this distinction is important as elevated gamma power The analyses of the peroneal nerve decompression surgery in Figures 6, 7 illustrate the neurodynamic heterogeneity within A an extended period of uncertainty, and show how this heterogeneity can be  used to describe the surgical procedure B in terms of a spatial and temporal neurodynamic trajectory. To explore the generality of these findings, a similar analysis C was performed upon another critical event during the operation which was the patient intubation procedure. The anesthesiologist FIGURE 7  |  Time x frequency vs. NI levels for the (A) F7, (B) O2, and (C) P7 who performed the patient intubation during the operation sensors. The NI levels are shown by the color bars to the right. previously performed three intubations under simulated conditions while acquiring EEG data that allowed neurodynamic comparisons across training modalities. The simulated and the live patient INTB segments were identified and isolated after bracketing them within 60-s data sections before and after the procedure to provide a dynamic context. Each of the INTB segments were above the IQR range for the performance indicating the procedure was one of importance for the anesthesiologist during both the simulations and in the operating room (Figure 8A). The four INTB segments ranged from 40 to 79  s in length and within each of the segments, there were peaks in the NI, often biphasic. One of the intubations (#1 of Figure 8B) was unsuccessful due to a blockage and the second intubation (#2) could not be confirmed as successful before the simulation ended. The other simulated and live patient intubations were successful. Aside from the elevated NI levels, there were no consistent defining features of the INTB procedures, which was not surprising with the temporal and intubation outcome differences among the trials. The analytic focus next shifted to the sensor NI levels during the INTB events. Because of the differences in the simulation and LPO EEG montages (Figure 1), these analyses contrasted the NI levels of the anterior and posterior sensors. These analyses were performed using the data from the INTB windows shown in Figure 8B. The anterior vs. posterior sensor regions’ NI levels for the simulation INTB events were not significantly different (Z = 0.77, p = 0.44, Wilcoxon), while the NI levels for the live patient INTB were nearly 3-fold greater at the anterior than posterior regions (Z = 2.02, p  <  0.05) (Figure  9). The anterior sensor NI levels were also significantly greater than the simulation groupings, indicating a skewing of the brain-wide neurodynamic organization toward the anterior regions during the live patient INTB procedure. The frequency band NI distributions were next generated across the 1–40  Hz spectrum shown in Figure 10. Frontiers in Psychology | www.frontiersin.org 2932 August 2019 | Volume 10 | Article 1660

Stevens et al. Team Neurodynamics in Simulation and Live Environments A B FIGURE 8  |  (A) The contexts of the INTB activities are shown by the area plots of the scenario NI; the periods of intubation are shown by horizontal lines. (B) This figure compares the neurodynamic information profiles of three simulated INTB attempts of varying difficulty with a live patient INTB attempt. has been associated with memory retrieval (Vergauwe and in both the simulation and live patient environments. This Cowan, 2014), whereas gamma power suppression has been provides an important validation of previous studies with associated with focused attention and while reading for military and healthcare teams where the team neurodynamics comprehension (Lachaux et  al., 2008; Ossandon et  al., 2011; were linked with speech (Gorman et  al., 2016), stressful Sato and Mizuhara, 2018). situations (Stevens et al., 2013), and expert performance ratings (Stevens and Galloway, 2017) during high-fidelity simulation Analyses were therefore performed using the high, average, training. They further suggest that developing models to track or low EEG values (i.e., −1, 1, or 3) rather than NI levels. the appearance of these fluctuations or estimate/predict their Figure 11 indicates that the elevated EEG beta-gamma NI magnitude and duration could have practical training levels found during the live patient INTB were due to low applications. For instance, providing these neurodynamic gamma EEG power values (H = 137, df = 3, p < 0.01) compared profiles to instructors prior to a debriefing following a training with the above average gamma power values during exercise could help focus the discussions around periods the simulation. where the team might have experienced uncertainty. Similarly, the periods of elevated NI could serve as triggers for providing DISCUSSION feedback in an intelligent tutoring setting for optimizing team health and performance. The results indicate that the sensor and frequency-averaged profiles of team and team member neurodynamics were similar While the overall neurodynamic profiles were similar under simulated and live patient conditions, according to the ideas Frontiers in Psychology | www.frontiersin.org 21303 August 2019 | Volume 10 | Article 1660

Stevens et al. Team Neurodynamics in Simulation and Live Environments FIGURE 9  |  NI levels at the anterior vs. posterior channels for the simulation (S) or the live patient (LP) INTB procedures. The frequency-averaged (18–40 Hz) NI levels were measured at the anterior (F3, Fz, F4) or posterior (P3, Pz, and P4) sensors for the simulation tasks, and the anterior (Fp1, Fpz, Fp2, F7, and F8) or posterior (P7, O1, Oz, O2, and P8) sensors for the live patient INTB. FIGURE 10  |  Frequency band distribution of NI for the INTB events. The FIGURE 11  |  Levels of EEG-gamma power during INTB events. The raw pooled low beta, high beta and gamma frequency bin NI levels from the live EEG values were determined for each of the intubation events; LP = live patient INTB were significantly greater than those from the simulations patient. (Mann Whitney, Z = 2.4, p = 0.01). behind hierarchal cognition, each NI peak is likely thought to begin by loading a sequence representation of neurodynamically heterogeneous. The appearance of patterns the task into memory, which controls the sequence and of elevated NI with the onset of meaningful events and their identify of the subtasks. Following the ideas of hierarchical decline after the task completion are consistent with the cognition, the component sequences are then executed idea they are neurodynamic representations of a set of (Schneider and Logan, 2006). procedures or subtasks needed to complete a task, i.e., a mental episode. Mental episodes are typically extended periods, An example of this heterogeneity, and the episodic nature with a defined beginning and ending, of focused deliberate of the final surgical procedure, is shown in Figures 7, 8 behavior during which a sequence of steps are completed where the neurodynamics revealed a change in the (Schneider and Logan, 2015). The execution of episodes is neurosurgeons cognitive state with the onset of the final surgical procedure. The primary focus for this neurodynamic reorganization was the occipital lobe at the 3–11 Hz frequencies. Frontiers in Psychology | www.frontiersin.org 21314 August 2019 | Volume 10 | Article 1660

Stevens et al. Team Neurodynamics in Simulation and Live Environments A second major cognitive state change occurred when the external world is associated with gamma rhythm suppression surgery was completed and the occipital lobe neurological in the default mode network (Ossandon et  al., 2011). This is organizations were replaced by a more heterogeneous frequency a series of brain regions linked with introspective thoughts profile at the P7 channel before returning to preoperation (Raichle et  al., 2001). levels. A similar neurodynamic analysis of the intubation procedure performed by the anesthesiologist suggests that Possible linkages between the reduced gamma rhythm levels each NI peak might show neurodynamic complexity at the we  have observed during the INTB event of the live patient sensor and frequency level. and previously reported spatially localized network and short- lived gamma suppression are difficult to speculate on from a The NI levels during the live patient INTB were unequally single sample. The possibility exists however that the INTB distributed between the anterior sensors where the levels with the live patient induced a more attentive state in the AN were significantly greater than those from the posterior than that provided by the simulations, suggesting a fundamental sensors. The anterior and posterior sensors’ NI levels from difference in the two environments. simulation attempts were not statistically different, but were intermediate to those at the anterior and posterior levels As expressed by the AN: “I was aware that the OR was a during the surgery. real patient and the lab case was just a simulation. I  felt the usual urgency in the real case to perform well as opposed to The finding of elevated neurodynamic organization in the the lab simulation where it’s more relaxed because you  know frontal regions during INTB may be  significant as frontal there isn’t anything important at stake.” As measures of individual regions have been implicated in the detection of unfavorable and team performance become more micro-scale and dynamic, outcomes, error correction, and resolution of uncertainty, all and simulations become extended into virtual environments, of which might be  expected to play a role during this critical these results argue for the (at least limited) need for parallel procedure (Ridderinkhof et  al., 2004; Murray and Rudebeck, studies in live environments to maximize the benefits from 2017). The EEG frequencies associated with the elevated frontal these emerging technologies. sensor NI were in the low beta – low gamma frequency range. Gamma EEG rhythms, or “gamma oscillations” emerge from ETHICS STATEMENT neuronal structures at rates from 30 to up to 300  Hz. Their rhythms are driven by balances of inhibitory GABAergic The study and the informed consent protocols were reviewed interneurons and excitatory glutamatergic neurons (Whittington and approved by the Biomedical IRB, San Diego, CA (Protocol et  al., 1995). Gamma oscillations occur alongside and in EEG01), and the Order of Saint Francis Healthcare Institutional proportion to perceptual processes/salience (Sedley and Review Board, Peoria IL. All participating subjects gave Cunningham, 2013) and are thought to be  pivotal in: (1) the written and informed consent to participate in the EEG search for information, or the refreshing of information within data collections and have their data (including images and the brain, and (2) the communication of this information across speech) anonymously analyzed per approved applicable regions of the brain. protocols. To maintain confidentiality, each subject was assigned a unique number known only to the investigators The suggestion of gamma rhythm involvement in the of the study, and subject identities were not shared. This search for information to populate short-term memory is based design complies with DHHS: protected human subject 45 on repeated observations showing decreased response speed CFR 46; FDA: informed consent 21 CFR 50. with the number of items in short-term memory, reaching a processing rate limit of 25–30 items per second (Vergauwe AUTHOR CONTRIBUTIONS and Cowan, 2014). These authors have proposed that information for features of one item are represented by groups of neurons RS and TG acquired and processed the EEG data for the that fire within a gamma cycle and this gamma-band simulation and live patient performances then performed the synchronization facilitates neural communication and neurodyamic modeling and generated and conducted the data synaptic plasticity. analysis. AW-D oversaw the development and implementation of the team simulation activities. All authors participated in Gamma rhythms do not act in isolation during this neural preparing the paper. communication, but become phase locked and nested within theta rhythms (~ 5–7 gamma per theta wave) or alpha oscillations FUNDING which serve to segment neuronal representations in time, and perhaps support their coordinated action across neuronal The studies were supported in part by the Jump Foundation assemblies (Bonnefond and Jensen, 2015). In these two instances, for Simulation Research and the Defense Advanced Research gamma activity increases. Projects Agency under contract W31P4QC0166 and the Illinois Neurological Institute. It is also becoming clear that attention-demanding tasks like reading for comprehension not only activate specific cortical regions, but also deactivate others that might interfere with the task either at local (Klimesch, 2012) or more distant cortical regions (Farooqi and Manly, 2018). Studies using intracerebral electrodes have suggested that focused interaction with the Frontiers in Psychology | www.frontiersin.org 21325 August 2019 | Volume 10 | Article 1660

Stevens et al. Team Neurodynamics in Simulation and Live Environments   Proc. Hum. Factors Ergon. Soc. Annu. Meet. 59, 235–239. doi: 10.1177/ 1541931215591048 REFERENCES Sanger, J., Muller, V., and Lindenberger, U. (2012). Intra-and interbrain synchronization and network properties when playing guitar in duets. Front. Aebersold, M. (2018). Simulation-based learning: no longer a novelty in Hum. Neurosci. 6:312. doi: 10.3389/fnhum.2012.00312 undergraduate education. Online J. Issues Nurs. 23. doi: 10.3912/OJIN. Sato, N., and Mizuhara, H. (2018). Successful encoding during natural reading Vol23No02PPT39 is associated with fixation related potential and large-scale network activation. eNeuro. 5. doi: 10.1523/ENEURO.0122-18.2018 Bonnefond, M., and Jensen, O. (2015). Gamma activity coupled to alpha phase Schmidt, E., Goldhaber-Fiebert, S. M., Ho, L. A., and McDonald, D. (2013). as a mechanism for top-down controlled gating. PLoS One 10:e012866. doi: Simulation exercises as a patient safety strategy: a systematic review. 10.1371/journal.pone.0128667 Ann. Intern. Med. 158, 426–432. doi: 10.7326/0003-4819-158-5- 201303051-00010 Buzaki, G. (2006). Rhythms of the brain. New York, NY: Oxford University Press. Schneider, D. W., and Logan, G. D. (2006). Hierarchical control of cognitive Caetano, G., Jousmaki, V., and Hari, R. (2007). Actor’s and observers processes: switching tasks in sequences. J. Exp. Psychol. Gen. 135, 623–640. doi: 10.1037/0096-3445.135.4.623 primary motor cortices stabilize similarly after seen or heard motor Schneider, D. W., and Logan, D. (2015). Chunking away task-switch costs: a actions. Proc. Natl. Acad. Sci. USA 104, 9058–9062. doi: 10.1073/ test of the chunk-point hypothesis. Psychon. Bull. Rev. 22, 884–889. doi: pnas.0702453104 10.3758/s13423-014-0721-3 Cooper, R., and Shallice, T. (2000). Contention scheduling and the control of Sedley, W., and Cunningham, M. O. (2013). Do cortical gamma oscillations routine activities. Cogn. Neuropsychol. 17, 297–338. doi: 10.1080/ promote or suppress perception? An under-asked question with an over- 026432900380427 assumed answer. Front. Hum. Neurosci. 7:595. doi: 10.3389/fnhum.2013.00595 Delorme, A., Palmer, J., Onton, J., Oostenveld, R., and Makeig, S. (2012). Shannon, C. E. (1948). A mathematical theory of communication. Bell Syst. Independent EEG sources are dipolar. PLoS One 7:e30135. doi: 10.1371/ Tech. J. 27, 379–423, 623–656. doi: 10.1002/j.1538-7305.1948.tb01338.x journal.pone.0030135 Shannon, C. E. (1951). Prediction and entropy of printed English. Bell Syst. Farooqi, A. A., and Manly, T. (2018). Hierarchical cognition causes task related Tech. J. 30, 50–64. doi: 10.1002/j.1538-7305.1951.tb01366.x deactivations but not just in the default mode regions. eNeuro. 5. doi: Staropoli, P. C., Gregori, N. Z., Junk, A. K., Galor, A., Goldhardt, R., Goldhagen, 10.1523/ENEURO.0008-18.2018 B. E., et al. (2018). Surgical simulation training reduces intraoperative cataract Gorman, J. C., Martin, M. J., Dunbar, T. A., Stevens, R. H., Galloway, T. L., surgery complications among residents. Simul. Healthc. 13, 11–15. doi: Amazeen, P. G., et al. (2016). Cross-level effects between neurophysiology 10.1097/SIH.0000000000000255 and communication during team training. Hum. Factors 58, 181–199. doi: Stephens, G., Silbert, L., and Hasson, U. (2010). Speaker-listener neural coupling 10.1177/0018720815602575 underlies successful communication. Proc. Natl. Acad. Sci. USA. 107, 14425–14430. Guastello, S. J., and Peressini, A. F. (2018). Development of a synchronization doi: 10.1073/pnas.1008662107 coefficient for biosocial interactions in groups and teams. Nonlinear Dynamics Stevens, R. H., and Galloway, T. (2014). Toward a quantitative description of Psychol. Life Sci. 48, 3–33. doi: 10.1177/1046496416675225 the neurodynamic organizations of teams. Soc. Neurosci. 9, 160–173. doi: Guastello, S. J., Pincus, D., and Gunderson, P. R. (2006). Electrodermal arousal 10.1080/17470919.2014.883324 between participants in a conversation: nonlinear dynamics for linkage effects. Stevens, R. H., and Galloway, T. (2015). Modeling the neurodynamic organizations Nonlinear Dynamics Psychol. Life Sci. 10, 365–399. and interactions of teams. Soc. Neurosci. 11, 123–139. doi: James, R. G., Ellison, C. J., and Crutchfield, J. P. (2011). Anatomy of a bit: 10.1080/17470919.2015.1056883 Information in a time series observation. Santa Fe Institute Working Paper Stevens, R. H., and Galloway, T. (2017). Are neurodynamic organizations a 11-05. arXiv:1105.2988v1 [cs.IT] [Preprint]. doi: 10.1063/1.3637494 fundamental property of teamwork? Front. Psychol. 8. doi: 10.3389/ Klimesch, W. (2012). Alpha-band oscillations, attention and controlled access fpsyg.2017.00644 to stored information. Trends Cogn. Sci. 16, 606–617. doi: 10.1016/j. Stevens, R. H., Galloway, T., Gorman, J., Willemsen-Dunlap, A., Halpin, D., tics.2012.10.007 and Grimm, D. (2018a). Human factors & ergonomics national meeting. Lachaux, J. P., Jung, J., Dreher, J. C., Bertrand, O., Minotti, L., Hoffman, D., Philadelphia, PA. 1, 167–172. et al. (2008). Silence is golden: transient neural deactivation in the prefrontal Stevens, R. H., Galloway, T., Halpin, D., and Willemsen-Dunlap, A. (2016). cortex during attentive reading. Cereb. Cortex 18, 443–450. doi: 10.1093/ Healthcare teams neurodyamically reorganize when resolving uncertainty. cercor/bhm085 Entropy 18:427. doi: 10.3390/e18120427 Magee, J. H. (2003). Validation of medical modeling & simulation training Stevens, R. H., Galloway, T., Lamb, J., Steed, R., and Lamb, C. (2017a). “Linking devices and systems. Stud. Health Technol. Inform. 94, 196–198. doi: team neurodynamic organizations with observational ratings of team 10.3233/978-1-60750-938-7-196 performance” in Innovative assessment of collaboration. eds. A. von Davier, Mullen, T. (2012). NITRC: CleanLine: Tool/Resource Info. Retrieved from: M. Zhu, and P. Kyllonen (New York, NY: Springer International Publishing). http://www.nitrc.org/projects/cleanline Stevens, R. H., Galloway, T., and Willemsen-Dunlap, A. (2017b). “Low level Murray, E. A., and Rudebeck, P. H. (2017). Specializations for reward-guided predictors of team dynamics: a neurodynamic approach” in Research in managing decision-making in the primate ventral prefrontal cortex. Nat. Rev. Neurosci. groups and teams research series: V. 18 team dynamics over time. eds. E. Salas, 19, 404–417. doi: 10.1038/s41583-018-0013-4 B. Vessey, and L. B. Landon (Emerald Publishing Limited), 71–92. Oostenveld, R., Fries, P., Maris, E., and Schoffelen, M. (2011). FieldTrip: open Stevens, R. H., Galloway, T., and Willemsen-Dunlap, A. (2018b). Quantitative source software for advanced analysis of MEG, EEG, and invasive modeling of individual, shared and team neurodynamic information. Hum. electrophysiological data. Comput. Intell. Neurosci. 2011:156869. doi: Factors 60, 1022–1034. doi: 10.1177/0018720818781623 10.1155/2011/156869 Stevens, R. H., Gorman, J. C., Amazeen, P., Likens, A., and Galloway, T. (2013). Ossandon, T., Jerbi, K., Vidal, J. R., Bayle, D. J., Henaff, M.-A., Jung, J., The organizational neurodynamics of teams. Nonlinear Dynamics Psychol. et al. (2011). Transient suppression of broadband gamma power in the Life Sci. 17, 67–86. default mode network is correlated with task complexity and subject Thomas, P. A., Kern, D. E., Hughes, M. T., and Chen, B. Y. (2015). Curriculum performance. J. Neurosci. 31, 14521–14530. doi: 10.1523/JNEUROSCI. development for medical education: A six-step approach. Baltimore, MD: Johns 2483-11.2011 Hopkins University Press. Raichle, M. E., MacLeod, A. M., Snyder, A. Z., Powers, W. J., Gusnard, D. A., Vergauwe, E., and Cowan, N. (2014). A common short-term memory retrieval and Shulman, G. L. (2001). A default mode of brain function. Proc. Natl. rate may describe many cognitive procedures. Front. Hum. Neurosci. 8. doi: Acad. Sci. USA 98, 676–682. doi: 10.1073/pnas.98.2.676 10.3389/fnhum.2014.00126 Ridderinkhof, K. R., Ullsperger, M., Crone, E. A., and Nieuwenhuis, S. (2004). The role of the medial frontal cortex in cognitive control. Science 306, 443–447. doi: 10.1126/science.1100301 Salas, E., Stevens, R., Gorman, J., Cooke, N. J., Guastello, S., and von Davier, A. (2015). What will quantitative measures of teamwork look like in 10 years? Frontiers in Psychology | www.frontiersin.org 21336 August 2019 | Volume 10 | Article 1660

Stevens et al. Team Neurodynamics in Simulation and Live Environments Whittington, M. A., Traub, R. D., and Jefferys, J. G. R. (1995). Synchronized Copyright © 2019 Stevens, Galloway and Willemsen-Dunlap. This is an oscillations in interneuron networks driven by metabotropic glutamate receptor open-access article distributed under the terms of the Creative Commons Attribution activation. Nature 373, 612–615. doi: 10.1038/373612a0 License (CC BY). The use, distribution or reproduction in other forums is permitted, provided the original author(s) and the copyright owner(s) are credited and that Conflict of Interest Statement: The authors declare that the research was conducted the original publication in this journal is cited, in accordance with accepted in the absence of any commercial or financial relationships that could be construed academic practice. No use, distribution or reproduction is permitted which does as a potential conflict of interest. not comply with these terms. Frontiers in Psychology | www.frontiersin.org 21347 August 2019 | Volume 10 | Article 1660

ORIGINAL RESEARCH published: 06 September 2019 doi: 10.3389/fcomm.2019.00050 The Evolution of Human-Autonomy Teams in Remotely Piloted Aircraft Systems Operations Mustafa Demir 1,2*, Nathan J. McNeese 3 and Nancy J. Cooke 1,2 1 Human Systems Engineering, Arizona State University, Mesa, AZ, United States, 2 The Cognitive Engineering Research Institute, Mesa, AZ, United States, 3 Human-Centered Computing, Clemson University, Clemson, SC, United States Edited by: The focus of this current research is 2-fold: (1) to understand how team interaction Eduardo Salas, in human-autonomy teams (HAT)s evolve in the Remotely Piloted Aircraft Systems Rice University, United States (RPAS) task context, and (2) to understand how HATs respond to three types of failures (automation, autonomy, and cyber-attack) over time. We summarize the findings from Reviewed by: three of our recent experiments regarding the team interaction within HAT over time in Gilbert Ernest Franco, the dynamic context of RPAS. For the first and the second experiments, we summarize Beacon College, United States general findings related to team member interaction of a three-member team over time, by comparison of HATs with all-human teams. In the third experiment, which Tara Behrend, extends beyond the first two experiments, we investigate HAT evolution when HATs are George Washington University, faced with three types of failures during the task. For all three of these experiments, measures focus on team interactions and temporal dynamics consistent with the theory United States of interactive team cognition. We applied Joint Recurrence Quantification Analysis, to communication flow in the three experiments. One of the most interesting and significant *Correspondence: findings from our experiments regarding team evolution is the idea of entrainment, Mustafa Demir that one team member (the pilot in our study, either agent or human) can change the communication behaviors of the other teammates over time, including coordination, [email protected] and affect team performance. In the first and second studies, behavioral passiveness of the synthetic teams resulted in very stable and rigid coordination in comparison to the Specialty section: all-human teams that were less stable. Experimenter teams demonstrated metastable This article was submitted to coordination (not rigid nor unstable) and performed better than rigid and unstable teams during the dynamic task. In the third experiment, metastable behavior helped Organizational Psychology, teams overcome all three types of failures. These summarized findings address three a section of the journal potential future needs for ensuring effective HAT: (1) training of autonomous agents on the principles of teamwork, specifically understanding tasks and roles of teammates, Frontiers in Communication (2) human-centered machine learning design of the synthetic agent so the agents can better understand human behavior and ultimately human needs, and (3) training of Received: 15 February 2019 human members to communicate and coordinate with agents due to current limitations Accepted: 23 August 2019 of Natural Language Processing of the agents. Published: 06 September 2019 Keywords: human-autonomy teaming, synthetic agent, team cognition, team dynamics, remotely piloted aircraft systems, unmanned air vehicle, artificial intelligence, recurrence quantification analysis Citation: Demir M, McNeese NJ and Cooke NJ (2019) The Evolution of Human-Autonomy Teams in Remotely Piloted Aircraft Systems Operations. Front. Commun. 4:50. doi: 10.3389/fcomm.2019.00050 Frontiers in Communication | www.frontiersin.org 2138 September 2019 | Volume 4 | Article 50

Demir et al. The Evolution of Human-Autonomy Teams INTRODUCTION and robotics (Cox, 2013; Goodrich and Yi, 2013; Chen and Barnes, 2014; Bartlett and Cooke, 2015; Zhang et al., 2015; Demir In general, teamwork can be defined as the interaction of two or et al., 2018c). However, considering an autonomous agent as a more heterogeneous and interdependent team members working teammate is challenging (Klein et al., 2004) and requires effective on a common goal or task (Salas et al., 1992). When team teamwork functions (McNeese et al., 2018): understanding its members interact dynamically with each other and with their own task, being aware of others’ tasks (Salas et al., 2005), and technological assets to complete a common goal, they act as a effective interaction (namely communication and coordination) dynamical system. Therefore, an essential part of a successful with other teammates (Gorman et al., 2010; Cooke et al., 2013). team is the ability of its members to effectively coordinate their Especially in dynamic task environments, team interaction plays behaviors over time. In the past, teamwork has been investigated an important role in teamwork and it requires some amount for all-human teams by considering team interactions (i.e., of pushing and pulling of information in a timely manner. communication and coordination) to understand team cognition However, the central issue to be addressed is more complex than (Cooke et al., 2013) and team situation awareness (Gorman et al., just pushing and pulling information; time is also a factor. This 2005, 2006). Presently, advancements in machine learning in behavioral complexity in dynamic task environments can be the development of autonomous agents are allowing agents to better understood from a dynamical systems perspective (Haken, interact more effectively with humans (Dautenhahn, 2007), to 2003; Thelen and Smith, 2007). make intelligent decisions, and to adapt to their task context over time (Cox, 2013). Therefore, autonomous agents are increasingly The Temporal Patterning of Team considered team members, rather than tools or assets (Fiore and Wiltshire, 2016; McNeese et al., 2018) and this has generated Interaction research in team science on Human-Autonomy Teams (HAT)s. Robotics science (Bristol, 2008) posits that complex behavior In this paper, we summarize findings from three of our three of an autonomous agent does not necessarily require complex recent experiments regarding the team interaction within the internal mechanisms in order to interact in the environment over HAT over time in the dynamic context of a Remotely Piloted time (Barrett, 2015). That is, the behavioral flexibility of a simple Aircraft System (RPAS). In the first and the second experiments, autonomous agent is contingent on the mechanics and wiring we summarize general findings related to the interaction of a of its sensors rather than its brain or other components (for an three-member team over time, by comparison of HATs with all- example see Braitenberg and Arbib, 1984). However, in order human teams. In the third experiment, which extends beyond the to produce complex behaviors, there are other elements than first two experiments, we investigate HAT evolution when HATs hardware, specifically interaction with the environment which are faced with a series of unexpected events (i.e., roadblocks) it is subject to. The behavioral complexity of an autonomous during the task: automation and autonomy failures and malicious agent is actually more than parts appear to be individually. This cyber-attacks. For all three of these experiments, measures focus complexity is a real challenge for robotics and cognitive scientists on team interactions (i.e., communication and coordination) and seeking to understand autonomous agents and their dynamic temporal dynamics consistent with the theory of interactive team interactions with both humans and the agent’s environment cognition (Cooke et al., 2013). Therefore, the goal of the current (Klein et al., 2004; Fiore and Wiltshire, 2016). Humans have a paper is to understand how team interaction in HATs develops similar dynamical complexity, as summarized by Simon (1969), over time, across routine and novel conditions, and how this team who stated, “viewed as behaving systems, [humans] are quite interaction relates to team effectiveness. simple. The apparent complexity of our behavior over time is largely a reflection of the complexity of the environment in which We begin by describing HATs as sociotechnical systems and we find ourselves” (p. 53). identify the challenges in capturing this dynamical complexity. Next, we introduce the RPAS synthetic task environment, and In order to better understand the complexity of autonomous three RPAS studies conducted in this environment. Then, we agents and their interactions with humans in their task summarize the findings from HATs and compare this evolution environment, we can consider the interactions as happening to that of all-human teams. within a dynamical system where an agent synchronizes with human team members in a dynamic task environment. In this Teaming With Autonomous Agents case, a dynamical system is a system which demonstrates a continuous state-dependent change (i.e., hysteresis: future state A HAT consists of a minimum of one person and causally depends on the current state of the system). Thus, one autonomous agent “coordinating and collaborating interactions are considered a state of the system whole rather than interdependently over time in order to successfully complete the individual components. A dynamical system can behave in a task” (McNeese et al., 2018). In this case, an autonomous many and different ways over time which move around within team member is considered to be capable of working alongside a multidimensional “state space.” Dynamical systems may favor human team member(s) by interacting with other team members a particular region of the state space—i.e., move into a reliable (Schooley et al., 1993; Krogmann, 1999; Endsley, 2015), making pattern of behavior—and, in such cases is considered to have its own decision about its actions during the task, and carrying transitioned to an “attractor state.” When the system moves out taskwork and teamwork (McNeese et al., 2018). In team beyond this state, it generally reverts to it in the future. The literature, it is clear that autonomous agents have grown more system then becomes more resilient (i.e., the attractor states get common in different contexts, e.g., software (Ball et al., 2010) Frontiers in Communication | www.frontiersin.org 2239 September 2019 | Volume 4 | Article 50

Demir et al. The Evolution of Human-Autonomy Teams stronger) to adapt to dynamic unexpected changes in the task the synthetic pilot (called Information) and navigates it to environment as it develops experience. However, if given a strong each waypoint, (2) the pilot controls the Remotely Piloted enough perturbation from the environment’s external forces, the Aircraft (RPA) and adjusts altitude and airspeed based on system may move into new patterns of behavior (Kelso, 1997; the photographer’s requests (called Negotiation), and (3) the Demir et al., 2018a). photographer photographs the target waypoints, adjusts the camera settings, and also shares information relating to photo With that in mind, HAT is a sociotechnical system in quality—i.e., whether or not the photo was “good”—to the which behaviors emerge via interactions between interdependent other two team members (called Feedback). Taking good autonomous and human team members over time. These photographs of designated target waypoints is the main goal for emerging behaviors are an example of entrainment, the effect all the teams, and it requires timely and effective information of time on team behavioral processes, and in turn team sharing among teammates. The photographer determines if a performance (McGrath, 1990). Replacing one human team role photo is good based on the photograph folder which shows with an autonomous agent can change the behavior of other examples of good photographs (in regard to camera settings, teammates and affect team performance over time. In the i.e., camera type, shutter speed, focus, aperture, and zoom). This sociotechnical system, human and autonomous team members timely effective coordination sequence for this task is called must synchronize and rhythmize their roles with the other team Information-Negotiation-Feedback (INF; Gorman et al., 2010). All members to achieve a team task over time. In order to do so, interactions occur within a text-based communications system it is necessary for the team to develop an emergent complexity (Cooke et al., 2007). which is resilient, adaptable, and includes fault-tolerant systems- level behavior in response to the dynamic task environment In the simulated RPAS task environment, the target waypoints (Amazeen, 2018; Demir et al., 2018a). were within areas referred to as Restricted Operating Zones (ROZ boxes) which have entry and exit waypoints that teams must pass Adaptive complex behavior of a team (as sociotechnical through to access the target waypoints. All studies had missions system) is considered within the realm of dynamical systems that could either be low workload (11–13 target waypoints (either linear or non-linear) and dynamical changes of the within five ROZ) or high workload (20 target waypoints within sociotechnical systems behavior can be measured via Non-linear seven ROZ). The number and length of missions varied as Dynamical Systems (NDS) methods. One commonly used NDS follows: In the first and the second experiments, all teams went method in team research is Recurrence Plots (RPs) and its through five 40 min missions with 15 min breaks in between extension Recurrence Quantification Analysis (RQA; Eckmann missions. Missions 1–4 were low workload, but Mission 5 was et al., 1987). The bivariate extension of RQA is Cross RQA and high workload in order to determine the teams’ performance multivariate extension is Joint RQA (JRQA; Marwan et al., 2002; strength. During the last study, teams went through ten 40 min Coco and Dale, 2014; Webber and Marwan, 2014). In general, missions which were divided into two sessions with 1 or 2 weeks RPs visualize the behavior trajectories of dynamical systems in in between. However, while in the first and second studies, the phase space and RQA evaluates how many recurrences there are first four missions had identical workloads, in the third study, the which use a phase space trajectory within a dynamical system. first nine missions had identical workloads and the 10th mission The experimental design of the RPAS team is conceptually in line was high workload. with JRQA and it is thus the method used for HAT research in this exploratory paper. Measures RPAS SYNTHETIC TASK ENVIRONMENT In the RPAS STE, we collected performance and process measures and then analyzed them with statistical and non-linear dynamical The synthetic teammate project (Ball et al., 2010) is a methods. In this way, we could first understand the nature of longtitudinal project which aims to replace a ground station all-human teams to prepare for the development of HATs. In team member with a fully-fledged autonomous agent. From a general, we collected the following measures for the following methodological perspective, all three of the experiments were three RPAS experiments (see Table 1; Cooke et al., 2007). Each conducted in the context of CERTT RPAS-STE (Cognitive of these measures was designed during a series of experiments Engineering Research on Team Tasks RPAS—Synthetic Task which were part of the synthetic teammate project. Environment; Cooke and Shope, 2004, 2005). CERTT RPAS-STE has various features and provides new hardware infrastructure to In RPAS studies, we considered team communication flow to support this study: (1) text chat capability for communications look at HAT patterns of interaction and their variation over between the human and synthetic participants, and (2) new time by using Joint Recurrence Plots (JRPs). JRPs are instances hardware consoles for three team members and two consoles for when two or more individual dynamical components show a two experimenters who oversee the simulation, inject roadblocks, simultaneous recurrence (pointwise product of reperesentative make observations, and code the observations. univariate RPs) and JRQA provides the quantity (and length) of recurrences in a dynamical system using phase space trajectory Task and Roles (Marwan et al., 2007). In this perspective, JRQA can be utilized for the purpose of examining variations between multiple teams The RPAS-STE task requires three different, interdependent in regard to how and why they, specifically how frequently team teammates working together to take good photos of the targets members synchronize their activities while communicating by (see Figure 1): (1) the navigator provides the flight plan to text message. That is, JRQA basically evaluates synchronization Frontiers in Communication | www.frontiersin.org 2340 September 2019 | Volume 4 | Article 50

Demir et al. The Evolution of Human-Autonomy Teams FIGURE 1 | Simulated RPAS task environment for each role, and task coordination (information-negotiation-feedback). The dashed line separates ground control station and simulated operational environment (from Demir et al., 2017; reprinted with permission). TABLE 1 | The RPAS measures. Description Measures Team performance A weighted combined score of team-level mission parameters, including time spent in warning and alarm states, number of missed targets, and rate of good target photographs per minute (which was weighted most heavily among the parameters). Target Processing Efficiency (TPE) Teams began each mission with a score of 1,000, and points were deducted based on the final values of the mission sub-scores Team process rating TPE takes into account time spent inside a target waypoint to get a good photo. Each team started with a maximum of 1,000 Team Situation Awareness (TSA) points then deducted the number of seconds spent in the target radius and 200 penalty points (for bad or missed photos) Team communication behaviors The rating comprises: (1) coordination—interacting with the right team member about the appropriate information in the right Team communication flow order; (2) timeliness—represents the ability of the team to sort through relevant data and interact expeditiously enough to Workload effectively deal with the target (to do this, interactions are evaluated in accordance to the relative position of the RPA to the target Post experiment question at that moment); and (3) communication quality—related to the clarity and uniqueness of the interactions since those two qualities are seen to minimize need for repetition Cooke et al. (2007). TSA is the degree to which the team members took action and overcame roadblocks (i.e., perturbation). If the team overcame the roadblock, it was coded as “1,” otherwise it was “0.” This measure indicates how a team can adapt to dynamic unexpected changes in the task environment as it develops experience The behaviors are classified into two groups: pushing or pulling of information among the team members It consists of each team member’s message sent time (by seconds) NASA Task Load Index (TLX; Hart and Staveland, 1988) It includes a series of questions about the backgrounds of team members (e.g., age, sex, automated system experience) and their impressions of the experiment and influence by means of looking at system interactions (Demir ideal window size based on the following order: (1) Determinism et al., 2018b). (DET) was estimated based on windows which increased by 1 s for each mission, and (2) DET variance was evaluated for each In RPAS studies, the time stamp for each message (as seconds) size of window and a 1 min window that was chosen according is used to evaluate the flow of communication between team to the average period in which DET no longer increased was members, resulting in multivariate binary data. We chose an Frontiers in Communication | www.frontiersin.org 241 September 2019 | Volume 4 | Article 50

Demir et al. The Evolution of Human-Autonomy Teams selected. This information was useful in order to visually and TABLE 2 | Experimental design for three RPAS studies. quantitatively represent any repeating structural elements within communication of the teams. Experiment Condition x Mission Design For all three experiments, we extracted seven measures from 1 2 (condition) × 5(40 min mission) Control: pilot was JRQA: recurrence rate, percent determinism (DET), longest diagonal line, entropy, laminarity, trapping time, and longest randomly selected vertical line. The measure which all three RPAS studies were interested in was DET, represented by formula (1) (Marwan participant et al., 2007), which we defined as the “ratio of recurrence points forming diagonal lines to all recurrence points in the upper Synthetic: pilot was triangle” (Marwan et al., 2007). Time periods during which the system repeated a sequence of states were represented in the randomly RP by diagonal lines. DET is able to characterize the level of organization present in the communications of a system by selected participant examining the dispersion of repeating points on the RP; systems which were highly deterministic repeated sequences of states 2 3 (condition) × 5(40 min mission) Control: pilot was many times (i.e., many diagonal lines on the RP) while systems that were mildly deterministic would only repeat a sequence of randomly selected states rarely (i.e., few diagonal lines). In Formula (1), l is the length of the diagonal line when its value is lmin and P(l) is the participant probability distribution of line lengths (Webber and Marwan, 2014). A 0% Determinism rate indicated that the time series never Synthetic: pilot was repeated, whereas a 100% Determinism rate indicated a perfectly repeating time series. ACT-R based model Experimenter: pilot was highly trained confederate researcher 3 No condition with 10 (40 min mission) Automation and Autonomy Failures, and Cyber Attack DET = N lP(l) (1) RPAS I: Human-Autonomy Teaming When l=lmin the Synthetic Agent Had Natural Language N lP(l) l=1 Capability THE RPAS EXPERIMENTS For the first experiment, the main question is whether the manipulation of team members’ beliefs about their pilot can In the first experiment, human team members collaborate with a be associated with team interactions and, ultimately, team “synthetic teammate” [a randomly selected human team member, performance for overcoming the roadblocks (Demir and Cooke, Wizard of Oz Paradigm; WoZ (Riek, 2012)] that communicates 2014; Demir et al., 2018c). Thus, there are two conditions in based on natural language. In the second experiment, a synthetic this experiment: synthetic and control, with 10 teams in each agent with limited communication behavior, the Adaptive, condition (total 20 teams). Sixty randomly selected participants Control of Thought-Rationale (ACT-R; Anderson, 2007), worked completed the experiment (Mage = 23, SDage = 6.39). In the with human team members. In the last experiment, similar synthetic condition, we simulated a “synthetic agent” using a to the first experiment, human team members communicated WoZ paradigm: one participant was chosen to be the pilot, and in and coordinated with a “synthetic teammate” (played this time therefore automatically and unknowingly became the synthetic by a highly trained experimenter who mimicked a synthetic agent. The other two team members were randomly assigned agent with a limited vocabulary; WoZ) in order to overcome to navigator and photographer roles and were informed that automation and autonomy failures, and malicious cyber-attack. there was a synthetic agent serving as the pilot. In this case, the Participants in all three experiments were undergraduate and navigator and photographer could not see the pilot when entering graduate students recruited from Arizona State University and or leaving the room, nor during the breaks. Since the pilot in were compensated $10/hour. In order to participate, students the control condition was a randomly assigned participant and were required to have normal or corrected-to-normal vision the other two team members knew this (all three roles signed and be fluent in English. The following table indicates the the consent forms together, and they all saw each other during experimental design and situation awareness index for each that time), communication developed naturally among the team of the conditions (see Table 2). This study was carried out members (again, the navigator and photographer roles were in accordance with the recommendations of The Cognitive randomly assigned). Engineering Research Institute Institutional Review Board under The Cognitive Engineering Research Institute (CERI, 2007). The In this study, we manipulated the beliefs of the navigator protocol was approved by The Cognitive Engineering Research and the photographer in that they were led to believe that Institute Institutional Review Board. All subjects gave written the third team member was not human, but a synthetic agent. informed consent in accordance with the Declaration of Helsinki. This was done in order to answer the question of whether the manipulation of that belief could affect team interactions and ultimately team effectiveness (Demir and Cooke, 2014; Cooke et al., 2016; Demir et al., 2018c). The key aspects of two articles of this study use several quantitative methods to understand team interaction and its relationship with team effectiveness across the conditions. In this specific experiment, the teams went through five 40 min missions (with a 15 min break after each) and we Frontiers in Communication | www.frontiersin.org 2542 September 2019 | Volume 4 | Article 50

Demir et al. The Evolution of Human-Autonomy Teams FIGURE 2 | Example JRP for two high performing UAV teams’ interactions (length 40 min): (A) control (Determinism: 46%) and (B) synthetic teams (Determinism: 77.6%) (from Demir et al., 2018c; reprinted with permission). collected the measures described in Table 1. We comprehensively teams, even though they could interact via natural language. The discussed the key findings in previous papers (Demir and Cooke, implication here is that merely believing that the pilot was a 2014; Demir et al., 2018c). not human resulted in more difficult planning for the synthetic teams, thus making it more difficult to effectively anticipate their As a dynamical analysis, we applied JRQA to binary teammates’ needs. communication flow time series data for 40 min missions in order to visually and quantitatively represent any repeating RPAS II: Human-Autonomy Teaming When structural elements within communication of the teams. In the following figure, we give two example JRP (one control and Humans Collaborate With ACT-R Based one synthetic team) for two RPAS teams’ interactions; these consist of three binary sequences (one for each team member) Synthetic Teammate that are each 40 min in length. The three binary sequences were created based on whether navigator, pilot, or photographer In the second experiment, the focal manipulation was of the sent a message in any given minute. If a message was sent pilot position resulting in three conditions: synthetic, control, or no message was sent, they was coded as “1” and “0,” and experimenter (10 teams for each condition). As indicated respectively. Based on the JRP and DET, the very short diagonals by the name, the synthetic condition had a synthetic team indicated that the control teams showed less predictable team member in the role of, which had been developed using ACT-R communication (Determinism: 46%) while the longer diagonals cognitive modeling architecture (Anderson, 2007); participants mean that the synthetic teams demonstrated more predictable in this condition had to communicate with the synthetic agent communication (Determinism: 77.6%; see Figure 2). Also, we in a manner void of ambiguous or cryptic elements due to its found that the predictability in synthetic teams had more limited language capability (Demir et al., 2015). In the control negative relationship with their performance on target processing condition, since the pilot was human, communication among (TPE), whereas this relationship was less negative in the control team members developed naturally. Finally, in the experimenter teams (Demir et al., 2018c). condition, the pilot was limited to using a coordination script specific to the role. Using the script, the experimenter pilot Overall findings from this first experiment (see Table 3) interacted with the other roles by asking questions at appropriate indicate that the teams which had been informed that their pilot times in order to promote adaptive and timely sharing of was actually a synthetic agent not only liked the pilot more, but information regarding critical waypoints. In all three conditions, also perceived lower workload, and assisted the pilot by giving the roles of navigator and photographer were randomly assigned. it more suggestions (Demir and Cooke, 2014). Based on the Therefore, 70 randomly selected participants completed the two goals of current paper, our findings indicate that (Demir second experiment (Mage = 23.7, SDage = 3.3). et al., 2018c) the control teams processed and coordinated more effectively at the targets to get good photographs (i.e., In the synthetic condition, the ACT-R based synthetic pilot target processing efficiency) than the synthetic teams and was designed based on interaction with team members and displayed a higher level of interaction while planning the task. interaction in the task environment, including adaptation of Team interaction was related to improved team effectiveness, various of English constructions, selection of apropos utterances, suggesting that the synthetic teams did not demonstrate enough discernment of whether or not communication was necessary, of the adaptive complex behaviors that were present in control and awareness of the current situation of the RPA, i.e., flying the RPA between waypoints on the simulated task environment Frontiers in Communication | www.frontiersin.org 2643 September 2019 | Volume 4 | Article 50

Demir et al. The Evolution of Human-Autonomy Teams TABLE 3 | Key findings from three RPAS experiments. Experiment Measures Results RPAS I (WoZ based Synthetic) Target processing efficiency Synthetic teams had poorer target processing efficiency than the control teams Team communication behaviors Control teams conducted more planning than the synthetic teams. Synthetic teams made more suggestions than the control teams RPAS II (ACT-R based Synthetic) Team communication flow Synthetic teams demonstrated more stable coordination dynamics Workload Synthetic teams had less workload Team performance Synthetic and control teams demonstrated same performance but were poorer than the experimenter teams Target processing efficiency Synthetic teams had poorer target processing efficiency than the control and experimenter teams. Experimenter teams were more efficient than the control teams Team situation awareness Synthetic and control teams performed equally to overcome the roadblocks, but poorer than the experimenter teams Team communication behaviors Synthetic teams pulled more information than they pushed, and pushing information was not as effective for their performance as the all-human teams. control and the RPAS III (WoZ based Synthetic) Team communication flow experimenter teams did more pushing than pulling, and the pushing information which was effective with their performance Team performance Synthetic teams demonstrated stable coordination dynamics, while experimenter teams Target processing efficiency were moderately stable and the control teams were unstable Team process rating Team performance increased across the missions Team situation awareness Target processing efficiency increased across the missions Target process rating increased across the missions Team communication behaviors Teams demonstrated better performance on overcoming automation and autonomy Team communication flow failures than the malicious attacks. Teams overcame an increasing number of automation failures across the missions, but a decreasing number of autonomy failures. Teams poorly All results use the α = 0.05 significance level. performed to overcome malicious cyber-attacks RPAS I: (Demir and Cooke, 2014; Demir et al., 2018c). Pushing information increased across the missions, while pulling decreased RPAS II: (Demir et al., 2016, 2017, 2018b; McNeese et al., 2018). Teams demonstrated better performance when they become more flexible during the RPAS III: (Cooke et al., 2018; Grimm et al., 2018a,b). failures (Ball et al., 2010). However, since the synthetic pilot still had message was sent in any minute, it was coded as “0.” The limited interaction capability, it was crucial that the navigator synthetic team in this example exhibited rigid communication and photographer made certain that their messages to the non- (higher determinism), whereas the control team demonstrated human teammate were void of ambiguous or cryptic elements. If an unstable communication pattern compared to the other two not, their synthetic teammate was unable to understand and, in teams. Taking into account the goals of this paper, in the synthetic some cases, malfunctioned (Demir et al., 2015). team, higher determinism tended to correspond to instances when all three team members were silent (see Figure 3A between In the second experiment, we explore and discuss team 30 and 35 min). For control teams, such varied communication interaction and effectiveness by comparing HATs with all-human patterns were not unanticipated since the pilot role was teams (i.e., control and experimenter teams). Here, we give randomly assigned. On the other hand, coordination behaviors a conceptual summary of findings from previous papers that of control teams, experimenter teams, and synthetic teams were compared human-autonomy and all-human teams on dynamics unstable, metastable, and rigid, respectively, as indexed by the (Demir et al., 2018a,b) and also their relationship with team percent DET from JRQA. Extreme team coordination dynamics situation awareness and team performance, via interaction (overly flexible or overly rigid) in the control and synthetic (Demir et al., 2016, 2017; McNeese et al., 2018). teams resulted in low team performance. Experimenter teams performed better in the simulated RPAS task environment due In Figure 3, three example JRPs from this study are depicted to metastability (Demir, 2017; Demir et al., 2018a,b). In addition for three teams’ communication for each condition (same to the dynamic findings, overall findings for this study showed as in the first RPAS study: three 40 min binary sequences) positive correlations between pushing information and both along with their calculated DET: Figure 3A—synthetic (DET team situation awareness and team performance. Additionally, = 52%), Figure 3B—control (DET = 34%), and Figure 3C— the all-human teams had higher levels than the synthetic teams experimenter (DET = 47%). Visible on the y-axis, instances of in regard to both pushing and pulling. By means of this study, we any messages sent by any of the three roles (navigator, pilot, or photographer) in any minute were coded as “1,” and if no Frontiers in Communication | www.frontiersin.org 2744 September 2019 | Volume 4 | Article 50

Demir et al. The Evolution of Human-Autonomy Teams FIGURE 3 | Example joint recurrence plots for three RPAS teams’ interactions in three conditions—length 40 min: (A) synthetic, (B) control, and (C) experimenter teams (from Demir et al., 2018b; reprinted with permission). saw that anticipation of other team members’ behaviors as well malicious cyber-attack—and had to overcome it within a set time as information requirements are important for effective Team limit. Automation failures were implemented as loss of displayed Situation Awareness (TSA) and team performance in HATs. information for one of the agents for a set period. Autonomy Developing mechanisms to enhance the pushing of information failures were implemented as comprehension or anticipation with HATs is necessary in order to increase the efficacy of failures on the part of the synthetic pilot. The malicious cyber- teamwork in such teams. attack was implemented near the end of the final mission as an attack on the synthetic pilot wherein it flew the RPA to a site RPAS III: Human-Autonomy Teaming When known to be a threat but claimed otherwise (Cooke et al., 2018; a Human Collaborates With a Synthetic Grimm et al., 2018a,b). Teammate Under Degraded Conditions The teams encountered three types of automation failures In the third experiment, the “synthetic” pilot position was present on either the pilot’s shared information data display, or filled by a well-trained experimenter (in a separate room—WoZ the photographer’s, e.g., there was an error in the current and paradigm) who mimicked the communication and coordination next waypoint information or in the distance and time from of a synthetic agent from the previous experiment (Demir et al., the current target waypoint. In order to overcome each failure, 2015). In the third experiment, 40 randomly selected participants team members were required to effectively communicate and (20 teams) completed the experiment (Mage = 23.3, SDage = coordinate with each other. Each of the automation failures were 4.04). In order to facilitate their effective communication with inserted individually at specific target waypoints from Missions the synthetic pilot, both the navigator and the photographer 2 through 10 (Mission 1 was the baseline mission and didn’t had a cheat sheet to use during the training and the task. The include any failures). Malicious cyber-attack was only applied on main manipulation and consideration of this study was team Mission 10. Therefore, Mission 10 was the most challenging. resilience, so at selected target waypoints teams faced one of three kinds of roadblocks—automation failure, autonomy failure, or Within the concept of dynamical systems analysis, two sample JRP are shown for the communication of high and low performing RPAS teams, which were indicated based on Frontiers in Communication | www.frontiersin.org 2845 September 2019 | Volume 4 | Article 50

Demir et al. The Evolution of Human-Autonomy Teams their target processing efficiency (TPE) scores during Mission the findings from dynamical systems contributed more insights 10 (three 40 min binary sequences). Additionally, the plots show to explain the dynamic complex behavior of HATs. the calculated DET for both teams; the first one performed well (DET = 48%) and the second performed poorly (DET In the first and second studies, behavioral passiveness of the = 54%). Accordingly, as shown in Figure 4A, although the synthetic teams resulted in very stable and rigid coordination in percentages of the DET scores were not too far apart, the comparison to the all-human teams, which were less stable. We communication of the high performing team was more rigid know that some degree of stability and instability is needed for than that of the low performing team. Interestingly each of the team effectiveness, but teams with too much of either performed team members in the high performing team communicated more poorly. In the second experiment, this issue is clearly seen across frequently during each one of the failures, than those of the other three conditions: synthetic, control, and experimenter teams. teams, and they overcame all of the failures they encountered, Experimenter teams demonstrated metastable coordination (not including the malicious cyber-attack. As for the low performing rigid nor unstable) and performed better, whereas the control and team (see Figure 4B), the members communicated more during synthetic teams demonstrated unstable and rigid coordination, the automation failure and they successfully overcame that respectively, and performed worse. Metastable coordination roadblock. Unfortunately, the same team did not communicate behavior of the experimenter teams may have helped them adapt to the same degree and with the same efficacy during the to the unexpected changes in the dynamic task environment. In remaining two roadblocks (autonomy failure and malicious addition to metastable coordination behavior, the experimenter cyber-attack). In fact, the navigator did not even participate teams also demonstrated effective team communication, pushing during the autonomy failure, and the photographer either failed and pulling information in a timely and constructive way. This to anticipate the needs of his teammates during the malicious type of metastable pattern was also discovered in different cyber-attack, the photographer was simply unaware of the failure. contexts using the entropy measure. For instance, a system This lack of team situation awareness resulted in poor TPE scores. functions better if there is a trade-off between its level of complexity and health functionality (Guastello, 2017). Another Based on the goals of current paper, when the HATs interact sample entropy analysis on neurophysiology shows that teams at effectively, they improve in their performance and process over the optimum level of organization exhibit metastable behavior in time and tend to push information or anticipate the information order to overcome unexpected changes in the task environment needs of others more as they gain experience. In addition, (Stevens et al., 2012). Sample entropy analysis also revealed that a dynamics of HATs differ in how they respond to failures. moderate amount of stability resulted in high team performance. When the HAT teams demonstrated more flexible behavior, they This finding also resembles the third experiment, moderately became more adaptive to the chaotic environment, and in turn stable behavior and timely anticipation of team members’ needs overcame more failures in the RPAS task environment. helped teams to overcome the three types of failures. However, one of the most important findings from these experiments CONCLUSION is entrainment. That is, one team member (in our case was the pilot). The goal of this current paper is 2-fold: first, to understand how team interaction in HATs evolves in the dynamic RPAS Through these studies it is clearly possibly to have successful task context and second, to observe how HATs respond to a HATs, but a more important question moving forward is how variety of failures (automation, autonomy, and malicious cyber- to achieve high levels of HAT performance. How can we attack) over time. One of the most significant findings from our ensure effective levels of communication, coordination, and experiments regarding team evolution is the idea of entrainment, situation awareness between humans and agents? In response that one team member (the pilot in our study, either synthetic or to this question, the authors propose three potential future human) can change the communication behaviors of the other needs for ensuring effective HATs: (1) training humans how to teammates over time, including coordination, and affect team communicate and coordinate with agents, (2) training agents on performance. In the communication context of this task, we the principles of teamwork, and (3) human-centered machine know that pushing information between the team members is learning design of the synthetic agent. In other words, for humans important and we know that, in general, the synthetic teammate and agents to interact with one other as team members, all was capable of communication and knew its own needs, but it participants must understand teamwork and be able to effectively did not know the needs of its counterparts in a timely manner, communicate and coordinate with the others; it’s not just one or especially during novel conditions. In the first experiment, the other. synthetic teams did not effectively plan during the task and, in turn, did not anticipate each others’ needs. Similarly, in the First, before participating in HATs, humans should be second experiment synthetic teams more often relied on pulling specifically trained on how to interact with the agent. In the information instead of anticipating each other’s needs in a timely future this training will be fundamentally important as the manner. Behavioral passiveness of the synthetic teams addresses types of available agents with which a person might team up team coordination dynamics which is a fundamental concept of vary greatly, with many variants in both cognitive modeling the ITC theory. Therefore, we applied one of the NDS methods, and machine learning. Understanding how to interact with JRQA, on communication flow from the three experiments and these agents is step one in ensuring effective HATs, because without meaningful communication, effective teamwork is impossible. In our studies, we specifically trained participants in how to properly interact with the synthetic agents in Frontiers in Communication | www.frontiersin.org 2946 September 2019 | Volume 4 | Article 50

Demir et al. The Evolution of Human-Autonomy Teams FIGURE 4 | Example Joint Recurrence Plots for two RPAS teams’ interactions: (A) high performing team (Mission 10—DET: 48%) overcame all three failures (automation, autonomy, and malicious cyber-attack); and (B) low performing team (Mission—DET: 54%) only overcame automation failure—Mission 10: length 40 min (from Grimm et al., 2018b; reprinted with permission). their teams. If we had not trained them how to interact, are. If you dig into the fundamentals of the synthetic agent in the interaction would have been significantly hindered due our studies, they did not understand the concept of teaming. to the participants not understanding the communication and Instead, it was capable of communication and understood coordination limitations of the synthetic agent. The training its own task with very little understanding of other team allowed them to successfully interact with the agent due to members’ tasks, let alone the team task. Moving forward, an understanding of the agent’s capabilities. In the future, the computer scientists and cognitive scientists need to work need for training humans to interact with agents will hopefully together to harness the power of machine learning to train decrease due to the increased availability and experience of agents to know what teamwork is (communication, coordination, interacting with agents and advancements in natural language awareness, etc.). An agent will never be able to adapt and processing. However, in the immediate future it will be adjust to dynamical characteristics such as coordination if it necessary to develop appropriate training specific to this type is not trained to conceptualize and taught how to apply that of interaction. knowledge first. Second, agents as team members must be programmed Finally, there is a significant need to have serious discussions and trained with a fundamental conceptualization of what on how the broader community should be developing these teamwork is and what the important principles of teamwork agents technically. Our agent was built on the ACT-R cognitive Frontiers in Communication | www.frontiersin.org 21407 September 2019 | Volume 4 | Article 50