REA-NDM-FONCSI 2021: REA-NDM-FONCSI JOINT INITIATIVE - BOUNCING FORWARD FROM GLOBAL CRISES AND CHALLENGES
PROGRAM FOR TUESDAY, JUNE 22ND
Days:
previous day
next day
all days

View: session overviewtalk overview

08:00-08:15 Session 12: Showcase from around the world - Australia

Sidney Dekker, David Provan

A conversation around

  • What happen since last symposium
  • what are your current projects
  • Current questions and possible solutions
  • What industry or other collaborations with end users you have
  • If relevant PhD topics of your students
Discussant:
Location: Main Room
08:15-09:00 Session 13: Graceful ‘Disruptability’: Is the capacity to innovate critical to sustaining healthcare operations under VUCA conditions?

Satyan Chari

Discussant:
Location: Main Room
08:15
Graceful ‘Disruptability’: Is the capacity to innovate critical to sustaining healthcare operations under VUCA conditions?

ABSTRACT. The notion of system resilience is increasingly invoked in healthcare as the societal impacts of the COVID-19 pandemic (and its flow-on effects) push us into progressively unfamiliar territory. Around the world,  health systems have been forced to respond to the rapidity of unfolding events by rethinking conventional (regulatory and normative) approaches to safety and system performance. In some instances, this has been accompanied by a (perhaps temporary) tolerance for more adaptive forms of work and decentralised decision-making. There are of course many positive stories of ingenuity and success embedded within this narrative. Yet, in countries hard hit by COVID-19, the overall experience of health professionals has been far from positive. Aside from the significant human toll of the pandemic on clinical workers, these times of volatility and uncertainty have engendered unsustainable levels of adaptive pressure on healthcare teams - creating anxiety, fear and often extending them far beyond their clinical capabilities in order to respond resiliently to emergent issues. Whilst frontline teams are inarguably best placed to respond resiliently to emerging threats, we might also consider how the field of resilience engineering (RE) can guide systems and workers as they reconfigure services and operations. Further, can RE theory and practice inform how frontline teams might 'gracefully disrupt' when extension of services is no longer viable due to transformed demands - and what are the capabilities that must be cultivated to do so effectively at scale? This talk engages with these key questions through the lens of work delivered by the Queensland Health Bridge Labs program which in 2020 brought together expertise in safety innovation, human factors and ergonomics and creative design to address various challenges at the blunt and sharp end of healthcare.

09:00-09:45 Session 14: Transportation

Lightning talk and fish bowl discussion

Location: Main Room
09:00
Learning from the positive – suggestions for potential improvements to maritime safety management approaches
PRESENTER: Johannah Olsson

ABSTRACT. The study presented in this article aimed to investigate how regulatory demands have shape safety management and safety management systems (SMS) in shipping companies. A case study focused on five companies operating in the Swedish maritime industry was carried out. Ten semi-structured interviews with respondents in management positions on shore and onboard were carried out during spring 2020. Originally the case study was supposed to include observations as well, but these were suspended due to COVID-19. The interviews were transcribed and analyzed with the help of the four resilience cornerstones; respond, anticipate, monitor, and learn. The results show that the companies, as well as the industry itself should be considered as complex systems, in which an additional layer of complexity is added by international and national rules and regulations. The work put into managing safety has become of such complex nature that for the most part, safety work is put into administrative tasks to ensure compliance with regulations. The results also highlight concerns among the management onboard that more time is dedicated to formulate routines and procedures than to actually making the work safer for those performing it. To compensate for the experienced gaps between work-as-imagined, or prescribed, and work-as-done questions based on the Resilience Assessment Grid (RAG) are presented. These can serve as a complement to the current safety management strategies, and may also give raise to a deeper cooperation between shore-based and ship-based personnel in shipping companies. This may enable a participatory approach in which safety management can be shaped bottom-up with as onboard personnel contributing essential knowledge on needs and preconditions to ensure safe operations while also taking the complexity of the system into account.

09:10
The Impact of Learning from Everyday Work and Decision Making on Aviation Maintenance

ABSTRACT. Aviation maintenance engineers are routinely required to work around challenges in the maintenance environment to deliver aircraft safely and efficiently. A process called Human Hazard Analysis is being used by the offshore helicopter community to explore the gaps between Work-As-Done on aircraft in-service with the Work-As-Imagined and Work-As-Prescribed during the design process. This presentation outlines the results of these workshops so far but also their limitations. Recent research into maintenance engineer decision-making has the potential to address some of these weaknesses. The results of these studies is outlined with corresponding recommendations on how this might be capitalised by the industry.

09:20
Complexity of field intervention in safety management: a case study in Air Traffic Control

ABSTRACT. This article relates the construction and findings of an intervention at an airport following a safety incident that uncovered a deviation of communication practice between two air traffic controllers for handling small aircraft traffic on a secondary runway. The intervention was carried out by coaching and empowering a local team so that they would work with the air traffic controllers to understand the reasons for the deviation, involving the training of controllers to observe normal work practices in the control tower and its effects on local safety management practices. The results in terms of adapting scientific observation methods to a local issue and subsequent behavioural changes in attitudes towards the deviation of formal procedure are discussed.

10:00-11:00 Session 15: Workshop - Development of a Practical Toolbox of Adaptation in Aviation and Air Traffic Control

Interactive workshop

(GR)

Location: Main Room
10:00
Development of a Practical Toolbox of Adaptation in Aviation and Air Traffic Control
PRESENTER: Stathis Malakis

ABSTRACT. Adaptation can be defined as making appropriate responses to changing situations while adaptability enables people to recognize and adapt fluidly to unexpected characteristics of a situation. Paradoxically, people at the front-line of organizations can intuitively recognize and understand cases of adaptation but there has been little guidance so far on how to learn, practice, evolve and transfer their experiences. The most common perspective is that people constitute a general adaptive capacity for organizations that compensate for shortfalls and problems at any level. Commercial aircraft operations and Air Traffic Control (ATC) are complex and tightly coupled systems moving passengers and cargo with safety and regularity worldwide. Although they are highly scripted environments, adaptation remains one of the most-valued concepts. At the operational level, commercial aviation and ATC share many common work demands of the classic Naturalistic Decision Making environments such as: rapidly escalating situations, multiple information resources, information-based uncertainty, time pressure, errors with high consequences, multifaceted decisions, trade-offs between interacting and conflicting goals, multiple stakeholders, hidden dynamic interdependencies and physiological stressors due to round the clock operations. The levels of safety in commercial aviation and ATC are high, but the adaptive expertise of their people is constrained because, firstly they are rarely exposed to exceptional situations and secondly relevant guidance is minimal. However, there exist many examples of extraordinary episodes of adaptation such as: the “Miracle of Hudson river” (i.e. US Airways Flight 1549 successful ditching at Hudson River in 2009 without any casualties) and the management of air traffic during ATC Zero on 11th of September 2001 when controllers managed safely the grounding of thousands of aircraft in the US skies at a short notice. In this line of reasoning, we explored the concept of adaptation in the commercial aircraft operations and the ATC through the lens of Resilience Engineering and Naturalistic Decision Making. We developed a model that includes cognitive and team processes such as: Steering, Sensemaking, Common Operating Picture, Coordination and Transfer of Control, Managing Changes and the Planning-Doing-Checking cycle. We analyzed cases of adaptive failures and successes and we developed a set of adaptation elements (i.e. a toolbox for adaptation). Our aim was to explore the nature of adaptation as a source of resilience in the commercial aircraft operations and the ATC and develop a practical toolbox with wider applications in diverse organizations and challenging situations.

11:15-12:00 Session 16: Showcase from around the world

Conversation between Sidney Dekker and David Provan

Discussants:
Location: Workshops
13:00-14:00 Session 17: Industry experiences

Serious Gaming Workshop, lightning talk, and fish bowl discussions.

Chair:
Location: Main Room
13:00
Introducing resilience engineering principles to tech companies

ABSTRACT. The tech industry has rapidly grown into being a domain that touches many parts of our lives, meaning that it has the potential to have very high consequence incidents, while the increased speed of operations can create an ever increasing time pressure; at the same time the tech industry is historically one that is resistant to importing knowledge from some other disciplines and domains. In this session, I'll share the tactics and strategies that have worked for me in introducing resilience engineering concepts to teams at high tech companies.

In the session, I'll draw upon my background as an engineer and incident responder in big tech corporations, a former EMT, and someone who studies resilience engineering and produces analyses for others. I'll discuss: Common challenges in tech, such as: Resistance to importing knowledge from other disciplines Relative newness of many companies and the industry as a whole The possibility of high consequences at very fast speeds, coupled with little regulation or guidance Encouragement to eschew traditional approaches Strategies that have been successful such as: Introducing the ideas of a sociotechnical system Exploring ways that automation can fail Building off of increasingly accepted work such as blameless approaches to incident learning Common ground in incident response I'll also explore subjects that tend to meet more resistance including: That increased automation doesn't necessarily mean decreased incidents or accidents Automation as a team player doesn't mean waiting for increased AI.

13:10
Applying the Principles of Resilience Engineering to Small and Medium Sized Businesses

ABSTRACT. Small to medium sized businesses are a significant contributor to the global economy. However, they are widely accepted to be more fragile than larger businesses, especially in terms of financial security and the coronavirus pandemic has exposed a need to build resilient performance of SMEs. This presentation explores how SMEs can move beyond the traditional, minimal approach to risk management, utilising the principles and techniques of resilience engineering to bounce forward from future shocks.

14:15-15:00 Session 18: Showcase from around the world - Brasil

Tarcisio Saurin, Eder Henriqson, Priscila Wachs, Rose Melo, Paulo Carvalho, Jose Gomes

  • Ongoing projects
  • Future prospects
15:15-16:00 Session 19A: The Right Way to Think About the Future
Location: Main Room
15:15
Naturalistic Decision Making - Thinking about the future
PRESENTER: Phil Tetlock
15:15-16:15 Session 19B: Workshop - Industry experiences

Implementing resilience engineering and safety II thinking into organisations

Chair:
Location: Workshops
15:15
Leveraging Mixed Methods Research Methodology for Resilience Engineering: A Case Study at American Airlines

ABSTRACT. Research methodology and specific research methods for Resilience Engineering (RE) research remains sparse within the published literature. We present a mixed methods approach to data collection and analysis to support a new Safety-II/RE-informed data stream within a commercial airline’s safety management system. This project found mixed methods to be an extremely versatile research approach that affords internal and external validity as well as numerous opportunities for secondary data analysis; additionally, corporate leadership were eager to leverage the frequent opportunities afforded by this multi-staged (sequential and convergent) research design for establishing and maintaining concurrence between various datasets to further validate the produced research conclusions. The designs of this project are specifically useful to those organizations which are unwilling or unable to completely abandon Safety-I work but are looking to improve operational safety through the incorporation of more modern approaches.

15:35
Developing a learning organisation
PRESENTER: Nippin Anand

ABSTRACT. Project Confidus received initial funding from the Scottish Oil & Gas Institute of Technology (OGTC) and emerges from the growing perception that most organisations invest substantial resources in reporting-based communication systems and yet, draw little benefit from such investments. It focuses on shifting from essentially one-way and highly fragmented communication practices, towards multi-way conversations. These conversations provide a much wider context on issues and therefore, generate enhanced potential for learning. The engagement in conversations produces reflection and sharing of knowledge with and between the various organisational levels. The use of dedicated technology aims to overcome time and geographical barriers and foster improved hierarchical, as well as peer-to-peer exchanges across remote assets.

15:55
A Case Study in Instantiating RAG: American Airlines Learning and Improvement Team (LIT)

ABSTRACT. Safety-II and Resilience Engineering professionals have many options for transforming industry safety programs, leveraging their expertise in the decision of which abstract theoretical model(s) to instantiate in a particular system. We present a case study and lessons learned from one large commercial airline’s effort to utilize Hollnagel’s Resilience Assessment Grid (RAG) in the transformation of their safety management system. Specifically, we will discuss the nuances of *how* the Learning and Improvement Team (LIT) transformed the RAG model into LIT’s LPAC model. We will share: our initial mindset as well as how it changed over the first two years; our deep dive into the epistemology of pilot work and how that was influential in developing LIT’s model and methodology; finding balance between corporate goals, organizational capabilities, safety theory, and rigorous methodology; and critiques we’ve received since going public with the program.

16:00-16:45 Session 20: Emerging technologies and collaborative work - Part B Day 2

Lightning talks and fish bowl discussion

Location: Main Room
16:00
Training for Macrocognitive Skills Awareness in Cybersecurity Professionals
PRESENTER: Hifsa Khan

ABSTRACT. Cybersecurity professionals are becoming increasingly important in all sectors due to the increase of security breaches in fields such as healthcare, government, and others (Hemphill & Longstreet, 2006). Cyber-defenders in particular carry out daunting and critical work to defend information systems from ever-increasing vulnerabilities (Shah et al., 2018). While cybersecurity experts are in high demand, there is not a clear educational path for emerging professionals to fill the gap in the market (Crumpler & Lewis, 2019). Furthermore, although research is advancing the state-of-the-art in the automation of network defense, little research has been conducted on the role of human cognition in the success of cybersecurity defenders, which remains critical in this complex environment. As a result of this, early-career cybersecurity defenders may not be well prepared for their roles initially (Crumpler & Lewis, 2019). In our presentation, we will present macrocognitive themes identified in our forthcoming publication (Schuster, in press) and describe their implications for research and practice. We will also propose a simple awareness training that introduces macrocognitive and metacognitive concepts to novice cybersecurity defenders to improve their performance and retention. In the proposed awareness training, we will first identify the macrocognitive concepts of interest using (Schuster, in press) as a guide and focusing on concepts that have the most support in the literature for cybersecurity and related fields. Next, we will define the skills and knowledge to be trained. We will illustrate our approach with an example. Finally, we will describe how we will evaluate our training at the reaction and learning levels defined by Kirkpatrick (1959, 1996). The proposed study serves two aims. First, it could provide evidence for the efficacy of metacognitive training for macrocognitive skills. Second, a distinguishing feature of the proposed study is the capture of qualitative feedback from subject-matter expert (SME) participants. This can inform research in cybersecurity training for macrocognition.

16:10
Supporting Resilience in Large-scale Multi-domain Operations as Learning Laboratories
PRESENTER: Martin Voshell

ABSTRACT. For military agencies worldwide, adversary threat capabilities continuously place new demands on weapon system training and mission and package employment that reveal critical integration challenges and new contexts for expertise and continuous learning. By looking at recent large-scale multi-domain air warfare exercises through the lens of the learning laboratory framework, this paper describes how next-generation data integration of multiple domains must be used to study and develop analytic insights into modeling resilience in complex mission environments especially in the context of integrated Live, Virtual, Constructive (LVC) training events.

16:20
A Systems-Resilience Approach to Technology Transition in High-Consequence Work Systems
PRESENTER: Kelly Neville

ABSTRACT. The rate at which new information technology fails to successfully transition is disconcertingly high. Many technologies only manage to transition through a forced and difficult process, and often produce system disruptions that pose serious threats to high-consequence systems (e.g., Finkelstein & Dowell, 1996; Goldstein, 2005; Patterson, Militello, Su, & Sarkar, 2016; Sherwood, Neville, McLean, Walwanis, & Bolton, 2020; Trist & Bamforth, 1951; White, Wastell, Broadhurst, & Hall, 2010; Wears, Cook, & Perry, 2006). We propose a systems-centered approach to technology transition and system modernization grounded in complex-systems science and resilience engineering. In an exploratory research effort, we developed an initial version of the Transform with Resilience during Upgrades to Socio-Technical Systems (TRUSTS) Framework. This framework, derived from a literature review, specifies general characteristics of complex systems that allow them to behave with resilience in complex, high-stakes operating environments. We have begun translating the framework into tools for guiding technology transition and system modernization. The toolset will help technology-development and transition teams evaluate and engineer into their designs the system-resilience impacts of new technologies starting with the concept development stage and through deployment. Our primary goal is to use the TRUSTS Framework and tools to significantly improve the success rate of technology transition and count the transition as a success not just because it is being used but because it has transitioned without compromising the receiving system’s ability to perform and adapt in high-demand conditions. Complementary efforts will focus on applying and adapting the TRUSTS Framework and toolset to benefit transition and modernization challenges faced in multiple high-consequence domains, including air transportation, healthcare, and national security.

17:00-20:00 Session 21: Emerging technologies and collaborative work - Day 2 - USA Facilitation

Discussion and two panels facilitated by USA with breaks.

John MERRIHEW, Veloxiti; Patty MCDERMOTT, the MITRE Corporation; Kerstin HARING, University of Denver

 

Discussant:
Location: Main Room
17:00
Panel: An Intelligent Assistant You Won't Want to Fire
PRESENTER: Cindy Dominguez

ABSTRACT. John MERRIHEW, Veloxiti; Patty McDermott, the MITRE Corporation; Kerstin Haring, University of Denver

People query intelligent assistants on personal devices to help with ever-expanding tasks and decisions throughout their day, yet it is difficult to imagine these assistants being reliable or effective enough to support decision makers in NDM-like settings. What is the current state of research applying this technology to classic NDM situations, and what are the potential payoffs and problematic pitfalls of a JARVIS-like software assistant for high-stakes decision makers? Are Intelligent Assistants being implemented today meeting needs for flexible and adaptive support of humans in dynamic, high-risk environments? In this panel we will convene an open and honest discussion on this topic. Recognizing the wealth of guidance that has emerged in applied research over the past few decades of work on agents, associates, and assistants, Panelists will describe their ongoing work in this area, highlighting ways forward and tough challenges that remain.

17:45
Panel: Advanced Automation panel
PRESENTER: Emilie Roth

ABSTRACT. We plan to assemble panellists from a range of U.S. government agencies such as the U.S. Army, the U.S. Food & Drug Administration, the Federal Aviation Administration, and Sandia National Labs to share the issues and challenges they face in developing, applying, studying and regulating advanced automation, artificial intelligence, machine learning in complex settings. These technologies are proposed for use in military operations, civilian aviation, railways, patient and physician-facing medical devices, and many others. While some themes are common across contexts, challenges vary depending on context, intended use, and intended user. Panelists will discuss the challenges related to integrating advanced automation in their domains, risks they anticipate, and human capabilities that should be preserved and supported.

18:30
Facilitated Discussion: Is Trust Overblown?

ABSTRACT. Trust is a buzzword in autonomous system design, with sponsors and managers often asking for it by name. The waters get murky as practitioners navigate the differences between trust and calibrated trust and the assessment techniques associated with each construct. In this session there is no audience - by attending you are part of a (large) roundtable so come lend your expertise and opinions! Through guided facilitation, the group will explore research on trust and calibrated trust; discuss whether the emphasis on ‘trusting the system’ is overblown; and share lessons learned on evaluating user trust and assessing whether users understand and use the system appropriately. A variety of facilitation techniques will be used to keep participants engaged and ensure that a diverse set of voices is heard. Attendees will walk away with concrete techniques for evaluating trust as well as advice for employing them. They will also have the opportunity to interact and connect with a wide variety of trust researchers that they may not have otherwise met.

19:00-22:00Social dinner (Toulouse)