previous day
all days

View: session overviewtalk overview

08:00-09:00 Session 27: Start - Day 4 - UK & Portugal and Lead Talk

Start - Day 4 - UK & Portugal - Julie Gore and Pedro Ferreira

Lead Talk by Jan-Maarten Schraagen and Gesa Praetorius

Location: Main Room
09:15-10:00 Session 28: Media and tech companies: Part A Day 4

Ligthning talk and fish bowl discussion

Location: Main Room
Designing for “Agility” in Envisioned Worlds: Concepts for Collaborative Intelligence in Human-Machine Teams
PRESENTER: Neelam Naikar

ABSTRACT. This paper explores two challenging design problems—that of designing “agile” sociotechnical systems, which can contend with very high levels of instability, uncertainty, and unpredictability, and that of designing future or envisioned systems, which do not yet exist. We suggest a design approach focusing on constraints or boundaries of successful operation, which can accommodate many possibilities for action, thereby providing a systematic basis for creating “agility” in the future system for dealing with dynamic, ambiguous environments. Given recent advances in artificial intelligence, we demonstrate some of these concepts with the decision ladder template from cognitive work analysis, showing how it may be applied in the design of “agile” human-machine intelligence. This design approach, which moves beyond standard approaches to function allocation, whether fixed or dynamic, toward ‘team player’, joint cognitive systems, and collaborative approaches to human-machine performance, has significant implications for research in artificial intelligence.

How Experts Know That Their Intuition is Right? Naturalistic Study on Executive Search Consultants

ABSTRACT. Previous studies on the role of intuition in experts’ decision-making allowed to discover strategy experts use to make correct decisions in situations in the absence of clear facts. This study aims to discover how expert executive search consultants know when to trust their intuition and how intuition influences their process of reasoning and decision-making. Six executive search consultants from Warsaw and London took part in the study (average experience = 13 years). Results of the study indicated that consultants use intuition to provide an initial appraisal of a candidate. Intuitive answer was found to be accompanied with a Feeling of Rightness (FOR) whose intensity was prompting the level of confidence in one’s intuition. Based on results, a decision-making model was created that depicts how FOR influences consultants’ process of reasoning and decision-making.

WAx on, WAx off: understanding the dynamics of cyber-socio-technical systems

ABSTRACT. The WAx is a conceptual framework that addresses the needs for understanding, modeling, and analyzing cyber-socio-technical systems, seen as the archetypal maximum expression of complexity achievable by collective systems formed by humans and technological artifacts (Patriarca et al., 2021).

The scope of current socio-technical systems is expanded by cyber elements, forcing reconsideration and re-evaluation of fundamental properties. The introduction of a kind of artifacts and agents (i.e., cyber-agents) objectively different in capability and rapidity of response, raises terrifically the number of possible non-linear interactions within modern systems. To this extent, Cyber-Socio-Technical Systems (CSTSs) sometimes include open AI issues (e.g., explainability, interpretability, and trust in technological artifacts) and exacerbate features related to adaptation, evolution, and, in general, unpredictability of complex adaptive systems. The understanding, analysis and, ultimately, the engineering of a CSTS, fundamental in the immediate future in terms of safety and security, passes through the definition of an appropriate systemic dimension of investigation. Risks of social manipulation, new types of warfare, or changes in power structures potentially attributable to cyber-artifacts come to mind in this sense. Relying on these observations, the WAx is structured to be impractical for reductionism and simplistic approaches that are patently unsuitable for CSTS risk and safety management. In a broader sense, the framework remains a conceptual support for reflection, useful for encouraging discussion and insight into the different system's performance. As such, it casts a light to remove biases on the dynamics involved in the different varieties of work, whether performed by humans or cybernetic agents.

10:15-11:00 Session 29: Models - Part A Day 4

Models - Part A Day 4

Location: Main Room
Modelling driver decisions to improve port traffic management during critical situations
PRESENTER: Ross Phillips

ABSTRACT. According to Flach & Voorhorst (2020) effective management of real-world situations can be achieved by treating human decisions according to meaning processing approaches to cognition. We describe a System Theoretic Process Analysis (STPA) of traffic management in a busy port area of the Netherlands. Reflecting on the meaning processing approach to cognition, we then generate and test a new set of prompts that the STPA analyst can use as they generate scenarios that to explain why human actors would carry out unsafe control actions. Then new prompts help the analyst consider the human controller as an individual embedded and in a dynamic ecology, with perceptions and actions emerging from interactions with that ecology. They help consider people as perceivers of affordances rather than whole processes; and as aware of the meaning of actions for several valued goals. On applying the new prompts to the port traffic management case, new insight was generated concerning how truck drivers will act to perceive (and not just perceive to act), seek out workarounds in the face of terminal closures, and attend to information that helps them decide whether diverting or ignoring diversions is most likely to lead to punctual delivery. While extending STPA analysis to account for the meaning processing approach, demands greater resource and understanding, there will be occasions when a less superficial approach to human control is desired by STPA practitioners.

Operational Resilience Management

ABSTRACT. Progressing digitalization and networking of systems and organization opens new potentials and opportunities. On the downside, the increasing complexity and interdependencies amplify the challenge to withstand adverse events. This puts pressure on classic risk management processes, which are getting to its limits when coping with such complex structures. This observation resulted into the emergence of the resilience concept, which strongly focuses on inherent capabilities of systems or organizations. In this context, several fundamental works formed a basic understanding of resilience and shaped the way for its actual application. The present work continues this path by proposing a framework to integrate resilience management into the actual operation of systems and organizations by providing an operative process for the provision and management of the fundamental resilience capabilities of responding, monitoring, anticipation and learning. Digital concepts, e.g. Digital Twins and the related data and information base, bear huge potential for further enhancement of resilience capabilities by anticipating management. Thus, they are deeply integrated into the framework to provide the capability of a self-adjusting resilience management and risk mitigation process.

13:30-14:00 Session 31: Upcoming book conversation "Advancing resilient performance"

Presentation of book by Editors Erik Hollnagel and Chris Nemeth and other authors

Location: Main Room
14:00-14:30 Session 32: The Power of Curiosity (USA) Gary Klein
Location: Main Room
The Power of Curiosity

ABSTRACT. Definition: Curiosity is the motivation to bridge a gap. The literature suggests this can be an information gap or a mental model gap. I think a better formulation is to consider a plausibility gap. Features of curiosity: Curiosity is superficial and transient and intense and impulsive. Superficial — it can arise, change focus, and end abruptly. (Waiting in a checkout line in a supermarket, we may become intensely curious about the latest news regarding a movie star’s marital woes — this curiosity disappears as soon as we step forward and away from the tabloids.) Transient — it reflects attention, and when attention captured by something else, curiosity disappears. Intense — people will work to satisfy their curiosity despite being hungry or thirsty Impulsive — something can evoke curiosity and stop us in our tracks. We may be tempted to buy that tabloid in the supermarket as long as we are looking at it Blocking curiosity: The research literature identifies common practices that can interfere with curiosity. Many of these inhibitors are initiated by instructors. Ask closed questions — calling for Yes/No answers. Focus on procedures. Emphasize memorization. Claim that there are right answers. Quickly correct students’ mistakes. Provide After-Action reviews that contain mostly statements and few questions. Focus on what the trainee is doing, and not why. Overwhelm students with details. Introduce too much complexity too early. (Too much confusion/complexity/ambiguity inhibits curiosity. The student stops being curious and instead tries to memorize all the material. Further, presenting too many open variables and loose ends can inhibit curiosity. Additional instructional behaviors that can inhibit curiosity include the following: Discourage questions from the students/trainees — make it clear that the class needs to get through all the material. Discourage class discussions. Provide complete explanations, leaving no space for student to engage in self-explanation. Provide explanations that make students feel stupid — use ridicule to make students afraid to appear dumb. Promoting curiosity. The collection of common inhibitors is fairly discouraging. However, the literature also identifies a number of factors that instructors and trainers can use to harness curiosity. These include the following: Pose a question; present contrasts and counterfactuals/hypotheticals; provide a sequence with an unknown resolution [e.g., who will win an election, or a race?). In addition, teachers can try to switch out of a mindset of criticizing students and instead wonder what caused the student to make a mistake.

14:30-15:30 Session 33: Media and tech companies: Part B Day 4

Lightning talk and fish bowl discussion

Location: Main Room
Surprises and adaptations in software development projects
PRESENTER: Michael Smith

ABSTRACT. The design and development of software systems is a complex process. Even a relatively small project may involve distributed teams of several different types of professions, dozens of different tools, and hundreds of different documents. Prescriptive system development processes, of which there are many, are focused on how to proactively reduce potential variation. By contrast, there has been little work exploring how to adapt to unanticipated variation - how to cope with surprises - in software project management. Surprises in the systems development process can impact the cost of development, the scope or quality of the product, the time of delivery, and the relationships between stakeholders. They can have positive or negative impacts, opening new opportunities or hindering progress forward. Responding to a surprise constitutes a test of the ability of the project team and parent organization to recognize and adapt. The sensitivity to meaningful variation and potential risks, the flexibility of work processes and resource allocation, and the expertise of the responsible practitioners all play important roles in the adaptation process. In this session we will begin by discussing patterns in problem detection (Klein), anomaly response (Woods & Hollnagel), and organizational surprises (Cunha, Clegg & Kamoche). We will share some examples from our own work at Mile Two and analyze them. The aim of the analysis will be to identify macrocognitive functions related to coping with surprises in this context, and how project or organizational resilience is evidenced in these examples. To make this an interactive session, at the start of the conference we will ask participants to share electronically their own accounts of surprises in systems development projects. We will include these in our analyses - ahead of time for earlier contributions, and in real time for late or concurrent contributions. We will use an online whiteboard or similar tool to actively show the real-time analysis.

Responses to overload as foundation for a resilient project and organizational health metric
PRESENTER: Marisa Bigelow

ABSTRACT. It can be difficult to understand changes to team performance in the moment and at a larger organizational scale, especially when adapting to unique evolving pressures and opportunities. To translate theory into practice and evaluate resilience at a project level, we are measuring four responses to saturation on a regular cadence and creating new project monitoring representations for communication and comparison at all organizational levels. The ongoing work exemplifies the convergence of contextualized decision-making with a resilience engineering framework for analysis.

Team Decision Making in Designing Software for Joint Cognitive Systems

ABSTRACT. This paper describes how Mile Two has attempted to integrate CSE with other disciplines in order to design software to support decision making in joint cognitive systems. The paper discusses three aspects of our approach: 1) Recognizing the value and limitation of different design perspectives; 2) Cross-training; and 3) Involving all disciplines at all stage of the design process - co-creation. The paper will discuss the successes and difficulties associated with implementing a total team process across different stages of design and development.

15:45-16:15 Session 34A: Models - Part B Day 4

Models - Part B Day 4

Location: Main Room
Using the Video-Occlusion Research Method to Guide the Design of a Perceptual-Cognitive Training Program for Law Enforcement Officers

ABSTRACT. Law enforcement officers often must make rapid decisions and actions to protect themselves and control a situation or a subject. While different agencies teach different techniques and philosophies for use-of-force and arrest-and-control, they are all enhanced by accurate and fast recognition of potentially attacking motions made by subjects. Whether subjects are civilians being questioned or suspects being detained, officers must respond with an appropriate scale and type of action. Slow or inappropriate actions can have highly negative results for subjects, officers, and their agencies. Unfortunately, training of rapid decision-action performance requires a high volume of practice that is difficult to acquire in training contexts that typically focus on “what you do” more than “what you see.” One possible solution is to target the Recognition component for concentrated practice using the Video-Occlusion method developed in sports science. Video-occlusion was validated by reliably differentiating expert performers from less expert performers. It was then developed into video-occlusion training applications. For decades researchers have observed that video-occlusion might contribute to training rapid decision-action performance in military and law enforcement contexts. This study compared the performance of experienced law enforcement officers and less experienced officers on a video-occlusion test of Attack Recognition. Participants in the study viewed 48 video examples of attacking and non-attacking movements by subjects. Video clips were edited to black (occluded) at various points in subjects’ movements, including occlusion just prior to the motion commencing. Participants identified the type of attacking (overhand punch, underhand punch, kick, or knife draw) or non-attacking motion. Findings validate the video-occlusion method and recommended specific occlusion points for recognition-only video-occlusion training of attack recognition that can be delivered on tablets and cell phones as a way to accelerate expertise.

Implications for World Model Development Based on Perceptual Cues and Strategies Used by an Experienced Pilot to Negotiate Wires Obstacles
PRESENTER: Katie Ernst

ABSTRACT. Helicopter or other rotorcraft operations fly almost continuously in close proximity to hazards and are just beginning to incorporate advanced automation that will rely on dynamic world models. Our work highlights the perceptual cues and strategies of human rotorcraft pilots in navigating complex airspace and identifies opportunities to support human knowledge and resilience to improve overall mission effectiveness and safety.

15:45-16:15 Session 34B: Showcase from around the world - USA

Mike Rayo, David Woods

A conversation around

  • What happen since last symposium
  • what are your current projects
  • Current questions and possible solutions
  • What industry or other collaborations with end users you have
  • If relevant PhD topics of your students
16:15-17:15 Session 35: RE-NDM - Tools-Methods

Lightning talks and fish bowl discussion - World café

Location: Main Room
Uncovering resilience abilities in maintenance teams for buildings with Functional Resonance Analysis Method
PRESENTER: Ivenio De Souza

ABSTRACT. Building maintenance comprises a set of complex socio-technical activities, with many interacting agents. Traditional tools for safety management make the results of such evaluations distant from real situations. Resilience Engineering (RE) argues that improvements in safety performance concern the ability to recognize and adapt to handle unanticipated perturbations. This study aims at increasing the understanding of everyday building maintenance activities for air conditioning systems to uncover resilience abilities developed by the maintenance team to deal with everyday challenges. The Functional Resonance Analysis Method (FRAM) has been applied to model these activities. The data collection comprised interviews and observations. As a contribution, this study outlined the potential of the FRAM model as the basis of in-depth and systematic analysis of daily performance, highlighting resilience abilities aligned with RE principles.

Use of Smart MMO Virtual Mega Worlds to Accelerate Innovative business Models and Expand Complex decision making in Organizations

ABSTRACT. For decades, our team has explored accelerating expertise through a specific kind of “gamification”, which emphasizes unstructured tasks with high stakes “non-negotiable” goals and opportunities for rapid trial and error iteration. This approach has resulted in the development of deep domain expertise. Recent work incorporating the use of Massive Multi-player virtual worlds with advanced behaviour tracking, which we pursued as a way of increasing the power of this method and scaling it, has yielded some unexpected benefits, for both accelerating learning and for organizational innovation. This paper discusses a case study of a Mining Portfolio Company that discovered an innovative way of using decision making enabled by the technology and the method.

Resilient Analysis Grid: A quantitative approach of healthcare provider’s perspective during COVID-19 pandemic
PRESENTER: Vanessa Bertoni

ABSTRACT. There are growing concerns about how healthcare systems can adapt in times of crisis. The overarching challenge lies in how resilience engineering could be used to analyze and improve the performance of healthcare systems concerning the Covid-19. This study aims to describe the relationship between resilience potentials and health and safety aspects and its consequences on quality and resilience in healthcare systems. This study has a quantitative methodological approach using a survey with the Resilience Analysis Grid as an approach to analyzing organizational resilience based on the idea that four potentials (responding, monitoring, learning, and anticipating) influence patient safety, occupational health and safety, and resilient system performance. As for this study results, anticipating and monitoring, overall resilience, and occupational health and safety are the variables that need more attention in healthcare systems. This study has a dyad of contributions, as a practice, evaluate the resilience in a pandemic time, and as theoretical, the identification of the importance of resilience four potential connections in healthcare systems

How can we help novice child protection social workers to see situations like experienced practitioners? A randomised controlled trial evaluation of the ShadowBox™ method using pre-recorded video feedback
PRESENTER: Andrew Whittaker

ABSTRACT. 1. Background:

Protecting children for abuse and neglect is a complex area of decision-making but frequent staff turnover has meant that many frontline child protection social workers are often relatively inexperienced. The ShadoxBoxTM method is an educational intervention that enables novice practitioners to gain decision-making skills quickly.

2. Aim:

This study was an RCT evaluation of an educational intervention for novice social workers that used the ShadoxBoxTM method adapted to include pre-recorded video feedback from an expert panel to test whether novice decision making would become more similar to experienced practitioners.

3. Methods:

The study was an RCT in which participants completed complex scenarios in a computer lab in control and intervention groups. The training method involved participants receiving feedback from highly experienced practitioners at each decision point within the scenarios. The intervention group received pre-recorded video feedback from a panel of highly experienced practitioners while the control group received no feedback.

4. Participants and Setting

Participants (n=83) were trainee social workers from a London university randomly allocated to control and intervention groups. Data was collected on computer stations using Qualtrics.

5. Results

Undertaking the scenarios improved both groups but greater improvement was seen in participants who had received video feedback from a panel of experienced practitioners. The results were promising, with participant accuracy increasing by 44% (from 31% to 75%) in the intervention group compared to an increase of 31% (from 32% to 63%) in the control group.

Considerable improvements were noted in both intervention and control groups, which suggest that scenario-based interventions can be a promising educational method as learning is rooted in real life scenarios and participants have the opportunity to reflect upon their decisions.

The qualitative findings are that novice participants make predictable errors, including: • Making shallow assumptions • Focusing exclusively on the parents rather than the child. • Jumping to early conclusions with insufficient information.

6. Conclusions

• ShadoxBoxTM training appears to be a promising intervention for improving decision making. • Novices benefitted from having concentrated exposure to complex scenarios focused on assessing risk and making professional judgements. • When this was augmented by direct feedback from a panel of highly experienced practitioners, these benefits were increased considerably. • The complexity of the scenarios also exposed the novices to real life pressures rather than the simplified versions used in decision research.

17:30-19:00 Session 36: Oberservations from Emerging talents and closure

Central discussion and experiences - What did we learn from successes and failures to sustain operations when challenges, which capabilities have been generated?

Location: Main Room