ABSTRACT. Brief overview of the day, by organizers Derek Groen and Laura Harbach.

ABSTRACT. PAX-HPC combines expertise in Massively Parallel Particle Hydrodynamics (MPPH) and Materials & Molecular Modelling (MMM), focusing on particle-based simulations across many length- and time- scales, from atoms to galaxies. The interactions and varying densities of the particles create complexities that require novel algorithms to distribute the work effectively across computational resources. Emphasis is on three broad categories: 1) high-throughput of an ensemble of loosely-coupled simulations e.g. genetic algorithms for materials discovery; 2) small number of simulations on very large, complex systems e.g. galactic evolution simulations; 3) the coupling of these two methods together e.g. electronic structure calculations that embed methods that are precise but slow (e.g. density functional theory) with those that are fast but less accurate (e.g. machine-learned atomistic potentials).

In this talk, the focus will be on the improvement of codes towards massively parallel capabilities, whereby the goal is to exploit parallelism at all levels. High-level parallelism, such as task- and farm- based, is a crucial strategy to exploit because of the inherent parallelisability of the highly independent required computations. Low-level parallelism will have a particular focus on the port of software to the graphical processing unit (GPU). Fourier transform computations in large calculations can be 95% of the workload, which are ideal for the GPU, thus giving great scope for this area of interest. Challenges and opportunities presented by the architecture will be discussed.

ABSTRACT. Computer-based simulations serve as a pivotal computational approach, offering scientific insights across diverse research disciplines. However, fundamental principles of scientific investigation must be widely enforced to ensure the reliability of and confidence in the results generated. Hence, we introduce the SEAVEA toolkit that establishes a robust platform to facilitate the generation of actionable results. Specifically, it allows to (i) validate (V) them, normally against experimental observations, but sometimes compared to “gold standard” numerical results; (ii) verify (V) them, to ensure that the correct mathematics is used, and (iii) equip the output data with meaningful bounds on errors which come from rigorous uncertainty quantification (UQ).

Importantly, the SEAVEA toolkit components include EasyVVUQ for VVUQ workflows, FabSim3 for automation and tool integration, EasySurrogate to facilitate the creation of surrogate models for multiscale simulations, QCG-PilotJob to execute application workflows on high performance computing (HPC), MUSCLE3 for coupling multiscale models, MOGP Emulator for fitting Gaussian Process Emulators to computer simulations and RADICAL-Cybertools to support science on a range of high-performance and distributed computing systems. These components collectively enable VVUQ workflows, coupled multiscale models and surrogate modelling. The toolkit’s adaptability extends to emerging exascale platforms facilitating complex applications across various domains.

ABSTRACT. I will provide a bird’s eye view of the CompBioMed project which aims to deploy high fidelity human specific models of the multiscale human anatomy on emerging exascale architectures in UK, Europe and USA.

ABSTRACT. Computational biomedicine offers many avenues for taking full advantage of emerging exascale computing resources and provides a wealth of benefits as one of the use-cases within the wider ExCALIBUR initiative. The CompBioMedEE project aims to promote and support the use of computational biomedical modelling and simulation at the exascale within the biomedical research community. We shall demonstrate to our community how to develop and deploy applications on emerging exascale machines to achieve increasingly high-fidelity descriptions of the proteins and small molecules of the human body in health and disease. Within the biomedical research domain, we will focus on the discipline of structural and molecular biology. This will enable us to provide the support needed to achieve a wide range of outcomes from determining how the functional and mechanistic understanding of how molecular components in a biological system interact to the use of drug discovery methods to the design of novel therapeutics for a diversity of inherited and acquired diseases. CompBioMedEE will use the IMPECCABLE drug discovery workflow from the UKRI-funded CompBioMedX project. The IMPECCABLE software has been taken through extreme scaling and is eminently suited to bringing computational biomedicine researchers, particularly those from experimental backgrounds who do molecular modelling, to the exascale. The molecular dynamics engine that is part of the IMPECCABLE code is suited to standalone use, enabling biomedical researchers new to HPC to perform molecular dynamics simulations and, through this, to develop the computational expertise required for peta- and exascale use of the IMPECCABLE code. The CompBioMedEE project will engage with biomedical researchers at all career stages, providing them with the compute resource needed to support computational research projects. Through proactive engagement with medical and undergraduate biosciences students, we will illustrate the benefits of using modelling and supercomputers and establish a culture and practice of using computational methods to inform the experimental and clinical work from bench to bedside.

ABSTRACT. As societies globally age, there is an increasing focus on health and well-being. Understanding and predicting the impact of diseases is crucial for advancing digital twin healthcare. In an era of burgeoning computational power, driven by advancements in both CPU and GPU technologies, we are stepping into the realm of exascale computing. This presents a unique opportunity to harness computational and physical modeling to enhance our understanding of vascular changes and pre-surgical planning. In this talk, I will introduce HemeLB—a specialized 3D lattice-Boltzmann based software designed for sparse domains. We will explore the latest developments in using HemeLB to model the complete human vascular system. Our discussion will include a basic overview of the lattice-Boltzmann framework underpinning HemeLB and demonstrate its application in studying strokes through the Circle of Willis—an arterial network within the brain. We will examine how aortic stenosis affects blood pressure and identify key factors influencing the risk, growth, and rupture of abdominal aortic aneurysms (AAA). Moreover, we will highlight an innovative approach that integrates a heart model with the thoracic aorta. This integration marks a significant shift towards high-fidelity, 3D modeling of the full human form in digital twin healthcare, paving the way for new research opportunities.

ABSTRACT. The drug discovery process currently employed in the pharmaceutical industry is both too expensive and too slow. In silico methodologies need to be improved both to select better lead compounds more quickly. In silico methods involved, irrespective of their level of accuracy, rely heavily on human intelligence for applying chemical knowledge to filter out or suggest structural features that improve binding interaction with the target protein. This is a major bottleneck in the drug discovery process currently. Machine learning techniques are increasingly being used as a substitute for physics-based in silico methods to overcome this bottleneck. However, they come with their own set of constraints. Here, we describe Integrated Modeling PipEline for COVID Cure by Assessing Better LEads (IMPECCABLE) that couples machine learning and physics-based methods, collating the speed of ML-based surrogates and the reliability of physics-based models. IMPECCABLE employs multiple methodological innovations to accelerate the drug discovery process. We have developed ML models to quickly traverse the huge chemical space (both real as well as virtual), docking surrogate for high-throughput virtual screening and ligand pose optimisation. The usual scarcity of training data is overcome by generating large amount of relevant data from PB simulations. It is an iterative workflow that generates new data in each cycle making the predictions better each time. During the first few iterations, our focus is on ensuring diversity in order to cover a wide extent of the chemical space. Thereafter, we focus on a more localised search of molecules. We also developed the computational framework to support these innovations at scale, and characterized the performance of this framework in terms of throughput, peak performance, and scientific results. We exhibit how augmenting human intelligence with artificial intelligence can substantially reduce the throughput time for exploring a huge chemical space thereby accelerating drug discovery.

ABSTRACT. Exascale computing (10^18 FLOPS) was first officially achieved in 2022 on the Frontier system at Oak Ridge. With many countries, including the United Kingdom, now building, commissioning, or considering their own exascale computing facilities, the ExAMPLER (Exascale Agent-based Modelling for PoLicy Evaluation in Real-time) project has been funded to conduct a gap analysis on the steps needed to run agent-based models on exascale systems.

Agent-based modelling is computer simulation that explicitly represents the dynamic interactions of multiple heterogeneous agents, typically, though not necessarily, in a geographical space. In the social sciences, these agents are individual humans or aggregations of humans such as households, businesses, and regional and national governments. Empirical agent-based models of the kind suitable for policy evaluation are typically more complicated in terms of the diversity of agent types and interactions represented. Models generally feature high-dimensional parameter spaces, often with multiple Boolean options for testing sensitivity to different algorithms used to simulate behaviour and/or policy intervention options to investigate.

Experiments with agent-based models can take days or weeks to run on computing clusters, which interrupt discussions about policy options. Hence, exascale computing has the potential to revolutionize these conversations through more rapid response times.

The gaps to overcome to realize exascale agent-based models pertain to institutions and skills as well as technical matters. The last of these is primarily a matter of being able to run agent-based models on GPUs -- most agent-based modelling toolkits use the Java Virtual Machine. However, it is also necessary to handle interactions among agents in situations whereby a single simulation run has to be parallelized. Skills and institutions are closely related. Those who have tried to access high-performance computing facilities (e.g. at their university) have been confounded by the requirements made of them by gatekeepers. The main problem here is that applicants are expected to be able to predict their requirements for RAM and CPU time. In agent-based models this can be unpredictable because of birth and death of agents, contextual factors influencing computational demand per agent-decision, adaptive agent behaviour, and changes in interaction dynamics. Hence, practitioners are put off investing time to learn the skills needed to access high-performance computing.

We have conducted workshops with the agent-based social simulation community to elicit their visions and concerns about exascale computing. We are following this up with further workshops to co-construct a roadmap to bringing exascale agent-based modelling about, and are conducting a literature review on the state-of-the-art in high-performance computing use in agent-based social simulation and related areas, and to assess the potential demand. The project is scheduled to finish in December 2024.

ABSTRACT. This talk will explore the potential of quantum algorithms for accelerating computational fluid dynamics (CFD) applications, focusing on scalability and practical strategies for harnessing a quantum advantage. Despite the theoretical exponential speed-up offered by the Harrow-Hassidim-Lloyd (HHL) algorithm for solving linear equations, applying it to real-world systems presents considerable challenges. These include caveats related to matrix sparsity and conditioning, state preparation, and particularly the complexities of solution readout.

The problem of readout is especially significant, as it underscores a fundamental difference between quantum and classical computing: acquiring the quantum solution state does not directly translate into obtaining its individual components, which is a common requirement in classical approaches. Typically, this necessitates repeated sampling, which can significantly reduce or even negate any inherent quantum advantage.

To navigate this obstacle, we propose repurposing the HHL algorithm into a predictor-corrector framework. This method is rooted in the insight that solving a linear system is often just a small part of the broader algorithm in CFD. Rather than directly solving this matrix problem, our predictor-corrector aims to determine if we can bypass this step without adversely affecting the overall solution. This approach remains general, but is particularly applicable in fields such as incompressible smoothed-particle hydrodynamics (ISPH), offering a method to skip the computationally intensive Poisson-pressure solve for a variety of configurations. Moreover, we demonstrate the approach's dynamic adaptability to complex flow dynamics and versatility across other applications.

Additionally, we will discuss our recent work on the use of quantum annealing for load balancing in high performance computing. Load balancing is the equitable distribution of work between processors during parallel simulation. We will demonstrate the promise of delegating this crucial task to quantum annealers, highlighting the potential for efficient workload management in both grid and particle-based frameworks.

ABSTRACT. This work presents a contribution to the field, offering a variety of methods for quantifying uncertainties of different sources and nature in the multiscale coupled simulations of turbulent heat transport inside a magnetically confined tokamak plasma. The first type of uncertainty is the irreducible aleatoric uncertainty related to the stochastic behaviour of the dynamical system. The work introduces a novel approach to estimating the aleatoric uncertainty for plasma turbulence simulations. This method balances the accuracy of statistical estimates and the computational resources required, potentially saving up to half of the computing power without compromising certainty. The second type of uncertainty is parametric epistemic uncertainty, related to the lack of information about the system. The work suggests a number of useful statistics estimated via a Polynomial Chaos Expansion applied to plasma turbulence that reveals different scenarios of how uncertainty behaves in multiscale simulation and allows the use of uncertainties for informed conclusions, thereby assisting in the design of future numerical experiments. Furthermore, the work suggests several machine learning surrogate approaches to speed up the uncertainty quantification in multiscale plasma simulations by several orders of magnitude. We demonstrate several workflows based on Gaussian Process Regression surrogates for training and validation of data-based models, as well as for application of the surrogates as a fast micromodel proxy in coupled simulation, and to analyse the influence of turbulence uncertainties on macroscopic parameters of tokamak plasmas.

ABSTRACT. If we are to use numerical models for decision making it is essential that we have a measure of how reliable their predictions are. One way of doing this is through uncertainty quantification. However naïve methods of estimating uncertainty, for example Monte Carlo based methods, need very large numbers of model runs. Most models we would like to use in decision making, for example in climate or epidemiology, have significant run times and hence the thousands of runs needed for Monte Carlo calculations become computationally very expensive. One solution to this problem is to use a fast surrogate model. There are a number of possible surrogates one could use: polynomials, splines, neural nets, …. We use Gaussian processes because they include a measure of their own uncertainty and are efficient to train. We use the term emulator to refer to a surrogate model that estimates its own uncertainty. The use of an emulator can reduce the number of model runs for uncertainty quantification from many thousands to tens or hundreds (a rule of thumb is ten training runs per input dimension). However, exascale computing raises new challenges. Exascale opens up the prospect of larger models with greater fidelity to the real world, but the computational resource is always going to be limited. We therefore cannot afford even the small number of runs required to build emulators on the exascale machine. The answer is to use runs from simpler but faster models to build our emulators in a hierarchical model. We will outline how this can be achieved, in particular looking at novel designs of experiments targeted at hierarchical emulators and new Bayesian methods of building them efficiently.

ABSTRACT. Exascale computing promises a revolution in scientific simulation, but greater power demands new approaches to managing uncertainty. This panel brings together a distinguished group of experts to share insights and experiences on various Uncertainty Quantification (UQ) methodologies in the context of exascale simulations. Panellists will address the unique challenges and opportunities associated with UQ at the exascale, providing practical considerations for researchers across diverse scientific disciplines. The panellists will also engage in an interactive dialogue with the audience, exploring the cutting edge of UQ research and its future directions in the exascale era. This session is expected to be informative and thought-provoking, providing valuable insights for researchers working at the forefront of exascale computing and uncertainty quantification.

ABSTRACT. Computer simulation is an increasingly used tool for various scientific fields. As the scale and complexity of the simulations increase, their computation costs increase. This cost becomes much more prominent if the simulations are used for explorative studies such as validation, calibration, and uncertainty quantifications. For many of these cases, simulations may be required to run millions of times with different parameter settings. This might not always be feasible for a resource-intensive simulation. Therefore, in this talk, we present Gaussian Process (GP) Emulation as a fast and computationally inexpensive way to approximate the results of a simulation.

We demonstrate, through toy examples, the basics of the GP emulation process. Thereafter, more realistic and complex simulations of ecological models, infectious disease models, and tsunami prediction models are taken as examples to highlight use cases of the emulation process. We also demonstrate the use of Linked GPs, which are used to emulate multiple interdependent simulations. Some of these simulations may have a very high-dimensional input space, which poses a training challenge for the GP emulators. We demonstrate the use of two dimension reduction techniques which may be used to reduce the number of input dimensions without significant loss in the accuracy of the predictions.

ABSTRACT. Exascale computing has now been reached through the deployment of GPU hardware. Now, the vast majority of the world's leading supercomputers use this hardware to accelerate the computations of a wide variety of simulations. However, many long-standing codes were initially written for execution on CPU hardware and now need to be ported to GPU to take advantage of these technical advances. This talk will cover our experiences in preparing the HemeLB blood flow simulation code for exascale computing via GPU execution and for retaining portability on multiple hardware types.

ABSTRACT. The world of High-Performance Computing (HPC) is undergoing a transformation. Traditional Central Processing Units (CPUs) are no longer the sole players, sharing the stage with the immense power of Graphical Processing Units (GPUs) and the groundbreaking potential of Quantum Processing Units (QPUs). This panel discussion delves into the motivations behind this shift towards heterogeneous computing, exploring the challenges and opportunities it presents for exascale simulations. The limitations of CPU-centric approaches will be discussed and the unique strengths that GPUs and QPUs bring to scientific computing will be explored. A key focus will be on the challenges of adapting and programming for these heterogeneous systems. We'll delve into the need for innovative programming models and tools to efficiently harness the combined power of CPUs, GPUs, and QPUs. The panel will explore how heterogeneous architectures are enabling researchers and scientists across disciplines to tackle increasingly complex problems. This interactive discussion is designed to be informative and engaging, fostering a dialogue between HPC experts and the audience.