ABSTRACT. The simulation of biological neural networks holds immense promise for advancing both neuroscience and artificial intelligence. Due to its high complexity, it requires powerful computers. However, the high proportion of communication and routing makes general-purpose pro- cessing architectures, as used in supercomputers, inefficient. Dedicated hardware, such as ASICs, on the other hand, can be specifically adapted to this type of workload. However, integrated circuits are rigid, thereby eliminating the use of future neuron models. To address this contradiction, this paper presents a programmable ar- chitecture for the computation of neuron models. Thanks to its Turing completeness, it enables embedding biological neural networks simula- tors into integrated circuits while simultaneously allowing adaptation of the neuron model. To assess suitability, both dedicated circuits and off- the-shelf processors are examined regarding AT efficiency. The proposed versatile architecture turns out to be up to 1800x more area efficient than a RISC-V processor, thereby playing a vital role in accelerating neuroscience simulation and research in AI.

ABSTRACT. We present NADA, a Network Attached Deep learning Accelerator. It provides a flexible hardware/software framework for training deep neural networks on ethernet-based FPGA clusters. The NADA hardware framework instantiates a dedicated entity for each layer. Features and gradients flow through these layers in a tightly pipelined manner. From a compact description of a model and target cluster, the NADA software framework generates specific configuration bitstreams for each particular FPGA in the cluster. We demonstrate the scalability and flexibility of our approach by mapping an example CNN onto a cluster consisting of three up to nine Intel Arria 10 FPGAs. To verify NADAs effectiveness for commonly used networks, we train MobileNetV2 on a six-node cluster. We address the inherent incompatibility of the tightly pipelined layer parallel approach with batch normalization by using online normalization instead.

ABSTRACT. This work offers a heuristic evaluation of the effects of variations in machine learning training regimes and learning paradigms on the energy consumption of computing, especially HPC hardware with a life-cycle aware perspective. While increasing data availability and innovation in high-performance hardware fuels the training of sophisticated models, it also fosters the fading perception of energy consumption and carbon emission. Therefore, the goal of this work is to raise awareness about the energy impact of general training parameters and processes, from learning rate over batch size to knowledge transfer. Multiple setups with different hyperparameter configurations are evaluated on three different hardware systems. Among many results, we have found out that even with the same model and hardware to reach the same accuracy, improperly set training hyperparameters consume up to 5 times the energy of the optimal setup. We also extensively examined the energy-saving benefits of learning paradigms including recycling knowledge through pretraining and sharing knowledge through multitask training.

ABSTRACT. This article presents the Sorting Composite Quantile Regression Neural Network (SCQRNN), an advanced quantile regression model designed to prevent quantile crossing and enhance computational efficiency. Integrating ad hoc sorting in training, the SCQRNN ensures non-intersecting quantiles, boosting model reliability and interpretability. We demonstrate that the SCQRNN not only prevents quantile crossing and reduces computational complexity but also achieves faster convergence than traditional models. This advancement meets the requirements of high-performance computing for sustainable, accurate computation. In organic computing, the SCQRNN enhances self-aware systems with predictive uncertainties, enriching applications across finance, meteorology, climate science, and engineering.

ABSTRACT. The Artificial DNA (ADNA) and Artificial Hormone System (AHS) together form a middleware that uses Organic Computing tech- niques to improve the robustness and adaptability of distributed embed- ded systems. These systems then have the properties of self-organization, self-healing, self-configuration and self-improvement. However, the adapt- ability of the system is limited by the rigidity of the ADNA, since it cannot be modified at runtime. Recent research approaches already as- sume the existence of a modifiable ADNA in their applications without actually implementing it or evaluating its behavior. In this paper, we present two crucial steps to extend the ADNA with the ability to allow run-time modifications. First, we describe the possible modifications that can be made at runtime, and how they can be implemented without making significant changes to the ADNA implementation. Second, we provide an experimental evaluation of this new feature and contextualize its behavior within the framework of traditional ADNA.

ABSTRACT. This article investigates the application of computer vision and graph-based models in solving mesh-based partial differential equations within high-performance computing environments. Focusing on structured, graded structured, and unstructured meshes, the study compares the performance and computational efficiency of three computer vision-based models against three graph-based models across three datasets. The research aims to identify the most suitable models for different mesh topographies, particularly highlighting the exploration of graded meshes, a less studied area. Results demonstrate that computer vision-based models, notably U-Net, outperform the graph models in prediction performance and efficiency in two (structured and graded) out of three mesh topographies. The study also reveals the unexpected effectiveness of computer vision-based models in handling unstructured meshes, suggesting a potential shift in methodological approaches for data-driven partial differential equation learning. The article underscores deep learning as a viable and potentially sustainable way to enhance traditional high-performance computing methods, advocating for informed model selection based on the topography of the mesh.