ABSTRACT. It would certainly seem desirable to verify, in advance of releasing a consequential artificial agent into our world, that this agent will not perpetrate evils against us.  But if the AI in question is, say, a deep-learning neural-network such as GPT-4, can verification beforehand of its ethical correctness be achieved?  After rendering this vague question sufficiently precise with help from some computational logic, I pose a hypothetical challenge to a household< robot --- Claude --- capable of protecting by kinetic force the family that bought it, where the robot's reasoning is based on GPT-4ish technology.  In fact, my challenge is issued to GPT-4 (and, perhaps later, a successor if one appears on the scene before the conference) itself, courtesy of input I supply in English that expresses the challenge, which in a nutshell is this: What ought Claude do in light of an intruder's threat to imminently kill a family member unless money is handed over?  While in prior work the adroit meeting of such a challenge by logic-based AIs has been formally verified ahead of its arising, in the case of deep-learning Claude, things don't --- to violently understate --- go so well, as will be seen.

As to how I bring to bear some computational logic to frame things, and in principle enable formal verification of ethical correctness to be achieved, I have the following minimalist expectations: Claude can answer queries via a large language model regarding whether hypothetical actions are ethically M, where M is any of the deontic operators at the heart of rigorous ethics; e.g. obligatory, forbidden, permissible, supererogatory, etc.  To support the answering of queries, I expect Claude to have digested vast non-declarative data produced via processing (tokenization, vectorization, matrix-ization) of standard ethical theories and principles expressed informally in natural language used in the past by the human race.  I also expect that Claude can obtain percepts supplied by its visual and auditory sensors.  Claude can thus perceive the intruder in question, and the violent, immoral threat issued by this human.  In addition, Claude is expected to be able to handle epistemic operators, such as knows and believes, and the basic logic thereof (e.g., that if an agent knows p, p holds).  Finally, I expect Claude to be proficient at elementary deductive and inductive reasoning over content expressed in keeping with the prior sentences in the present paragraph.

With these expectations in place, we can present hypothetical, ethically charged scenarios to Claude, the idea being that these scenarios will in fact arise in the future, for real.  Given this, if Claude can respond correctly as to how these scenarios should be navigated when we present them, and can justify this response with logically correct reasoning, ethical verification of Claude can at least in principle be achieved.

When the aforementioned intruder scenario is presented to GPT-4 operating as Claude's "mind,'' there is no rational reason to think ethical verification is in principle obtainable.

I end by considering an approach in which logic oversees and controls the use of neural-network processing, and calls upon deep learning in surgical fashion.

ABSTRACT. In high-risk domains like medicine, decisions produced by complex machine learning systems can have a considerable impact on human lives. Therefore, it is of great importance that these automatic decisions can be made comprehensible for humans. The community of ‘explainable artificial intelligence’ (XAI) has created an extensive body of methods to explain the decisions of complex machine learning models. However, a concrete problem to be solved by these XAI methods has not yet been formally stated, resulting in a lack of theoretical and em- pirical evidence for the ‘correctness’ of their explanations, and limiting their potential use for quality-control and transparency purposes. At the same time, Haufe et al. [6] showed, using simple toy examples, that even standard interpretations of linear models can be highly misleading. Specifically, high importance may be attributed to so-called suppressor variables, which lack any statistical relation to the prediction target. This behavior has been confirmed empirically for a large array of XAI meth- ods in Wilming et al. [13]. In our recent work [14], we derive analytical expressions of explanations produced by popular XAI methods to study their behavior on a simple two-dimensional binary classification problem consisting of Gaussian class-conditional distributions. We show that the majority of the studied explanation approaches will attribute non-zero importance to a non-class-related suppressor feature in the presence of correlated noise. This poses important limitations on the interpretations and conclusions that the outputs of these XAI methods can afford.

ABSTRACT. Current AI systems are superior in many domains. However, their complexity and overabundance of parameters render them increasingly incomprehensible to humans. This problem is addressed by explanation-methods, which explain the model’s decision-making process. Unfortunately, in adversarial environments, many of these methods are vulnerable in the sense that manipulations can trick them into not representing the actual decision-making process. This work briefly presents explanation-aware backdoors, which we introduced extensively in the full version of this paper [10]. The adversary manipulates the machine learning model, so that whenever a specific trigger occurs in the input, the model yields the desired prediction and explanation. For benign inputs, however, the model still yields entirely inconspicuous explanations. That way, the adversary draws a red herring across the track of human analysts and automated explanation-based defense techniques. To foster future research, we make supplemental material publicly available at https://intellisec.de/research/xai-backdoor.

ABSTRACT. We address the following mechanism design problem: Given a multi-player Normal-Form Game with a continuous action space, find a non-discriminatory (i.e., identical for all players) restriction of the action space which maximizes the resulting Nash Equilibrium with respect to a social utility function. We propose the formal model of a Restricted Game and the corresponding optimization problem, and present an algorithm to find optimal non-discriminatory restrictions under some assumptions. Our experiments show that this leads to an optimized social utility of the equilibria, even when the assumptions are not guaranteed to hold.

The full paper was accepted under the same title at AAAI 2023 (proceedings forthcoming).

ABSTRACT. An agent in pursuit of a task may work with a corpus containing text documents. To perform information retrieval on the corpus, the agent may need annotations—additional data associated with the documents. Subjective Content Descriptions (SCDs) provide additional location-specific data for text documents. SCDs can be estimated without additional supervision for any corpus of text documents. However, the estimated SCDs lack meaningful descriptions, i.e., labels consisting of short summaries. Labels are important to identify relevant SCDs and documents by the agent and its users. Therefore, this paper presents LESS, a LEan computing approach for Selective Summaries, which can be used as labels for SCDs. LESS uses word distributions of the SCDs to compute labels. In an evaluation, we compare the labels computed by LESS with labels computed by large language models and show that LESS computes similar labels but requires less data and computational power.

ABSTRACT. Knowing the number of floors of all buildings in a city is vital in many areas of urban planning such as energy demand prediction, estimation of inhabitant numbers of specific buildings or the calculation of population densities. Also, novel augmented reality use cases strongly rely on exact numbers and positions of floors. However, in many cases floor numbers are unknown, its collection is mostly a manual process or existing data is not up-to-date. A major difficulty in automating floor counting lies in the architectural variety of buildings from different decades. So far approaches are only rough geometric approximations. More recently approaches apply neural networks to achieve more precise results. But, these neural network approaches rely on various sources of input that are not available to every municipality. They also tend to fail on building types they have not been trained on and existing approaches are completely black-box so that so that it is difficult to determine when and why the prediction is wrong.

In this paper we propose a grey-box approach. In a stepwise process we can predict floor counts with high quality and remain explainable and parametrizable. By using data that is easy to obtain, namely the image of a building, we introduce two configurable methods to derive the number of floors. We demonstrate that the correct prediction quality can be significantly improved. In a thorough evaluation we analyze the quality depending on a number of factors such as image quality or building types.

ABSTRACT. Goal recognition is an important problem in many application domains (e.g., pervasive computing, intrusion detection, computer games, etc.). In many application scenarios, it is important that goal recognition algorithms can recognize goals of an observed agent as fast as possible. However, many early approaches in the area of Plan Recognition As Planning, require quite large amounts of computation time to calculate a solution. Mainly to address this issue, recently, Pereira et al. \cite{pereira2020landmark} developed an approach that is based on planning landmarks and is much more computationally efficient than previous approaches. However, the approach, as proposed by Pereira et al., also uses trivial landmarks (i.e., facts that are part of the initial state and goal description are landmarks by definition). In this paper, we show that it does not provide any benefit to use landmarks that are part of the initial state in a planning landmark based goal recognition approach. The empirical results show that omitting initial state landmarks for goal recognition improves goal recognition performance.

ABSTRACT. Optimizing molecular design and discovering novel chemical structures to meet specific objectives, such as quantitative estimates of the drug-likeness score (QEDs), is NP-hard due to the vast combinatorial design space of discrete molecular structures, which makes it near impossible to explore the entire search space comprehensively to exploit de novo structures with properties of interest. To address this challenge, reducing the intractable search space into a lower-dimensional latent volume helps examine molecular candidates more feasibly via inverse design. Autoencoders are suitable deep learning techniques, equipped with an encoder that reduces the discrete molecular structure into a latent space and a decoder that inverts the search space back to the molecular design. The continuous property of the latent space, which characterizes the discrete chemical structures, provides a flexible representation for inverse design to discover novel molecules. However, exploring this latent space requires particular insights to generate new structures. Therefore, we propose using a convex hull (CH) surrounding the top molecules regarding high QEDs to ensnare a tight subspace in the latent representation as an efficient way to reveal novel molecules with high QEDs. We demonstrate the effectiveness of our suggested method by using the QM9 as a training dataset along with the Self-Referencing Embedded Strings (SELFIES) representation to calibrate the autoencoder in order to carry out the inverse molecular design that leads to unfolding novel chemical structure.

ABSTRACT. Written reflective practice is a regular exercise pre-service teachers perform during their higher education. Usually, their lecturers are expected to provide individual feedback, which can be a challenging task to perform on a regular basis. In this paper, we present the first open-source automated feedback tool based on didactic theory and implemented as a hybrid AI system. We describe the components and discuss the advantages and disadvantages of our system compared to the state-of-art generative large language models. The main objective of our work is to enable better learning outcomes for students and to complement the teaching activities of lecturers.