ABSTRACT. Neural style transfer has been used as a popular graphics processing technique in recent years, by both digital artists in their practice and within commercial software systems. We investigate ways to blend styles both through interpolation of styles in a single neural model and via overlaying different stylisations. With the application of producing preset stylisation filters for the development of a casual creator app, we experiment with various MAP-Elites quality/diversity approaches to evolving style transfer blends with particular properties while maintaining diversity in the population.

ABSTRACT. In this paper we look at the concept of complexity as it applies to generative art and design. Complexity has many different, discipline specific definitions, such as complexity in physical systems (entropy), algorithmic measures of information complexity and the field of ``complex systems''. We apply a series of different complexity measures to three different generative art datasets and look at the correlations between complexity and individual aesthetic judgement by the artist (in the case of two datsets) or physically evaluated complexity of 3D forms. Our results show that the degree of correlation is different for each set and measure, indicating that there is no overall ``better'' measure. However, specific measures do perform well on specific datasets, indicating that careful choice can benefit the value of using such measures. We conclude by discussing the value or complexity measures in generative and evolutionary art and reinforce recent findings in neuroimaging and psychology that suggest human aesthetic judgement in informed by many extrinsic factors beyond the measurable properties of the object being judged.

ABSTRACT. Image enhancement is an image processing procedure in which the original information of the image is improved. It can be used to alter an image in several different ways, for instance, by highlighting a specific feature in order to ease post-processing analyses by a human or machine. In this work, we show our approach to image enhancement for digital real-estate-marketing. The aesthetic quality of the images for real-estate marketing is critical since it is the only input that clients have once browsing for options. Thus, improving and ensuring the aesthetic quality of the images is crucial for marketing success. The problem is that each set of images, even for the same real-estate item, is often taken under diverse conditions making it hard to find one solution that fits all. State of the art image enhancement pipelines applies a set of filters that tend to solve specific issues, so it is still hard to generalise that solves all type of issues encountered. With this in mind, we propose a Genetic Programming approach for the evolution of image enhancement pipelines, based on image filters from the literature. We report a set of experiments in image enhancement of real state images and analysed the results. The overall results suggest that it is possible to attain suitable pipelines that enhance the image visually and according to a set of image quality assessment metrics. The evolved pipelines show improvements across the validation metrics showing that it is possible to create image enhancement pipelines automatically. Moreover, during the experiments, some of the created pipelines end up creating non-photorealistic rendering effects in a moment of computational serendipity. Thus, we further analysed the different evolved non-photorealistic solutions, showing the potential of applying the evolved pipelines in other types of images.

ABSTRACT. In this work we formulate and propose an extended version of the multi-agent Swarm Grammar (SG) model for the generation of virtual worlds. It unfolds a comparatively small database into a complex world featuring terrain, vegetation and bodies of water. This approach allows for adaptivity of generated assets to their environment, unbounded worlds and interactivity in their generation. In order to evaluate the model, we conducted sensitivity analyses at a local interaction scale. In addition, at a global scale, we investigated two virtual environments, discussing notable interactions, recurring configuration patterns, and obstacles in working with SGs. These analyses showed that SGs can create visually interesting virtual worlds, but require further work in ease of use. Lastly we identified which future extensions might shrink required database sizes.

ABSTRACT. Reinforcement Learning (RL) offers a promising solution when dealing with the general problem of optimal decision and control of agents that interact with uncertain environments. A major challenge of existing algorithms is the slow rate of convergence and long training times especially when dealing with high-dimensional state and action spaces. In our work, we leverage evolutionary computing as a competitive alternative to training deep neural networks for RL problems. We present a novel distributed algorithm based on efficient model encoding which enables the intuitive application of genetic operators. Another contribution is the application of crossover operator in two neural networks in the encoded space. Preliminary results demonstrate a considerable reduction of trainable parameters and memory requirements while maintaining comparable performance with DQN and A3C when evaluated on Atari games, resulting in an overall significant speedup.

ABSTRACT. Following the development of Information Technology (IT) techniques, data and the knowledge behind data have been increased exponentially today. The goals to better manage and share such massive big data become more and more critical in many prominent Artificial Intelligence (AI)-based smart industries. Smart healthcare is a typical example in these cases. Efficient management of data sharing apparently can lead to better diagnosis, illness prevention and epidemic monitoring. However, appropriate and robust management of sharing sensitive data among different stakeholders operating within the ecosystem still poses challenges of geographical boundaries and compliance to diverse data sharing and access rules across continents. To address the complexity of large-scale data sharing and provide an efficient solution, our study proposes a bio-inspired autonomic agent-based framework capable of leveraging the large-scale distributed data sharing infrastructure with multiple stakeholders whilst, at the same time, supporting the development of future data integration ecosystems. Our current study selects smart healthcare as a use case to discuss the deployment of our data sharing framework and use this example to demonstrate the potential advantage of this framework but the framework itself also has good extensibility to other interdisciplinary scenarios and domains.

ABSTRACT. This work presents an improvement of a procedural content generation system with general purposes but adapted to the design of levels in Metroidvania games using a model of the preferences and experience of the designers. This system obtains promising results about the accomplishment of the designer preferences, but the generated diversity is lower. Therefore, we propose the addition of the fitness sharing and a local search process to obtain quality and diverse solutions. In the calculation of the sharing value we need a distance value between individuals, therefore, we propose the use of a quantum circuit to get this distance.

ABSTRACT. We analyze the sensitivity of a memetic algorithm to the parameter governing the stochastic application of local search (based on Sharpe index) in the context of constrained portfolio optimization, and compare it to non-memetic proposals.

ABSTRACT. Evolutionary computation has risen as a promising approach to propose neural network architectures without human interference, relieving researchers and practitioners from considerable efforts related to the many possible options of parameters and hyper-parameters. However, the high computational cost from that approach is a serious challenge for the application and even research of such methods. In this work, we address this issue by empirically analyzing the short-term effects of weight initialization strategies on the performance of neural networks that creates a highly changing space due to constant interventions from the evolutionary algorithms. We performed experiments with the CoDeepNEAT algorithm on the CIFAR-10 and MNIST datasets. While results are somewhat below start-of-the-art performances, we achieved higher than 99.20% in the MNIST dataset and 84.60% in the CIFAR-10 dataset with significantly less computational time, which suggests that focusing on short-term effects of the weight initialization functions is a promising direction for co-evolution of neural network architectures.

ABSTRACT. The plant propagation algorithm, a crossoverless populationbased metaheuristic, performs significantly better when its fitness function is deterministically adjusted throughout the optimization process.

ABSTRACT. In the plant propagation algorithm, we deploy an adaptive version of its sigmoidal fitness function which increasingly steepens while running. It increases at a linear rate, and optimal rate windows are identified for the five two-dimensional benchmark test functions, on which the algorithm significantly outperforms earlier results.

ABSTRACT. We discussBeauty, a hybrid biological-technological artwork, currently in development by Phylum, an experimental research collective an experimental research collective specializing in cultural production informed by the intersections of science, technology and the arts (and of which the authors are members of). The work is based upon an artificial intelligence agent that uses deep reinforcement learning to interact with and alter cultures of pattern-forming social bacteria in order to make them more aesthetically pleasing

ABSTRACT. Text-to-image generation involves the automatic production of images which somehow reflect a given word or phrase. Recently, a new approach to this task has become available, which employs generative adversarial networks (GANs) for image generation, with the latent vector inputs found by a backpropagation search. The search is directed by a loss function calculated by another pair of pre-trained models called CLIP, which can estimate how appropriately a generated image reflects a text prompt. We compare and contrast the usage of CLIP in guiding various GANs for artistic image production, highlighting the huge potential for this approach, as well as current limitations.

ABSTRACT. We present a generative deep learning tool that can produce symbolic musical compositions with interesting passages that can be considered extreme in various ways. Generation is via tuning and then seeding a pre-trained GPT-2 model, and we experiment with iterative seeding schemes. We generate thousands of musical segments, analyse and categorise them in terms of attributes involving note duration, repetitive figures and intervals, and then curate and concatenate passages into short musical pieces. In contrast to other approaches, we embrace extreme passages with respect to these musical attributes, and build pieces with extremity arcs via a user interface to the tool. We describe here our methodology, experiments and results which led to a collection of generated pieces on SoundCloud.

ABSTRACT. In this paper, we resort to the TensorFlow framework to investigate the benefits of applying data vectorization and fitness caching methods to domain evaluation in Genetic Programming. For this purpose, an independent engine was developed, TensorGP, along with a testing suite to extract comparative timing results across different architectures and amongst both iterative and vectorized approaches. Our performance benchmarks demonstrate that by exploiting the TensorFlow eager execution model, performance gains of up to two orders of magnitude can be achieved on a parallel approach running on dedicated hardware when compared to a standard iterative approach.

ABSTRACT. Designing robots by hand can be costly and time consuming, especially if the robots have to be created with novel materials, or be robust to internal or external changes. In order to create robots automatically, without the need for human intervention, it is necessary to optimise both the behaviour and the body design of the robot. However, when co-optimising the morphology and controller of a locomoting agent the morphology tends to converge prematurely, reaching a local optimum. Approaches such as explicit protection of morphological innovation have been used to reduce this problem, but it might also be possible to increase exploration of morphologies using a more indirect approach. We explore how changing the environment the agent locomotes in affects the convergence of morphologies. The agents' morphologies and controllers are co-optimised, while the environments the agents locomote in are evolved open-endedly with the Paired Open-Ended Trailblazer (POET). We compare the diversity, fitness and robustness of agents evolving in environments generated by POET to agents evolved in handcrafted curricula of environments. We show that agents evolving in open-endedly evolving environments exhibit larger morphological diversity in their population than agents evolving in hand crafted curricula of environments. POET proved capable of creating a curriculum of environments which encouraged both diversity and quality in the population. This suggests that POET can be a promising approach to reduce premature convergence in co-optimisation of morphology and controllers.

ABSTRACT. Biological and artificial embodied agents behave by acquiring information through sensors, processing that information, and acting on the environment. The sensory apparatus, i.e., the location on the body of the sensors and the kind of information the sensors are able to capture, has a great impact on the agent ability of exhibiting complex behaviors. While in Nature the sensory apparatus is the result of a long-lasting evolution, in artificial agents (robots) it is usually the result of a design choice. However, when the agents are complex and the design space is large, making that choice can be hard. In this paper, we explore the possibility of evolving the sensory apparatus of voxel-based soft robots (VSRs), a kind of simulated robots composed of multiple deformable components. VSRs, due to their intrinsic modularity, allow for great freedom in how to shape the robot body, brain, and sensory apparatus. We consider a set of sensors that allow the agent to sense itself and the environment (using vision and touch) and we show, experimentally, that the effectiveness of the sensory apparatus depends on the shape of the body and on the actuation capability, i.e., the VSR strength. Then we show that evolutionary optimization is able to evolve an effective sensory apparatus, even when constraints on the availability of the sensors are posed. By extending the adaptation to the sensory apparatus, beyond the body shape and the brain, we believe that our study takes a step forward to the ambitious path towards self-building robots.

ABSTRACT. In this work, we develop a set of behavioral and conditional modules for the use with behavior trees. We present AutoMoDe-Cedrata, an automatic modular design method that automatically assembles and fine-tunes these modules into behavior trees that control robot swarms. We test Cedrata on three missions and, to gain further insights on its effectiveness, we design control software for the same missions using AutoMoDe-Maple, another automatic design method, and by a group of human designers. Results show that the proposed modules allow for well-performing behavior trees. Yet, Cedrata had difficulties automatically generating control software that performs similarly well as the one generated by human designers, especially when involving communication.

ABSTRACT. Researchers in the field of nature-inspired optimization have recently proposed multi-population asynchronous algorithms that distribute the evolutionary process among heterogeneous search paradigms. These algorithms execute the optimization strategy by reading streams of messages containing solution populations from message queues. After searching for a small number of iterations, new evolved populations are generated and sent back to a queue. Current research suggests that when we have many population-processing algorithms communicating in parallel, parameters intensifying exploration or exploitation in each population strike a dynamic that balances the two, exploring and exploiting simultaneously, maintaining an overall diversity, and improving the search. In this work, we propose a simple reactive migration, population-generation, and processing method for the asynchronous processing of multi-population, multi-strategy algorithms that achieves an improvement over homogeneous configurations. We evaluate this method by comparing a heterogeneous ensemble of multi-populations against a homogeneous solution consisting of Genetic Algorithms (GAs) and Particle Swarm Optimization (PSO) populations. We evaluate both solutions using five problems from the noiseless BBOB testbed for the optimization of continuous functions. Results show that compared with other asynchronous homogeneous population-based algorithms, this method offers better performance in terms of the number of evaluations needed to find a solution and the number of instances where it found it.

ABSTRACT. Current developments in the field of Artistic Style Transfer use the information encoded in pre-trained neural networks to extract properties from images in an unsupervised process. This neural style transfer works well with art and paintings but only produces limited results when dealing with highly structured data. Characteristics of the extracted information directly define the quality of the generated artifact and traditionally require the user to do manual fine-tuning. This paper uses current methods of deep learning to analyze the properties embedded in the network, group filter responses into semantic classes and extract an optimized layer set for artistic style transfer, to improve the artifact generation with a potentially unsupervised preprocessing step.

ABSTRACT. This paper examines and showcases a framework to generate artworks using evolutionary algorithms. Based on the idea of an abstract artistic process stylized images are generated from different input images without human supervision. After explaining the underlying concept, the solution space of different styles is explored and its properties are discussed. Given this insights into the framework, current shortcomings are evaluated and improvements are discussed.

ABSTRACT. We introduce a new framework for interacting with and manipulating deep generative models that we call network bending. We present a comprehensive set of deterministic transformations that can be inserted as distinct layers into the computational graph of a trained generative neural network and applied during inference. In addition, we present a novel algorithm for analysing the deep generative model and clustering features based on their spatial activation maps. This allows features to be grouped together based on spatial similarity in an unsupervised fashion. This results in the meaningful manipulation of sets of features that correspond to the generation of a broad array of semantically significant aspects of the generated images. We outline this framework, demonstrating our results on state of the art deep generative models trained on several image datasets, showing how it allows for the direct manipulation of semantically meaningful aspects of the generative process as well as allowing for broad range of expressive outcomes.

ABSTRACT. This paper presents and evaluates co-creative drawing scenarios in which a user is asked to provide a small hand-drawn pattern which then is interactively extended with the support of a trained neural model. We show that it is possible to use one-shot trained Transformer Neural Networks to generate stroke-based images and that these trained models can successfully be used for design assisting tasks.

ABSTRACT. Standard grammatical evolution (GE) uses a one-point crossover ("ripple crossover") that exchanges codons between two genotypes. The two resulting genotypes are then mapped to their respective phenotypes using a Backus-Naur form grammar. This article studies how different types of grammars affect the resulting individuals of a ripple crossover. We distinguish different grammars based on the expected number of non-terminals chosen when mapping genotype codons to phenotypes, $B_{avg}$. The grammars only differ in $B_{avg}$ but can express the same phenotypes. We perform crossover operations on the genotypes and find that grammars with $B_{avg} > 1$ lead to high numbers of either very small trees or invalid individuals. Due to the re-sampling of the invalid individuals, the algorithmic runtime is higher compared to grammars with a small $B_{avg}$, despite being able to express the same phenotypes. In grammars with $B_{avg} \leq 1$, the bias towards small trees is reduced and instead, the frequency of valid large trees is increased. Our results give insights on favorable grammar designs and underline the central role of grammar design in GE.

ABSTRACT. When faced with a new dataset, most practitioners begin by performing exploratory data analysis to discover interesting patterns and characteristics within data. Techniques such as association rule mining are commonly applied to uncover relationships between features (attributes) of the data. However, association rules are primarily designed for use on binary or categorical data, due to their use of rule-based machine learning. A large proportion of real-world data is continuous in nature, and discretisation of such data leads to inaccurate and less informative association rules. In this paper, we propose an alternative approach called feature relationship mining (FRM), which uses a genetic programming approach to automatically discover symbolic relationships between continuous or categorical features in data. To the best of our knowledge, our proposed approach is the first such symbolic approach to discovering relationships between features. Empirical testing on a variety of real-world datasets shows the proposed method is able to find high-quality, simple feature relationships which can be easily interpreted and which provide clear and non-trivial insight into data.

ABSTRACT. Designing Neural Network Architectures requires expert knowledge and extensive parameter searches. Neural Architecture Search (NAS) aims to change that by automating the design process. It is important that these approaches are reproducible so they can be used in real-life scenarios. In our work, we reproduce a genetic programming approach to designing convolutional neural networks called CGP-CNN. We show that this is difficult and requires many changes to the training scheme, reducing real-life applicability. We achieve a final accuracy of 90.6% ± 0.005, substantially lower than the reported 93.7%±0.005. This negates some of the benefits of using CGP-CNN for NAS. We establish a random search as a consensus baseline and show that it produces similar results to the evolutionary method of CGP-CNN. To assess the adaptability and generality of the presented algorithm, it is applied to CIFAR-100 and SVHN with a final accuracy of 63.1% and 95.6%, respectively. We extend the investigated NAS by two methods for predicting candidate fitnesses from partial learning curves. This improves CGP-CNN runtime efficiency by a factor of 1.69.

ABSTRACT. Scalability problems have hindered the progress of Evolvable Hardware in tackling complex circuits. The two key issues are the amount of testing (for example, a 64-bit × 64-bit add-shift multiplier problem has 2^(64+64) test cases) and low level that hardware works at; a circuit to implement 64-bit × 64-bit add-shift multiplier would require approximately 33,234 gates when synthesized using yosys. We use Grammatical Evolution and SystemVerilog, a Hardware Description Language (HDL), to evolve parameterized adder, multiplier and selective parity circuits with default input bit-width sizes of 64-bit + 64-bit, 64-bit × 64-bit and 128-bit respectively. These are substantially larger than the current state of the art for evolutionary approaches, specifically, 6.4× (adder), 10.7× (multiplier), and 6.7× (parity). We are able to scale so dramatically because our use of an HDL permits us to operate at a far higher level of abstraction than most other approaches. This has the additional benefit that no further evolutionary experiments are needed to design different input bit-width sizes of the same circuit as is the case for existing EHW approaches. Thus, one can evolve once and reuse multiple times, simply by specifying the newly desired input/output bit-width sizes during module instantiation. For example, 32-bit × 32-bit and 256-bit × 256-bit multipliers can be instantiated from an evolved parameterized multiplier. We also adopt a method for reducing testing from Digital Circuit Design known as corner case testing. Circuit designers rely heavily on this to avoid time-consuming exhaustive testing; we demonstrate a simple way to identify and use corner cases for evolutionary testing and show that it enables the generation of massively complex circuits with a huge number of inputs. We obtain successful results (ranging from 72% to 100%) on each benchmark and all three problems were tackled without resorting to the use of any standard decomposition methods due to our ability to use high-level programming constructs and operators available in SystemVerilog.

ABSTRACT. Grammar-guided Genetic Programming is a common approach for program synthesis where the user's intent is given by a set of input/output examples. For use in real-world software development, the generated programs must work on previously unseen test cases too. Therefore, we study in this work the generalizability of programs synthesized by grammar-guided GP with lexicase selection. As benchmark, we analyze proportionate and tournament selection too. We find that especially for program synthesis problems with a low output cardinality (e.g., a Boolean output) lexicase selection overfits the training cases and does not generalize well to unseen test cases. An analysis using common software metrics shows for such a problem that lexicase selection generates more complex programs with many code lines and a heavier use of control structures compared to the other studied selection methods. Nevertheless, the generalizability can be improved when we do not stop a GP run as usual after a first program is found that solves all training cases correctly, but give GP more time to find further solution candidates (also solving correctly all training cases) and select the smallest program (measured with different software metrics) out of these.

ABSTRACT. DenseNet architectures have demonstrated impressive performance in image classification tasks, but limited research has been conducted on using character-level DenseNet (char-DenseNet) architectures for text classification tasks. It is not clear what DenseNet architectures are optimal for text classification tasks. The iterative task of designing, training and testing of char-DenseNets is an NP-Hard problem that requires expert domain knowledge. Evolutionary deep learning (EDL) has been used to automatically design CNN architectures for the image classification domain, thereby mitigating the need for expert domain knowledge. This study demonstrates the first work on using EDL to evolve char-DenseNet architectures for text classification tasks. A novel genetic programming-based algorithm (GP-Dense) coupled with an indirect-encoding scheme, facilitates the evolution of performant char-DenseNet architectures. The algorithm is evaluated on two popular text datasets, and the best-evolved models are benchmarked against four current state-of-the-art character-level CNN and DenseNet models. Results indicate that the algorithm evolves performant models for both datasets that outperform two of the state-of-the-art models in terms of model accuracy and three of the state-of-the-art models in terms of parameter size.

ABSTRACT. Feature selection is an essential preprocessing step in data mining and machine learning. A feature selection task can be treated as a multi-objective optimization problem which simultaneously minimizes the classification error and the number of selected features. Many existing feature selection approaches including multi-objective methods neglect that there exists multiple optimal solutions in feature selection. There can be multiple different optimal feature subsets which achieve the same or similar classification performance. Furthermore, when using evolutionary multi-objective optimization for feature selection, a crowding distance metric is typically used to play a role in environmental selection. However, some existing calculations of crowding metrics based on continuous/numeric values are inappropriate for feature selection since the search space of feature selection is discrete. Therefore, this paper proposes a new environmental selection method to modify the calculation of crowding metrics. The proposed approach is expected to help a multi-objective feature selection algorithm to find multiple potential optimal feature subsets. Experiments on sixteen different datasets of varying difficulty show that the proposed approach can find more diverse feature subsets achieving the same classification performance without deteriorating performance regarding hypervolume and inverted generational distance.

ABSTRACT. In order to address scalability issues, which can be a challenge for Deep Learning methods, we propose Wide Learning of Diverse Architectures --- a model that scales horizontally rather than vertically, enabling distributed learning. We propose a distributed version of a quality-diversity evolutionary algorithm (MAP-Elites) to evolve an architecturally diverse ensemble of shallow networks, each of which extracts a feature vector from the data. These features then become the input to a single shallow network which is optimised using gradient descent to solve a classification task. The technique is shown to perform well on two benchmark classification problems (MNIST and CIFAR). Additional experiments provide insight into the role that diversity plays in contributing to the performance of the repertoire.

ABSTRACT. Ensembles of predictors have been generally found to have better performance than single predictors. Although diversity is widely thought to be an important factor in building successful ensembles, there have been contradictory results in the literature regarding the influence of diversity on the generalisation error. Fundamental to this may be the way diversity itself is defined. We present two new diversity measures, based on the idea of ambiguity, obtained from the bias-variance decom- position by using the cross-entropy error or the hinge-loss. If patterns on which to train ensemble members are selected by random sampling, we find that generalisation error is negatively correlated with diversity at high sampling rates, but positively correlated at low sampling rates corresponding to high diversity. We use evolutionary optimisers to select the subsets of patterns for predictor training by maximising these diver- sity measures on training data. Evaluation of their generalisation per- formance on a range of classification datasets from the literature shows that the ensembles obtained by maximising the cross-entropy diversity measure generalise well, enhancing the performance of small ensembles. Contrary to expectation, we find that there is no correlation between whether a pattern is selected and its proximity to the decision boundary.

ABSTRACT. Planning is a powerful approach to reinforcement learning with several desirable properties. However, it requires a model of the world, which is not readily available in many real-life problems.

In this paper, we propose to learn a world model that enables \emph{Evolutionary Planning in Latent Space} (EPLS). We use a Variational Auto Encoder (VAE) to learn a compressed latent representation of individual observations and extend a Mixture Density Recurrent Neural Network (MDRNN) to learn a stochastic, multi-modal forward model of the world that can be used for planning. We use the Random Mutation Hill Climbing (RMHC) to find a sequence of actions that maximize expected reward in this learned model of the world.

We demonstrate how to build a model of the world by bootstrapping it with rollouts from a random policy and iteratively refining it with rollouts from an increasingly accurate planning policy using the learned world model. After a few iterations of this refinement, our planning agents are better than standard model-free reinforcement learning approaches demonstrating the viability of our approach.

ABSTRACT. Indirect encoding is a promising area of research in machine learning/evolutionary computation, however, it is rarely able to achieve performance on par with state of the art directly encoded methods. One of the most important properties of indirect encoding is the ability to control exploration during learning by transforming random genotypic variation into an arbitrary distribution of phenotypic variation. This gives indirect encoding a capacity to learn to be adaptable in a way which is not possible for direct encoding. However, during normal objective based learning, there is no direct selection for adaptability, which results in not only a missed opportunity to improve the ability to learn, but often degrading it too. The recent meta learning algorithm MAML makes it possible to directly and efficiently optimize for the ability to adapt. This paper demonstrates that even when indirect encoding can be detrimental to performance in the case of normal learning, when selecting for the ability to adapt, indirect encoding can outperform direct encoding in a fair comparison. The indirect encoding technique Hypernetwork was used on the task of few shot image classification on the Omniglot dataset. The results show the importance of directly optimizing for adaptability in realizing the powerful potential of indirect encoding.

ABSTRACT. Abstract. In this paper, we introduce a Model-based Algorithm Turning Engine, namely MATE, where the parameters of an algorithm are represented as expressions of the features of a target optimisation problem. In contrast to most static (feature-independent) algorithm tuning engines such as irace and SPOT, our approach aims to derive the best parameter configuration of a given algorithm for a specific problem, exploiting the relationships between the algorithm parameters and the features of the problem. We formulate the problem of finding the relationships between the parameters and the problem features as a symbolic regression problem and we use genetic programming to extract these expressions in a human-readable form. For the evaluation, we apply our approach to the configuration of the (1+1) EA and RLS algorithms for the OneMax, LeadingOnes, BinValue and Jump optimisation problems, where the theoretically optimal algorithm parameters to the problems are available as functions of the features of the problems. Our study shows that the found relationships typically comply with known theoretical results – this demonstrates (1) the potential of model-based parameter tuning as an alternative to existing static algorithm tuning engines, and (2) its potential to discover relationships between algorithm performance and problem features in human-readable form.

ABSTRACT. We study evolutionary algorithms in a dynamic setting, where for each generation a different fitness function is chosen, and selection is performed with respect to the current fitness function. Specifically, we consider Dynamic BinVal, in which the fitness functions for each generation is given by the linear function BinVal, but in each generation the order of bits is randomly permuted. For the $(1+1)$-EA it was known that there is an efficiency threshold $c_0$ for the mutation parameter, at which the runtime switches from quasilinear to exponential. Previous empirical evidence suggested that for larger population size $\mu$, the threshold may increase. We prove that this is at least the case in an $\eps$-neighborhood around the optimum: the threshold of the $(\mu+1)$-EA becomes arbitrarily large if the $\mu$ is chosen large enough.

However, the most surprising result is obtained by a second order analysis for $\mu=2$: the threshold \emph{in}creases with increasing proximity to the optimum. In particular, the hardest region for optimization is \emph{not} around the optimum.

ABSTRACT. Teachers often give their students a head start. In programming courses they demonstrate how to solve either a problem related to one coming up on the problem set, or a problem that is possible to re-purpose, by making minor variations. We explore how to give Genetic Programming (GP) a head start to synthesize a programming problem. Our method uses a related problem and introduces a schedule that directs it to solve the related problem first either fully or to some extent first, or at the same time. In addition, if the related problem’s solutions are written by students or evolved by GP, we explore the extent to which initializing the GP population with some of these solutions provides a head start. We find that having a population solve one programming problem before working to solve a related programming problem helps to a greater extent as the targeted problems and the intermediate problems themselves are selected to be more challenging.

ABSTRACT. GP convergence is viewed via information theory as software plasticity and functional resilience to source code changes. On average 100 evolved functions remove test case effectiveness and fail to propagate crossover infected errors. Considerable savings in bloated tree GP runs are given by exploiting population convergence with existing data structures in the absence of side effects leading to near linear O(gens) runtime. With multi-threading and SIMD AVX parallel computing a 16 core desktop can deliver the equivalent of 571 billion GPop/s.

ABSTRACT. Grammatical Evolution (GE) is one of the most popular Genetic Programming (GP) variants, and it has been used with success in several problem domains. Since the original proposal, many enhancements have been proposed to GE in order to address some of its main issues and improve its performance.

In this paper we propose Probabilistic Grammatical Evolution (PGE), which introduces a new genotypic representation and new mapping mechanism for GE. In concrete, we resort to a Probabilistic Context-Free Grammar (PCFG) where its probabilities are adapted during the evolutionary process, taking into account the productions chosen to construct the fittest individual. The genotype is a list of real values, where each value represents the likelihood of selecting a derivation rule. We evaluate the performance of PGE in two regression problems and compare it with GE and Structured Grammatical Evolution (SGE) .

The results show that PGE has a a better performance than GE, with statistically significant differences, and achieved similar performance when comparing with SGE, surpassing it in some scenarios.

ABSTRACT. Code smells (a.k.a. anti-patterns) are manifestations of poor design solutions that could deteriorate the software maintainability and evolution. Despite the high number of existing detection methods, the issue of class label uncertainty is usually omitted. Indeed, two human experts may have different degrees of uncertainty about the smelliness of a particular software class not only for the smell detection task but also for the smell type identification one. Thus, this uncertainty should be taken into account and then processed by detection tools. Unfortunately, these latter usually reject and/or ignore uncertain data that correspond to software classes (i.e. dataset instances) with uncertain labels. This practice could considerably degrade the detection/identification process effectiveness. Motivated by this observation and the interesting performance of the Possibilistic K-NN (PK-NN) classifier in dealing with uncertain data, we propose a new possibilistic evolutionary detection method, named ADIPOK (Anti-patterns Detection and Identification using Possibilistic Optimized K-NNs), that is able to deal with label uncertainty using some concepts stemming from the Possibility theory. ADIPOK is validated using a possibilistic base of smell examples that simulates the subjectivity of software engineers' opinions’ uncertainty. The statistical analysis of the obtained results on a set of comparative experiments with respect to four state-of-the-art methods show the merits of our proposed method.

ABSTRACT. We study the evolution of rules that define how to assign tasks to workstations in heuristic procedures for assembly line balancing. In assembly line balancing, a set of partially ordered tasks has to be assigned to workstations. The variant we consider, known as the assembly line worker assignment and balancing problem (ALWABP), has a fixed number of machines and workers, and different workers need different times to execute the tasks. A solution is an assignment of tasks and workers to workstations satisfying the partial order of the tasks, and the problem is to find a solution that maximizes the production rate of the assembly line. These problems are often solved by station-based assignment procedures, which use heuristic rules to select the tasks to be assigned to stations. There are many selection rules available in the literature. We show how efficient rules can be evolved, and demonstrate that rules evolved for simple assignment procedures are also effective in stochastic heuristic procedures using beam search, leading to improved heuristics.

ABSTRACT. Managing construction projects is a complex, resource-intense and risky task that involves the organization and management of people skilled in the design and completion of construction projects. Embarking on a construction project means to plan the allocation of resources and labour, while ensuring that the output (e.g. a new building) meets a certain quality, and is delivered in time and within budget without breaching contractual obligations. We formulate a simplified version of this task as a constrained multi-objective optimization problem, and then used a non-dominated sorting genetic algorithm to tackle the problem. In addition to providing a formal definition of the problem, further contributions of this work include the validation of the methodology using real data of construction projects varying in scale and resource-utilisation; the use of real data is scarce in the construction project management area. We also perform a scenario-based analysis to understand how the approach reacts to changing environmental parameters (such as availability of resources). Finally, we discuss practical implications. Our empirical analysis highlights that the proposed approach improves significantly in terms of project budget, quality, and duration targets, when compared with the industry standard.

ABSTRACT. Quality-diversity (QD) algorithms that return a large archive of elite solutions to a problem provide insights into how high-performing solutions are distributed throughout a feature-space defined by a user --- they are often described as illuminating the feature-space, providing a qualitative illustration of relationships between features and objective quality. However, if there are 1000s of solutions in an archive, extracting a succinct set of rules that capture these relationships in a quantitative manner (i.e. as a set of rules) is challenging. We propose two methods for the automated generation of rules from data contained in an archive; the first uses Genetic Programming and the second, a rule-induction method known as CN2. Rules are generated from large archives of data produced by running MAP-Elites on an urban logistics problem. A quantitative and qualitative evaluation that includes the end-user demonstrate that the rules are capable of fitting the data, but also highlights some mismatches between the model used by the optimiser and that assumed by the user.

ABSTRACT. Proteins are vital macro-molecules for every living organism. As the experimental determination of protein structures is costly and time-consuming, computational methods became an interesting way to predict proteins' shape based on their amino acid sequence. Through the years, metaheuristics have been employed to the protein structure prediction problem, with different characteristics and different knowledge sources. However, these methods are heavily dependent on parameter tuning, where wrong parameters might cause poor performance. In order to deal with the non-trivial task of parameter tuning, adaptive strategies were proposed, leaving the algorithm to choose its parameters for each optimization step. Although adaptive metaheuristics are widely applied to benchmark problems, only a few were tested in the PSP problem. In order to contribute to the analysis of adaptive metaheuristics in the PSP problem, we explore in this work the capability of one of the CEC'14 winners: the Success-History based Adaptive Differential Evolution algorithm on the tertiary protein structure prediction problem. We tested the SHADE algorithm in nine different proteins and compared the algorithm to the other four classical metaheuristics. Moreover, we enhanced the SHADE with domain knowledge from APL, which other works have not yet explored. Our results enlarge the body of research in adaptive methods for the PSP problem, showing that SHADE is very competitive compared to non-adaptive methods and related works.

ABSTRACT. This paper examines the optimisation of traffic signals to prioritise public transportation (busses) in real-time. A novel representation for the traffic signal prioritisation problem is introduced. Through the novel representa-tion a creative evolutionary process, while ensuring safe solutions that comply with real-world traffic signal constraints is possible. The proposed system finds near-optimal solutions in 20 seconds, enabling real-time op-timisation. The author examines a specific junction in Hamburg, Germany. Based on real-world traffic data a variety of different problem scenarios ranging from low to exceptional traffic saturations are generated. In collabo-ration with domain experts a fitness function is defined to reduce the jour-ney time of a bus while maintaining an overall stable traffic system. Candi-date solutions are evaluated using the microscopic traffic simulator SUMO allowing for precise optimisation and addressing of the flow prediction problem. The results show good scaling of the proposed system, with more significant improvements in more congested scenarios. Given the results, future research on bigger and multiple road junctions is motivated. This work contributes to the field in four ways. Firstly, by defining a re-al-world problem containing the actual intersection layout and traffic signal parameters. Secondly by presenting a software design that integrates highly efficient SUMO simulation into an evolutionary algorithm. Thirdly, by in-troducing a novel representation that allows creative, unconventional solu-tions while ensuring compliance with traffic signal regulations at all times. Lastly, by testing the suggested approach on various problem scenarios of the real-world problem.

ABSTRACT. Motivated by a real-world application, we consider an Assembly Job Shop Scheduling Problem (AJSSP), with three objectives: product quality, product quantity, and first product lead time. Using real-world inspection data, we demonstrate the ability to model product quality transformations during assembly jobs via genetic programming by considering the quality attributes of subparts. We investigate integrating quality transformation models into an AJSSP. Through the use of the de facto standard multi-objective evolutionary algorithm, NSGA-II, and a novel genotype to handle the constraints, we describe an evolutionary approach to optimizing all stated objectives. This approach is empirically shown to outperform random search and hill climbing in both performance and usability metrics expected to be valuable to administrators involved in plant scheduling and operations.

ABSTRACT. Real-valued genotypes together with the variation operators, mutation and crossover, constitute some of the fundamental building blocks of Evolutionary Algorithms. Real-valued genotypes are utilized in a broad range of contexts, from weights in Artificial Neural Networks to parameters in robot control systems. Shared between most uses of real-valued genomes is the need for limiting the range of individual parameters to allowable bounds. In this paper we will illustrate the challenge of limiting the parameters of real-valued genomes and analyze the most promising method to properly limit these values. We utilize both empirical as well as benchmark examples to demonstrate the utility of the proposed method and through a literature review show how the insight of this paper could impact other research within the field. The proposed method requires minimal intervention from Evolutionary Algorithm practitioners and behaves well under repeated application of variation operators, leading to better theoretical properties as well as significant differences in well-known benchmarks.

ABSTRACT. Research on automatic music generation lacks consideration of the originality of musical outputs, creating risks of plagiarism and/or copyright infringement. We present the originality report - a set of analyses for measuring the extent to which an algorithm copies from the input music on which it is trained. First, a baseline is constructed, determining the extent to which human composers borrow from themselves and each other in some existing music corpus. Second, we apply a similar analysis to musical outputs of runs of MAIA Markov and Music Transformer generation algorithms, and compare the results to the baseline. Third, we investigate how originality varies as a function of Transformer's training epoch. Results from the second analysis indicate that the originality of Transfomer's output is below the 95%-confidence interval of the baseline. Musicological interpretation of the analyses shows that the Transformer model obtained via the conventional stopping criteria produces single-note repetition patterns, resulting in outputs of low quality and originality, while in later training epochs, the model tends to overfit, producing copies of excerpts of input pieces. We recommend the originality report as a new means of evaluating algorithm training processes and outputs in future, and question the reported success of language-based deep learning models for music generation. Supporting materials (code, dataset) will be made available via https://osf.io/96emr/.

ABSTRACT. In contrast to most recent models that generate an entire image at once, the paper introduces a new architecture for generating images one pixel at a time using a Compositional Pattern-Producing Network (CPPN) as the generator part in a Generative Adversarial Network (GAN), allowing for effective generation of visually interesting images with artistic value, at arbitrary resolutions independent of the dimensions of the training data. The architecture, as well as accompanying (hyper-)parameters, for training CPPNs using recent GAN stabilisation techniques is shown to generalise well across many standard datasets. Rather than relying on just a latent noise vector (entangling various features with each other), mutual information maximisation is utilised to get disentangled representations, removing the requirement to use labelled data and giving the user control over the generated images. A web application for interacting with pre-trained models was also created, unique in the offered level of interactivity with an image-generating GAN.

ABSTRACT. This paper reports the implementation of machine learning techniques in the identification of pure painting pigments applying spectral data obtained from both the paint tubes used and the paintings produced by Portuguese artist Amadeo de Souza Cardoso. It illustrates the rationales and advantages behind the application of more accurate artificial mixing by subtractive mixing on the reference pigments as well as the use of Root Mean Square Error (RMSE) for distinguishing especially the mixtures that contain white and black, so that a more holistic machine learning approach can be applied; notably, the experiment of neural network for discerning black and white pigments, which later could be applied for both pure and mixed pigment identification. Other machine learning techniques like Decision Tree and Support Vector Machine are also exploited and compared in terms of the identification of pure pigments. In addition, this paper proposes the solution to the common problem of highly-imbalanced and limited data in the analysis of historical artwork field.

ABSTRACT. This work presents a novel genetic programming system for multi-class classification, called progressively insular cooperative genetic programming (PIC GP). Based on the idea that effective multiclass classification can be obtained by appropriately joining classifiers that are highly specialized on the single classes, PIC GP evolves, at the same time, two populations. The first population contains individuals called specialists, and each specialist is optimized on one specific target class. The second population contains higher-level individuals, called teams, that join specialists to obtain the final algorithm prediction. By means of three simple parameters, PIC GP can tune the amount of cooperation between specialists of different classes. The first part of the paper is dedicated to a study of the influence of these parameters on the evolution dynamics. The obtained results indicate that PIC GP achieves the best performance when the evolution begins with a high level of cooperation between specialists of different classes, and then this type of cooperation is progressively decreased, until only specialists of the same class can cooperate between each other. The last part of the work is dedicated to an experimental comparison between PIC GP and a set of state-of-the-art classification algorithms. The presented results indicate that PIC GP outperforms the majority of its competitors on the studied test problems.

ABSTRACT. The promotion and maintenance of the population diversity in a Genetic Programming (GP) algorithm was proved to be an important part of the evolutionary process. Such diversity maintenance improves the exploration capabilities of the GP algorithm, which as a consequence improves the quality of the found solutions by avoiding local optima. This paper aims to further investigate and prove the efficacy of a GP heuristic proposed in a previous work: the Inclusive Genetic Programming (IGP). Such heuristic can be classified as a niching technique, which performs the evolutionary operations like crossover, mutation and selection by considering the individuals belonging to different niches in order to maintain and exploit a certain degree of diversity in the population, instead of evolving the niches separately to find different local optima. A comparison between a standard formulation of GP and the IGP is carried out on nine different benchmarks coming from synthetic and real world data. The obtained results highlight how the greater diversity in the population, measured in terms of entropy, leads to better results on both training and test data, showing that an improvement on the generalization capabilities is also achieved.

ABSTRACT. Quality Diversity (QD) algorithms are a class of population-based evolutionary algorithms designed to generate sets of solutions that are both fit and diverse. In this paper, we describe a strategy for applying QD concepts to the generation of decision tree ensembles by optimizing collections of trees for both individually accurate and collectively diverse predictive behavior. We compare three variants of this QD strategy with two existing ensemble generation strategies over several classificationcation data sets. We then briefly highlight the effectsect of the evolutionary algorithm at the core of the strategy. The examined algorithms generate ensembles with distinct predictive behaviors as measured by classification accuracy and intrinsic diversity. The plotted behaviors hint at highly data-dependent relationships between these metrics. QD-based strategies are suggested as a means to optimize classifier ensembles along this performance curve along with other suggestions for future work.

ABSTRACT. The traditional pharmaceutical industry is being challenged by our growing ability to create personalised medicines targeting specific sub-populations and individuals. With a higher level of personalisation, an already critical supply chain is facing additional challenges by the very sensitive nature of products. Nevertheless, studies concerned with the efficient development and delivery of these products are scarce. Thus, this paper presents the case of personalised medicine and the challenges imposed by its mass delivery. We propose a multi-objective mathematical model for the location-allocation problem with two interdependent facility types in the case of personalised medicine products. We show its practical applications through a cell and gene therapy case study. A multi-objective genetic with a novel population initialisation procedure is used as solution method.

ABSTRACT. For several medical treatments, it is possible to observe transcriptional variations in gene expressions between responders and non-responders. Modelling the correlation between such variations and the patent's response to drugs as a system of Ordinary Differential Equations could be invaluable to improve the efficacy of treatments, and would represent an important step towards personalized medicine. Two main obstacles lie on this path: (i) the number of genes is too large to straightforwardly analyze their interactions; (ii) defining the correct parameters for the mathematical models of gene interaction is a complex optimization problem, even when a limited number of genes is involved. In this paper, we propose a novel approach to creating mathematical models able to explain patients' response to treatment from transcriptional variations. The approach is based on (i) a feature selection algorithm, set to identify a minimal set of gene expressions that are highly correlated with treatment outcome; (ii) a state-of-the-art evolutionary optimizer, Covariance Matrix Adaptation Evolution Strategy, applied to finding the parameters of the mathematical model characterizing the relationship between gene expressions and patient responsiveness. The proposed methodology is tested on real-world data describing responsiveness of asthma patients to Omalizumab, a humanized monoclonal antibody that binds to immunoglobulin E. In this case study, the presented approach is shown able to identify 5 genes (out of 28,402) that are transcriptionally relevant to predict treatment outcomes, and to deliver a compact mathematical model that is able to explain the interaction between the different genes involved.

ABSTRACT. Depression affects an estimated 300 million people around the globe. Early detection of depression and associated mental health problems constitutes one of the best prevention methods when trying to reduce the disease's incidence. Information collected by tracking smartphone use behaviour and using ecological momentary assesments (EMA) can be used together with machine learning techniques to identify patterns indicative of depression and predict its appearance, contributing in this way to its early detection. However many of these techniques fail to identify the importance and relationships between the factors used to reach their prediction outcome. In this paper we propose the use of Bayesian networks (BN) as a tool to analyse and model data collected using EMA and smartphone measured behaviours. We compare the performance of BN against results obtained using support vector regression and random forest. The comparison is done in terms of efficacy, efficiency and insight. Results show that no significant difference in performance was found between the models. However, BN presented clear advantages in terms of efficacy and insight given its probability factorization, graphical representation and ability to infer under uncertainty.