Understand the fundamentals and develop your own AI solutions in this updated edition packed with many new examples Key
967 193 11MB
English Pages 578 [579] Year 2020
Table of contents :
Cover
Copyright
Packt Page
Contributors
Table of Contents
Preface
Chapter 1: Getting Started with NextGeneration Artificial Intelligence through Reinforcement Learning
Reinforcement learning concepts
How to adapt to machine thinking and become an adaptive thinker
Overcoming reallife issues using the threestep approach
Step 1 – describing a problem to solve: MDP in natural language
Watching the MDP agent at work
Step 2 – building a mathematical model: the mathematical representation of the Bellman equation and MDP
From MDP to the Bellman equation
Step 3 – writing source code: implementing the solution in Python
The lessons of reinforcement learning
How to use the outputs
Possible use cases
Machine learning versus traditional applications
Summary
Questions
Further reading
Chapter 2: Building a Reward Matrix – Designing Your Datasets
Designing datasets – where the dream stops and the hard work begins
Designing datasets
Using the McCullochPitts neuron
The McCullochPitts neuron
The PythonTensorFlow architecture
Logistic activation functions and classifiers
Overall architecture
Logistic classifier
Logistic function
Softmax
Summary
Questions
Further reading
Chapter 3: Machine Intelligence – Evaluation Functions and Numerical Convergence
Tracking down what to measure and deciding how to measure it
Convergence
Implicit convergence
Numerically controlled gradient descent convergence
Evaluating beyond human analytic capacity
Using supervised learning to evaluate a result that surpasses human analytic capacity
Summary
Questions
Further reading
Chapter 4: Optimizing Your Solutions with KMeans Clustering
Dataset optimization and control
Designing a dataset and choosing an ML/DL model
Approval of the design matrix
Implementing a kmeans clustering solution
The vision
The data
The strategy
The kmeans clustering program
The mathematical definition of kmeans clustering
The Python program
Saving and loading the model
Analyzing the results
Bot virtual clusters as a solution
The limits of the implementation of the kmeans clustering algorithm
Summary
Questions
Further reading
Chapter 5: How to Use Decision Trees to Enhance KMeans Clustering
Unsupervised learning with KMC with large datasets
Identifying the difficulty of the problem
NPhard – the meaning of P
NPhard – the meaning of nondeterministic
Implementing random sampling with minibatches
Using the LLN
The CLT
Using a Monte Carlo estimator
Trying to train the full training dataset
Training a random sample of the training dataset
Shuffling as another way to perform random sampling
Chaining supervised learning to verify unsupervised learning
Preprocessing raw data
A pipeline of scripts and ML algorithms
Step 1 – training and exporting data from an unsupervised ML algorithm
Step 2 – training a decision tree
Step 3 – a continuous cycle of KMC chained to a decision tree
Random forests as an alternative to decision trees
Summary
Questions
Further reading
Chapter 6: Innovating AI with Google Translate
Understanding innovation and disruption in AI
Is AI disruptive?
AI is based on mathematical theories that are not new
Neural networks are not new
Looking at disruption – the factors that are making AI disruptive
Cloud server power, data volumes, and web sharing of the early 21st century
Public awareness
Inventions versus innovations
Revolutionary versus disruptive solutions
Where to start?
Discover a world of opportunities with Google Translate
Getting started
The program
The header
Implementing Google's translation service
Google Translate from a linguist's perspective
Playing with the tool
Linguistic assessment of Google Translate
AI as a new frontier
Lexical field and polysemy
Exploring the frontier – customizing Google Translate with a Python program
knearest neighbor algorithm
Implementing the KNN algorithm
The knn_polysemy.py program
Implementing the KNN function in Google_Translate_Customized.py
Conclusions on the Google Translate customized experiment
The disruptive revolutionary loop
Summary
Questions
Further reading
Chapter 7: Optimizing Blockchains with Naive Bayes
Part I – the background to blockchain technology
Mining bitcoins
Using cryptocurrency
PART II – using blockchains to share information in a supply chain
Using blockchains in the supply chain network
Creating a block
Exploring the blocks
Part III – optimizing a supply chain with naive Bayes in a blockchain process
A naive Bayes example
The blockchain anticipation novelty
The goal – optimizing storage levels using blockchain data
Implementation of naive Bayes in Python
Gaussian naive Bayes
Summary
Questions
Further reading
Chapter 8: Solving the XOR Problem with a Feedforward Neural Network
The original perceptron could not solve the XOR function
XOR and linearly separable models
Linearly separable models
The XOR limit of a linear model, such as the original perceptron
Building an FNN from scratch
Step 1 – defining an FNN
Step 2 – an example of how two children can solve the XOR problem every day
Implementing a vintage XOR solution in Python with an FNN and backpropagation
A simplified version of a cost function and gradient descent
Linear separability was achieved
Applying the FNN XOR function to optimizing subsets of data
Summary
Questions
Further reading
Chapter 9: Abstract Image Classification with Convolutional Neural Networks (CNNs)
Introducing CNNs
Defining a CNN
Initializing the CNN
Adding a 2D convolution layer
Kernel
Shape
ReLU
Pooling
Next convolution and pooling layer
Flattening
Dense layers
Dense activation functions
Training a CNN model
The goal
Compiling the model
The loss function
The Adam optimizer
Metrics
The training dataset
Data augmentation
Loading the data
The testing dataset
Data augmentation on the testing dataset
Loading the data
Training with the classifier
Saving the model
Next steps
Summary
Questions
Further reading and references
Chapter 10: Conceptual Representation Learning
Generating profit with transfer learning
The motivation behind transfer learning
Inductive thinking
Inductive abstraction
The problem AI needs to solve
The ￼ gap concept
Loading the trained TensorFlow 2.x model
Loading and displaying the model
Loading the model to use it
Defining a strategy
Making the model profitable by using it for another problem
Domain learning
How to use the programs
The trained models used in this section
The trained model program
Gap – loaded or underloaded
Gap – jammed or open lanes
Gap datasets and subsets
Generalizing the ￼ (the gap conceptual dataset)
The motivation of conceptual representation learning metamodels applied to dimensionality
The curse of dimensionality
The blessing of dimensionality
Summary
Questions
Further reading
Chapter 11: Combining Reinforcement Learning and Deep Learning
Planning and scheduling today and tomorrow
A realtime manufacturing process
Amazon must expand its services to face competition
A realtime manufacturing revolution
CRLMM applied to an automated apparel manufacturing process
An apparel manufacturing process
Training the CRLMM
Generalizing the unit training dataset
Food conveyor belt processing – positive p￼ and negative n￼ gaps
Running a prediction program
Building the RLDLCRLMM
A circular process
Implementing a CNNCRLMM to detect gaps and optimize
Qlearning – MDP
MDP inputs and outputs
The optimizer
The optimizer as a regulator
Finding the main target for the MDP function
A circular model – a streamlike system that never starts nor ends
Summary
Questions
Further reading
Chapter 12: AI and the Internet of Things (IoT)
The public service project
Setting up the RLDLCRLMM model
Applying the model of the CRLMM
The dataset
Using the trained model
Adding an SVM function
Motivation – using an SVM to increase safety levels
Definition of a support vector machine
Python function
Running the CRLMM
Finding a parking space
Deciding how to get to the parking lot
Support vector machine
The itinerary graph
The weight vector
Summary
Questions
Further reading
Chapter 13: Visualizing Networks with TensorFlow 2.x and TensorBoard
Exploring the output of the layers of a CNN in two steps with TensorFlow
Building the layers of a CNN
Processing the visual output of the layers of a CNN
Analyzing the visual output of the layers of a CNN
Analyzing the accuracy of a CNN using TensorBoard
Getting started with Google Colaboratory
Defining and training the model
Introducing some of the measurements
Summary
Questions
Further reading
Chapter 14: Preparing the Input of Chatbots with Restricted Boltzmann Machines (RBMs) and Principal Component Analysis (PCA)
Defining basic terms and goals
Introducing and building an RBM
The architecture of an RBM
An energybased model
Building the RBM in Python
Creating a class and the structure of the RBM
Creating a training function in the RBM class
Computing the hidden units in the training function
Random sampling of the hidden units for the reconstruction and contractive divergence
Reconstruction
Contrastive divergence
Error and energy function
Running the epochs and analyzing the results
Using the weights of an RBM as feature vectors for PCA
Understanding PCA
Mathematical explanation
Using TensorFlow's Embedding Projector to represent PCA
Analyzing the PCA to obtain input entry points for a chatbot
Summary
Questions
Further reading
Chapter 15: Setting Up a Cognitive NLP UI/CUI Chatbot
Basic concepts
Defining NLU
Why do we call chatbots "agents"?
Creating an agent to understand Dialogflow
Entities
Intents
Context
Adding fulfillment functionality to an agent
Defining fulfillment
Enhancing the cogfilmdr agent with a fulfillment webhook
Getting the bot to work on your website
Machine learning agents
Using machine learning in a chatbot
Speechtotext
Texttospeech
Spelling
Why are these machine learning algorithms important?
Summary
Questions
Further reading
Chapter 16: Improve the Emotional Intelligence Deficiencies of Chatbots
From reacting to emotions, to creating emotions
Solving the problems of emotional polysemy
The greetings problem example
The affirmation example
The speech recognition fallacy
The facial analysis fallacy
Small talk
Courtesy
Emotions
Data logging
Creating emotions
RNN research for future automatic dialog generation
RNNs at work
RNN, LSTM, and vanishing gradients
Text generation with an RNN
Vectorizing the text
Building the model
Generating text
Summary
Questions
Further reading
Chapter 17: Genetic Algorithms in Hybrid Neural Networks
Understanding evolutionary algorithms
Heredity in humans
Our cells
How heredity works
Evolutionary algorithms
Going from a biological model to an algorithm
Basic concepts
Building a genetic algorithm in Python
Importing the libraries
Calling the algorithm
The main function
The parent generation process
Generating a parent
Fitness
Display parent
Crossover and mutation
Producing generations of children
Summary code
Unspecified target to optimize the architecture of a neural network with a genetic algorithm
A physical neural network
What is the nature of this mysterious SFNN?
Calling the algorithm cell
Fitness cell
ga_main() cell
Artificial hybrid neural networks
Building the LSTM
The goal of the model
Summary
Questions
Further reading
Chapter 18: Neuromorphic Computing
Neuromorphic computing
Getting started with Nengo
Installing Nengo and Nengo GUI
Creating a Python program
A Nengo ensemble
Nengo neuron types
Nengo neuron dimensions
A Nengo node
Connecting Nengo objects
Visualizing data
Probes
Applying Nengo's unique approach to critical AI research areas
Summary
Questions
References
Further reading
Chapter 19: Quantum Computing
The rising power of quantum computers
Quantum computer speed
Defining a qubit
Representing a qubit
The position of a qubit
Radians, degrees, and rotations
The Bloch sphere
Composing a quantum score
Quantum gates with Quirk
A quantum computer score with Quirk
A quantum computer score with IBM Q
A thinking quantum computer
Representing our mind's concepts
Expanding MindX's conceptual representations
The MindX experiment
Preparing the data
Transformation functions – the situation function
Transformation functions – the quantum function
Creating and running the score
Using the output
Summary
Questions
Further reading
Appendix: Answers to the Questions
Chapter 1 – Getting Started with NextGeneration Artificial Intelligence through Reinforcement Learning
Chapter 2 – Building a Reward Matrix – Designing Your Datasets
Chapter 3 – Machine Intelligence – Evaluation Functions and Numerical Convergence
Chapter 4 – Optimizing Your Solutions with KMeans Clustering
Chapter 5 – How to Use Decision Trees to Enhance KMeans Clustering
Chapter 6 – Innovating AI with Google Translate
Chapter 7 – Optimizing Blockchains with Naive Bayes
Chapter 8 – Solving the XOR Problem with a Feedforward Neural Network
Chapter 9 – Abstract Image Classification with Convolutional Neural Networks (CNNs)
Chapter 10 – Conceptual Representation Learning
Chapter 11 – Combining Reinforcement Learning and Deep Learning
Chapter 12 – AI and the Internet of Things
Chapter 13 – Visualizing Networks with TensorFlow 2.x and TensorBoard
Chapter 14 – Preparing the Input of Chatbots with Restricted Boltzmann Machines (RBMs) and Principal Component Analysis (PCA)
Chapter 15 – Setting Up a Cognitive NLP UI/CUI Chatbot
Chapter 16 – Improve the Emotional Intelligence Deficiencies of Chatbots
Chapter 17 – Genetic Algorithms in Hybrid Neural Networks
Chapter 18 – Neuromorphic Computing
Chapter 19 – Quantum Computing
Other Books You
May Enjoy
Index
Artificial Intelligence By Example Second Edition
Acquire advanced AI, machine learning, and deep learning design skills
Denis Rothman
BIRMINGHAM  MUMBAI
Artificial Intelligence By Example Second Edition
Copyright © 2020 Packt Publishing All rights reserved. No part of this book may be reproduced, stored in a retrieval system, or transmitted in any form or by any means, without the prior written permission of the publisher, except in the case of brief quotations embedded in critical articles or reviews. Every effort has been made in the preparation of this book to ensure the accuracy of the information presented. However, the information contained in this book is sold without warranty, either express or implied. Neither the author, nor Packt Publishing or its dealers and distributors, will be held liable for any damages caused or alleged to have been caused directly or indirectly by this book. Packt Publishing has endeavored to provide trademark information about all of the companies and products mentioned in this book by the appropriate use of capitals. However, Packt Publishing cannot guarantee the accuracy of this information. Producer: Tushar Gupta Acquisition Editor – Peer Reviews: Divya Mudaliar Content Development Editor: Dr. Ian Hough Technical Editor: Saby D'silva Project Editor: Kishor Rit Proofreader: Safis Editing Indexer: Tejal Daruwale Soni Presentation Designer: Pranit Padwal First published: May 2018 Second edition: February 2020 Production reference: 1270220 Published by Packt Publishing Ltd. Livery Place 35 Livery Street Birmingham B3 2PB, UK. ISBN 9781839211539 www.packt.com
packt.com
Subscribe to our online digital library for full access to over 7,000 books and videos, as well as industry leading tools to help you plan your personal development and advance your career. For more information, please visit our website.
Why subscribe?
• Spend less time learning and more time coding with practical eBooks and Videos from over 4,000 industry professionals • Learn better with Skill Plans built especially for you • Get a free eBook or video every month • Fully searchable for easy access to vital information • Copy and paste, print, and bookmark content
Did you know that Packt offers eBook versions of every book published, with PDF and ePub files available? You can upgrade to the eBook version at www.Packt.com and as a print book customer, you are entitled to a discount on the eBook copy. Get in touch with us at [email protected] for more details. At www.Packt.com, you can also read a collection of free technical articles, sign up for a range of free newsletters, and receive exclusive discounts and offers on Packt books and eBooks.
Contributors About the author
Denis Rothman graduated from Sorbonne University and ParisDiderot
University, writing one of the very first word2matrix embedding solutions. He began his career authoring one of the first AI cognitive natural language processing (NLP) chatbots applied as a language teacher for Moët et Chandon and other companies. He authored an AI resource optimizer for IBM and apparel producers. He then authored an advanced planning and scheduling (APS) solution used worldwide. "I want to thank the corporations who trusted me from the start to deliver artificial intelligence solutions and share the risks of continuous innovation. I also thank my family, who believed I would make it big at all times."
About the reviewers
Carlos Toxtli is a humancomputer interaction researcher who studies the impact of artificial intelligence in the future of work. He studied a Ph.D. in Computer Science at the University of West Virginia and a master's degree in Technological Innovation and Entrepreneurship at the Monterrey Institute of Technology and Higher Education. He has worked for some international organizations such as Google, Microsoft, Amazon, and the United Nations. He has also created companies that use artificial intelligence in the financial, educational, customer service, and parking industries. Carlos has published numerous research papers, manuscripts, and book chapters for different conferences and journals in his field. "I want to thank all the editors who helped make this book a masterpiece."
Kausthub Raj Jadhav graduated from the University of California, Irvine, where he specialized in intelligent systems and founded the Artificial Intelligence Club. In his spare time, he enjoys powerlifting, rewatching Parks and Recreation, and learning how to cook. He solves hard problems for a living.
Table of Contents Prefacexiii Chapter 1: Getting Started with NextGeneration Artificial Intelligence through Reinforcement Learning 1 Reinforcement learning concepts How to adapt to machine thinking and become an adaptive thinker Overcoming reallife issues using the threestep approach Step 1 – describing a problem to solve: MDP in natural language Watching the MDP agent at work
Step 2 – building a mathematical model: the mathematical representation of the Bellman equation and MDP From MDP to the Bellman equation
2 4 5 7
8
10
10
Step 3 – writing source code: implementing the solution in Python The lessons of reinforcement learning How to use the outputs
14 16 18
Machine learning versus traditional applications Summary Questions Further reading
23 24 24 25
Possible use cases
Chapter 2: Building a Reward Matrix – Designing Your Datasets
Designing datasets – where the dream stops and the hard work begins Designing datasets Using the McCullochPitts neuron The McCullochPitts neuron The PythonTensorFlow architecture
[i]
20
27 28 29 29 31 35
Table of Contents
Logistic activation functions and classifiers Overall architecture Logistic classifier Logistic function Softmax Summary Questions Further reading
Chapter 3: Machine Intelligence – Evaluation Functions and Numerical Convergence
Tracking down what to measure and deciding how to measure it Convergence Implicit convergence Numerically controlled gradient descent convergence
Evaluating beyond human analytic capacity Using supervised learning to evaluate a result that surpasses human analytic capacity Summary Questions Further reading
Chapter 4: Optimizing Your Solutions with KMeans Clustering Dataset optimization and control Designing a dataset and choosing an ML/DL model Approval of the design matrix
Implementing a kmeans clustering solution The vision The data The strategy
35 35 36 37 38 42 43 43
45 46 48
49 49
56 60 64 65 65
67 68 69
70
74 74
75 76
The kmeans clustering program
77
Saving and loading the model Analyzing the results
84 85
The mathematical definition of kmeans clustering The Python program
Bot virtual clusters as a solution The limits of the implementation of the kmeans clustering algorithm
Summary Questions Further reading
Chapter 5: How to Use Decision Trees to Enhance KMeans Clustering
Unsupervised learning with KMC with large datasets [ ii ]
78 80
86 87
88 88 89
91 92
Table of Contents
Identifying the difficulty of the problem
94
Implementing random sampling with minibatches Using the LLN The CLT
95 96 96
NPhard – the meaning of P NPhard – the meaning of nondeterministic
Using a Monte Carlo estimator
94 95
97
Trying to train the full training dataset Training a random sample of the training dataset Shuffling as another way to perform random sampling Chaining supervised learning to verify unsupervised learning
98 98 100 102
A pipeline of scripts and ML algorithms
103
Preprocessing raw data
Step 1 – training and exporting data from an unsupervised ML algorithm Step 2 – training a decision tree Step 3 – a continuous cycle of KMC chained to a decision tree
Random forests as an alternative to decision trees Summary Questions Further reading
103 105 106 110
114 118 118 119
Chapter 6: Innovating AI with Google Translate
121
AI is based on mathematical theories that are not new Neural networks are not new
124 124
Understanding innovation and disruption in AI Is AI disruptive?
Looking at disruption – the factors that are making AI disruptive
Cloud server power, data volumes, and web sharing of the early 21st century Public awareness
Inventions versus innovations Revolutionary versus disruptive solutions Where to start? Discover a world of opportunities with Google Translate Getting started The program The header Implementing Google's translation service
Google Translate from a linguist's perspective Playing with the tool Linguistic assessment of Google Translate
AI as a new frontier Lexical field and polysemy Exploring the frontier – customizing Google Translate with a Python program [ iii ]
123 123
125
125 126
126 127 127 128 128 128
128 129
130
131 131
135 135 137
Table of Contents
knearest neighbor algorithm
Implementing the KNN algorithm The knn_polysemy.py program Implementing the KNN function in Google_Translate_Customized.py Conclusions on the Google Translate customized experiment The disruptive revolutionary loop
Summary Questions Further reading
Chapter 7: Optimizing Blockchains with Naive Bayes
Part I – the background to blockchain technology Mining bitcoins Using cryptocurrency PART II – using blockchains to share information in a supply chain Using blockchains in the supply chain network Creating a block Exploring the blocks Part III – optimizing a supply chain with naive Bayes in a blockchain process A naive Bayes example The blockchain anticipation novelty The goal – optimizing storage levels using blockchain data
Implementation of naive Bayes in Python Gaussian naive Bayes
Summary Questions Further reading
Chapter 8: Solving the XOR Problem with a Feedforward Neural Network The original perceptron could not solve the XOR function XOR and linearly separable models Linearly separable models The XOR limit of a linear model, such as the original perceptron
Building an FNN from scratch Step 1 – defining an FNN Step 2 – an example of how two children can solve the XOR problem every day Implementing a vintage XOR solution in Python with an FNN and backpropagation A simplified version of a cost function and gradient descent Linear separability was achieved
[ iv ]
138
139 142 144 152 153
153 154 154
157 158 159 160 161 164 165 166 167 167
169 170
173
173
177 177 178
179 180 181
181 182
184 184 185 189
191 194
Table of Contents
Applying the FNN XOR function to optimizing subsets of data Summary Questions Further reading
Chapter 9: Abstract Image Classification with Convolutional Neural Networks (CNNs) Introducing CNNs Defining a CNN Initializing the CNN Adding a 2D convolution layer Kernel Shape ReLU
196 202 203 203
205 206 207 209 210
210 215 215
Pooling Next convolution and pooling layer Flattening Dense layers
218 219 220 220
Dense activation functions
221
Training a CNN model The goal Compiling the model
221 222 223
The training dataset
226
The testing dataset
228
Training with the classifier Saving the model
229 230
The loss function The Adam optimizer Metrics
223 225 226
Data augmentation Loading the data
227 227
Data augmentation on the testing dataset Loading the data
Next steps
228 228
230
Summary Questions Further reading and references
Chapter 10: Conceptual Representation Learning Generating profit with transfer learning The motivation behind transfer learning Inductive thinking Inductive abstraction The problem AI needs to solve
[v]
231 231 231
233 234 235
235 235 236
Table of Contents
The 𝚪𝚪 gap concept Loading the trained TensorFlow 2.x model
Loading and displaying the model Loading the model to use it Defining a strategy Making the model profitable by using it for another problem
Domain learning How to use the programs
The trained models used in this section The trained model program
237 238
238 242 245 246
247 247
248 248
Gap – loaded or underloaded Gap – jammed or open lanes Gap datasets and subsets
249 251 253
The motivation of conceptual representation learning metamodels applied to dimensionality
254
Generalizing the 𝚪𝚪 (the gap conceptual dataset) The curse of dimensionality The blessing of dimensionality
Summary Questions Further reading
Chapter 11: Combining Reinforcement Learning and Deep Learning Planning and scheduling today and tomorrow A realtime manufacturing process Amazon must expand its services to face competition A realtime manufacturing revolution
253
254 255
256 257 257
259 260 262
262 263
CRLMM applied to an automated apparel manufacturing process An apparel manufacturing process Training the CRLMM
266 267 269
Building the RLDLCRLMM A circular process Implementing a CNNCRLMM to detect gaps and optimize Qlearning – MDP
274 275 276 277
Generalizing the unit training dataset Food conveyor belt processing – positive p𝜸𝜸 and negative n𝜸𝜸 gaps Running a prediction program
MDP inputs and outputs
269 270 274
278
The optimizer
281
A circular model – a streamlike system that never starts nor ends
286
The optimizer as a regulator Finding the main target for the MDP function
[ vi ]
281 284
Table of Contents
Summary Questions Further reading
Chapter 12: AI and the Internet of Things (IoT) The public service project Setting up the RLDLCRLMM model Applying the model of the CRLMM The dataset Using the trained model
291 291 292
293 294 295 297
298 300
Adding an SVM function Motivation – using an SVM to increase safety levels Definition of a support vector machine Python function Running the CRLMM Finding a parking space Deciding how to get to the parking lot
301 302 303 305 307 307 310
Summary Questions Further reading
315 316 316
Support vector machine The itinerary graph The weight vector
Chapter 13: Visualizing Networks with TensorFlow 2.x and TensorBoard Exploring the output of the layers of a CNN in two steps with TensorFlow Building the layers of a CNN Processing the visual output of the layers of a CNN Analyzing the visual output of the layers of a CNN
Analyzing the accuracy of a CNN using TensorBoard Getting started with Google Colaboratory Defining and training the model Introducing some of the measurements Summary Questions Further reading
Chapter 14: Preparing the Input of Chatbots with Restricted Boltzmann Machines (RBMs) and Principal Component Analysis (PCA) Defining basic terms and goals
[ vii ]
311 313 314
317 318 319 323
327
334 334 336 339 341 342 342
343 344
Table of Contents
Introducing and building an RBM The architecture of an RBM An energybased model Building the RBM in Python
Creating a class and the structure of the RBM Creating a training function in the RBM class Computing the hidden units in the training function Random sampling of the hidden units for the reconstruction and contractive divergence Reconstruction Contrastive divergence Error and energy function
345 346 347 350
350 350 351 352 353 354 354
Running the epochs and analyzing the results Using the weights of an RBM as feature vectors for PCA Understanding PCA
355 357 362
Using TensorFlow's Embedding Projector to represent PCA Analyzing the PCA to obtain input entry points for a chatbot Summary Questions Further reading
367 370 372 373 373
Mathematical explanation
Chapter 15: Setting Up a Cognitive NLP UI/CUI Chatbot Basic concepts Defining NLU Why do we call chatbots "agents"? Creating an agent to understand Dialogflow Entities Intents Context Adding fulfillment functionality to an agent Defining fulfillment Enhancing the cogfilmdr agent with a fulfillment webhook Getting the bot to work on your website Machine learning agents Using machine learning in a chatbot Speechtotext Texttospeech Spelling Why are these machine learning algorithms important? Summary Questions Further reading [ viii ]
363
375 376 376 376 377 378 382 387 392 393 394 397 398 398 398 399 401 403 404 405 405
Table of Contents
Chapter 16: Improving the Emotional Intelligence Deficiencies of Chatbots From reacting to emotions, to creating emotions Solving the problems of emotional polysemy The greetings problem example The affirmation example The speech recognition fallacy The facial analysis fallacy
Small talk
407 408 408
409 410 410 411
412
Courtesy Emotions
412 415
Data logging Creating emotions RNN research for future automatic dialog generation RNNs at work
415 418 423 424
Text generation with an RNN Vectorizing the text Building the model Generating text Summary Questions Further reading
426 426 427 429 431 432 432
RNN, LSTM, and vanishing gradients
Chapter 17: Genetic Algorithms in Hybrid Neural Networks Understanding evolutionary algorithms Heredity in humans Our cells How heredity works
425
433 434 434
435 435
Evolutionary algorithms
436
Building a genetic algorithm in Python
440
Unspecified target to optimize the architecture of a neural network with a genetic algorithm
451
Going from a biological model to an algorithm Basic concepts Importing the libraries Calling the algorithm The main function The parent generation process Generating a parent Fitness Display parent Crossover and mutation Producing generations of children Summary code
[ ix ]
437 437 440 441 441 442 442 443 444 445 447 450
Table of Contents A physical neural network What is the nature of this mysterious SFNN? Calling the algorithm cell Fitness cell ga_main() cell
Artificial hybrid neural networks Building the LSTM The goal of the model Summary Questions Further reading
Chapter 18: Neuromorphic Computing Neuromorphic computing Getting started with Nengo Installing Nengo and Nengo GUI Creating a Python program A Nengo ensemble Nengo neuron types Nengo neuron dimensions A Nengo node
Connecting Nengo objects Visualizing data Probes Applying Nengo's unique approach to critical AI research areas Summary Questions References Further reading
Chapter 19: Quantum Computing
The rising power of quantum computers Quantum computer speed Defining a qubit Representing a qubit The position of a qubit Radians, degrees, and rotations The Bloch sphere
Composing a quantum score
451 452 453 454 455
456 457 458 459 460 460
461 462 463 464 466 466
467 468 468
470 470 475 479 482 483 483 483
485 486 487 490 490 491
492 493
494
Quantum gates with Quirk A quantum computer score with Quirk A quantum computer score with IBM Q
494 496 497
A thinking quantum computer Representing our mind's concepts
500 500 [x]
Table of Contents
Expanding MindX's conceptual representations The MindX experiment Preparing the data Transformation functions – the situation function Transformation functions – the quantum function Creating and running the score Using the output
Summary Questions Further reading
Appendix: Answers to the Questions
Chapter 1 – Getting Started with NextGeneration Artificial Intelligence through Reinforcement Learning Chapter 2 – Building a Reward Matrix – Designing Your Datasets Chapter 3 – Machine Intelligence – Evaluation Functions and Numerical Convergence Chapter 4 – Optimizing Your Solutions with KMeans Clustering Chapter 5 – How to Use Decision Trees to Enhance KMeans Clustering Chapter 6 – Innovating AI with Google Translate Chapter 7 – Optimizing Blockchains with Naive Bayes Chapter 8 – Solving the XOR Problem with a Feedforward Neural Network Chapter 9 – Abstract Image Classification with Convolutional Neural Networks (CNNs) Chapter 10 – Conceptual Representation Learning Chapter 11 – Combining Reinforcement Learning and Deep Learning Chapter 12 – AI and the Internet of Things Chapter 13 – Visualizing Networks with TensorFlow 2.x and TensorBoard Chapter 14 – Preparing the Input of Chatbots with Restricted Boltzmann Machines (RBMs) and Principal Component Analysis (PCA) Chapter 15 – Setting Up a Cognitive NLP UI/CUI Chatbot Chapter 16 – Improving the Emotional Intelligence Deficiencies of Chatbots Chapter 17 – Genetic Algorithms in Hybrid Neural Networks Chapter 18 – Neuromorphic Computing Chapter 19 – Quantum Computing
Other Books You May Enjoy Index
[ xi ]
500 501
501 501 504 504 506
507 507 508
509 509 511 512 513 515 516 518 519 521 522 524 525 527 528 529 530 531 532 534
537 541
Preface This second edition of Artificial Intelligence By Example will take you through the main aspects of presentday artificial intelligence (AI) and beyond! This book contains many revisions and additions to the key aspects of AI described in the first edition: • The theory of machine learning and deep learning including hybrid and ensemble algorithms. • Mathematical representations of the main AI algorithms including natural language explanations making them easier to understand. • Reallife case studies taking the reader inside the heart of ecommerce: manufacturing, services, warehouses, and delivery. • Introducing AI solutions that combine IoT, convolutional neural networks (CNN), and Markov decision process (MDP). • Many open source Python programs with a special focus on the new features of TensorFlow 2.x, TensorBoard, and Keras. Many modules are used, such as scikitlearn, pandas, and more. • Cloud platforms: Google Colaboratory with its free VM, Google Translate, Google Dialogflow, IBM Q for quantum computing, and more. • Use of the power of restricted Boltzmann machines (RBM) and principal component analysis (PCA) to generate data to create a meaningful chatbot. • Solutions to compensate for the emotional deficiencies of chatbots.
[ xiii ]
Preface
• Genetic algorithms, which run faster than classical algorithms in specific cases, and genetic algorithms used in a hybrid deep learning neural network. • Neuromorphic computing, which reproduces our brain activity with models of selective spiking ensembles of neurons in models that reproduce our biological reactions. • Quantum computing, which will take you deep into the tremendous calculation power of qubits and cognitive representation experiments. This second edition of Artificial Intelligence By Example will take you to the cutting edge of AI and beyond with innovations that improve existing solutions. This book will make you a key asset not only as an AI specialist but a visionary. You will discover how to improve your AI skills as a consultant, developer, professor, a curious mind, or any person involved in artificial intelligence.
Who this book is for
This book contains a broad approach to AI, which is expanding to all areas of our lives. The main machine learning and deep learning algorithms are addressed with reallife Python examples extracted from hundreds of AI projects and implementations. Each AI implementation is illustrated by an open source program available on GitHub and cloud platforms such as Google Colaboratory. Artificial Intelligence By Example, Second Edition is for developers who wish to build solid machine learning programs that will optimize production sites, services, IoT and more. Project managers and consultants will learn how to build input datasets that will help the reader face the challenges of reallife AI. Teachers and students will have an overview of the key aspects of AI, along with many educational examples. Artificial Intelligence By Example, Second Edition will help anybody interested in AI to understand how systems to build solid, productive Python programs.
[ xiv ]
Preface
What this book covers
Chapter 1, Getting Started with NextGeneration Artificial Intelligence through Reinforcement Learning, covers reinforcement learning through the Bellman equation based on the MDP. A case study describes how to solve a delivery route problem with a human driver and a selfdriving vehicle. This chapter shows how to build an MDP from scratch in Python. Chapter 2, Building a Reward Matrix – Designing Your Datasets, demonstrates the architecture of neural networks starting with the McCullochPitts neuron. The case study describes how to use a neural network to build the reward matrix used by the Bellman equation in a warehouse environment. The process will be developed in Python using logistic, softmax, and onehot functions. Chapter 3, Machine Intelligence – Evaluation Functions and Numerical Convergence, shows how machine evaluation capacities have exceeded human decisionmaking. The case study describes a chess position and how to apply the results of an AI program to decisionmaking priorities. An introduction to decision trees in Python shows how to manage decisionmaking processes. Chapter 4, Optimizing Your Solutions with KMeans Clustering, covers a kmeans clustering program with Lloyd's algorithm and how to apply it to the optimization of automatic guided vehicles. The kmeans clustering program's model will be trained and saved. Chapter 5, How to Use Decision Trees to Enhance KMeans Clustering, begins with unsupervised learning with kmeans clustering. The output of the kmeans clustering algorithm will provide the labels for the supervised decision tree algorithm. Random forests will be introduced. Chapter 6, Innovating AI with Google Translate, explains the difference between a revolutionary innovation and a disruptive innovation. Google Translate will be described and enhanced with an innovative knearest neighborsbased Python program. Chapter 7, Optimizing Blockchains with Naive Bayes, is about mining blockchains and describes how blockchains function. We use naive Bayes to optimize the blocks of supply chain management (SCM) blockchains by predicting transactions to anticipate storage levels.
[ xv ]
Preface
Chapter 8, Solving the XOR Problem with a Feedforward Neural Network, is about building a feedforward neural network (FNN) from scratch to solve the XOR linear separability problem. The business case describes how to group orders for a factory. Chapter 9, Abstract Image Classification with Convolutional Neural Networks (CNNs), describes CNN in detail: kernels, shapes, activation functions, pooling, flattening, and dense layers. The case study illustrates the use of a CNN using a webcam on a conveyor belt in a foodprocessing company. Chapter 10, Conceptual Representation Learning, explains conceptual representation learning (CRL), an innovative way to solve production flows with a CNN transformed into a CRL metamodel (CRLMM). The case study shows how to use a CRLMM for transfer and domain learning, extending the model to other applications. Chapter 11, Combining Reinforcement Learning and Deep Learning, combines a CNN with an MDP to build a solution for automatic planning and scheduling with an optimizer with a rulebased system. The solution is applied to apparel manufacturing showing how to apply AI to reallife systems. Chapter 12, AI and the Internet of Things (IoT), explores a support vector machine (SVM) assembled with a CNN. The case study shows how selfdriving cars can find an available parking space automatically. Chapter 13, Visualizing Networks with TensorFlow 2.x and TensorBoard, extracts information of each layer of a CNN and displays the intermediate steps taken by the neural network. The output of each layer contains images of the transformations applied. Chapter 14, Preparing the Input of Chatbots with Restricted Boltzmann Machines (RBM) and Principal Component Analysis (PCA), explains how to produce valuable information using an RBM and a PCA to transform raw data into chatbotinput data. Chapter 15, Setting Up a Cognitive NLP UI/CUI Chatbot, describes how to build a Google Dialogflow chatbot from scratch using the information provided by an RBM and a PCA algorithm. The chatbot will contain entities, intents, and meaningful responses. [ xvi ]
Preface
Chapter 16, Improving the Emotional Intelligence Deficiencies of Chatbots, explains the limits of a chatbot when dealing with human emotions. The Emotion options of Dialogflow will be activated along with Small Talk to make the chatbot friendlier. Chapter 17, Genetic Algorithms in Hybrid Neural Networks, enters our chromosomes, finds our genes, and helps you understand how our reproduction process works. From there, it is shown how to implement an evolutionary algorithm in Python, a genetic algorithm (GA). A hybrid neural network will show how to optimize a neural network with a GA. Chapter 18, Neuromorphic Computing, describes what neuromorphic computing is and then explores Nengo, a unique neuromorphic framework with solid tutorials and documentation. This neuromorphic overview will take you into the wonderful power of our brain structures to solve complex problems. Chapter 19, Quantum Computing, will show quantum computers are superior to classical computers, what a quantum bit is, how to use it, and how to build quantum circuits. An introduction to quantum gates and example programs will bring you into the futuristic world of quantum mechanics. Appendix, Answers to the Questions, provides answers to the questions listed in the Questions section in all the chapters.
To get the most out of this book Artificial intelligence projects rely on three factors:
• Understanding the subject the AI project will be applied to. To do so, go through a chapter to pick up the key ideas. Once you understand the key ideas of a case study described in the book, try to see how an AI solution can be applied to reallife examples around you. • The mathematical foundations of the AI algorithms. Do not skip the mathematics equations if you have the energy to study them. AI relies heavily on mathematics. There are plenty of excellent websites that explain the mathematics used in this book. • Development. An artificial intelligence solution can be directly used on an online cloud platform machine learning site such as Google. We can access these platforms with APIs. In the book, Google Cloud is used several times. Try to create an account of your own to explore several cloud platforms to understand their potential and their limits. Development remains critical for AI projects. [ xvii ]
Preface
Even with a cloud platform, scripts and services are necessary. Also, sometimes, writing an algorithm is mandatory because the readytouse online algorithms are insufficient for a given problem. Explore the programs delivered with the book. They are open source and free.
Technical requirements
The following is a nonexhaustive list of the technical requirements for running the codes in this book. For a more detailed chapterwise list, please refer to this link: https://github.com/PacktPublishing/ArtificialIntelligenceByExampleSecondEdition/blob/master/Technical%20Requirements.csv. Package
Website
Python
https://www.python.org/
NumPy
https://pypi.org/project/numpy/
Matplotlib
https://pypi.org/project/matplotlib/
pandas
https://pypi.org/project/pandas/
SciPy
https://pypi.org/project/scipy/
scikitlearn
https://pypi.org/project/scikitlearn/
PyDotPlus Google API
https://pypi.org/project/pydotplus/ https://developers.google.com/docs/api/ quickstart/python
html
https://pypi.org/project/html/
TensorFlow 2
https://pypi.org/project/tensorflow/
Keras
https://pypi.org/project/Keras/
Pillow
https://pypi.org/project/Pillow/
Imageio
https://pypi.org/project/imageio/
Pathlib
https://pypi.org/project/pathlib/
OpenCVPython
https://pypi.org/project/opencvpython/
Google Dialogflow
https://dialogflow.com/
DEAP
https://pypi.org/project/deap/
bitstring
https://pypi.org/project/bitstring/
nengo
https://pypi.org/project/nengo/
nengogui
https://pypi.org/project/nengogui/
IBM Q
https://www.research.ibm.com/ibmq/
Quirk
http://algassert.com/2016/05/22/quirk.html
[ xviii ]
Preface
Download the example code files
You can download the example code files for this book from your account at www. packt.com/. If you purchased this book elsewhere, you can visit www.packtpub. com/support and register to have the files emailed directly to you. You can download the code files by following these steps: 1. Log in or register at http://www.packt.com. 2. Select the Support tab. 3. Click on Code Downloads. 4. Enter the name of the book in the Search box and follow the onscreen instructions. Once the file is downloaded, please make sure that you unzip or extract the folder using the latest version of: • WinRAR / 7Zip for Windows • Zipeg / iZip / UnRarX for Mac • 7Zip / PeaZip for Linux The code bundle for the book is also hosted on GitHub at https://github.com/ PacktPublishing/ArtificialIntelligenceByExampleSecondEdition. In case there's an update to the code, it will be updated on the existing GitHub repository. We also have other code bundles from our rich catalog of books and videos available at https://github.com/PacktPublishing/. Check them out!
Download the color images
We also provide a PDF file that has color images of the screenshots/diagrams used in this book. You can download it here: https://static.packtcdn.com/ downloads/9781839211539_ColorImages.pdf.
[ xix ]
Preface
Conventions used
There are a number of text conventions used throughout this book. CodeInText: Indicates code words in text, database table names, folder names,
filenames, file extensions, pathnames, dummy URLs, user input, and Twitter handles. For example; "The decision tree program, decision_tree.py, reads the output of the KMC predictions, ckmc.csv." A block of code is set as follows: # load dataset col_names = ['f1', 'f2','label'] df = pd.read_csv("ckmc.csv", header=None, names=col_names) if pp==1: print(df.head())
When we wish to draw your attention to a particular part of a code block, the relevant lines or items are set in bold: for i in range(0,1000): xf1=dataset.at[i,'Distance'] xf2=dataset.at[i,'location'] X_DL = [[xf1,xf2]] prediction = kmeans.predict(X_DL)
Any commandline input or output is written as follows: Selection: BnVYkFcRK Fittest: 0 This generation Fitness: 0 Time Difference: 0:00:00.000198
Bold: Indicates a new term, an important word, or words that you see on the screen, for example, in menus or dialog boxes, also appear in the text like this. For example: "When you click on SAVE, the Emotions progress bar will jump up."
Warnings or important notes appear like this.
Tips and tricks appear like this.
[ xx ]
Preface
Get in touch
Feedback from our readers is always welcome. General feedback: If you have questions about any aspect of this book, mention the book title in the subject of your message and email us at [email protected] packtpub.com. Errata: Although we have taken every care to ensure the accuracy of our content, mistakes do happen. If you have found a mistake in this book we would be grateful if you would report this to us. Please visit, www.packtpub.com/support/ errata, selecting your book, clicking on the Errata Submission Form link, and entering the details. Piracy: If you come across any illegal copies of our works in any form on the Internet, we would be grateful if you would provide us with the location address or website name. Please contact us at [email protected] with a link to the material. If you are interested in becoming an author: If there is a topic that you have expertise in and you are interested in either writing or contributing to a book, please visit authors.packtpub.com.
Reviews
Please leave a review. Once you have read and used this book, why not leave a review on the site that you purchased it from? Potential readers can then see and use your unbiased opinion to make purchase decisions, we at Packt can understand what you think about our products, and our authors can see your feedback on their book. Thank you! For more information about Packt, please visit packt.com.
[ xxi ]
1 Getting Started with NextGeneration Artificial Intelligence through Reinforcement Learning Nextgeneration AI compels us to realize that machines do indeed think. Although machines do not think like us, their thought process has proven its efficiency in many areas. In the past, the belief was that AI would reproduce human thinking processes. Only neuromorphic computing (see Chapter 18, Neuromorphic Computing), remains set on this goal. Most AI has now gone beyond the way humans think, as we will see in this chapter. The Markov decision process (MDP), a reinforcement learning (RL) algorithm, perfectly illustrates how machines have become intelligent in their own unique way. Humans build their decision process on experience. MDPs are memoryless. Humans use logic and reasoning to think problems through. MDPs apply random decisions 100% of the time. Humans think in words, labeling everything they perceive. MDPs have an unsupervised approach that uses no labels or training data. MDPs boost the machine thought process of selfdriving cars (SDCs), translation tools, scheduling software, and more. This memoryless, random, and unlabeled machine thought process marks a historical change in the way a former human problem was solved.
[1]
Getting Started with NextGeneration Artificial Intelligence through Reinforcement Learning
With this realization comes a yet more mindblowing fact. AI algorithms and hybrid solutions built on IoT, for example, have begun to surpass humans in strategic areas. Although AI cannot replace humans in every field, AI combined with classical automation now occupies key domains: banking, marketing, supply chain management, scheduling, and many other critical areas. As you will see, starting with this chapter, you can occupy a central role in this new world as an adaptive thinker. You can design AI solutions and implement them. There is no time to waste. In this chapter, we are going to dive quickly and directly into reinforcement learning through the MDP. Today, AI is essentially mathematics translated into source code, which makes it difficult to learn for traditional developers. However, we will tackle this approach pragmatically. The goal here is not to take the easy route. We're striving to break complexity into understandable parts and confront them with reality. You are going to find out right from the outset how to apply an adaptive thinker's process that will lead you from an idea to a solution in reinforcement learning, and right into the center of gravity of the next generation of AI.
Reinforcement learning concepts AI is constantly evolving. The classical approach states that: • AI covers all domains • Machine learning is a subset of AI, with clustering, classification, regression, and reinforcement learning • Deep learning is a subset of machine learning that involves neural networks However, these domains often overlap and it's difficult to fit neuromorphic computing, for example, with its subsymbolic approach, into these categories (see Chapter 18, Neuromorphic Computing). In this chapter, RL clearly fits into machine learning. Let's have a brief look into the scientific foundations of the MDP, the RL algorithm we are going to explore. The main concepts to keep in mind are the following: • Optimal transport: In 1781, Gaspard Monge defined transport optimizing from one location to another using the shortest and most costeffective path; for example, mining coal and then using the most costeffective path to a factory. This was subsequently generalized to any form of path from point A to point B. [2]
Chapter 1
• Boltzmann equation and constant: In the late 19th century, Ludwig Boltzmann changed our vision of the world with his probabilistic distribution of particles beautifully summed up in his entropy formula: S = k * log W S represents the entropy (energy, disorder) of a system expressed. k is the Boltzmann constant, and W represents the number of microstates. We will explore Boltzmann's ideas further in Chapter 14, Preparing the Input of Chatbots with Restricted Boltzmann Machines (RBMs) and Principal Component Analysis (PCA). • Probabilistic distributions advanced further: Josiah Willard Gibbs took the probabilistic distributions of large numbers of particles a step further. At that point, probabilistic information theory was advancing quickly. At the turn of the 19th century, Andrey Markov applied probabilistic algorithms to language, among other areas. A modern era of information theory was born. • When Boltzmann and optimal transport meet: 2011 Fields Medal winner, Cédric Villani, brought Boltzmann's equation to yet another level. Villani then went on to unify optimal transport and Boltzmann. Cédric Villani proved something that was somewhat intuitively known to 19th century mathematicians but required proof. Let's take all of the preceding concepts and materialize them in a realworld example that will explain why reinforcement learning using the MDP, for example, is so innovative. Analyzing the following cup of tea will take you right into the next generation of AI:
Figure 1.1: Consider a cup of tea
[3]
Getting Started with NextGeneration Artificial Intelligence through Reinforcement Learning
You can look at this cup of tea in two different ways: 1. Macrostates: You look at the cup and content. You can see the volume of tea in the cup and you could feel the temperature when holding the cup in your hand. 2. Microstates: But can you tell how many molecules are in the tea, which ones are hot, warm, or cold, their velocity and directions? Impossible right? Now, imagine, the tea contains 2,000,000,000+ Facebook accounts, or 100,000,000+ Amazon Prime users with millions of deliveries per year. At this level, we simply abandon the idea of controlling every item. We work on trends and probabilities. Boltzmann provides a probabilistic approach to the evaluation of the features of our real world. Materializing Boltzmann in logistics through optimal transport means that the temperature could be the ranking of a product, the velocity can be linked to the distance to delivery, and the direction could be the itineraries we will study in this chapter. Markov picked up the ripe fruits of microstate probabilistic descriptions and applied it to his MDP. Reinforcement learning takes the huge volume of elements (particles in a cup of tea, delivery locations, social network accounts) and defines the probable paths they take. The turning point of human thought occurred when we simply could not analyze the state and path of the huge volumes facing our globalized world, which generates images, sounds, words, and numbers that exceed traditional software approaches. With this in mind, we can start exploring the MDP.
How to adapt to machine thinking and become an adaptive thinker
Reinforcement learning, one of the foundations of machine learning, supposes learning through trial and error by interacting with an environment. This sounds familiar, doesn't it? That is what we humans do all our lives—in pain! Try things, evaluate, and then continue; or try something else. In real life, you are the agent of your thought process. In reinforcement learning, the agent is the function calculating randomly through this trialanderror process. This thought process function in machine learning is the MDP agent. This form of empirical learning is sometimes called Qlearning.
[4]
Chapter 1
Mastering the theory and implementation of an MDP through a threestep method is a prerequisite. This chapter will detail the threestep approach that will turn you into an AI expert, in general terms: 1. Starting by describing a problem to solve with reallife cases 2. Then, building a mathematical model that considers reallife limitations 3. Then, writing source code or using a cloud platform solution This is a way for you to approach any project with an adaptive attitude from the outset. This shows that a human will always be at the center of AI by explaining how we can build the inputs, run an algorithm, and use the results of our code. Let's consider this threestep process and put it into action.
Overcoming reallife issues using the threestep approach
The key point of this chapter is to avoid writing code that will never be used. First, begin by understanding the subject as a subject matter expert. Then, write the analysis with words and mathematics to make sure your reasoning reflects the subject and, most of all, that the program will make sense in real life. Finally, in step 3, only write the code when you are sure about the whole project. Too many developers start writing code without stopping to think about how the results of that code are going to manifest themselves within reallife situations. You could spend weeks developing the perfect code for a problem, only to find out that an external factor has rendered your solution useless. For instance, what if you coded a solarpowered robot to clear snow from the yard, only to discover that during winter, there isn't enough sunlight to power the robot! In this chapter, we are going to tackle the MDP (Q function) and apply it to reinforcement learning with the Bellman equation. We are going to approach it a little differently to most, however. We'll be thinking about practical application, not simply code execution. You can find tons of source code and examples on the web. The problem is, much like our snow robot, such source code rarely considers the complications that come about in reallife situations. Let's say you find a program that finds the optimal path for a drone delivery. There's an issue, though; it has many limits that need to be overcome due to the fact that the code has not been written with reallife practicality in mind. You, as an adaptive thinker, are going to ask some questions: • What if there are 5,000 drones over a major city at the same time? What happens if they try to move in straight lines and bump into each other? [5]
Getting Started with NextGeneration Artificial Intelligence through Reinforcement Learning
• Is a dronejam legal? What about the noise over the city? What about tourism? • What about the weather? Weather forecasts are difficult to make, so how is this scheduled? • How can we resolve the problem of coordinating the use of charging and parking stations? In just a few minutes, you will be at the center of attention among theoreticians who know more than you, on one hand, and angry managers who want solutions they cannot get on the other. Your reallife approach will solve these problems. To do that, you must take the following three steps into account, starting with really getting involved in the reallife subject. In order to successfully implement our reallife approach, comprised of the three steps outlined in the previous section, there are a few prerequisites: • Be a subject matter expert (SME): First, you have to be an SME. If a theoretician geek comes up with a hundred TensorFlow functions to solve a drone trajectory problem, you now know it is going to be a tough ride in which reallife parameters are constraining the algorithm. An SME knows the subject and thus can quickly identify the critical factors of a given field. AI often requires finding a solution to a complex problem that even an expert in a given field cannot express mathematically. Machine learning sometimes means finding a solution to a problem that humans do not know how to explain. Deep learning, involving complex networks, solves even more difficult problems. • Have enough mathematical knowledge to understand AI concepts: Once you have the proper natural language analysis, you need to build your abstract representation quickly. The best way is to look around and find an everyday life example and make a mathematical model of it. Mathematics is not an option in AI, but a prerequisite. The effort is worthwhile. Then, you can start writing a solid piece of source code or start implementing a cloud platform ML solution. • Know what source code is about as well as its potential and limits: MDP is an excellent way to go and start working on the three dimensions that will make you adaptive: describing what is around you in detail in words, translating that into mathematical representations, and then implementing the result in your source code. With those prerequisites in mind, let's look at how you can become a problemsolving AI expert by following our practical threestep process. Unsurprisingly, we'll begin at step 1. [6]
Chapter 1
Step 1 – describing a problem to solve: MDP in natural language
Step 1 of any AI problem is to go as far as you can to understand the subject you are asked to represent. If it's a medical subject, don't just look at data; go to a hospital or a research center. If it's a private security application, go to the places where they will need to use it. If it's for social media, make sure to talk to many users directly. The key concept to bear in mind is that you have to get a "feel" for the subject, as if you were the real "user." For example, transpose it into something you know in your everyday life (work or personal), something you are an SME in. If you have a driver's license, then you are an SME of driving. You are certified. This is a fairly common certification, so let's use this as our subject matter in the example that will follow. If you do not have a driver's license or never drive, you can easily replace moving around in a car by imagining you are moving around on foot; you are an SME of getting from one place to another, regardless of what means of transport that might involve. However, bear in mind that a reallife project would involve additional technical aspects, such as traffic regulations for each country, so our imaginary SME does have its limits. Getting into the example, let's say you are an ecommerce business driver delivering a package in a location you are unfamiliar with. You are the operator of a selfdriving vehicle. For the time being, you're driving manually. You have a GPS with a nice color map on it. The locations around you are represented by the letters A to F, as shown in the simplified map in the following diagram. You are presently at F. Your goal is to reach location C. You are happy, listening to the radio. Everything is going smoothly, and it looks like you are going to be there on time. The following diagram represents the locations and routes that you can cover:
Figure 1.2: A diagram of delivery routes
The guidance system's state indicates the complete path to reach C. It is telling you that you are going to go from F to B to D, and then to C. It looks good! [7]
Getting Started with NextGeneration Artificial Intelligence through Reinforcement Learning
To break things down further, let's say: • The present state is the letter s. s is a variable, not an actual state. It can be one of the locations in L, the set of locations: L = {A, B, C, D, E, F} We say present state because there is no sequence in the learning process. The memoryless process goes from one present state to another. In the example in this chapter, the process starts at location F. • Your next action is the letter a (action). This action a is not location A. The goal of this action is to take us to the next possible location in the graph. In this case, only B is possible. The goal of a is to take us from s (present state) to s' (new state). • The action a (not location A) is to go to location B. You look at your guidance system; it tells you there is no traffic, and that to go from your present state, F, to your next state, B, will take you only a few minutes. Let's say that the next state B is the letter B. This next state B is s'. At this point, you are still quite happy, and we can sum up your situation with the following sequence of events:
s, a, s' The letter s is your present state, your present situation. The letter a is the action you're deciding, which is to go to the next location; there, you will be in another state, s'. We can say that thanks to the action a, you will go from s to s'. Now, imagine that the driver is not you anymore. You are tired for some reason. That is when a selfdriving vehicle comes in handy. You set your car to autopilot. Now, you are no longer driving; the system is. Let's call that system the agent. At point F, you set your car to autopilot and let the selfdriving agent take over.
Watching the MDP agent at work
The selfdriving AI is now in charge of the vehicle. It is acting as the MDP agent. This now sees what you have asked it to do and checks its mapping environment, which represents all the locations in the previous diagram from A to F. In the meantime, you are rightly worried. Is the agent going to make it or not? You are wondering whether its strategy meets yours. You have your policy P—your way of thinking—which is to take the shortest path possible. Will the agent agree? What's going on in its machine mind? You observe and begin to realize things you never noticed before.
[8]
Chapter 1
Since this is the first time you are using this car and guidance system, the agent is memoryless, which is an MDP feature. The agent doesn't know anything about what went on before. It seems to be happy with just calculating from this state s at location F. It will use machine power to run as many calculations as necessary to reach its goal. Another thing you are watching is the total distance from F to C to check whether things are OK. That means that the agent is calculating all the states from F to C. In this case, state F is state 1, which we can simplify by writing s1; B is state 2, which we can simplify by writing s2; D is s3; and C is s4. The agent is calculating all of these possible states to make a decision. The agent knows that when it reaches D, C will be better because the reward will be higher for going to C than anywhere else. Since it cannot eat a piece of cake to reward itself, the agent uses numbers. Our agent is a real number cruncher. When it is wrong, it gets a poor reward or nothing in this model. When it's right, it gets a reward represented by the letter R, which we'll encounter during step 2. This actionvalue (reward) transition, often named the Q function, is the core of many reinforcement learning algorithms. When our agent goes from one state to another, it performs a transition and gets a reward. For example, the transition can be from F to B, state 1 to state 2, or s1 to s2. You are feeling great and are going to be on time. You are beginning to understand how the machine learning agent in your selfdriving car is thinking. Suddenly, you look up and see that a traffic jam is building up. Location D is still far away, and now you do not know whether it would be good to go from D to C or D to E, in order to take another road to C, which involves less traffic. You are going to see what your agent thinks! The agent takes the traffic jam into account, is stubborn, and increases its reward to get to C by the shortest way. Its policy is to stick to the initial plan. You do not agree. You have another policy. You stop the car. You both have to agree before continuing. You have your opinion and policy; the agent does not agree. Before continuing, your views need to converge. Convergence is the key to making sure that your calculations are correct, and it's a way to evaluate the quality of a calculation. A mathematical representation is the best way to express this whole process at this point, which we will describe in the following step.
[9]
Getting Started with NextGeneration Artificial Intelligence through Reinforcement Learning
Step 2 – building a mathematical model: the mathematical representation of the Bellman equation and MDP Mathematics involves a whole change in your perspective of a problem. You are going from words to functions, the pillars of source coding.
Expressing problems in mathematical notation does not mean getting lost in academic math to the point of never writing a single line of code. Just use mathematics to get a job done efficiently. Skipping mathematical representation will fasttrack a few functions in the early stages of an AI project. However, when the real problems that occur in all AI projects surface, solving them with source code alone will prove virtually impossible. The goal here is to pick up enough mathematics to implement a solution in reallife companies. It is necessary to think through a problem by finding something familiar around us, such as the itinerary model covered early in this chapter. It is a good thing to write it down with some abstract letters and symbols as described before, with a meaning an action, and s meaning a state. Once you have understood the problem and expressed it clearly, you can proceed further. Now, mathematics will help to clarify the situation by means of shorter descriptions. With the main ideas in mind, it is time to convert them into equations.
From MDP to the Bellman equation
In step 1, the agent went from F, or state 1 or s, to B, which was state 2 or s'. A strategy drove this decision—a policy represented by P. One mathematical expression contains the MDP state transition function:
Pa(s, s') P is the policy, the strategy made by the agent to go from F to B through action a. When going from F to B, this state transition is named the state transition function: • a is the action • s is state 1 (F), and s' is state 2 (B) The reward (right or wrong) matrix follows the same principle:
Ra(s, s')
[ 10 ]
Chapter 1
That means R is the reward for the action of going from state s to state s'. Going from one state to another will be a random process. Potentially, all states can go to any other state. Each line in the matrix in the example represents a letter from A to F, and each column represents a letter from A to F. All possible states are represented. The 1 values represent the nodes (vertices) of the graph. Those are the possible locations. For example, line 1 represents the possible moves for letter A, line 2 for letter B, and line 6 for letter F. On the first line, A cannot go to C directly, so a 0 value is entered. But, it can go to E, so a 1 value is added. Some models start with 1 for impossible choices, such as B going directly to C, and 0 values to define the locations. This model starts with 0 and 1 values. It sometimes takes weeks to design functions that will create a reward matrix (see Chapter 2, Building a Reward Matrix – Designing Your Datasets). The example we will be working on inputs a reward matrix so that the program can choose its best course of action. Then, the agent will go from state to state, learning the best trajectories for every possible starting location point. The goal of the MDP is to go to C (line 3, column 3 in the reward matrix), which has a starting value of 100 in the following Python code: # Markov Decision Process (MDP)  The Bellman equations adapted to # Reinforcement Learning import numpy as ql # R is The Reward Matrix for each state R = ql.matrix([ [0,0,0,0,1,0], [0,0,0,1,0,1], [0,0,100,1,0,0], [0,1,1,0,1,0], [1,0,0,1,0,0], [0,1,0,0,0,0] ])
Somebody familiar with Python might wonder why I used ql instead of np. Some might say "convention," "mainstream," "standard." My answer is a question. Can somebody define what "standard" AI is in this fastmoving world! My point here for the MDP is to use ql as an abbreviation of "Qlearning" instead of the "standard" abbreviation of NumPy, which is np. Naturally, beyond this special abbreviation for the MDP programs, I'll use np. Just bear in mind that conventions are there to break so as to set ourselves free to explore new frontiers. Just make sure your program works well!
[ 11 ]
Getting Started with NextGeneration Artificial Intelligence through Reinforcement Learning
There are several key properties of this decision process, among which there is the following: • The Markov property: The process does not take the past into account. It is the memoryless property of this decision process, just as you do in a car with a guidance system. You move forward to reach your goal. • Unsupervised learning: From this memoryless Markov property, it is safe to say that the MDP is not supervised learning. Supervised learning would mean that we would have all the labels of the reward matrix R and learn from them. We would know what A means and use that property to make a decision. We would, in the future, be looking at the past. MDP does not take these labels into account. Thus, MDP uses unsupervised learning to train. A decision has to be made in each state without knowing the past states or what they signify. It means that the car, for example, was on its own at each location, which is represented by each of its states. • Stochastic process: In step 1, when state D was reached, the agent controlling the mapping system and the driver didn't agree on where to go. A random choice could be made in a trialanderror way, just like a coin toss. It is going to be a headsortails process. The agent will toss the coin a significant number of times and measure the outcomes. That's precisely how MDP works and how the agent will learn. • Reinforcement learning: Repeating a trialanderror process with feedback from the agent's environment. • Markov chain: The process of going from state to state with no history in a random, stochastic way is called a Markov chain. To sum it up, we have three tools: • Pa(s, s'): A policy, P, or strategy to move from one state to another • Ta(s, s'): A T, or stochastic (random) transition, function to carry out that action • Ra(s, s'): An R, or reward, for that action, which can be negative, null, or positive T is the transition function, which makes the agent decide to go from one point to another with a policy. In this case, it will be random. That's what machine power is for, and that is how reinforcement learning is often implemented.
[ 12 ]
Chapter 1
Randomness
Randomness is a key property of MDP, defining it as a stochastic process. The following code describes the choice the agent is going to make: next_action = int(ql.random.choice(PossibleAction,1)) return next_action
The code selects a new random action (state) at each episode.
The Bellman equation
The Bellman equation is the road to programming reinforcement learning. The Bellman equation completes the MDP. To calculate the value of a state, let's use Q, for the Q actionreward (or value) function. The pseudo source code of the Bellman equation can be expressed as follows for one individual state:
𝑄𝑄(𝑠𝑠) = 𝑅𝑅(𝑠𝑠) + 𝛾𝛾 ∗ max(𝑠𝑠 ′ )
The source code then translates the equation into a machine representation, as in the following code: # The Bellman equation Q[current_state, action] = R[current_state, action] + gamma * MaxValue
The source code variables of the Bellman equation are as follows: • Q(s): This is the value calculated for this state—the total reward. In step 1, when the agent went from F to B, the reward was a number such as 50 or 100 to show the agent that it's on the right track. • R(s): This is the sum of the values up to that point. It's the total reward at that point. • 𝛾𝛾 = gamma: This is here to remind us that trial and error has a price. We're wasting time, money, and energy. Furthermore, we don't even know whether the next step is right or wrong since we're in a trialanderror mode. Gamma is often set to 0.8. What does that mean? Suppose you're taking an exam. You study and study, but you don't know the outcome. You might have 80 out of 100 (0.8) chances of clearing it. That's painful, but that's life. The gamma penalty, or learning rate, makes the Bellman equation realistic and efficient. • max(s'): s' is one of the possible states that can be reached with Pa(s, s'); max is the highest value on the line of that state (location line in the reward matrix). [ 13 ]
Getting Started with NextGeneration Artificial Intelligence through Reinforcement Learning
At this point, you have done twothirds of the job: understanding the reallife (process) and representing it in basic mathematics. You've built the mathematical model that describes your learning process, and you can implement that solution in code. Now, you are ready to code!
Step 3 – writing source code: implementing the solution in Python
In step 1, a problem was described in natural language to be able to talk to experts and understand what was expected. In step 2, an essential mathematical bridge was built between natural language and source coding. Step 3 is the software implementation phase. When a problem comes up—and rest assured that one always does—it will be possible to go back over the mathematical bridge with the customer or company team, and even further back to the natural language process if necessary. This method guarantees success for any project. The code in this chapter is in Python 3.x. It is a reinforcement learning program using the Q function with the following reward matrix: import numpy as ql R = ql.matrix([ [0,0,0,0,1,0], [0,0,0,1,0,1], [0,0,100,1,0,0], [0,1,1,0,1,0], [1,0,0,1,0,0], [0,1,0,0,0,0] ]) Q = ql.matrix(ql.zeros([6,6])) gamma = 0.8
R is the reward matrix described in the mathematical analysis. Q inherits the same structure as R, but all values are set to 0 since this is a learning matrix. It will progressively contain the results of the decision process. The gamma variable is a double reminder that the system is learning and that its decisions have only an 80% chance of being correct each time. As the following code shows, the system explores the possible actions during the process: agent_s_state = 1 # The possible "a" actions when the agent is in a given state [ 14 ]
Chapter 1 def possible_actions(state): current_state_row = R[state,] possible_act = ql.where(current_state_row >0)[1] return possible_act # Get available actions in the current state PossibleAction = possible_actions(agent_s_state)
The agent starts in state 1, for example. You can start wherever you want because it's a random process. Note that the process only takes values > 0 into account. They represent possible moves (decisions). The current state goes through an analysis process to find possible actions (next possible states). You will note that there is no algorithm in the traditional sense with many rules. It's a pure random calculation, as the following random.choice function shows: def ActionChoice(available_actions_range): if(sum(PossibleAction)>0): next_action = int(ql.random.choice(PossibleAction,1)) if(sum(PossibleAction) 1: Max_State = int(ql.random.choice(Max_State, size = 1)) else: Max_State = int(Max_State) MaxValue = Q[action, Max_State] # Q function Q[current_state, action] = R[current_state, action] + gamma * MaxValue # Rewarding Q matrix reward(agent_s_state,action,gamma)
You can see that the agent looks for the maximum value of the next possible state chosen at random. [ 15 ]
Getting Started with NextGeneration Artificial Intelligence through Reinforcement Learning
The best way to understand this is to run the program in your Python environment and print() the intermediate values. I suggest that you open a spreadsheet and note the values. This will give you a clear view of the process. The last part is simply about running the learning process 50,000 times, just to be sure that the system learns everything there is to find. During each iteration, the agent will detect its present state, choose a course of action, and update the Q function matrix: for i in range(50000): current_state = ql.random.randint(0, int(Q.shape[0])) PossibleAction = possible_actions(current_state) action = ActionChoice(PossibleAction) reward(current_state,action,gamma) # Displaying Q before the norm of Q phase print("Q :") print(Q) # Norm of Q print("Normed Q :") print(Q/ql.max(Q)*100)
The process continues until the learning process is over. Then, the program will print the result in Q and the normed result. The normed result is the process of dividing all values by the sum of the values found. print(Q/ql.max(Q)*100) norms Q by dividing Q by q1.max(Q)*100. The result comes out as a normed percentage. You can run the process with mdp01.py.
The lessons of reinforcement learning
Unsupervised reinforcement machine learning, such as the MDPdriven Bellman equation, is toppling traditional decisionmaking software location by location. Memoryless reinforcement learning requires few to no business rules and, thus, doesn't require human knowledge to run. Being an adaptive nextgeneration AI thinker involves three prerequisites: the effort to be an SME, working on mathematical models to think like a machine, and understanding your source code's potential and limits.
[ 16 ]
Chapter 1
Machine power and reinforcement learning teach us two important lessons: • Lesson 1: Machine learning through reinforcement learning can beat human intelligence in many cases. No use fighting! The technology and solutions are already here in strategic domains. • Lesson 2: A machine has no emotions, but you do. And so do the people around you. Human emotions and teamwork are an essential asset. Become an SME for your team. Learn how to understand what they're trying to say intuitively and make a mathematical representation of it for them. Your job will never go away, even if you're setting up solutions that don't require much development, such as AutoML. AutoML, or automated machine learning, automates many tasks. AutoML automates functions such as the dataset pipeline, hyperparameters, and more. Development is partially or totally suppressed. But you still have to make sure the whole system is well designed. Reinforcement learning shows that no human can solve a problem the way a machine does. 50,000 iterations with random searching is not an option for a human. The number of empirical episodes can be reduced dramatically with a numerical convergence form of gradient descent (see Chapter 3, Machine Intelligence – Evaluation Functions and Numerical Convergence). Humans need to be more intuitive, make a few decisions, and see what happens, because humans cannot try thousands of ways of doing something. Reinforcement learning marks a new era for human thinking by surpassing human reasoning power in strategic fields. On the other hand, reinforcement learning requires mathematical models to function. Humans excel in mathematical abstraction, providing powerful intellectual fuel to those powerful machines. The boundaries between humans and machines have changed. Humans' ability to build mathematical models and evergrowing cloud platforms will serve online machine learning services. Finding out how to use the outputs of the reinforcement learning program we just studied shows how a human will always remain at the center of AI.
[ 17 ]
Getting Started with NextGeneration Artificial Intelligence through Reinforcement Learning
How to use the outputs
The reinforcement program we studied contains no trace of a specific field, as in traditional software. The program contains the Bellman equation with stochastic (random) choices based on the reward matrix. The goal is to find a route to C (line 3, column 3) that has an attractive reward (100): # Markov Decision Process (MDP) – The Bellman equations adapted to # Reinforcement Learning with the Q actionvalue(reward) matrix import numpy as ql # R is The Reward Matrix for each state R = ql.matrix([ [0,0,0,0,1,0], [0,0,0,1,0,1], [0,0,100,1,0,0], [0,1,1,0,1,0], [1,0,0,1,0,0], [0,1,0,0,0,0] ])
That reward matrix goes through the Bellman equation and produces a result in Python: Q : [[ 0. 0. 0. 0. 258.44 0. ] [ 0. 0. 0. 321.8 0. 207.752] [ 0. 0. 500. 321.8 0. 0. ] [ 0. 258.44 401. 0. 258.44 0. ] [ 207.752 0. 0. 321.8 0. 0. ] [ 0. 258.44 0. 0. 0. 0. ]] Normed Q : [[ 0. 0. 0. 0. 51.688 0. ] [ 0. 0. 0. 64.36 0. 41.5504] [ 0. 0. 100. 64.36 0. 0. ] [ 0. 51.688 80.2 0. 51.688 0. ] [ 41.5504 0. 0. 64.36 0. 0. ] [ 0. 51.688 0. 0. 0. 0. ]]
The result contains the values of each state produced by the reinforced learning process, and also a normed Q (the highest value divided by other values). As Python geeks, we are overjoyed! We made something that is rather difficult work, namely, reinforcement learning. As mathematical amateurs, we are elated. We know what MDP and the Bellman equation mean. However, as natural language thinkers, we have made little progress. No customer or user can read that data and make sense of it. Furthermore, we cannot explain how we implemented an intelligent version of their job in the machine. We didn't. [ 18 ]
Chapter 1
We hardly dare say that reinforcement learning can beat anybody in the company, making random choices 50,000 times until the right answer came up. Furthermore, we got the program to work, but hardly know what to do with the result ourselves. The consultant on the project cannot help because of the matrix format of the solution. Being an adaptive thinker means knowing how to be good in all steps of a project. To solve this new problem, let's go back to step 1 with the result. Going back to step 1 means that if you have problems either with the results themselves or understanding them, it is necessary to go back to the SME level, the reallife situation, and see what is going wrong. By formatting the result in Python, a graphics tool, or a spreadsheet, the result can be displayed as follows:
A
A
B
C
D
E
F




258.44

B



321.8

207.752
C


500
321.8


D

258.44
401.

258.44

E
207.752


321.8


F

258.44




Now, we can start reading the solution: • Choose a starting state. Take F, for example. • The F line represents the state. Since the maximum value is 258.44 in the B column, we go to state B, the second line. • The maximum value in state B in the second line leads us to the D state in the fourth column. • The highest maximum of the D state (fourth line) leads us to the C state. Note that if you start at the C state and decide not to stay at C, the D state becomes the maximum value, which will lead you back to C. However, the MDP will never do this naturally. You will have to force the system to do it. You have now obtained a sequence: F>B>D>C. By choosing other points of departure, you can obtain other sequences by simply sorting the table.
[ 19 ]
Getting Started with NextGeneration Artificial Intelligence through Reinforcement Learning
A useful way of putting it remains the normalized version in percentages, as shown in the following table: A
B
C
D
E
F
A




51.68%

B



64.36%

41.55%
C


100%
64.36%


D

51.68%
80.2%

51.68%

E
41.55%


64.36%


F

51.68%




Now comes the very tricky part. We started the chapter with a trip on the road. But I made no mention of it in the results analysis. An important property of reinforcement learning comes from the fact that we are working with a mathematical model that can be applied to anything. No human rules are needed. We can use this program for many other subjects without writing thousands of lines of code.
Possible use cases
There are many cases to which we could adapt our reinforcement learning model without having to change any of its details.
Case 1: optimizing a delivery for a driver, human or not This model was described in this chapter.
Case 2: optimizing warehouse flows
The same reward matrix can apply to go from point F to C in a warehouse, as shown in the following diagram:
Figure 1.3: A diagram illustrating a warehouse flow problem
[ 20 ]
Chapter 1
In this warehouse, the F>B>D>C sequence makes visual sense. If somebody goes from point F to C, then this physical path makes sense without going through walls. It can be used for a video game, a factory, or any form of layout.
Case 3: automated planning and scheduling (APS)
By converting the system into a scheduling vector, the whole scenery changes. We have left the more comfortable world of physical processing of letters, faces, and trips. Though fantastic, those applications are social media's tip of the iceberg. The real challenge of AI begins in the abstract universe of human thinking. Every single company, person, or system requires automatic planning and scheduling (see Chapter 12, AI and the Internet of Things (IoT)). The six A to F steps in the example of this chapter could well be six tasks to perform in a given unknown order represented by the following vector x:
𝑥𝑥1 𝑥𝑥2 𝑥𝑥3 𝑥𝑥 = 𝑥𝑥 4 𝑥𝑥5 [𝑥𝑥6 ]
The reward matrix then reflects the weights of constraints of the tasks of vector x to perform. For example, in a factory, you cannot assemble the parts of a product before manufacturing them. In this case, the sequence obtained represents the schedule of the manufacturing process.
Cases 4 and more: your imagination
By using physical layouts or abstract decisionmaking vectors, matrices, and tensors, you can build a world of solutions in a mathematical reinforcement learning model. Naturally, the following chapters will enhance your toolbox with many other concepts. Before moving on, you might want to imagine some situations in which you could use the A to F letters to express some kind of path. To help you with these mind experiment simulations, open mdp02.py and go to line 97, which starts with the following code that enables a simulation tool. nextc and nextci are simply variables to remember where the path begins and will end. They are set to 1 so as to avoid 0, which is a location. [ 21 ]
Getting Started with NextGeneration Artificial Intelligence through Reinforcement Learning
The primary goal is to focus on the expression "concept code." The locations have become any concept you wish. A could be your bedroom, and C your kitchen. The path would go from where you wake up to where you have breakfast. A could be an idea you have, and F the end of a thinking process. The path would go from A (How can I hang this picture on the wall?) to E (I need to drill a hole) and, after a few phases, to F (I hung the picture on the wall). You can imagine thousands of paths like this as long as you define the reward matrix, the "concept code," and a starting point: """# Improving the program by introducing a decisionmaking process""" nextc=1 nextci=1 conceptcode=["A","B","C","D","E","F"]
This code takes the result of the calculation, labels the result matrix, and accepts an input as shown in the following code snippet: origin=int(input( "index number origin(A=0,B=1,C=2,D=3,E=4,F=5): "))
The input only accepts the label numerical code: A=0, B=1 … F=5. The function then runs a classical calculation on the results to find the best path. Let's takes an example. When you are prompted to enter a starting point, enter 5, for example, as follows: index number origin(A=0,B=1,C=2,D=3,E=4,F=5): 5
The program will then produce the optimal path based on the output of the MDP process, as shown in the following output: Concept Path > F > B > D > C
Try multiple scenarios and possibilities. Imagine what you could apply this to: • An ecommerce website flow (visit, cart, checkout, purchase) imagining that a user visits the site and then resumes a session at a later time. You can use the same reward matrix and "concept code" explored in this chapter. For example, a visitor visits a web page at 10 a.m., starting at point A of your website. Satisfied with a product, the visitor puts the product in a cart, which is point E of your website. Then, the visitor leaves the site before going to the purchase page, which is C. D is the critical point. Why didn't the visitor purchase the product? What's going on? [ 22 ]
Chapter 1
You can decide to have an automatic email sent after 24 hours saying: "There is a 10% discount on all purchases during the next 48 hours." This way, you will target all the visitors stuck at D and push them toward C. • A sequence of possible words in a sentence (subject, verb, object). Predicting letters and words was one of Andrey Markov's first applications 100+ years ago! You can imagine that B is the letter "a" of the alphabet. If D is "t," it is much more probable than F if F is "o," which is less probable in the English language. If an MDP reward matrix is built such as B leads to D or F, B can thus either go to D or to F. There are thus two possibilities, D or F. Andrey Markov would suppose, for example, that B is a variable that represents the letter "a," D is a variable that represents the letter "t" and F is a variable that represents the letter "o." After studying the structure of a language closely, he would find that the letter "a" would more likely be followed by "t" than by "o" in the English language. If one observes the English language, it is more likely to find an "at" sequence than an "ao" sequence. In a Markov decision process, a higher probability will be awarded to the "at" sequence and a lower one to "ao." If one goes back to the variables, the BD sequence will come out as more probable than the BF sequence. • And anything you can find that fits the model that works is great!
Machine learning versus traditional applications
Reinforcement learning based on stochastic (random) processes will evolve beyond traditional approaches. In the past, we would sit down and listen to future users to understand their way of thinking. We would then go back to our keyboard and try to imitate the human way of thinking. Those days are over. We need proper datasets and ML/DL equations to move forward. Applied mathematics has taken reinforcement learning to the next level. In my opinion, traditional software will soon be in the museum of computer science. The complexity of the huge volumes of data we are facing will require AI at some point. An artificial adaptive thinker sees the world through applied mathematics translated into machine representations. Use the Python source code example provided in this chapter in different ways. Run it and try to change some parameters to see what happens. Play around with the number of iterations as well. Lower the number from 50,000 down to where you find it fits best. Change the reward matrix a little to see what happens. Design your reward matrix trajectory. This can be an itinerary or decisionmaking process. [ 23 ]
Getting Started with NextGeneration Artificial Intelligence through Reinforcement Learning
Summary
Presently, AI is predominantly a branch of applied mathematics, not of neurosciences. You must master the basics of linear algebra and probabilities. That's a difficult task for a developer used to intuitive creativity. With that knowledge, you will see that humans cannot rival machines that have CPU and mathematical functions. You will also understand that machines, contrary to the hype around you, don't have emotions; although we can represent them to a scary point in chatbots (see Chapter 16, Improving the Emotional Intelligence Deficiencies of Chatbots). A multidimensional approach is a prerequisite in an AI/ML/DL project. First, talk and write about the project, then make a mathematical representation, and finally go for software production (setting up an existing platform or writing code). In real life, AI solutions do not just grow spontaneously in companies as some hype would have us believe. You need to talk to the teams and work with them. That part is the real fulfilling aspect of a project—imagining it first and then implementing it with a group of reallife people. MDP, a stochastic random actionreward (value) system enhanced by the Bellman equation, will provide effective solutions to many AI problems. These mathematical tools fit perfectly in corporate environments. Reinforcement learning using the Q actionvalue function is memoryless (no past) and unsupervised (the data is not labeled or classified). MDP provides endless avenues to solve reallife problems without spending hours trying to invent rules to make a system work. Now that you are at the heart of Google's DeepMind approach, it is time to go to Chapter 2, Building a Reward Matrix – Designing Your Datasets, and discover how to create the reward matrix in the first place through explanations and source code.
Questions
The answers to the questions are in Appendix B, with further explanations: 1. Is reinforcement learning memoryless? (Yes  No) 2. Does reinforcement learning use stochastic (random) functions? (Yes  No) 3. Is MDP based on a rule base? (Yes  No) 4. Is the Q function based on the MDP? (Yes  No) 5. Is mathematics essential to AI? (Yes  No)
[ 24 ]
Chapter 1
6. Can the BellmanMDP process in this chapter apply to many problems? (Yes  No) 7. Is it impossible for a machine learning program to create another program by itself? (Yes  No) 8. Is a consultant required to enter business rules in a reinforcement learning program? (Yes  No) 9. Is reinforcement learning supervised or unsupervised? (Supervised  Unsupervised) 10. Can Qlearning run without a reward matrix? (Yes  No)
Further reading
• Andrey Markov: https://www.britannica.com/biography/AndreyAndreyevichMarkov
• The Markov process: https://www.britannica.com/science/Markovprocess
[ 25 ]
2 Building a Reward Matrix – Designing Your Datasets Experimenting and implementation comprise the two main approaches of artificial intelligence. Experimenting largely entails trying readytouse datasets and black box, readytouse Python examples. Implementation involves preparing a dataset, developing preprocessing algorithms, and then choosing a model, the proper parameters, and hyperparameters. Implementation usually involves white box work that entails knowing exactly how an algorithm works and even being able to modify it. In Chapter 1, Getting Started with NextGeneration Artifcial Intelligence through Reinforcement Learning, the MDPdriven Bellman equation relied on a reward matrix. In this chapter, we will get our hands dirty in a white box process to create that reward matrix. An MDP process cannot run without a reward matrix. The reward matrix determines whether it is possible to go from one cell to another, from A to B. It is like a map of a city that tells you if you are allowed to take a street or if it is a oneway street, for example. It can also set a goal, such as a place that you would like to visit in a city, for example. To achieve the goal of designing a reward matrix, the raw data provided by other systems, software, and sensors needs to go through preprocessing. A machine learning program will not provide efficient results if the data has not gone through a standardization process.
[ 27 ]
Building a Reward Matrix – Designing Your Datasets
The reward matrix, R, will be built using a McCullochPitts neuron in TensorFlow. Warehouse management has grown exponentially as ecommerce has taken over many marketing segments. This chapter introduces automated guided vehicles (AGVs), the equivalent of an SDC in a warehouse to store and retrieve products. The challenge in this chapter will be to understand the preprocessing phase in detail. The quality of the processed dataset will influence directly the accuracy of any machine learning algorithm. This chapter covers the following topics: • The McCullochPitts neuron will take the raw data and transform it • Logistic classifiers will begin the neural network process • The logistic sigmoid will squash the values • The softmax function will normalize the values • The onehot function will choose the target for the reward matrix • An example of AGVs in a warehouse The topics form a list of tools that, in turn, form a pipeline that will take raw data and transform it into a reward matrix—an MDP.
Designing datasets – where the dream stops and the hard work begins
As in the previous chapter, bear in mind that a reallife project goes through a threedimensional method in some form or other. First, it's important to think and talk about the problem in need of solving without jumping onto a laptop. Once that is done, bear in mind that the foundation of machine learning and deep learning relies on mathematics. Finally, once the problem has been discussed and mathematically represented, it is time to develop the solution. First, think of a problem in natural language. Then, make a mathematical description of a problem. Only then should you begin the software implementation.
[ 28 ]
Chapter 2
Designing datasets
The reinforcement learning program described in the first chapter can solve a variety of problems involving unlabeled classification in an unsupervised decisionmaking process. The Q function can be applied to drone, truck, or car deliveries. It can also be applied to decision making in games or real life. However, in a reallife case study problem (such as defining the reward matrix in a warehouse for the AGV, for example), the difficulty will be to produce an efficient matrix using the proper features. For example, an AGV requires information coming from different sources: daily forecasts and realtime warehouse flows. The warehouse manages thousands of locations and hundreds of thousands of inputs and outputs. Trying to fit too many features into the model would be counterproductive. Removing both features and worthless data requires careful consideration. A simple neuron can provide an efficient way to attain the standardization of the input data. Machine learning and deep learning are frequently used to preprocess input data for standardization purposes, normalization, and feature reduction.
Using the McCullochPitts neuron
To create the reward matrix, R, a robust model for processing the inputs of the huge volumes in a warehouse must be reduced to a limited number of features. In one model, for example, the thousands to hundreds of thousands of inputs can be described as follows: • Forecast product arrivals with a low priority weight: w1 = 10 • Confirmed arrivals with a high priority weight: w2 = 70 • Unplanned arrivals decided by the sales department: w3 = 75 • Forecasts with a high priority weight: w4 = 60 • Confirmed arrivals that have a low turnover and so have a low weight: w5 = 20
[ 29 ]
Building a Reward Matrix – Designing Your Datasets
The weights have been provided as constants. A McCullochPitts neuron does not modify weights. A perceptron neuron does as we will see beginning with Chapter 8, Solving the XOR Problem with a Feedforward Neural Network. Experience shows that modifying weights is not always necessary. These weights form a vector, as shown here:
𝑤𝑤1 10 𝑤𝑤2 70 𝑥𝑥 = 𝑤𝑤3 = 75 𝑤𝑤4 60 [𝑤𝑤5 ] [20]
Each element of the vector represents the weight of a feature of a product stored in optimal locations. The ultimate phase of this process will produce a reward matrix, R, for an MDP to optimize itineraries between warehouse locations. Let's focus on our neuron. These weights, used through a system such as this one, can attain up to more than 50 weights and parameters per neuron. In this example, 5 weights are implemented. However, in reallife case, many other parameters come into consideration, such as unconfirmed arrivals, unconfirmed arrivals with a high priority, confirmed arrivals with a very low priority, arrivals from locations that probably do not meet security standards, arrivals with products that are potentially dangerous and require special care, and more. At that point, humans and even classical software cannot face such a variety of parameters. The reward matrix will be size 6×6. It contains six locations, A to F. In this example, the six locations, l1 to l6, are warehouse storage and retrieval locations. A 6×6 reward matrix represents the target of the McCullochPitts layer implemented for the six locations. When experimenting, the reward matrix, R, can be invented for testing purposes. In reallife implementations, you will have to find a way to build datasets from scratch. The reward matrix becomes the output of the preprocessing phase. The following source code shows the input of the reinforcement learning program used in the first chapter. The goal of this chapter describes how to produce the following reward matrix that we will be building in the next sections. # R is The Reward Matrix for each location in a warehouse (or any other problem) R = ql.matrix([ [0,0,0,0,1,0], [0,0,0,1,0,1], [0,0,100,1,0,0], [0,1,1,0,1,0], [1,0,0,1,0,0], [0,1,0,0,0,0] ]) [ 30 ]
Chapter 2
For the warehouse example that we are using as for any other domain, the McCullochPitts neuron sums up the weights of the input vector described previously to fill in the reward matrix. Each location will require its neuron, with its weights.
INPUTS > WEIGHTS > BIAS > VALUES • Inputs are the flows in a warehouse or any form of data. • Weights will be defined in this model. • Bias is for stabilizing the weights. Bias does exactly what it means. It will tilt weights. It is very useful as a referee that will keep the weights on the right track. • Values will be the output.
There are as many ways as you can imagine to create reward matrices. This chapter describes one way of doing it that works.
The McCullochPitts neuron
The McCullochPitts neuron dates back to 1943. It contains inputs, weights, and an activation function. Part of the preprocessing phase consists of selecting the right model. The McCullochPitts neuron can represent a given location efficiently. The following diagram shows the McCullochPitts neuron model:
Figure 2.1: The McCullochPitts neuron model
[ 31 ]
Building a Reward Matrix – Designing Your Datasets
This model contains several input x weights that are summed to either reach a threshold that will lead, once transformed, to the output, y = 0, or 1. In this model, y will be calculated in a more complex way. MCP.py written with TensorFlow 2 will be used to illustrate the neuron.
In the following source code, the TensorFlow variables will contain the input values (x), the weights (W), and the bias (b). Variables represent the structure of your graph: # The variables x = tf.Variable([[0.0,0.0,0.0,0.0,0.0]], dtype = tf.float32) W = tf.Variable([[0.0],[0.0],[0.0],[0.0],[0.0]], dtype = tf.float32) b = tf.Variable([[0.0]])
In the original McCullochPitts artificial neuron, the inputs (x) were multiplied by the following weights: 𝑛𝑛
𝑤𝑤1 𝑥𝑥1 + ⋯ + 𝑤𝑤𝑛𝑛 𝑥𝑥𝑛𝑛 = ∑ 𝑤𝑤𝑗𝑗 𝑥𝑥𝑗𝑗 𝑗𝑗=1
The mathematical function becomes a function with the neuron code triggering a logistic activation function (sigmoid), which will be explained in the second part of the chapter. Bias (b) has been added, which makes this neuron format useful even today, shown as follows: # The Neuron def neuron(x, W, b): y1=np.multiply(x,W)+b y1=np.sum(y1) y = 1 / (1 + np.exp(y1)) return y
Before starting a session, the McCullochPitts neuron (1943) needs an operator to set its weights. That is the main difference between the McCullochPitts neuron and the perceptron (1957), which is the model for modern deep learning neurons. The perceptron optimizes its weights through optimizing processes. Chapter 8, Solving the XOR Problem with a Feedforward Neural Network, describes why a modern perceptron was required.
[ 32 ]
Chapter 2
The weights are now provided, and so are the quantities for each input value, which are stored in the x vector at l1, one of the six locations of this warehouse example:
𝑤𝑤1 10 𝑤𝑤2 70 𝑥𝑥 = 𝑤𝑤3 = 75 𝑤𝑤4 60 𝑤𝑤 [ 5 ] [20]
The weight values will be divided by 100, to represent percentages in terms of 0 to 1 values of warehouse flows in a given location. The following code deals with the choice of one location, l1 only, its values, and parameters: # The x_1 = w_t = b_1 =
data [[10, 2, 1., 6., 2.]] [[.1, .7, .75, .60, .20]] [1.0]
The neuron function is called, and the weights (w_t) and the quantities (x_1) of the warehouse flow are entered. Bias is set to 1 in this model. No session needs to be initialized; the neuron function is called: # Computing the value of the neuron value=neuron(x_1,w_t,b_1)
The neuron function neuron will calculate the value of the neuron. The program returns the following value: value for threshold calculation:0.99999
This value represents the activity of location l1 at a given date and a given time. This example represents only one of the six locations to compute. For this location, the higher the value, the closer to 1, the higher the probable saturation rate of this area. This means there is little space left to store products at that location. That is why the reinforcement learning program for a warehouse is looking for the least loaded area for a given product in this model. Each location has a probable availability: A = Availability = 1 – load The probability of a load of a given storage point lies between 0 and 1.
[ 33 ]
Building a Reward Matrix – Designing Your Datasets
High values of availability will be close to 1, and low probabilities will be close to 0, as shown in the following example: >>> print("Availability of location x:{0:.5f}".format( ...
round(availability,5)))
Availability of location x:0.00001
For example, the load of l1 has a probable rounded load of 0.99, and its probable availability is 0.002 maximum. The goal of the AGV is to search and find the closest and most available location to optimize its trajectories. l1 is not a good candidate at that day and time. Load is a keyword in production or service activities. The less available resources have the highest load rate. When all six locations' availabilities have been calculated by the McCullochPitts neuron—each with its respective quantity inputs, weights, and bias—a location vector of the results of this system will be produced. Then, the program needs to be implemented to run all six locations and not just one location through a recursive use of the one neuron model:
A(L) = {a(l1),a(l2),a(l3),a(l4),a(l5),a(l6)} The availability, 1 – output value of the neuron, constitutes a sixline vector. The following vector, lv, will be obtained by running the previous sample code on all six locations.
0.0002 0.2 0.9 𝑙𝑙𝑣𝑣 = 0.0001 0.4 [ 0.6 ]
As shown in the preceding formula, lv is the vector containing the value of each location for a given AGV to choose from. The values in the vector represent availability. 0.0002 means little availability; 0.9 means high availability. With this choice, the MDP reinforcement learning program will optimize the AGV's trajectory to get to this specific warehouse location. The lv is the result of the weighing function of six potential locations for the AGV. It is also a vector of transformed inputs.
[ 34 ]
Chapter 2
The PythonTensorFlow architecture
Implementation of the McCullochPitts neuron can best be viewed as shown in the following graph:
Figure 2.2: Implementation of the McCullochPitts neuron
A data flow graph will also help optimize a program when things go wrong as in classical computing.
Logistic activation functions and classifiers
Now that the value of each location of L = {l1, l2, l3, l4, l5, l6} contains its availability in a vector, the locations can be sorted from the most available to the least available location. From there, the reward matrix, R, for the MDP process described in Chapter 1, Getting Started with NextGeneration Artifcial Intelligence through Reinforcement Learning, can be built.
Overall architecture
At this point, the overall architecture contains two main components: 1. Chapter 1: A reinforcement learning program based on the valueaction Q function using a reward matrix that will be finalized in this chapter. The reward matrix was provided in the first chapter as an experiment, but in the implementation phase, you'll often have to build it from scratch. It sometimes takes weeks to produce a good reward matrix.
[ 35 ]
Building a Reward Matrix – Designing Your Datasets
2. Chapter 2: Designing a set of 6×1 neurons that represents the flow of products at a given time at six locations. The output is the availability probability from 0 to 1. The highest value indicates the highest availability. The lowest value indicates the lowest availability. At this point, there is some reallife information we can draw from these two main functions through an example: • An AGV is automatically moving in a warehouse and is waiting to receive its next location to use an MDP, to calculate the optimal trajectory of its mission. • An AGV is using a reward matrix, R, that was given during the experimental phase but needed to be designed during the implementation process. • A system of six neurons, one per location, weighing the real quantities and probable quantities to give an availability vector, lv, has been calculated. It is almost ready to provide the necessary reward matrix for the AGV. To calculate the input values of the reward matrix in this reinforcement learning warehouse model, a bridge function between lv and the reward matrix, R, is missing. That bridge function is a logistic classifier based on the outputs of the n neurons that all perform the same tasks independently or recursively with one neuron. At this point, the system: • Took corporate data • Used n neurons calculated with weights • Applied an activation function The activation function in this model requires a logistic classifier, a commonly used one.
Logistic classifier
The logistic classifier will be applied to lv (the six location values) to find the best location for the AGV. This method can be applied to any other domain. It is based on the output of the six neurons as follows:
input × weight + bias What are logistic functions? The goal of a logistic classifier is to produce a probability distribution from 0 to 1 for each value of the output vector. As you have seen so far, artificial intelligence applications use applied mathematics with probable values, not raw outputs. [ 36 ]
Chapter 2
The main reason is that machine learning/deep learning works best with standardization and normalization for workable homogeneous data distributions. Otherwise, the algorithms will often produce underfitted or overfitted results. In the warehouse model, for example, the AGV needs to choose the best, most probable location, li. Even in a wellorganized corporate warehouse, many uncertainties (late arrivals, product defects, or some unplanned problems) reduce the probability of a choice. A probability represents a value between 0 (low probability) and 1 (high probability). Logistic functions provide the tools to convert all numbers into probabilities between 0 and 1 to normalize data.
Logistic function
The logistic sigmoid provides one of the best ways to normalize the weight of a given output. The activation function of the neuron will be the logistic sigmoid. The threshold is usually a value above which the neuron has a y = 1 value; or else it has a y = 0 value. In this model, the minimum value will be 0. The logistic function is represented as follows:
1 1 + 𝑒𝑒 −𝑥𝑥
• e represents Euler's number, or 2.71828, the natural logarithm. • x is the value to be calculated. In this case, s is the result of the logistic sigmoid function. The code has been rearranged in the following example to show the reasoning process that produces the output, y, of the neuron: y1=np.multiply(x,W)+b y1=np.sum(y1) y = 1 / (1 + np.exp(y1)) #logistic Sigmoid
Thanks to the logistic sigmoid function, the value for the first location in the model comes out squashed between 0 and 1 as 0.99, indicating a high probability that this location will be full. To calculate the availability of the location once the 0.99 value has been taken into account, we subtract the load from the total availability, which is 1, as follows:
Availability = 1 – probability of being full (value) Or availability = 1 – value [ 37 ]
Building a Reward Matrix – Designing Your Datasets
As seen previously, once all locations are calculated in this manner, a final availability vector, lv, is obtained.
0.0002 0.2 0.9 𝑙𝑙𝑣𝑣 = 0.0001 0.4 [ 0.6 ]
When analyzing lv, a problem has stopped the process. Individually, each line appears to be fine. By applying the logistic sigmoid to each output weight and subtracting it from 1, each location displays a probable availability between 0 and 1. However, the sum of the lines in lv exceeds 1. That is not possible. A probability cannot exceed 1. The program needs to fix that. Each line produces a [0, 1] solution, which fits the prerequisite of being a valid probability. In this case, the vector lv contains more than one value and becomes a probability distribution. The sum of lv cannot exceed 1 and needs to be normalized. The softmax function provides an excellent method to normalize lv. Softmax is widely used in machine learning and deep learning. Bear in mind that mathematical tools are not rules. You can adapt them to your problem as much as you wish as long as your solution works.
Softmax
The softmax function appears in many artificial intelligence models to normalize data. Softmax can be used for classification purposes and regression. In our example, we will use it to find an optimized goal for an MDP. In the case of the warehouse example, an AGV needs to make a probable choice between six locations in the lv vector. However, the total of the lv values exceeds 1. lv requires normalization of the softmax function, S. In the source code, the lv vector will be named y.
𝑆𝑆(𝑦𝑦𝑖𝑖 ) =
𝑒𝑒 𝑦𝑦𝑖𝑖
∑𝑛𝑛𝑗𝑗=1 𝑒𝑒 𝑦𝑦𝑖𝑖
[ 38 ]
Chapter 2
The following code used is SOFTMAX.py. 1. y represents the lv vector: # y is the vector of the scores of the lv vector in the warehouse example: y = [0.0002, 0.2, 0.9,0.0001,0.4,0.6]
2. 𝑒𝑒 𝑦𝑦𝑖𝑖 is the exp(i) result of each value in y (lv in the warehouse example), as follows: y_exp = [math.exp(i) for i in y] 𝑛𝑛
3. ∑
𝑒𝑒 𝑦𝑦𝑖𝑖 is the sum of 𝑒𝑒 𝑦𝑦𝑖𝑖 as shown in the following code:
𝑗𝑗=1
sum_exp_yi = sum(y_exp)
Now, each value of the vector is normalized by applying the following function: softmax = [round(i / sum_exp_yi, 3) for i in y_exp]
0.0002 0.111 0.2 0.135 0.9 0.273 𝑙𝑙𝑣𝑣 = → softmax(𝑙𝑙𝑣𝑣 ) → 0.0001 0.111 0.4 0.165 [ 0.6 ] [0.202]
softmax(lv) provides a normalized vector with a sum equal to 1, as shown in this compressed version of the code. The vector obtained is often described as containing logits. The following code shows one version of a softmax function: def softmax(x): return np.exp(x) / np.sum(np.exp(x), axis=0)
lv is now normalized by softmax(lv) as follows.
0.0002 0.111 0.2 0.135 0.9 0.273 𝑙𝑙𝑣𝑣 = → softmax(𝑙𝑙𝑣𝑣 ) → 0.0001 0.111 0.4 0.165 [ 0.6 ] [0.202] [ 39 ]
Building a Reward Matrix – Designing Your Datasets
The last part of the softmax function requires softmax(lv) to be rounded to 0 or 1. The higher the value in softmax(lv), the more probable it will be. In clearcut transformations, the highest value will be close to 1, and the others will be closer to 0. In a decisionmaking process, the highest value needs to be established as follows: print("7C. Finding the highest value in the normalized y vector : ",ohot)
The output value is 0.273 and has been chosen as the most probable location. It is then set to 1, and the other, lower values are set to 0. This is called a onehot function. This onehot function is extremely helpful for encoding the data provided. The vector obtained can now be applied to the reward matrix. The value 1 probability will become 100 in the R reward matrix, as follows:
0.0002 0.111 0 0 0.2 0.135 0 0 0.9 0.273 1 100 𝑙𝑙𝑣𝑣 = → softmax(𝑙𝑙𝑣𝑣 ) → → onehot(𝑙𝑙𝑣𝑣 ) → → 𝑅𝑅 → 0.0001 0.111 0 0 0.4 0.165 0 0 [ 0.6 ] [0.202] [0] [ 0 ]
The softmax function is now complete. Location l3 or C is the best solution for the AGV. The probability value is multiplied by 100, and the reward matrix, R, can now receive the input. Before continuing, take some time to play around with the values in the source code and run it to become familiar with the softmax function.
We now have the data for the reward matrix. The best way to understand the mathematical aspect of the project is to draw the result on a piece of paper using the actual warehouse layout from locations A to F. Locations={l1A, l2B, l3C, l4D, l5E, l6F}
Line C of the reward matrix ={0, 0, 100, 0, 0, 0}, where C (the third value) is now the target for the selfdriving vehicle, in this case, an AGV in a warehouse.
Figure 2.3: Illustration of a warehouse transport problem
[ 40 ]
Chapter 2
We obtain the following reward matrix, R, described in Chapter 1, Getting Started with NextGeneration Artificial Intelligence through Reinforcement Learning: State/values
A
B
C
D
E
F
A




1

B



1

1
C


100
1


D

1
1

1

E
1


1


F

1




This reward matrix is exactly the one used in the Python reinforcement learning program using the Q function from Chapter 1. The output of this chapter is thus the input of the R matrix. The 0 values are there for the agent to avoid those values. The 1 values indicate the reachable cells. The 100 in the C×C cell is the result of the softmax output. This program is designed to stay close to probability standards with positive values, as shown in the following R matrix taken from the mdp01.py of Chapter 1: R = ql.matrix([ [0,0,0,0,1,0], [0,0,0,1,0,1], [0,0,100,1,0,0], [0,1,1,0,1,0], [1,0,0,1,0,0], [0,1,0,0,0,0] ])
At this point: • The output of the functions in this chapter generated a reward matrix, R, which is the input of the MDP described in Chapter 1, Getting Started with NextGeneration Artificial Intelligence through Reinforcement Learning. • The MDP process was set to run for 50,000 episodes in Chapter 1. • The output of the MDP has multiple uses, as we saw in this chapter and Chapter 1. The building blocks are in place to begin evaluating the execution and performances of the reinforcement learning program, as we will see in Chapter 3, Machine Intelligence – Evaluation Functions and Numerical Convergence.
[ 41 ]
Building a Reward Matrix – Designing Your Datasets
Summary
Using a McCullochPitts neuron with a logistic activation function in a onelayer network to build a reward matrix for reinforcement learning shows one way to preprocess a dataset. Processing reallife data often requires a generalization of a logistic sigmoid function through a softmax function, and a onehot function applied to logits to encode the data. Machine learning functions are tools that must be understood to be able to use all or parts of them to solve a problem. With this practical approach to artificial intelligence, a whole world of projects awaits you. This neuronal approach is the parent of the multilayer perceptron that will be introduced starting in Chapter 8, Solving the XOR Problem with a Feedforward Neural Network. This chapter went from an experimental black box machine learning and deep learning to white box implementation. Implementation requires a full understanding of machine learning algorithms that often require finetuning. However, artificial intelligence goes beyond understanding machine learning algorithms. Machine learning or deep learning require evaluation functions. Performance or results cannot be validated without evaluation functions, as explained in Chapter 3, Machine Intelligence – Evaluation Functions and Numerical Convergence. In the next chapter, the evaluation process of machine intelligence will be illustrated by examples that show the limits of human intelligence and the rise of machine power.
[ 42 ]
Chapter 2
Questions
1. Raw data can be the input to a neuron and transformed with weights. (Yes  No)
2. Does a neuron require a threshold? (Yes  No) 3. A logistic sigmoid activation function makes the sum of the weights larger. (Yes  No) 4. A McCullochPitts neuron sums the weights of its inputs. (Yes  No) 5. A logistic sigmoid function is a log10 operation. (Yes  No) 6. A logistic softmax is not necessary if a logistic sigmoid function is applied to a vector. (Yes  No) 7. A probability is a value between –1 and 1. (Yes  No)
Further reading
• The original McCullochPitts neuron 1943 paper: http://www.cse. chalmers.se/~coquand/AUTOMATA/mcp.pdf
• TensorFlow variables: https://www.tensorflow.org/beta/guide/ variables
[ 43 ]
3 Machine Intelligence – Evaluation Functions and Numerical Convergence Two issues appear when a reward matrix (R)driven MDP produces results. These issues can be summed up in two principles. Principle 1: AI algorithms often surpass humans in classification, prediction, and decisionmaking areas. The key executive function of human intelligence, decisionmaking, relies on the ability to evaluate a situation. No decision can be made without measuring the pros and cons and factoring the parameters. Humanity takes great pride in its ability to evaluate. However, in many cases, a machine can do better. Chess represents our pride in our thinking ability. A chessboard is often present in movies to symbolize human intelligence. Today, not a single chess player can beat the best chess engines. One of the extraordinary core capabilities of a chess engine is the evaluation function; it takes many parameters into account more precisely than humans. Principle 2: Principle 1 leads to a very tough consequence. It is sometimes possible and other times impossible for a human to verify the results that an AI algorithm produces, let alone ensemble metaalgorithms.
[ 45 ]
Machine Intelligence – Evaluation Functions and Numerical Convergence
Principle 1 has been difficult to detect because of the media hype surrounding face and object recognition. It is easy for a human to check whether the face or object the ML algorithm was supposed to classify was correctly classified. However, in a decisionmaking process involving many features, principle 2 rapidly appears. In this chapter, we will identify what results and convergence to measure and decide how to measure it. We will also explore measurement and evaluation methods. This chapter covers the following topics: • Evaluation of the episodes of a learning session • Numerical convergence measurements • An introduction to numerical gradient descent • Decision tree supervised learning as an evaluation method The first thing is to set evaluation goals. To do this, we will decide what to measure and how.
Tracking down what to measure and deciding how to measure it
We will now tackle the tough task of finding the factors that can make a system go wrong. The model built in the previous chapters can be summed up as follows: 0.0002 0.111 0 0 0.2 0.135 0 0 0.9 0.273 1 100 𝑙𝑙𝑣𝑣 = → softmax(𝑙𝑙𝑣𝑣 ) → → onehot → → 𝑅𝑅 → → gamma → 𝑄𝑄 → Results 0.0001 0.111 0 0 0.4 0.165 0 0 [ 0.6 ] [0.202] [0] [ 0 ]
From lv, the availability vector (capacity in a warehouse, for example), to R, the process creates the reward matrix from the raw data (Chapter 2, Building a Reward Matrix – Designing Your Datasets) required for the MDP reinforcement learning program (Chapter 1, Getting Started with NextGeneration Artificial Intelligence through Reinforcement Learning). As described in the previous chapter, a softmax(lv) function is applied to lv. In turn, a onehot(softmax(lv)) is applied, which is then converted into the reward value R, which will be used for the Q (Qlearning) algorithm.
[ 46 ]
Chapter 3
The MDPdriven Bellman equation then runs from reading R (the reward matrix) to the results. Gamma is the learning parameter, Q is the Qlearning function, and the results represent the final value of the states of the process. The parameters to be measured are as follows: • The company's input data. Readytouse datasets such as MNIST are designed to be efficient for an exploration phase. These readymade datasets often contain some noise (unreliable data) to make them realistic. The same process must be achieved with raw company data. The only problem is that you cannot download a corporate dataset from somewhere. You have to build timeconsuming datasets. • The weights and biases that will be applied. • The activation function (a logistic function or other). • The choices to make after the onehot process. • The learning parameter. • Episode management through convergence. • A verification process through interactive random checks and independent algorithms such as supervised learning to control unsupervised algorithms. In reallife company projects, a system will not be approved until tens of thousands of results have been produced. In some cases, a corporation will approve the system only after hundreds of datasets with millions of data samples have been tested to be sure that all scenarios are accurate. Each dataset represents a scenario that consultants can work on with parameter scripts. The consultant introduces parameter scenarios that are tested by the system and measured. In decisionmaking systems with up to 200 parameters, a consultant will remain necessary for months in an industrial environment. A reinforcement learning program will be on its own to calculate events. However, even then, consultants are needed to manage the hyperparameters. In reallife systems, with high financial stakes, quality control will always remain essential. Measurement should thus apply to generalization more than simply applying to a single or few datasets. Otherwise, you will have a natural tendency to control the parameters and overfit your model in a toogoodtobetrue scenario.
[ 47 ]
Machine Intelligence – Evaluation Functions and Numerical Convergence
For example, say you wake up one morning and look at the sky. The weather is clear, the sun is shining, and there are no clouds. The next day, you wake up and you see the same weather. You write this down in a dataset and send it off to a customer for weather prediction. Every time the customer runs the program, it predicts clear sunny skies! That what overfitting leads to! This explains why we need large datasets to fully understand how to use an algorithm or illustrate how a machine learning program works. Beyond the reward matrix, the reinforcement program in the first chapter had a learning parameter 𝜆𝜆 = 0.8, shown in mdp03.py, which is used for this section: # Gamma: It's a form of penalty or uncertainty for learning # If the value is 1, the rewards would be too high. # This way the system knows it is learning. gamma = 0.8
The 𝜆𝜆 learning parameter in itself needs to be closely monitored because it introduces uncertainty into the system. This means that the learning process will always remain a probability, never a certainty. One might wonder why this parameter is not just taken out. Paradoxically, that will lead to even more global uncertainty. The more the 𝜆𝜆 learning parameter tends to 1, the more you risk overfitting your results. Overfitting means that you are pushing the system to think it's learning well when it isn't. It's exactly like a teacher who gives high grades to everyone in the class all the time. The teacher would be overfitting the gradestudent evaluation process, and nobody would know whether the students have learned anything. The results of the reinforcement program need to be measured as they go through episodes. The range of the learning process itself must be measured. All of these measurements will have a significant effect on the results obtained. The best way to start is by measuring the quality of convergence of the system. If the system provides good convergence, you might avoid the headache of having to go back and check everything.
Convergence
Convergence measures the distance between the present state of a training session and the goal of the training session. In a reinforcement learning program, an MDP, for example, there is no training data, so there is no target data to compare to.
[ 48 ]
Chapter 3
However, two methods are available: 1. Implicit convergence: In this case, we run the training for a large number of episodes, 50,000, for example. We know through trial and error that the program will reach a solution by then. 2. Numerically controlled gradient descent: We measure the training progress at each episode and stop when it is safe to do so.
Implicit convergence
In the last part of mdp01.py in the first chapter, a range of 50,000 was implemented. In this chapter, we will run mdp03.py. In the last part of mdp01.py, the idea was to set the number of episodes at such a level that meant that convergence was certain. In the following code, the range (50000) is a constant: for i in range(50000): current_state = ql.random.randint(0, int(Q.shape[0])) PossibleAction = possible_actions(current_state) action = ActionChoice(PossibleAction) reward(current_state,action,gamma)
Convergence, in this case, will be defined as the point at which no matter how long you run the system, the Q result matrix will not change anymore. By setting the range to 50000, you can test and verify this. As long as the reward matrices remain homogeneous, this will work. If the reward matrices strongly vary from one scenario to another, this model will produce unstable results. Try to run the program with different ranges. Lower the ranges until you see that the results are not optimal.
Numerically controlled gradient descent convergence
In this section, we will use mdp03.py, a modified version of mdp01.py explored in Chapter 1, with an additional function: numerically controlled gradient descent. Letting the MDP train for 50,000 will produce a good result but consume unnecessary CPU. Using a numerically controlled gradient descent evaluation function will save a lot of episodes. Let's see how many.
[ 49 ]
Machine Intelligence – Evaluation Functions and Numerical Convergence
First, we need to define the gradient descent function based on a derivative. Let's have a quick review of what a derivative is.
𝑓𝑓(𝑥𝑥 + ℎ) − 𝑓𝑓(𝑥𝑥) ℎ
h is the value of the step of the function. Imagine that h represents each line of a bank account statement. If we read the statement line by line, h = 1. If we read two lines at a time, h = 2. Reading the present line of the bank account statement = f(x) = a certain amount. When you read the next line of the bank account, the function is (f + h) = the amount after f(x). If you had 100 units of currency in your bank account at f(x) and spent 10 units of currency, on the next line, x + h, you would have f(x + h) = 90 units of currency left. The gradient provides the direction of your slope: up, down, or constant. In this case, we can say that the slope, the gradient, is doing downward, as shown in the following graph, which illustrates the decreasing values of y(cost, loss) as x increases (training episodes):
Figure 3.1: Plotting the decreasing cost/loss values as training episodes increase
We also need to know by how much your bank account is changing – how much the derivative is worth. In this case, derivative means by how much the balance of your bank account is changing on each line of your bank statement. In this case, you spent 10 units of currency in one bank statement line, so the derivative at this value of x (line in your bank account) = –10. [ 50 ]
Chapter 3
In the following code of the Bellman equation as seen in Chapter 1, Getting Started with NextGeneration Artifcial Intelligence through Reinforcement Learning, the step of the loop is 1 as well: for i in range(sec):
Since i = 1, h = 1 in our gradient descent calculation can be simplified:
𝑓𝑓(𝑥𝑥 + 1) − 𝑓𝑓(𝑥𝑥) = 𝑓𝑓(𝑥𝑥 + 1) − 𝑓𝑓(𝑥𝑥) 1
We now define f(x) in the following code: conv=Q.sum()
conv is the sum of the 6×6 Q matrix that is slowly filling up as the MDP training progresses. Thus f(x) = conv=Q.sum() = sum of Q. The function adds up all the values in Q to have a precise value of the state of the system at each i.
f(x) = the state of the system at i – 1 f(x + 1) is the value of the system at i: Q.sum()
We must remember that the Q matrix is increasing progressively as the MDP process continues to train. We measure the distance between two steps, h. This distance will decrease. Now we have: f(x + 1) – f(x) = Q.sum()+conv • First we implement additional variables for our evaluation function, which uses gradient descent at line 83 of mdp01.py: ci=0 # convergence counter which counts the number of episodes conv=0 # sum of Q at state 1 and then every x episodes nc=1 # numerical convergence activated to perform numericalcontrolled gradient descent xi=100 # xi episode optimizer: stop as soon as convergence reached + xix(unknown) sec=2500 # security number of episodes for this matrix size brought down from 50,000 to 2,500 cq=ql.zeros((2500, 1))
• nc=1 activates the evaluation function, and ci begins to count the episodes it will take with this function: for i in range(sec): [ 51 ]
Machine Intelligence – Evaluation Functions and Numerical Convergence current_state = ql.random.randint(0, int(Q.shape[0])) PossibleAction = possible_actions(current_state) action = ActionChoice(PossibleAction) reward(current_state,action,gamma) ci+=1 # convergence counter incremented by 1 at each state if(nc==1): # numerical convergence activated
• At the first episode, i==1, f(x)= Q.sum() as planned: if(i==1): conv=Q.sum()
# at state one, conv is activated # conv= the sum of Q
• f(x + 1) = Q.sum()+conv is applied: print("Episode",i,"Local derivative:",Q.sum()+conv,...
• The distance, the absolute value of the derivative, is displayed and stored because we will be using it to plot a figure with Matplotlib: print(... "Numerical Convergence value estimator", Q.sum()conv) cq[i][0]=Q.sum()conv
• xi=100 plays a critical role in this numerically controlled gradient descent function. Every xi, the process stops to check the status of the training process: if(ci==xi):
# every 100 episodes the system checks to see...
There are two possible cases: a) and b). Case a) As long as the local derivative is >0 at each episode, the MDP continues its training process: if(conv!=Q.sum()): # if the sum of Q changes... conv=Q.sum() # ...the training isn't over, conv is updated ci=0
# ...the convergence counter is set to
O
[ 52 ]
Chapter 3
The output will display varying local derivatives: Episode 1911 Local derivative: 9.094947017729282e13 Numerical Convergence value estimator 9.094947017729282e13 Episode 1912 Local derivative: 9.094947017729282e13 Numerical Convergence value estimator 9.094947017729282e13 Episode 1913 Local derivative: 1.3642420526593924e12 Numerical Convergence value estimator 1.3642420526593924e12
Case b) When the derivative value reaches a constant value for xi episodes, the MDP has been trained and the training can now stop: if(conv==Q.sum()): # ...if the sum of Q has changed print(i,conv,Q.sum()) # ...if it hasn't the training is over break
# ...the system stops training
The output will display a constant derivate, xi, before the training stops: Episode 2096 Local derivative: 0.0 Numerical Convergence value estimator 0.0 Episode 2097 Local derivative: 0.0 Numerical Convergence value estimator 0.0 Episode 2098 Local derivative: 0.0 Numerical Convergence value estimator 0.0 Episode 2099 Local derivative: 0.0 Numerical Convergence value estimator 0.0
When the training is over, the number of training episodes is displayed: number of episodes: 2099
2,099 is a lot less than the 50,000 implicit convergence episodes, which proves the efficiency of this numerically controlled gradient descent method.
[ 53 ]
Machine Intelligence – Evaluation Functions and Numerical Convergence
At the end of the learning process, you can display a Matplotlib figure containing the convergence level of each episode that we had stored in cq=ql.zeros((2500, 1)): cq[i][0]=Q.sum()conv
The figure is displayed with a few lines of code: import matplotlib.pyplot as plt plt.plot(cq) plt.xlabel('Episodes') plt.ylabel('Convergence Distances') plt.show()
Figure 3.2: A plot demonstrating numerical convergence
This graph shows the numerical convergence. As you can see in the graph, the cost or loss decreases as the number of training episodes increases, as explained earlier in this chapter. Please note the following properties of this gradient descent method: • The number of episodes will vary from one training session to another because the MDP is a random process. • The training curve at local episodes is sometimes erratic because of the random nature of the training process. Sometimes, the curve will go up instead of down locally. In the end, it will reach 0 and stay there.
[ 54 ]
Chapter 3
• If the training curve increases locally, there is nothing you can do. An MDP does no backpropagation to modify weights, parameters, or strategies, as we will see when we look at artificial neural networks (ANNs), for example, in Chapter 8, Solving the XOR Problem with a Feedforward Neural Network. No action is required in an MDP process. You can try to change the learning rate or go back and check your reward matrix and the preprocessing phase implemented on the raw datasets. • If the training curve does not reach 0 and stay there, check the learning parameters, the reward matrix, and the preprocessing phase implemented on the raw datasets. You might even have to go back and check the noise (defective data or missing data) in the initial datasets. Once the MDP training is over, do some random tests using the functionality provided at line 145 and explained in Chapter 1: origin=int(input("index number origin(A=0,B=1,C=2,D=3,E=4,F=5): "))
For example, when prompted for an input, enter 1 and see if the result is correct, as shown in the following output: index number origin(A=0,B=1,C=2,D=3,E=4,F=5): 1 …/… print("Path:") > B > D > C
This random test verification method will work efficiently with a relatively small reward matrix. However, this approach will prove difficult with a size 25×25 reward matrix, for example. The machine easily provides a result. But how can we evaluate it? In that case, we have reached the limit of human analytic capacity. In the preceding code, we entered a starting point and obtained an answer. With a small reward matrix, it is easy to visually check and see if the answer is correct. When analyzing 25 × 25 = 625 cells, it would take days to verify the results. For the record, bear in mind that when Andrey Markov invented his approach over 100 years ago, he used a pen and paper! However, we have computers, so we must use an evaluation algorithm to evaluate the results of our MDP process. The increasing volumes of data and parameters in a global world have made it impossible for humans to outperform the evergrowing intelligence of machines.
[ 55 ]
Machine Intelligence – Evaluation Functions and Numerical Convergence
Evaluating beyond human analytic capacity
An efficient manager has a high evaluation quotient. A machine often has a better one in an increasing number of fields. The problem for a human is to understand the evaluation machine intelligence has produced. Sometimes a human will say "that's a good machine thinking result" or "that's a bad result," without being able to explain why or determine whether there is a better solution. Evaluation is one of the major keys to efficient decisionmaking in all fields: from chess, production management, rocket launching, and selfdriving cars to data center calibration, software development, and airport schedules. We'll explore a chess scenario to illustrate the limits of human evaluation. Chess engines are not highlevel deep learningbased software. They rely heavily on evaluations and calculations. They evaluate much better than humans, and there is a lot to learn from them. The question now is to know whether any human can beat a chess engine or not. The answer is no. To evaluate a position in chess, you need to examine all the pieces, their quantitative value, their qualitative value, the cooperation between pieces, who owns each of the 64 squares, the king's safety, the bishop pairs, the knight positioning, and many other factors. Evaluating a position in a chess game shows why machines are surpassing humans in quite a few decisionmaking fields. The following scenario is after move 23 in the KramnikBluebaum 2017 game. It cannot be correctly evaluated by humans. It contains too many parameters to analyze and too many possibilities.
[ 56 ]
Chapter 3
Figure 3.3: Chess example scenario
It is white's turn to play, and a close analysis shows that both players are lost at this point. In a tournament like this, they must each continue to keep a poker face. They often look at their position with a confident face to hide their dismay. Some even shorten their thinking time to make their opponent think they know where they are going. These unsolvable positions for humans are painless to solve with chess engines, even for cheap, highquality chess engines on a smartphone. This can be generalized to all human activity that has become increasingly complex, unpredictable, and chaotic. Decisionmakers will increasingly rely on AI to help them make the right choices. No human can play chess and evaluate the way a chess engine does by simply calculating the positions of the pieces, their squares of liberty, and many other parameters. A chess engine generates an evaluation matrix with millions of calculations.
[ 57 ]
Machine Intelligence – Evaluation Functions and Numerical Convergence
The following table is the result of an evaluation of only one position among many others (real and potential). Position evaluated
0,3
White
34 Initial Position Value position
Quality Total Value Value
Pawn
a2
a2
1
a2b2 small pawn island
0,05
1,05
Pawn
b2
b2
1
a2b2 small pawn island
0,05
1,05
Pawn
c2
x
0
Captured
0
0
Pawn
d2
d4
1
Occupies center, defends Be5
0,25
1,25
Pawn
e2
e2
1
Defends Qf3
0,25
1,25
Pawn
f2
x
0
Captured
0
0
Pawn
g2
g5
1
Unattacked, attacking 2 squares
0,3
1,3
Pawn
h2
h3
1
Unattacked, defending g4
0,1
1,1
Rook
a1
c1
5
Occupying cfile, attacking 1 b7 with Nd5Be5
6
Knight
b1
d5
3
Attacking Nb6, 8 squares
0,5
3,5
BishopDS
c1
e5
3
Central position, 10 squares, attacking c7
0,5
3,5
Queen
d1
f3
9
Battery with Bg2, defending Ne5, XRay b7
1
11
King
e1
h1
0
Xrayed by Bb6 on a7g1 diagonal
0,5
0,5
BishopWS
f1
g2
3
Supporting Qf3 in defense and attack
0,5
3,5
Knight
g1
x
0
Captured
0
0
Rook
h1
x
0
Captured
0
0
5
34
29
White: 34
The value of the position of white is 34. White
34
Black
33,7 [ 58 ]
Chapter 3
Initial position
Position
Value
Quality Value
Total Value
Pawn
a7
a7
1
a7b7 small pawn island
0,05
1,05
Pawn
b7
b7
1
a7b7 small pawn island
0,05
1,05
Pawn
c7
Pawn
d7
x
0
Captured
0
0
x
0
Captured
0
0
Pawn
e7
f5
1
Doubled, 2 squares
0
1
Pawn
f7
f7
1
0
1
Pawn
g7
g6
1
Defending f5 but abandoning Kg8
0
1
Pawn
h7
h5
1
Well advanced with f5,g6
0,1
1,1
Rook
a8
d8
5
Semiopen dfile attacking Nd5
2
7
Knight
b8
x
0
Captured
0
0
BishopDS
c8
b6
3
Attacking d4, 3 squares
0,5
3,5
Queen
d8
e6
9
Attacking d4,e5, a bit cramped
1,5
10,5
King
e8
g8
0
f6,h6, g7,h8 attacked
1
1
BishopWS
f8
x
0
Captured, white lost bishop pair
0,5
0,5
Knight
g8
e8
3
Defending c7,f6,g7
1
4
Rook
h8
f8
5
Out of play
2
3
2,7
Black: 33,7
31
The value of black is 33.7. So white is winning by 34 – 33.7 = 0.3. The evaluation system can easily be represented with two McCullochPitts neurons, one for black and one for white. Each neuron would have 30 weights = {w1,w2 … w30}, as shown in the previous table. The sum of both neurons requires an activation function that converts the evaluation into 1/100th of a pawn, which is the standard measurement unit in chess. Each weight will be the output of squares and piece calculations. Then the MDP can be applied to Bellman's equation with a random generator of possible positions. Presentday chess engines contain this type of brute calculation approach. They don't need more to beat humans. [ 59 ]
Machine Intelligence – Evaluation Functions and Numerical Convergence
No human, not even world champions, can calculate these positions with this accuracy. The number of parameters to take into account overwhelms them each time they reach positions like these. They then play more or less randomly with a possibly good idea in mind. The chances of success against a chess engine resemble a lottery sometimes. Chess experts discover this when they run humanplayed games with powerful chess engines to see how the game plays out. The players themselves now tend to reveal their incapacity to provide a deep analysis when asked why they made a questionable move. It often takes hours to go through a game, its combinations and find the reasons of a bad move. In the end, the players will often use a machine to help them understand what happened. The positions analyzed here represent only one possibility. A chess engine will test millions of possibilities. Humans can test only a few. Measuring a result like this has nothing to do with natural human thinking. Only machines can think like that. Not only do chess engines solve the problem, but they are also impossible to beat. Principle 1: At one point, there are problems humans face that only machines can solve. Principle 2: Sometimes, it will be possible to verify the result of an ML system, sometimes not. However, we must try to find ways to check the result. One solution to solve the problem of principle 2 is to verify an unsupervised algorithm with a supervised algorithm through random samples.
Using supervised learning to evaluate a result that surpasses human analytic capacity
More often than not, an AI solution exceeds a human's capacity to analyze a situation in detail. It is often too difficult for a human to understand the millions of calculations a machine made to reach a conclusion and explain it. To solve that problem, another AI, ML, or DL algorithm will provide assisted AI capability. Let's suppose the following: • The raw data preprocessed by the neural approach of Chapter 2, Building a Reward Matrix – Designing Your Datasets, works fine. The reward matrix looks fine.
[ 60 ]
Chapter 3
• The MDPdriven Bellman equation provides good reinforcement training results. • The convergence function and values work. • The results on this dataset look satisfactory but the results are questioned. A manager or user will always come up with a killer question: how can you prove that this will work with other datasets in the future and confirm 100% that the results are reliable? The only way to be sure that this whole system works is to run thousands of datasets with hundreds of thousands of product flows. The idea now is to use supervised learning to create an independent way of checking the results. One method is to use a decision tree to visualize some key aspects of the solution and be able to reassure the users and yourself that the system is reliable. Decision trees provide a white box approach with powerful functionality. In this section, we will limit the exploration to an intuitive approach. In Chapter 5, How to Use Decision Trees to Enhance KMeans Clustering, we will go into the theory of decision trees and random trees and explore more complex examples. In this model, the features of the input are analyzed so that we can classify them. The analysis can be transformed into decision trees depending on realtime data, to create a distribution representation to predict future outcomes. For this section, you can run the following program: Decision_Tree_Priority_classifier.py
Or the following Jupyter notebook on Google Colaboratory: DTCH03.ipynb
Google Colaboratory might have the two following packages installed: import collections # from Python library container datatypes import pydotplus # a Python Interface to Graphviz's Dot language.(dotV command line
This could help you avoid installing them locally, which might take some time if you get a Graphviz requirement message.
[ 61 ]
Machine Intelligence – Evaluation Functions and Numerical Convergence
Both programs produce the same decision tree image: warehouse_example_decision_tree.png
The intuitive description of this decision tree approach runs in 5 steps: Step 1: Represent the features of the incoming orders to store in a warehouse – for example: features = [ 'Priority/location', 'Volume', 'Flow_optimizer' ]
In this case, we will limit the model to three properties: • Priority/location, which is the most important property in a warehouse flow in this model • Volumes to transport • Optimizing priority – the financial and customer satisfaction property Step 2: Provide priority labels for the learning dataset: Y = ['Low', 'Low', 'High', 'High', 'Low', 'Low']
Step 3: Providing the dataset input matrix, which is the output matrix of the reinforcement learning program. The values have been approximated but are enough to run the model. They simulate some of the intermediate decisions and transformations that occur during the decision process (ratios applied, uncertainty factors added, and other parameters). The input matrix is X: X = [[256, [320, [500, [400, [320, [256,
1,0], 1,0], 1,1], 1,0], 1,0], 1,0]]
The features in step 1 apply to each column. The values in step 2 apply to every line. The values of the third column [0,1] are discrete indicators for the training session. Step 4: Run a standard decision tree classifier. This classifier will distribute the representations (distributed representations) into two categories: • The properties of highpriority orders • The properties of lowpriority orders [ 62 ]
Chapter 3
There are many types of algorithms. In this case, a standard sklearn function is called to do the job, as shown in the following source code: classify = tree.DecisionTreeClassifier() classify = classify.fit(X,Y)
Step 5: Visualization separates the orders into priority groups. Visualizing the tree is optional but provides a trendy white box approach. You will have to use: • import collections, a Python container library. • import pydotplus, a Python interface to Graphviz's dot language. You can choose to use Graphviz directly with other variations of this source code. The source code will take the nodes and edges of the decision tree, draw them, and save the image in a file as follows: info = tree.export_graphviz(classify,feature_names=features, out_file=None, filled=True,rounded=True) graph = pydotplus.graph_from_dot_data(info) edges = collections.defaultdict(list) for edge in graph.get_edge_list(): edges[edge.get_source()].append(int(edge.get_destination())) for edge in edges: edges[edge].sort() for i in range(2): dest = graph.get_node(str(edges[edge][i]))[0] graph.write_png(.png)
The file will contain this intuitive decision tree:
Figure 3.3: A decision tree
[ 63 ]
Machine Intelligence – Evaluation Functions and Numerical Convergence
The image produces the following information: • A decision tree represented as a graph that has nodes (the boxes) and edges (the lines). • When gini=0, this box is a leaf; the tree will grow no further. • gini means Gini impurity. At an intuitive level, Gini impurity will focus on the highest values of Gini impurity to classify the samples. We will go into the theory of Gini impurity in Chapter 5, How to Use Decision Trees to Enhance KMeans Clustering. • samples = 6. There are six samples in the training dataset: °
Priority/location >result:Bonjour. Je m'appelle Usty! source='The weather is nice today' >>>result: Le temps est beau aujourd'hui source='Ce professor me chercher des poux.' >>>result: This professor is looking for lice!
The first two examples look fine in French, although the second translation is a bit strange. But in the third test, the expression chercher des poux means looking for trouble in English, and not looking for lice, as translated into French. A linguistic assessment of Google Translate will now be made.
Linguistic assessment of Google Translate Assessing Google Translate correctly will lead directly to its limits. Limits are the boundaries researchers crave! We are frontiersmen! An expertlevel assessment will lead the project team to the frontier and beyond. To do this, we will first explore some linguistic methods.
Lexical field theory
Lexical fields describe word fields. A word only acquires its full meaning when interpreted within a context. This context often goes beyond a few other words or even a sentence. Chercher des poux translated as such means look for lice. But in French, it can mean looking for trouble or literally looking for lice. The result that Google Translate comes up with contains three basic problems. source='chercher des poux' >>result: look for lice
[ 131 ]
Innovating AI with Google Translate
Problem 1 – the lexical field: There is no way of knowing whether this means looking for lice or looking for trouble without a context. Problem 2 – metaphors or idiomatic expressions: Suppose you have to translate this is giving you a headache. There is no way of knowing whether it is a physical problem or a metaphor meaning this is driving you crazy. These two idiomatic expressions happen to have the same metaphors when translated into French. But the lice metaphor in French means nothing in English. Problem 3: chercher is an infinitive in French, and the result should have been looking for lice in English. But entering chercher des limites est intéressant provides the right verb form, which is looking for: source='Chercher des limites est intéressant.' >>>result:Looking for boundaries is interesting.
The answer is correct because is splits the sentence into two, making it easier for Google Translate to identify chercher as the first part of a sentence, thus using looking in English. Lexical fields vary from language to language, but so does jargon.
Jargon
Jargon arises when fields specialize. In AI, the expression hidden neurons is jargon. This expression means nothing to a lawyer, for example. A lawyer may think you have hidden intelligence on the subject somewhere or are hiding money in a cryptocurrency named hidden neuron. In the same way, if somebody asks an AI expert to explain the exact meaning of filing a motion, that would prove difficult. In a corporate legal environment, beyond using Google Translate as a dictionary, translating sentences might be risky if only a random number of results prove to be correct. If we add the jargon variations to the lexical variations from one language to another, we can see that wordtoword translation does not work when a word is in a context. Translating is thus more than just finding the most similar words in the language we are translating to.
Translating is not just translating but interpreting
Sometimes, translating requires interpreting, as shown with the following sentence taken from a standard description of French commercial law:
[ 132 ]
Chapter 6 source='Une SAS ne dispense pas de suivre les recommandations en vigueur autour des pratiques commerciales.' >>>result:An SAS does not exempt from following the recommendations in force around commercial practices.
The French sentence refers to a type of company; SAS is similar to company types like inc., ltd., and so on. In English, SAS means Special Air Service. Then comes the grammar, which does not sound right. A translator would write better English and also specify what an SAS is: An SAS (a type of company in France) must follow the recommendations that cover commercial practices. Translating often means interpreting, and not simply translating words. In this case, a legal translator may interpret the text in a contract and go as far as writing: The COMPANY must respect the legal obligation to treat all customers fairly. The legal translator will suggest that COMPANY be defined at the beginning of the contract to avoid confusion, such as the one Google Translate just made. When reading about NLP, chatbots, and translation, everything seems easy. However, working on Google Translate can easily turn into a nightmare! Let's take one last example: The project team is all ears
Google Translate provides the output in French: source:"The project team is all ears". >>>result: L'équipe de projet est tout ouïe.
In French, as in English, it is better to say project team and not use of to say the team of the project. In French, we have équipe projet (équipe (team) appears before project). From our examples so far, we can see that Google Translate is: • Sometimes correct • Sometimes wrong • Sometimes partly correct and partly wrong The problem now is how to know which category a translation is in. [ 133 ]
Innovating AI with Google Translate
How to know whether a translation is correct
How can you check a translation if you do not know the language? Be careful. If Google Translate provides randomly correct answers in another language, then you have no way of knowing whether the translation is reliable or not. If you can't be confident that Google Translate is going to be correct, you may find yourself in difficult situations; even sending the opposite of what you mean to somebody important to you. You may misunderstand a sentence you are trying to understand. In a transportation company, for example, you could write an email stating that the coach stopped and people were complaining: source='The coach stopped and everybody was complaining.'
Google Translate, for lexical field reasons, got it wrong and translated coach as a sports trainer in French, which would give a completely different meaning to the sentence: result: L'entraîneur s'est arrêté et tout le monde se plaignait..
Now, the situation can get worse. To help Google Translate, let's add some context. source='The coach broke down and stopped and everybody was complaining.'
This answer is worse. Google Translate translates broke down correctly with the French expression en panne but still translates coach as entraineur (trainer) in French, meaning the trainer broke down, not the coach (bus). result: L'entraîneur est tombé en panne et s'est arrêté et tout le monde se plaignait.
Google will no doubt continue to improve the program as it has done since 2006. For now, however, a human translator will find hundreds of expressions Google Translate cannot deal with yet. Understanding a sentence in your native language can prove difficult when a word or an expression has several meanings. Adding a translation function to the issue makes it even more difficult to provide a reliable answer. And that is where we reach the frontier, just beyond the cutting edge, as we have established the limits of Google Translate.
[ 134 ]
Chapter 6
In the next section, we'll look at some ways we could improve standard Google Translate results. Although no silver bullet exists to verify a translation, we will explore methods to improve the process. We will find a way to improve Google Translate and implement it.
AI as a new frontier
Google has a great, but limited, translation program. Use the flaws to innovate! AI research and development has just scratched the surface of the innovations to come. First, implement an AI solution. Then, use it for what it is. But don't accept its limits. Don't be negative about it. Innovate! Imagine ideas or listen to other ideas you like and build solutions in a team! Google might even publish your solutions! Improving Google Translate for any translation is impossible. A realistic approach is to focus on customizing Google Translate for a given domain, such as the transportation company in this example. In the next section, we will focus on ways to customize Google Translate.
Lexical field and polysemy
Google_Translate_Customized.py will provide ideas on how to improve Google
Translate in a specific area. This section focuses on the transportation vocabulary error Google Translate made. Once again, Google may rapidly correct this error, but the method can be applied to the many remaining errors. A lexical field contains words that form sets and subsets. They differ from one language to another. A language itself forms a set and contains subsets of lexical fields.
Colder countries have more words describing water in its frozen form than tropical countries where snow hardly ever falls. A lexical field of cold could be a subset of C: C = {ice, hail, snowflakes, snowman, slushy, powder, flake, snowball, blizzard, melting, crunch … n} The curse of dimensionality applies here. Words contain an incredible number of dimensions and definitions. To translate certain expressions, Google Translate suppresses their dimensions and reduces them. Google Translate often uses ngrams to translate. An ngram is a fixedlength sequence of tokens. Tokens can be a word, a character, or even a numerical representation of words and characters. [ 135 ]
Innovating AI with Google Translate
The probability that token n means something is calculated given the preceding/ following n – x or n + x tokens. x is a variable depending on the algorithm applied. For example, slushy has a special meaning in the expression slushy snow. The snow is partly melting, it's watery and making a slushing sound when we walk through it. Melting is only one component of the meaning of slush. Google Translate, at this point, will only translate slushy snow in French by:
neige (snow) fondante (melting) Google Translate will also translate melting snow by:
neige (snow) fondante (melting) To translate slushy into French, you have to use a phrase. To find that phrase, you need some imagination or have some parsed (searched) novels or other forms of speech representations. That takes time and resources. It will most probably take years before Google Translate reaches an acceptable native level in all the languages publicized. Another dimension to take into account is polysemy. Polysemy means a word can have several very different meanings in a language. The equivalent word in another language may simply have one meaning or other, very different meanings. "Go + over" in English can mean go over a bridge or go over some notes. At this point (hopefully it will improve by the time you read this book), it is translated in both cases in French by aller sur. This means to go on (not over), which is incorrect in both cases. Prepositions in English constitute a field in themselves, generating many meanings with the same word. The verb go can have a wide list of meanings: go up (upstairs), go up (stock market), go down (downstairs), go down (fall apart), and many more possibilities besides. The prototype customized program starts with defining X. A small dataset to translate that will be more than enough to get things going: X=['Eating fatty food can be unhealthy.', 'This was a catch22 situation.', 'She would not lend me her tote bag', 'He had a chip on his shoulder', 'The market was bearish yesterday', 'That was definitely wrong', 'The project was compromised but he pulled a rabit out of his hat', 'So just let the chips fall where they may', 'She went the extra mile to satisfy the customer', [ 136 ]
Chapter 6 'She bailed out when it became unbearable', 'The term person includes one or more individuals, labor unions, partnerships, associations, corporations, legal representatives, mutual companies, jointstock companies, trusts, unincorporated organizations, trustees, trustees in bankruptcy, or receivers.', 'The coach broke down, stopped and everybody was complaining']
If you find spelling mistakes or minor mistakes, do not correct them during the training phase. Some amount of noise is required to reproduce human and machine errors to avoid overfitting.
Google Translate will automatically translate these sentences. X1, as implemented in the following code, defines some keywords statistically related to the sentences; it applies the ngram probability theory described previously. X1=['grasse', 'insoluble', 'sac', 'aggressif', 'marché', 'certainement', 'chapeau', 'advienne', 'supplémentaire', 'parti', 'personne', 'bus']
Each line in X1 goes with the corresponding line in X. As explained, this only remains a probability and may not be correct. We are not seeking perfection at this point but an improvement. Let's explore how we can improve Google Translate by customizing translations by implementing a KNN in a Python program.
Exploring the frontier – customizing Google Translate with a Python program
Now it's time to add some customized novelties. The use of the vectors in this section will be explained in the next section, again through the source code that uses them. [ 137 ]
Innovating AI with Google Translate
A trigger vector will force the program to try an alternate method to translate a mistranslated sentence. When the sentence has been identified, and if its value in X2 is equal to 1, it triggers a deeper translation function, as implemented here: X2=[0,0,0,1,0,0,0,0,0,0,0,1]
0 and 1 are flags. Each value represents a line in X. Note for developers: To use this method correctly, all the values of this vector should be set to 1. That will automatically trigger several alternate methods to translate Google Translate errors. A lot of work remains to be done here!
The example is taken from a transportation business. A transportation phrase dictionary should be built. In this case, a general phrase_translation dictionary has been implemented with one expression, as shown in the following array. phrase_translation=['','','','Il est agressif','','','','','','','','']
What remains to be done in order to fill up this dictionary? • Scan all the documents of the company—emails, letters, contracts, and every form of written documents. • Store the embedded words and sentences. • Train the team to use the program to improve it by providing feedback (the right answer) in a learning interface when the system returns incorrect answers. What Google Translate cannot do on a global scale, you can implement at a local scale to improve the system significantly. Now that we have defined a method, we will dig into the KNN algorithm.
knearest neighbor algorithm
No matter how you address a linguistic problem, it will always boil down to the concept of context. When somebody does not understand somebody else, they say: "you took my words out of their context," or "that is not what I meant; let me explain."
[ 138 ]
Chapter 6
As explained before, in many cases, you cannot translate a word or expression without a lexical field. The difficulty remains proportional to the polysemy property, as the program will show. Using the KNN algorithm as a classification method can prove extremely useful. Any language interpretation (translation or chatbot) will have to use a contextoriented algorithm. By finding the words closest (neighbors) to each other, KNN will create the lexical fields required to interpret a language. Even better, when provided with the proper datasets, it will solve the polysemy problem, as shown in the upcoming sections.
Implementing the KNN algorithm
Generally, a word requires a context to mean something. Looking for "neighbors" close by provides an efficient way to determine where the word belongs. KNN is supervised because it uses the labels of the data provided to train its algorithm. KNN, in this case, is used for classification purposes. For a given point p, KNN will calculate the distances to all other points. Then, k represents the knearest neighbors to take into account. Let's clear this up by means of an example. In English, the word "coach" can mean a trainer on a football field, a bus, or a railroad passenger car. In a transportation company, "coach" will mostly be a bus that should not be confused with a trainer: • Step 1: Parsing (examining in a detailed manner) texts with "coach" as a bus and "coach" as a trainer. Thus, the program is searching for three target words: trainer, bus, and coach. • Step 2: Finding some words that appear close to the target words we are searching for. As recommended, do the following: °
Parse all the company documents you can use with a standard program.
°
Use a Python function such as if(ngram in the source) then store the data.
[ 139 ]
Innovating AI with Google Translate
In this case, the V1.csv file shown in the following output excerpt provided with the source code contains the result of such a parsing function: broke,road,stopped,shouted,class 1,3.5,6.4,9,trainer 1,3.0,5.4,9,trainer 1,3.2,6.3,9,trainer ... 6.4,6.2,9.5,1.5,bus 2,3.2,9,1,bus 6.4,6.2,9.5,1.5,bus ... 3.3,7.3,3.0,2.5,coach 4.7,5.7,3.1,3.7,coach 2.0,6.0,2.7,3.1,coach
Generating files such as V1.csv is not in the scope of this chapter or book. However, you can start, among other sites, by exploring scikitlearn's text document functionality at the following link: https://scikitlearn.org/stable/tutorial/text_analytics/working_with_ text_data.html
The program parsed emails, documents, and contracts. Each line represents the result of parsing one document. The numbers represent the occurrences (number of times the word was present). The numbers have been "squashed" (divided again and again) to remain small and manageable. For more on how to work with text data, please click on the scikitlearn link in the previous paragraph. Progressively, the words that came out with "trainer" were "shouted" more than "stopped." For a bus, "broke" (broken down as in breaking down), "road," and "stopped" appeared more than "shout." "Coach" appeared on an average of "shouted," "stopped," "road," and "broke" because it could be either a trainer or a bus, hence the problem we face when translating this word. The polysemy (several meanings) of "coach" can lead to poor translations. The KNN algorithm loaded the V1.csv file that contains the data to be trained and finds the following result:
[ 140 ]
Chapter 6
Figure 6.1: Result from the KNN algorithm
The knn_polysemy.py program determined the following: • The verb "broke" in blue has a better chance of applying to a bus (x axis value > 6) than to a trainer (x axis value < 4). However, "coach" remains above "trainer" and below "bus" because it can be both. • The word "road" follows the same logic as the blue chart. • The verb "stopped" can apply to a trainer and more to a bus. "Coach" remains undecided again. • The verb "shouted" applies clearly to a trainer more than a bus. "Coach" remains undecided again. Note that the coordinates of each point in these charts are as follows: • y axis: bus = 1, coach = 2, and trainer = 3. • x axis: The value represents the "squashed" occurrence (the number of times the word appeared) values. This is the result of the search for those words in many sources.
[ 141 ]
Innovating AI with Google Translate
When a new point, a data point named Pn is introduced into the system, it will find its nearest neighbor(s) depending on the value of k. The KNN algorithm will calculate the Euclidean distance between Pn and all the other points from P1 to Pn – 1 using the Euclidean distance formula. The k in KNN represents the number of "nearest neighbors" the algorithm will take into account for classification purposes. The Euclidean distance (d1) between two given points, for example, between Pn(x1, y1) and P1(x2, y2), is: 2
𝑑𝑑1 (𝑃𝑃𝑛𝑛 , 𝑃𝑃1 ) = √(𝑥𝑥1 − 𝑥𝑥2 )2 + (𝑦𝑦1 − 𝑦𝑦2 )2
Considering the number of distances to calculate, a function such as the one provided by sklearn.neighbors proves necessary.
The knn_polysemy.py program
The program imports the V1.csv file described previously, prints a few lines, and prepares the labels in the correct arrays in their respective x axis and y axis, as shown in this source code example: import pandas as pd from matplotlib import pyplot as plt from sklearn.neighbors import KNeighborsClassifier # Import data df = pd.read_csv('V1.csv') print (df.head()) # KNN Classification labels X = df.loc[:,'broke':'shouted'] Y = df.loc[:,'class']
Then the model is trained, as shown in the following code: # Trains the model knn = KNeighborsClassifier() knn.fit(X,Y)
Once the model is trained, a prediction is requested and is provided by the following code: # Requesting a prediction #broke and stopped are #activated to see the best choice of words to fit these features. # brock and stopped were found in the sentence to be interpreted. # In X_DL as in X, the labels are : broke, road, stopped,shouted. X_DL = [[9,0,9,0]] [ 142 ]
Chapter 6 prediction = knn.predict(X_DL) print ("The prediction is:",str(prediction).strip('[]'))
This is the result displayed: The prediction is: 'bus'
The initial data is plotted for visualization purposes, as implemented in the following code: #Uses the same V1.csv because the parsing has # been checked and is reliable as "dataset lexical rule base". df = pd.read_csv('V1.csv') # Plotting the relation of each feature with each class figure,(sub1,sub2,sub3,sub4) = plt.subplots( 4,sharex=True,sharey=True) plt.suptitle('knearest neighbors') plt.xlabel('Feature') plt.ylabel('Class') X = df.loc[:,'broke'] Y = df.loc[:,'class'] sub1.scatter(X, Y,color='blue',label='broke') sub1.legend(loc=4, prop={'size': 5}) sub1.set_title('Polysemy') X = df.loc[:,'road'] Y = df.loc[:,'class'] sub2.scatter(X, Y,color='green',label='road') sub2.legend(loc=4, prop={'size': 5}) X = df.loc[:,'stopped'] Y = df.loc[:,'class'] sub3.scatter(X, Y,color='red',label='stopped') sub3.legend(loc=4, prop={'size': 5}) X = df.loc[:,'shouted'] Y = df.loc[:,'class'] sub4.scatter(X, Y,color='black',label='shouted') sub4.legend(loc=4, prop={'size': 5}) figure.subplots_adjust(hspace=0) plt.show()
A compressed version of this program has been introduced in Google_Translate_
Customized.py, as shown here:
def knn(polysemy,vpolysemy,begin,end): df = pd.read_csv(polysemy+'.csv') X = df.loc[:,'broke':'shouted'] Y = df.loc[:,'class'] [ 143 ]
Innovating AI with Google Translate knn = KNeighborsClassifier() knn.fit(X,Y) prediction = knn.predict(vpolysemy) return prediction
The description is as follows: • polysemy is the name of the file to read because it can be any file. • vpolysemy is the vector that needs to be predicted. • In future, in the todo list, begin should replace broke and end should replace shouted so that the function can predict the values of any vector. • The KNN classifier is called and the prediction returned. Now that we have prepared the KNN classifier function, we can customize Google Translate.
Implementing the KNN function in Google_ Translate_Customized.py
This program requires more time and research because of the concepts of linguistics involved. The best way to grasp the algorithm is to run it in order. Google Translate offers various translation methods. We will focus on two of them in the following code: #print('PhraseBased Machine Translation(PBMT)model:base'): #m='base' print('Neural Machine Translation model:nmt')
These are explained as follows: • Phrasebased machine translation (PBMT): This translates the whole sequence of words. The phrase, or rather phraseme (multiword expression), is not always quite a sentence. • Neural machine translation (NMT): This uses neural networks such as a recurrent neural network (RNN), which will be detailed later in this book. This method goes beyond the phraseme and takes the whole sentence into account. In terms of the dataset presented in this chapter, this neural network method provides slightly better results. Both methods and Google's other approaches are interesting, but Google Translate still requires additional customized algorithms to reach an acceptable level of quality in many cases. In this chapter, we are exploring one approach with a KNN, but you can use others as long as they work. [ 144 ]
Chapter 6
As you have seen so far, the subject is extremely complex if you take the lexical fields and structures of the many languages, their regional variations, and jargon into account.
Step 1 – translating the X dataset line by line from English into French The following code calls the translation function:
for xi in range(len(X)): source=X[xi] targetl="fr";m='nmt' result = g_translate(source,targetl,m)
The code is explained as follows: • xi is the line number in X. • source is the xi line in X. • targetl is the target language, in this case, fr (French). • m is the method (PBMT or NMT), as described previously. In this case, nmt is applied. • Then, the Google Translate function is called as described earlier in this chapter. The result is stored in the result variable.
Step 2 – backtranslation
How can somebody know the correctness of a translation from language L1 to language L2 if L1 is the person's native language, and L2 is a language the person does not understand at all? This is one of the reasons, among others, that translators often use backtranslation to check translations:
Translation = Initial translation from L1 to L2 Backtranslation = Translation back from L2 to L1 If the initial text is not obtained, then there is probably a problem. In this case, the length of the initial sentence L1 can be compared to the length of the same sentence translated back to L1. The following code calls backtranslation: back_translate=result back_translate = g_translate(back_translate,targetl,m) print("source:",source,":",len(source)) print("result:",result) print("target:",back_translate,":",len(back_translate)) [ 145 ]
Innovating AI with Google Translate
Length comparison can be used to improve the algorithm:
Length of the initial ngram = Length of the backtranslation If it's equal, then the translation may be correct. If not, it could be incorrect. Of course, more methods must be applied during each translation. However, a method that leads to improvement is already a good step. In this case, the source (initial sentence) is compared to the backtranslation in the following code: if(source == back_translate): print("true") if((term not in words)and (xi!=4)): t+=1 else: f+=1;print("false")
• t is a True counter. • f is a False counter. The first line of X runs as follows: source: Eating fatty food can be unhealthy. : 35 result: Manger de la nourriture grasse peut être malsain. target: Eating fat food can be unhealthy. : 33 false
Eating fatty food is backtranslated as eating fat food, which is slightly wrong. Something may be wrong. The French sentence sounds wrong, too. Fatty food cannot be translated as such. Usually, the common sentence is manger gras, meaning eating (manger) fatty (gras), which cannot be translated into English as such. The X array referred to in this section starts at line 8: X=['Eating fatty food can be unhealthy.', 'This was a catch22 situation.', 'She would not lend me her tote bag', 'He had a chip on his shoulder', ....]
Several phrases come back with a false translation, for example, X[4], 'He had a chip on his shoulder'. I programmed a phrasebased translation using a trigger in the False condition in the following code. [ 146 ]
Chapter 6 else: f+=1;print("false") if(X2[xi]>0): DT=deeper_translate(source,xi) dt+=1
Since I did not write a complete application for this book, but just some examples that can be extended in the future, I used X2 as a trigger. If X2[x1]>0, then the deeper_translate function is activated.
Step 3 – deeper translation with phrasebased translations deeper_translate has two arguments:
• source: The initial sentence to translate • x1: The target sentence In this case, the problem to solve is an idiomatic expression that exists in English but does not exist in French: source: He had a chip on his shoulder : 29 result: Il avait une puce sur son épaule target: He had a chip on his shoulder : 29 false
To have a chip on the shoulder means to have an issue with something or somebody. It expresses some form of tension. Google translated chip by assuming computer chip, or puce in French, which means both computer chip and flea. The translation is meaningless. Chip enters three categories and should be labeled as such: • Idiomatic expression • Jargon • Polysemy At this point, the following function I created simulates the phrasebased solution to implement deeper translations. def deeper_translate(source,index): dt=source deeper_response=phrase_translation[index] if(len(deeper_response)0): DT=deeper_translate(source,xi) dt+=1
In the present state of the prototype, only example four activates a phrasebased translation. Otherwise, True is accepted. If False is the case, the deeper translation is only activated for two cases in this sample code. The flag is in X2 (0 or 1). [ 148 ]
Chapter 6
deeper_translate is called for either the phrasebased translation (described previously) or the KNN routine, which is activated if the phrasebased translation did not work.
If the translation did not work, an ngram is prepared for the KNN algorithm, as shown in the following code: if(len(deeper_response)0): polysemy='V1' begin=str(V1[0]).strip('[]');end=str(V1[3]).strip('[]') sememe=knn(polysemy,vpolysemy,begin,end)
• polysemy indicates the KNN file to open. • begin is the first label of the V1 vector and end is the last label of the V1 vector. • sememe is the prediction we expect.
[ 149 ]
Innovating AI with Google Translate
Now, a condensed version of the KNN algorithm is called, as described previously for knn_polysemy.py, in the following code: def knn(polysemy,vpolysemy,begin,end): df = pd.read_csv(polysemy+'.csv') X = df.loc[:,begin:end] Y = df.loc[:,'class'] knn = KNeighborsClassifier() knn.fit(X,Y) prediction = knn.predict(vpolysemy) return prediction
The example, in this case, is the polysemy feature of a coach, as explained in the KNN section. The output will be produced as follows: Source: The coach broke down, stopped and everybody was complaining : 59 result: L'entraîneur est tombé en panne, s'est arrêté et tout le monde se plaignait target: The coach broke down, stopped, and everyone was complaining : 59 term: bus false
The translation is false because Google Translate returns trainer instead of bus. The term bus is identical in English and French. The KNN routine returned bus in English as the correct word to use when broke down and stopped were found, as shown in the KNN section. The goal of the rest of the source code in the deeper_translate function is to replace coach—the word increasing the polysemy feature to translate—with a better word (limited polysemy) to translate: sememe. The sememe variable is initialized by the KNN function in the following code: sememe=knn(polysemy,vpolysemy,begin,end) for i in range(2): if(V1_class[i] in source): replace=str(V1_class[i]).strip('[]') sememe=str(sememe).strip('[]') dtsource = source.replace(replace,sememe) targetl="fr";m='base' result = g_translate(dtsource,targetl,m) print('polysemy narrowed result:',result, ":Now true") [ 150 ]
Chapter 6
The function replaces coach by bus found by the KNN algorithm in the English sentence and then asks Google Translate to try again. The correct answer is returned. Instead of trying to translate a word with too many meanings (polysemy), the deeper_translate function first replaces the word with a better word (less polysemy). Better results will often be attained.
Step 3.1 – adding a frequentist error probability function
A frequentist error probability function is added to measure performance, as shown in the following code: def frequency_p(tnumber,cnumber): ff=cnumber/tnumber #frequentist interpretation and probability return ff
• cnumber is the number of false answers returned by Google Translate. • tnumber is the number of sentences translated. • ff gives a straightforward error (translation) probability, ETP. The function is called when a translation is false, or f>0, as implemented in the following code: if(f>0): B1=frequency_p(xi+1,f) #error detection probability before deep translation B2=frequency_p(xi+1,fdt) #error detection probability after deep translation if(f>0): print("ETP before DT",round(B1,2), "ETP with DT",round(B2,2)) else: print('Insufficient data in probability distribution')
• B1 is the error (translation) probability (ETP) before the deeper_translate function is called. • B2 is the ETP after the deeper_translate function is called. At the end of the program, a summary is displayed, as shown in the following output: print("Summary") print('Neural Machine Translation model:nmt') print('Google Translate:',"True:",t,"False:",f,'ETP',round(f/ len(X),2)) print('Customized Google Translate:',"True:",t,"False:",f[ 151 ]
Innovating AI with Google Translate dt,'ETP',round((fdt)/len(X),2)) a=2.5;at=t+a;af=fa #subjective acceptance of an approximate result print('Google Translate acceptable:',"True:",at,"False:",af,'ETP',roun d(af/len(X),2)) #The error rate should decrease and be stabilized as the KNN knowledge base increases print('Customized Google Translate acceptable:',"True:",at,"False:", afdt,'ETP',round((afdt)/len(X),2))
• A subjective acceptance of an approximate result has been added to increase the true probability. • The error rate should decrease as the quality of the KNN knowledge base increases. In frequent probability theory, this means that a stabilized prediction rate should be reached. We've come to the end of our attempts to improve Google Translate. Let's consider some of the conclusions following our experiment.
Conclusions on the Google Translate customized experiment The final error (translation) probability produced is interesting, as shown in the following output: >>Summary>>Neural Machine Translation model:nmt >>Google Translate: True: 2 False: 8 ETP 0.67 >>Customized Google Translate: True: 2 False: 7 ETP 0.58 >>Google Translate acceptable: True: 4.5 False: 5.5 ETP 0.46 >>Customized Google Translate acceptable: True: 4.5 False: 4.5 ETP 0.38
Even with its NMT model, Google Translate is still struggling. This provides great opportunities for AI linguists, as shown with some of the methods presented to improve Google Translate at a local level that could go even further. This experiment with Google Translate shows that Google has just scratched the surface of reallife translations that sound right to the native speakers that receive these translations. It would take a real company project to get this on track with a financial analysis of its profitability before consuming resources.
[ 152 ]
Chapter 6
The disruptive revolutionary loop
As you can now see, Google Translate, like all AI solutions, has its limits. Once this limit has been reached, you are at the cutting edge. Cross the border into AI Frontierland; innovate on your own or with a team. If you work for a corporation, you can create a revolutionary customized solution for hundreds of users. It does not have to go public. It can remain a strong asset to your company. At some point or other, the revolutionary addon will reach beyond the company, and others will use it. It will become disruptive. Finally, others will reach the limit of your nowdisruptive solution. They will then innovate and customize it in their corporation as a revolutionary solution. This is what I call the disruptive revolutionary loop. It is challenging and exciting because it means that AI developers will not all be replaced in the near future by AutoAI bots! Designing a solution does not mean it will be an invention, an innovation, revolutionary, or disruptive. But that does not really matter. What a company earns with a solution represents more than the novelty of what it sells as long as it is profitable. That is rule number 1. That said, without innovating in its market, that company will not survive through the years. If a product requires quality for security reasons, it should remain in its invention state as long as necessary. If a product can produce sales at the low end of the market before its total completion, then the company should sell it. The company will acquire a reputation for innovation, get more money to invest, and take over the territory of its competitors.
Summary
Google Translate is a good example of disruptive marketing. As shown, the theory, the model, and even the cloud architecture are over 10 years old. But each time one of the hundreds of millions of users stumbles across it, it creates more disruption by hooking the user onto Google solutions. The user will come back again and again to view more advertisements, and everyone is happy! AI has only just begun. Google Translate has been around since 2006. However, the results still leave room for developers, linguists, and mathematicians to improve upon. Google has added a neural network and offers other models to improve translations by analyzing whole sentences. How long will it take to be really reliable? In the meantime, the world community is moving AI forward beyond the cutting edge into Frontierland. [ 153 ]
Innovating AI with Google Translate
In this chapter, we first carefully explored the difference between inventing and innovation. An innovation has an impact on the rest of the market. An invention is just the starting point of an innovation. We saw that a revolutionary solution could be a technical breakthrough. But that revolutionary solution will only become disruptive when it spreads out to the rest of the market. We then studied some basic linguistics principles that could help us understand Google Translate, its limits, and how to improve translation errors. We finally implemented a customized translation tool using a KNN to work around Google Translate errors. In the next chapter, Chapter 7, Optimizing Blockchains with Naive Bayes, we will go further in our investigation of the new frontier of AI by using blockchains to make predictions in corporate environments.
Questions
1. Is it better to wait until you have a topquality product before putting it on the market? (Yes  No) 2. Considering the investment made, a new product should always be priced high to reach the top segment of the market. (Yes  No) 3. Inventing a new solution will make it known in itself. (Yes  No) 4. AI can solve most problems without using standard nonlearning algorithms. (Yes  No) 5. Google Translate can satisfactorily translate all languages. (Yes  No) 6. If you are not creative, it is no use trying to innovate. (Yes  No) 7. If you are not a linguist, it is no use bothering with trying to improve Google Translate. (Yes  No) 8. Translating is too complicated to understand. (Yes  No) 9. AI has already reached its limits. (Yes  No)
Further reading
• The Harvard Business Review on disruptive innovations can be found here: https://hbr.org/2015/12/whatisdisruptiveinnovation
• Google Translate documentation can be found here: https://cloud. google.com/translate/docs/
[ 154 ]
Chapter 6
• Google AlphaGo Zero: https://deepmind.com/blog/article/alphagozerostartingscratch
• KNN documentation: http://scikitlearn.org/stable/modules/ neighbors.html#neighbors
• Insights on translation ANNs: https://research.googleblog. com/2016/09/aneuralnetworkformachine.html
• Insights on EnglishFrench translations: http://www.oneskyapp.com/blog/ frenchtranslationchallenges/
• More on how to work with text data to build datasets for your algorithms: https://scikitlearn.org/stable/tutorial/text_analytics/ working_with_text_data.html
[ 155 ]
7 Optimizing Blockchains with Naive Bayes In this threepart chapter, we will use blockchains to optimize a supply chain using naive Bayes. To achieve this goal, we will first start by understanding how a blockchain is generated using cryptocurrency as an example. Blockchains have entered corporations and are here to stay. Hundreds of major corporations have implemented IBM Hyperledger. Suppliers of these corporations will gradually join the network. Corporate blockchains will provide work for many years to come thanks to the millions of lines of code to update with new features and maintain. Mining cryptocurrency represents the most known use of blockchains. Cryptocurrencies are growing around the world. This chapter starts by explaining how the mining aspect of blockchains works, using bitcoins as an example. We will then move on and analyze how to use blockchains for a different purpose than generating cryptocurrency. Corporations use blockchains to record transactions between companies, for example. IBM was founded in 1911, making it the most experienced company in its field today. Google, Amazon, and Microsoft also offer historymaking machine learning platforms. However, IBM offers machine learning platforms that are supported by the 100+ years of experience that the company can bring to bear.
[ 157 ]
Optimizing Blockchains with Naive Bayes
Some of IBM's many years in the computer and software market were bumpy. Some terrible decisions caused the company a lot of problems across the world. IBM learned from those mistakes and now offers robust solutions, including IBM Hyperledger for blockchain solutions. IBM advocates using blockchains for corporate transactions. We will finally move to the third part of this chapter, which explains what blockchains mean for companies around the world and how to use the information blockchains to provide optimizing algorithms with artificial intelligence. Naive Bayes will be applied to a blockchain sample to optimize stock levels. The following topics will be covered in this chapter: • The background of blockchain • Using blockchains to mine bitcoins • Using blockchains for business transactions • How the blocks of a blockchain provide a unique way to share information between companies • Applying artificial intelligence to the blocks of a blockchain to predict and suggest transactions • Naive Bayes • How to use naive Bayes on blocks of a blockchain to predict further transactions and blocks Let's begin with a short introduction to blockchain.
Part I – the background to blockchain technology
In this section, we will go through cryptocurrency mining with blockchains. Producing bitcoins with blockchains made the technology disruptive. The purpose of this section is to understand how the blockchain adventure started before moving on to subsequent uses of blockchain technology. Blockchain technology will transform transactions in every field. Blockchains appeared in 2008. Nobody knows for sure who invented them. Each block contains an encrypted hash of its predecessor (previous block), the DateTime (timestamp) data, and the information regarding the transaction.
[ 158 ]
Chapter 7
For more than 1,000 years, transactions have been mostly local bookkeeping systems. For the past 100 years, even though the computer age changed the way information was managed, things did not change that much. Each company continued to keep its transactions to itself, only sharing some information through tedious systems. Blockchain makes a transaction block visible to the global network it has been generated in. The fundamental concepts to keep in mind are sharing and privacy control. The two ideas seem to create a cognitive dissonance, something impossible to solve. Yet it has been solved, and it will change the world. When a block (a transaction of any kind) is generated, it is shared with the entire network. Permissions to read the information within that block remain manageable and thus private if the regulator of that block wants the information to stay private. Whether the goal is to mine bitcoins through blocks or use blocks for transactions, artificial intelligence will enhance this innovation shortly. In the coming sections, we'll talk about blockchain and its applications in some further detail. Understanding how to produce blocks for cryptocurrency will enable us to grasp blockchain technology. Once we understand blockchain technology, it is easier to see how this secure encrypted method can be applied to any type of business transaction beyond cryptocurrencies. Let's go mining first!
Mining bitcoins
Creating a block in a blockchain does not necessarily have to generate a bitcoin, which is a form of transaction like any other. But understanding how to mine cryptocurrency provides a good way to understand blockchains and how to apply them to many other fields. Mining a bitcoin means creating a mathematical block for a valid transaction and adding this block to the chain; the blockchain:
Blockchain = {block1, block2, the block just added … blockn} The blockchain cannot go back in time. It is like a timedating feature in life. At minute m, you do something, at minute m + 1 something else, at minute m + n something else, and so on. You cannot travel back in time. What is done is done. When a block is added to the bitcoin chain, there is no way of undoing the transaction.
[ 159 ]
Optimizing Blockchains with Naive Bayes
The global network of bitcoin mining consists of nodes. With the appropriate software, you leave a port open, allocate around 150+ GB of disk space, and generate new blocks. The nodes communicate with each other, and the information is relayed to the other nodes around the whole network. For a node to be a miner, it must solve complex mathematical puzzles that are part of the bitcoin program. To solve the puzzle, the software must find a number that fits in a specific range when combined with the data in the block being generated. The number is passed through a hash function. You can call the number a nonce, and it is used only once. For example, an integer between 0 and 4,294,967,296 for a bitcoin must be generated. The process is random. The software generates a number, passes it through the hash function, and sends it out to the network. The first miner who produces a number in the expected range informs the whole network that that particular block has been generated. The rest of the network stops working on that block and moves on to another one. The reward for the miner is naturally paid out in bitcoins. It represents a lot of money, but it's hard to get, considering the competition in the network and the cost required (CPU, electricity, disk space, and time) to produce correct results. A constant balance must be maintained between the cost of mining a bitcoin, or any other cryptocurrency, and the amount received for mining it.
Having talked about mining bitcoin, we'll briefly discuss actually using cryptocurrency.
Using cryptocurrency
Be very careful with cryptocurrency. There are now 1,500+ cryptocurrencies. The concept sounds fascinating, but the result remains currency. Currency can be volatile, and you can lose your life's savings in less than an hour if a crash occurs. Golden rule: If you cannot resist investing in cryptocurrencies, do not invest more than you can afford to lose.
[ 160 ]
Chapter 7
That being said, to use cryptocurrency, first set up a wallet to store your bitcoins, for example. The wallet can be online, through a provider, or even offline. Once that is done, you will be able to purchase bitcoins as you wish in hard cash or using credit cards, debit cards, and transfers. Remember, you are buying these currencies like any other currency with all the potential, but also all the risks involved. In this section, we saw how the blockchain era began with bitcoin production to understand how to mine cryptocurrencies through bitcoins using a blockchain. With the original way blockchains are created in mind, we can apply blockchains to many other fields. In the second part of our threepart chapter, we will see how to use blockchains beyond cryptocurrencies. We will apply them to supply chains.
PART II – using blockchains to share information in a supply chain
In Part I – the background to blockchain technology, we saw how to use blockchains to mine cryptocurrencies. That prerequisite to entering the world of blockchains having been achieved, this section will show how to use blockchains in a supply chain. This will not involve cryptocurrencies. It opens the path to innovating blockchains with AI. A supply chain is a chain of production and service that gets a product from a starting point to the consumer. For example, take a roll of fabric (cloth) that is shipped from India to Tunisia. In Tunisia, the fabric is cut into patterns and assembled as clothing. Then the clothing is shipped to France where it is packaged in a box with a brand printed on it. It then goes on to be stored in a warehouse to be shipped to a physical shop or directly to an online customer. We can sum this supply chain up as follows:
Cloth from India > cut and assembled in Tunisia > shipped to France > packaged > stored > shipped to a shop or directly to a consumer A supply chain process such as an apparel production and delivery system involves thousands of people along the way: production sites, transport personnel, warehouse employees, and management teams for each step of the chain. This is where a modern blockchain technology comes in handy to track all of the transactions down in one system. [ 161 ]
Optimizing Blockchains with Naive Bayes
In the following example, we will take six companies named A, B, C, D, E, and F. In Chapter 1, Getting Started with NextGeneration Artificial Intelligence through Reinforcement Learning, we used a size six reward matrix. In that chapter, we used A, B, C, D, E, and F to represent locations in a Markov decision process for reinforcement learning. At the end of the chapter, we saw that the reward matrix could be applied to many different domains. In our case, we will refer to the six items (A, B, C, D, E, and F) as companies and their locations (one per company in this example) in a supply chain process. Each company, A to F, that is a member of the supply chain using blockchains can optimize its activity. Instead of each company having separate transaction ledgers, one central blockchain will contain all of the transactions (blocks): • How much and when the cloth left India • When it got on a ship and how long it took • When the ship got to Tunisia • … all of the intermediate activities right down to the consumer With such data, the blocks (records of the transactions) of a blockchain have become machine learning goldmines to detect information and make predictions. One profitable reason for using this system is to reduce stock levels. Piling up unsold goods to anticipate sales is costly and can ruin a company's profit. On the contrary, if each company in the supply chain can see the stock levels and real needs of the other partners through the blocks of the blockchain, they can fit their stock levels to the exact need and save huge amounts of money! Let's see how this works using IBM software as an example. IBM Blockchain, based on Hyperledger Fabric, provides a way for companies around the world to share a blockchain transaction network without worrying about mining or using cryptocurrencies. The system is based on the Linux Foundation project. Hyperledger is an open source collaborative project hosted by the Linux Foundation. At this level, Hyperledger uses blockchains to guarantee secure transactions without trying to optimize the cryptocurrency aspect. The software generates blocks in a blockchain network shared by all the parties involved, but they do not have to purchase cryptocurrencies in the currency sense—only in the technological sense. In the following graph, we will use the six nodes (companies A to F) to illustrate how the Markov decision process we studied in Chapter 1, Getting Started with NextGeneration Artificial Intelligence through Reinforcement Learning, can be applied to a blockchain: [ 162 ]
Chapter 7
Figure 7.1: Markov decision process graph
Each node represents a company that takes part in an IBM Hyperledger network set up for those six companies, as described in the following table: Company
Activity
ML weight
A buys and sells clothing and also other products in this network.
Provide goods to the network but keep stock levels down
Stock levels
B buys and sells fabric and also other products in this network.
Provide goods to the network but keep stock levels down
Stock levels
C buys and sells buttons and also other products in this network.
Provide goods to the network but keep stock levels down
Stock levels
D buys and sells printed fabric and also other products in this network.
Provide goods to the network but keep stock levels down
Stock levels
E buys and sells accessories (belts, bracelets) and also other products in this network.
Provide goods to the network but keep stock levels down
Stock levels
F buys and sells packaging boxes and also other products in this network.
Provide goods to the network but keep stock levels down
Stock levels
The structure of the table is as follows: • Company contains six companies, {A, B, C, D, E, F}, that have different activities. Each company, in this example, only has one location, so A to F are company locations as well. [ 163 ]
Optimizing Blockchains with Naive Bayes
• Activity: Part of this group of companies to supply their members, but making sure the costly stock levels are kept down. • ML weight represents a classification by ML of the stock levels to make predictions. The key to profit generation, in our example, is to track the stock levels of each member (A to F) of the supply chain. If the stock levels of company F go down, for example, the other members of the network (A to E) can anticipate and deliver only the limited necessary amount for F's stock levels to reach an acceptable level again. With millions of commercial transactions per year with a huge amount of transportation (truck, train, boat, air), it is increasingly complicated to manage this type of network effectively in the 21st century without a solution like IBM Hyperledger. With IBM Hyperledger, the companies have one online transaction ledger with smart contracts (online) and realtime tracking.
The transactions are secure; they can be private or public among the members of the network, and they provide realtime optimization information for an artificial intelligence solution.
Using blockchains in the supply chain network
IBM Hyperledger provides artificial intelligence developers with a unique advantage over any other dataset—a 100% reliable dataset updated in real time. Each company member (A to F) of the network will create a block for each transaction so that an AI analyst can have access to the data to make forecasts. To make the system work, each member of the network (A to F) will create blocks in the blockchain so that others can view the information recorded and use the information to make decisions.
[ 164 ]
Chapter 7
Creating a block
A block is formed using the method described for mining a bitcoin, except that this time, currency is not a goal. The goal is a secure transaction with a smart contract when necessary. The following screenshot is a standard IBM interface that can be customized:
Figure 7.2: Standard IBM interface
You can see the individual and unique blocks that make up a blockchain. Each block in IBM Hyperledger has a unique number. In this example, it is 111 with a zoom on block 103. A block in the supply chain network in our graph (A to F) can be the purchase of a product X with a contract. The transaction can be between companies A and B, for example. The next block can be the transportation of that product to another location from A to B, for example, within the blockchain network. The information attached to that block is in the Hyperledger repository: company name (A to F), address, phone number, transaction description, and any other type of data required by the network of companies. Each block can be viewed by all or some depending on the permission properties attached to it.
[ 165 ]
Optimizing Blockchains with Naive Bayes
Exploring the blocks
Exploring the blocks provides an artificial intelligence program with a gold mine: a reallife, and 100% reliable, dataset. The interesting part for AI optimization is the block information, as described in the following screenshot. The present block was added to the chain along with the previous block and the transaction code:
Figure 7.3: Exploring the blockchain
Notice that the block hash (see the preceding mining section) of a given block is linked to the previous block hash and possesses a unique transaction code. Once we have our set of blocks as data (the type of transaction, date, amount), we can start to build an AI prediction algorithm. IBM provides extraction possibilities through scripts and other tools. Once we have a dataset containing blocks of the blockchain, the goal of the prediction will be to determine the stock level category to see whether a fellow member of the supply chain network (A to F) needs to be replenished.
The blocks of the blockchain provide data that can be used to predict the type of stock levels that the network will be facing after each transaction. Each company will be able to run a prediction algorithm that uses the existing data to predict the potential replenishing required. For example, company A might detect that company B needs more of its cloth. [ 166 ]
Chapter 7
Considering the amount of data, a prediction algorithm such as naive Bayes can do the job.
Part III – optimizing a supply chain with naive Bayes in a blockchain process Naive Bayes will use some of the critical information as features to optimize warehouse storage and product availability in a realtime process.
The Naive Bayes learning function will learn from the previous blocks on how to predict the next blocks (supplying another company that needs more stock) that should be inserted in the blockchain. The blocks will be inserted in a dataset just like any other form of timestamped data to make predictions. Naive Bayes is based on Bayes' theorem. Bayes' theorem applies conditional probability, defined as follows:
𝑃𝑃(𝐴𝐴𝐵𝐵) =
𝑃𝑃(𝐵𝐵𝐴𝐴)𝑃𝑃(𝐴𝐴) 𝑃𝑃(𝐵𝐵)
• P(AB) is a posterior probability, the probability of A after having observed some events (B). It is also a conditional probability: the likelihood of A happening given B has already happened. • P(BA) is the probability of B given the prior observations A. It is also a conditional probability: the likelihood of B happening given A has already happened. • P(A) is the probability of A prior to the observations. • P(B) is the probability of the predictions. Naive Bayes, although based on Bayes' theorem, assumes that the features in a class are independent of each other. In many cases, this makes predictions more practical to implement. The statistical presence of features, related or not, will produce a prediction. As long as the prediction remains sufficiently efficient, naive Bayes provides a good solution.
A naive Bayes example
In this section, we will first illustrate naive Bayes with a mathematical example before writing the Python program. The goal of this section is just to understand the concepts involved in naive Bayes. [ 167 ]
Optimizing Blockchains with Naive Bayes
This section is not a development chapter but a chapter to understand the underlying concepts of the reallife example and the mathematics used to make predictions. The blockchains in this chapter represent information on the stock levels based on manufactured goods in the apparel industry. For more, read Chapter 12, AI and the Internet of Things (IoT), which describes an AIoptimized apparel manufacturing process in detail. In this section, we will focus on storage. The load of a sewing station in the apparel industry is expressed in quantities in stock keep units (SKUs). An SKU, for example, can be product P: a pair of jeans of a given size. Once the garment has been produced, it goes into storage. At that point, a block in a blockchain can represent that transaction with two useful features for a machine learning algorithm: • The day the garment was stored • The total quantity of that SKU garment now in storage A block in a blockchain contains both the day (timestamp) and quantity stored in the transaction information. Since the blockchain contains the storage blocks of all A, B, C, D, E, and F corporate locations that are part of the network, a machine learning program can access the data and make predictions. The goal is to spread the stored quantities of the given product evenly over the six locations, as represented in the following histogram:
Figure 7.4: Histogram of the storage level of the product distributed over six locations
[ 168 ]
Chapter 7
This screenshot shows the storage level of product P distributed over six locations. Each location in this blockchain network is a hub. A hub in supply chain management (SCM) is often an intermediate storage warehouse. For example, to cover the area of these six locations, the same product will be stored at each location. This way, local trucks can come and pick the goods for delivery. The goal is to have an available product P at point location L (A to F) when needed. For example, the delivery time from a location L to a location point a1 (a store or home) will only take a few hours. If A did not store P, then the finer consumer would have to wait for the product to travel from C to A, for example. If the blockchain network is well organized, one location can specialize in producing product P (best production costs) and evenly distribute the quantity stored in the six locations, including itself. By having evenly balanced storage quantities at all locations, the system will flow in a continuous delivery process. In our example, a company = its location (A to F). Using the blocks of the blockchain of these six members, we can predict when a given storage point (A to F) requires replenishing to reduce waiting times, for example. Using blockchains to optimize the storage levels is an efficient new approach to reducing costs while delivering to customers faster.
The blockchain anticipation novelty
In former days, all the warehouses at those six locations (A to F) had to ensure the minimum storage level for each location. Since they did not know what was happening in their supply chain network in real time, they often stored more products than required, which increased their costs, or did not store enough, leading to delivery problems. In a world of realtime production and selling, distributors need to predict demand. The system needs to be demanddriven. Naive Bayes can solve that problem. It will take the first two features into account: • DAY: The day the garment was stored • STOCK: The total quantity of that SKU garment now in storage Then it will add a novelty—the number of blocks related to product P.
[ 169 ]
Optimizing Blockchains with Naive Bayes
A high number of blocks at a given date means that this product was in demand in general (production, distribution). The more blocks there are, the more transactions there are. Also, if the storage levels (the STOCK feature) are diminishing, this is an indicator; it means storage levels must be replenished. The DAY feature timestamps the history of the product. The block feature is named BLOCKS. Since all share the blockchain, a machine learning program can access reliable global data in seconds. The dataset reliability provided by blockchains constitutes a motivation in itself to optimize storage levels using the blocks of the blockchains as datasets.
The goal – optimizing storage levels using blockchain data
The goal is to maintain stock at low levels by providing fast delivery when product requests are made. To make a decision, the ML solution will analyze the blocks of a blockchain in real time. The program will take the DAY, STOCK, and BLOCKS (number of) features for a given product P and produce a result. The result predicts whether this product P will be in demand. If the answer is yes (or 1), the demand for this product requires anticipation.
Step 1 – defining the dataset
The dataset contains raw data from prior events in a sequence, which makes it perfect for prediction algorithms. Blocks can be extracted using scripts from IBM Hyperledger, for example. This constitutes a unique opportunity to see the data of all companies without having to build a database. The raw dataset will look like the following list:
Figure 7.5: Raw dataset
[ 170 ]
Chapter 7
This dataset contains the following: • Blocks of product P present in the blockchain on day x having scanned the blockchain back by 30 days. No means no significant amounts of blocks have been found. Yes means a significant number of blocks have been found. If blocks have been found, this means that there is a demand for this product somewhere along the blockchain. • Some_blocks means that blocks have been found, but they are too sparse to be taken into account without overfitting the prediction. However, Yes will contribute to the prediction as well as No. • No_blocks means there is no demand at all, sparse or otherwise (Some_ blocks), numerous (Blocks) or not. This means trouble for this product, P. The goal is to avoid predicting demand on sparse (Some_blocks) or absent (No_ blocks) products. This example is trying to predict a potential Yes for numerous blocks for this product P. Only if Yes is predicted can the system trigger the
automatic demand process (see the Implementation of naive Bayes in Python section later in the chapter).
Step 2 – calculating the frequency
Looking at the following frequency table provides additional information:
Figure 7.6: Frequency table
The Yes and No statuses of each feature (Blocks, Some_blocks, or No_blocks) for a given product P for a given period (past 30 days) have been grouped by frequency. The sum is on the bottom line for each No feature and Yes feature. For example, Yes and No_blocks add up to 2.
[ 171 ]
Optimizing Blockchains with Naive Bayes
Some additional information will prove useful for the final calculation: • The total number of samples = 10. • The total number of Yes samples = 8. • The total number of No samples = 2.
Step 3 – calculating the likelihood
Now that the frequency table has been calculated, the following likelihood table is produced using that data:
Figure 7.7: Likelihood table
The table contains the following statistics: • • • • •
No = 2 = 20% = 0.2 Yes = 8 = 80%=0.8 Some_blocks = 2 = 20%=0.2 No_blocks = 3 = 30%=0.3 Blocks = 5 = 50%=0.5
Blocks represent an important proportion of the samples, which means that along with Some_blocks, the demand looks good.
Step 4 – applying the naive Bayes equation
The goal now is to represent each variable of the Bayes' theorem in a naive Bayes equation to obtain the probability of having demand for product P and trigger a purchase scenario for the blockchain network. Bayes' theorem can be expressed as follows:
• • • • •
𝑃𝑃(𝐴𝐴𝐵𝐵) =
𝑃𝑃(𝐵𝐵𝐴𝐴)𝑃𝑃(𝐴𝐴) 𝑃𝑃(𝐵𝐵)
P(YesBlocks) = P(BlocksYes) * P(Yes)/P(Blocks) P(Yes) = 8/10 = 0.8 P(Blocks) = 5/10 = 0.5 P(BlocksYes) = 4/8 = 0.5 P(YesBlocks) = (0.5*0.8)/0.5 = 0.8 [ 172 ]
Chapter 7
The demand looks acceptable. However, penalties are necessary, and other factors must be considered as well (transportation availability through other block exploration processes). This example and method show the concept of the naive Bayes approach. This example is meant to be nothing other than a simplified mathematical explanation of the philosophy of Bayes' theorem. By doing this calculation from scratch, we now know the basic concepts of Bayes' theorem: using prior values to predict future events taking several features into account. We will now move from this theoretical approach to the implementation phase. We will build a program using naive Bayes in Python.
Implementation of naive Bayes in Python
This section shows how to use a stock level optimizer version of naive Bayes. Blockchains provide exceptionally reliable datasets for ML miners looking for areas to optimize and generate profit. Choosing the right model remains the key challenge.
Gaussian naive Bayes
We will be implementing Gaussian naive Bayes because it fits the apparel industry. For example, if you sell dresses, there will be a target middle size S. The marketing department knows that this size S will represent most of the sales. The larger S + n sizes and small S – n will generate fewer sales, creating a Gaussian curve. In implementation mode, a dataset with raw data from the blockchain will be used without the feature interpretation function of naive Bayes in the following table: DAY
STOCK
BLOCKS
DEMAND
10
1455
78
1
11
1666
67
1
12
1254
57
1
14
1563
45
1
15
1674
89
1
10
1465
89
1
12
1646
76
1
15
1746
87
2
12
1435
78
2
[ 173 ]
Optimizing Blockchains with Naive Bayes
Each line represents a block: • DAY: The day of the period analyzed in the dataset. In this case, we are analyzing the days of a given month, which represents a financial period. No other information is required to run a calculation. In other cases, a dd/mm/ yyyy format can be used. You can also just use a counter (1 to n) from day 1 of the start of a period and run it over several weeks, months, or years. • STOCK: The total inputs in a given location (A, B, or … F) found in the blocks and totaled on that day. Since this represents the inputs of one location and only one, no location information is required. In other cases, the location can be added. • BLOCKS: The number of blocks containing product P for location A, for example. A high number of blocks in the BLOCK column and a low number of quantities in the STOCK column mean that demand is high. • DEMAND = 1. The proof of demand is a transaction block that contains a purchase in the past. These transaction blocks provide vital information. A low number of blocks in the BLOCK column, and a high number of quantities in the STOCK column, mean that the demand is low. • DEMAND = 2. Proof that no transaction was found. The limit of naive Bayes In some cases, DEMAND = 1 when the stock is high and the blocks are low. That's why strict correlation is not so useful. This would be the limit of naive Bayes, which just analyzes the statistics and learns how to predict, ignoring the actual conditional probabilities, the interactions between features.
In this section, we described the dataset and features that will be taken into account to write a naive Bayes program with readytouse algorithms.
The Python program
We now know what a blockchain is and how to use the blocks of a blockchain to optimize the stock levels of the locations of a network of companies (A to F). We also know the basic concepts of Bayes' theorem and naive Bayes.
[ 174 ]
Chapter 7
With this in mind, we can build a Python program to predict the stock level categories of incoming blocks of a blockchain. These predictions will help local managers increase their stock levels to meet the demand of their partners (companies A to F) in the supply chain they are part of. The Python naive_bayes_blockchains.py program uses a sklearn class. Consider the following snippet: import numpy as np import pandas as pd from sklearn.naive_bayes import GaussianNB
It reads the dataset into the data structure. The following code reads data_BC.csv into df: #Reading the data df = pd.read_csv('data_BC.csv') print("Blocks of the Blockchain") print(df.head())
It prints the top of the file in the following output: Blocks of the Blockchain DAY STOCK BLOCKS DEMAND 0 10 1455 78 1 1 11 1666 67 1 2 12 1254 57 1 3 14 1563 45 1 4 15 1674 89 1
It prepares the training set, using X to find and predict Y in the following code: # Prepare the training set X = df.loc[:,'DAY':'BLOCKS'] Y = df.loc[:,'DEMAND']
It chooses the class and trains the following clfG model: #Choose the class clfG = GaussianNB() # Train the model clfG.fit(X,Y)
[ 175 ]
Optimizing Blockchains with Naive Bayes
The program then takes some blocks of the blockchain, makes predictions, and prints them using the following clfG.predict function: # Predict with the model(return the class) print("Blocks for the prediction of the AF blockchain") blocks=[[14,1345,12], [29,2034,50], [30,7789,4], [31,6789,4]] print(blocks) prediction = clfG.predict(blocks) for i in range(4): print("Block #",i+1," Gauss Naive Bayes Prediction:", prediction[i])
The blocks are displayed, and the following predictions are produced. 2 means no demand for the moment; 1 will trigger a purchase block: Blocks for the prediction of the AF blockchain [[14, 1345, 12], [29, 2034, 50], [30, 7789, 4], [31, 6789, 4]] Block # 1 Gauss Naive Bayes Prediction: 1 Block # 2 Gauss Naive Bayes Prediction: 2 Block # 3 Gauss Naive Bayes Prediction: 2 Block # 4 Gauss Naive Bayes Prediction: 2
This is a replenishment program. It will mimic the demand. When no demand is found, nothing happens; when demand is found, it triggers a purchase block. Some chain stores know the number of garments purchased on a given day (or week or another unit) and automatically purchase that amount. Others have other purchasing rules. Finding business rules is part of the consulting aspect of a project. In this section, we implemented naive Bayes to predict the categories of the incoming blocks. If the demand is high, then a supply chain manager will know that more products need to be stored. If the demand is low, the manager will avoid storing more products. Blocks in a blockchain provide reliable datasets for a program that scans the blocks 24/7 and generates recommendations in real time.
[ 176 ]
Chapter 7
Summary
The reliable sequence of blocks in a blockchain has opened the door to endless machine learning algorithms. Naive Bayes appears to be a practical way to start optimizing the blocks of a blockchain. It calculates correlations and makes predictions by learning the independent features of a dataset, irrespective of whether the relationship is conditional or not. This freestyle prediction approach fits the openminded spirit of blockchains that are being propagated by the millions today with limitless resources. IBM Hyperledger takes blockchain's "Frontierland" development to another level with the Linux Foundation project. IBM also offers a cloud platform and services. IBM, Microsoft, Amazon, and Google provide cloud platforms with an arsenal of disruptive machine learning algorithms. This provides a smooth approach to your market or department, along with the ability to set up a blockchain prototype online in a short space of time. With this approach, you can enter some additional prototype data in the model, export the data, or use an API to read the block sequences. Then, you will be able to apply machine learning algorithms to these reliable datasets. The only limit is our imagination. The next chapter will lead us into more AI power as we explore the world of neural networks.
Questions
1. Cryptocurrency is the only use of blockchains today. (Yes  No) 2. Mining blockchains can be lucrative. (Yes  No) 3. Blockchains for companies cannot be applied to sales. (Yes  No) 4. Smart contracts for blockchains are more accessible to write than standard offline contracts. (Yes  No) 5. Once a block is in a blockchain network, everyone in the network can read the content. (Yes  No) 6. A block in a blockchain guarantees that absolutely no fraud is possible. (Yes  No) 7. There is only one way of applying Bayes' theorem. (Yes  No) 8. Training a naive Bayes dataset requires a standard function. (Yes  No) 9. Machine learning algorithms will not change the intrinsic nature of the corporate business. (Yes  No) [ 177 ]
Optimizing Blockchains with Naive Bayes
Further reading
• For more on naive Bayes on scikitlearn's website: https://scikitlearn. org/stable/modules/naive_bayes.html
• To explore IBM's Hyperledger solution: https://www.ibm.com/ blockchain/hyperledger.html
[ 178 ]
8 Solving the XOR Problem with a Feedforward Neural Network In the course of a corporate project, there always comes the point when a problem that seems impossible to solve hits you. At that point, you try everything you've learned, but it doesn't work for what's asked of you. Your team or customer begins to look elsewhere. It's time to react. In this chapter, an impossibletosolve business case regarding material optimization will be resolved successfully with a handmade version of a feedforward neural network (FNN) with backpropagation. Feedforward networks are one of the key building blocks of deep learning. The battle around the XOR function perfectly illustrates how deep learning regained popularity in corporate environments. XOR is an exclusive OR function that we will explore later in this chapter. The XOR FNN illustrates one of the critical functions of neural networks: classification. Once information becomes classified into subsets, it opens the doors to prediction and many other functions of neural networks, such as representation learning. An XOR FNN will be built from scratch to demystify deep learning from the start. A vintage, startfromscratch method will be applied, blowing the deep learning hype off the table.
[ 179 ]
Solving the XOR Problem with a Feedforward Neural Network
The following topics will be covered in this chapter: • Explaining the XOR problem • How to handbuild an FNN • Solving XOR with an FNN • Classification • Backpropagation • A cost function • Cost function optimization • Error loss • Convergence Before we begin building an FNN, we'll first introduce XOR and its limitations in the first artificial neural model.
The original perceptron could not solve the XOR function
The original perceptron was designed in the 1950s and improved in the late 1970s. The original perceptron contained one neuron that could not solve the XOR function. An XOR function means that you have to choose an exclusive OR (XOR). This can be difficult to grasp, as we're not used to thinking about the way in which we use or in our everyday lives. In truth, we use or interchangeably as either inclusive or exclusive all of the time. Take this simple example: If a friend were to come and visit me, I may ask them, "Would you like tea or coffee?" This is basically the offer of tea XOR coffee; I would not expect my friend to ask for both tea and coffee! My friend will choose one or the other.
[ 180 ]
Chapter 8
I may follow up my question with, "Would you like milk or sugar?" In this case, I would not be surprised if my friend wanted both. This is an inclusive or. XOR, therefore, means "You can have one or the other, but not both." We will develop these concepts in the chapter through more examples. To solve this XOR function, we will build an FNN. Once the feedforward network for solving the XOR problem is built, it will be applied to an optimization example. The material optimizing example will choose the best combinations of dimensions among billions to minimize the use of corporate resources with the generalization of the XOR function. First, a solution to the XOR limitation of a perceptron must be clarified.
XOR and linearly separable models
In the late 1960s, it was mathematically proven that a perceptron could not solve an XOR function. Fortunately, today, the perceptron and its neocognitron version form the core model for neural networking. You may be tempted to think, so what? However, the entire field of neural networks relies on solving problems such as this to classify patterns. Without pattern classification, images, sounds, and words mean nothing to a machine.
Linearly separable models
The McCullochPitts 1943 neuron (see Chapter 2, Building a Reward Matrix – Designing Your Datasets) led to Rosenblatt's 195759 perceptron and the 1960 WidrowHoff adaptive linear element (Adaline). These models are linear models based on an f(x, w) function that requires a line to separate results. A perceptron cannot achieve this goal and thus cannot classify many objects it faces.
[ 181 ]
Solving the XOR Problem with a Feedforward Neural Network
A standard linear function can separate values. Linear separability can be represented in the following graph:
Figure 8.1: Linearly separable patterns
Imagine that the line separating the preceding dots and the part under it represent a picture that needs to be represented by a machine learning or deep learning application. The dots above the line represent clouds in the sky; the dots below the line represent trees on a hill. The line represents the slope of that hill. To be linearly separable, a function must be able to separate the clouds from the trees to classify them. The prerequisite to classification is separability of some sort, linear or nonlinear.
The XOR limit of a linear model, such as the original perceptron A linear model cannot solve the XOR problem expressed as follows in a table: Value of x1
Value of x2
Output
1
1
0
0
0
0
1
0
1
0
1
1
[ 182 ]
Chapter 8
Lines 3 and 4 show an exclusive OR (XOR). Imagine that you are offering a child a piece of cake OR a piece of candy (1 or 1): • Case 1: The child answers: "I want candy or nothing at all!" (0 or 1). That's exclusive OR (XOR)! • Case 2: The child answers: "I want a cake or nothing at all!" (1 or 0). That's an exclusive OR (XOR) as well! The following graph shows the linear inseparability of the XOR function represented by one perceptron:
Figure 8.2: Linearly inseparable patterns
The values of the table represent the Cartesian coordinates in this graph. The circles with a cross at (1, 1) and (0, 0) cannot be separated from the circles at (1, 0) and (0, 1). That's a huge problem. It means that Frank Rosenblatt's f(x, w) perceptron cannot separate, and thus can not classify, these dots into clouds and trees. Thus, in many cases, the perceptron cannot identify values that require linear separability. Having invented the most powerful neural concept of the twentieth century—a neuron that can learn—Frank Rosenblatt had to bear with this limitation through the 1960s. As explained with the preceding cakeORcandy example, the absence of an XOR function limits applications in which you must choose exclusively between two options. There are many "it'seitherthatornothing" situations in reallife applications. For a selfdriving car, it could be either turn left or turn right, but don't swerve back and forth while making the decision! We will solve this limitation with a vintage solution, starting by building, and later implementing, an FNN. [ 183 ]
Solving the XOR Problem with a Feedforward Neural Network
Building an FNN from scratch
Let's perform a mind experiment. Imagine we are in 1969. We have today's knowledge but nothing to prove it. We know that a perceptron cannot implement the exclusive OR function XOR. We have an advantage because we now know a solution exists. To start our experiment, we only have a pad, a pencil, a sharpener, and an eraser waiting for us. We're ready to solve the XOR problem from scratch on paper before programming it. We have to find a way to classify those dots with a neural network.
Step 1 – defining an FNN
We have to be unconventional to solve this problem. We must forget the complicated words and theories of the twentyfirst century. We can write a neural network layer in highschool format. A hidden layer will be:
h1 = x * w OK. Now we have one layer. A layer is merely a function. This function can be expressed as:
f(x, w) In which x is the input value, and w is some value to multiply x by. Hidden means that the computation is not visible, just as x = 2 and x + 2 is the hidden layer that leads to 4. At this point, we have defined a neural network in three lines: • Input x. • Some function that changes its value, like 2 × 2 = 4, which transformed 2. That is a layer. And if the result is superior to 2, for example, then great! The output is 1, meaning yes or true. Since we don't see the computation, this is the hidden layer. • An output. f(x, w) is the building block of any neural network. "Feedforward" means that we will be going from layer 1 to layer 2, moving forward in a sequence. Now that we know that basically any neural network is built with values transformed by an operation to become an output of something, we need some logic to solve the XOR problem.
[ 184 ]
Chapter 8
Step 2 – an example of how two children can solve the XOR problem every day
An example follows of how two children can solve the XOR problem using a straightforward everyday example. I strongly recommend this method. I have taken very complex problems, broken them down into small parts to a child's level, and often solved them in a few minutes. Then, you get the sarcastic answer from others such as "Is that all you did?" But, the sarcasm vanishes when the solution works over and over again in highlevel corporate projects. First, let's convert the XOR problem into a candy problem in a store. Two children go to the store and want to buy candy. However, they only have enough money to buy one pack of candy. They have to agree on a choice between two packs of different candy. Let's say pack one is chocolate and the other is chewing gum. Then, during the discussion between these two children, 1 means yes, 0 means no. Their budget limits the options of these two children: • Going to the store and not buying any chocolate or chewing gum = (no, no) = (0, 0). That's not an option for these children! So the answer is false. • Going to the store and buying both chocolate and chewing gum = (yes, yes) = (1, 1). That would be fantastic, but that's not possible. It's too expensive. So, the answer is, unfortunately, false. • Going to the store and either buying chocolate or chewing gum = (1, 0 or 0, 1) = (yes or no) or (no or yes). That's possible. So, the answer is true. Imagine the two children. The eldest one is reasonable. The younger one doesn't really know how to count yet and wants to buy both packs of candy. We express this on paper: • x1 (eldest child's decision, yes or no, 1 or 0) * w1 (what the elder child thinks). The elder child is thinking this, or: x1 * w1 or h1 = x1 * w1 The elder child weighs a decision like we all do every day, such as purchasing a car (x = 0 or 1) multiplied by the cost (w1). • x2 (the younger child's decision, yes or no, 1 or 0) * w3 (what the younger child thinks). The younger child is also thinking this, or: x2 * w3 or h2 = x2 * w3
[ 185 ]
Solving the XOR Problem with a Feedforward Neural Network
Theory: x1 and x2 are the inputs. h1 and h2 are neurons (the result of a calculation). Since h1 and h2 contain calculations that are not visible during the process, they are hidden neurons. h1 and h2 thus form a hidden layer. w1 and w3 are weights that represent how we "weigh" a decision, stating that something is more important than something else.
Now imagine the two children talking to each other. Hold it a minute! This means that now, each child is communicating with the other: • x1 (the elder child) says w2 to the younger child. Thus, w2 = this is what I think and am telling you: x1 * w2 • x2 (the younger child) says, "please add my views to your decision," which is represented by w4: x2 * w4 We now have the first two equations expressed in highschoollevel code. It's what one thinks plus what one says to the other, asking the other to take that into account: h1=(x1*w1)+(x2*w4) #II.A.weight of hidden neuron h1 h2=(x2*w3)+(x1*w2) #II.B.weight of hidden neuron h2
h1 sums up what is going on in one child's mind: personal opinion + the other child's opinion. h2 sums up what is going on in the other child's mind and conversation: personal opinion + the other child's opinion. Theory: The calculation now contains two input values and one hidden layer. Since, in the next step, we are going to apply calculations to h1 and h2, we are in a feedforward neural network. We are moving from the input to another layer, which will lead us to another layer, and so on. This process of going from one layer to another is the basis of deep learning. The more layers you have, the deeper the network is. The reason h1 and h2 form a hidden layer is that their output is just the input of another layer.
[ 186 ]
Chapter 8
For this example, we don't need complicated numbers in an activation function such as logistic sigmoid, so we state whether the output values are less than 1 or not: if h1 + h2 >= 1 then y1 = 1 if h1 + h2 < 1 then y2 = 0 Theory: y1 and y2 form a second hidden layer. These variables can be scalars, vectors, or matrices. They are neurons.
Now, a problem comes up. Who is right? The elder child or the younger child? The only way seems to be to play around, with the weights W representing all the weights. Weights in a neural network work like weights in our everyday lives. We weigh decisions all the time. For example, there are two books to purchase, and we will "weigh" our decisions. If one is interesting and cheaper, it will weigh more or less in our decision, for example. The children in our case agree on purchasing at least something, so from now on, w3 = w2, w4 = w1. The younger and elder child will thus share some of the decision weights. Now, somebody has to be an influencer. Let's leave this hard task to the elder child. The elder child, being more reasonable, will continuously deliver the bad news. You have to subtract something from your choice, represented by a minus (–) sign. Each time they reach the point hi, the eldest child applies a critical negative view on purchasing packs of candy. It's –w of everything comes up to be sure not to go over the budget. The opinion of the elder child is biased, so let's call the variable a bias, b1. Since the younger child's opinion is biased as well, let's call this view a bias too, b2. Since the eldest child's view is always negative, –b1 will be applied to all of the eldest child's thoughts. When we apply this decision process to their view, we obtain:
h1 = y1 * –b1 h2 = y2 * b2 Then, we just have to use the same result. If the result is >=1, then the threshold has been reached. The threshold is calculated as shown in the following function:
y = h1 + h2 [ 187 ]
Solving the XOR Problem with a Feedforward Neural Network
We will first start effectively finding the weights, starting by setting the weights and biases to 0.5, as follows:
w1 = 0.2; w2 = 0.5; b1 = 0.5 w3 = w2; w4 = w1; b2 = b1 It's not a full program yet, but its theory is done. Only the communication going on between the two children is making the difference; we will focus on only modifying w2 and b1 after a first try. It works on paper after a few tries. We now write the basic mathematical function, which is, in fact, the program itself on paper: #Solution to the XOR implementation with #a feedforward neural network(FNN) #I.Setting the first weights to start the process w1=0.5;w2=0.5;b1=0.5 w3=w2;w4=w1;b2=b1 #II.hidden layer #1 and its output h1=(x1*w1)+(x2*w4) #II.A.weight of hidden neuron h1 h2=(x2*w3)+(x1*w2) #II.B.weight of hidden neuron h2 #III.threshold I, hidden layer 2 if(h1>=1): h1=1 if(h1=1): h2=1 if(h2=1): y=1 if(y=1):h1=1 if(h1=1):h2=1 if(h20): subsets+=1 print("Subset:",subsets,"size subset #",x1," and ","size subset #",x2,"result:",result[0],"order #"," and ",s1,"order #",s2) if(subsets>=8333): break
When the 8,333 subsets have been found respecting the smallerlarger size distribution, the system stops, as shown in the following output: Subset: 8330 size subset # 1 and size subset # 0 result: 1 order # and 53154 order # 14310 Subset: 8331 size subset # 1 and size subset # 0 result: 1 order # and 473411 order # 196256 Subset: 8332 size subset # 1 and size subset # 0 result: 1 order # and 133112 order # 34827 Subset: 8333 size subset # 0 and size subset # 1 result: 1 order # and 470291 order # 327392
This example proves the point. Simple solutions can solve very complex problems. Two main functions, among some minor ones, must be added: • After each choice, the orders chosen must be removed from the 500,000order dataset. When an order has been selected, processing it again will generate errors in the global results. This will preclude choosing the same order twice and reduce the number of choices to be made. • An optimization function to regroup the results for production purposes, for example. The idea is not to run through the records randomly, but to organize them by sets. This way, each set can be controlled independently.
[ 201 ]
Solving the XOR Problem with a Feedforward Neural Network
Application information: • The core calculation part of the application is fewer than 50 lines long. • With a few control functions and arrays, the program might reach 200 lines maximum. The goal of the control functions is to check and see whether the results reach the overall goal. For example, every 1,000 records, a local result could be checked to see whether it fits the overall goal. • This results in easy maintenance for a team. Optimizing the number of lines of code to create a powerful application can prove to be very efficient for many business problems.
Summary
Building a small neural network from scratch provides a practical view of the elementary properties of a neuron. We saw that a neuron requires an input that can contain many variables. Then, weights are applied to the values with biases. An activation function then transforms the result and produces an output. Neural networks, even one or twolayer networks, can provide reallife solutions in a corporate environment. A reallife business case was implemented using complex theory broken down into small functions. Then, these components were assembled to be as minimal and profitable as possible. It takes talent to break a problem down into elementary parts and find a simple, powerful solution. It requires more effort than just typing hundreds to thousands of lines of code to make things work. A wellthought through algorithm will always be more profitable, and software maintenance will prove more costeffective. Customers expect quickwin solutions. Artificial intelligence provides a large variety of tools that satisfy that goal. When solving a problem for a customer, do not look for the best theory, but the simplest and fastest way to implement a profitable solution, no matter how unconventional it seems. In this case, an enhanced FNN perceptron solved a complex business problem. In the next chapter, we will explore a convolutional neural network (CNN). We will build a CNN with TensorFlow 2.x, layer by layer, to classify images.
[ 202 ]
Chapter 8
Questions
1. Can the perceptron alone solve the XOR problem? (Yes  No) 2. Is the XOR function linearly nonseparable? (Yes  No) 3. One of the main goals of layers in a neural network is classification. (Yes  No) 4. Is deep learning the only way to classify data? (Yes  No) 5. A cost function shows the increase in the cost of a neural network. (Yes  No) 6. Can simple arithmetic be enough to optimize a cost function? (Yes  No) 7. A feedforward network requires inputs, layers, and an output. (Yes  No) 8. A feedforward network always requires training with backpropagation. (Yes  No)
9. In reallife applications, solutions are only found by following existing theories. (Yes  No)
Further reading
• Linear separability: http://www.ece.utep.edu/research/webfuzzy/ docs/kkthesis/kkthesishtml/node19.html
[ 203 ]
9 Abstract Image Classification with Convolutional Neural Networks (CNNs) The invention of convolutional neural networks (CNNs) applied to vision represents by far one of the most innovative achievements in the history of applied mathematics. With their multiple layers (visible and hidden), CNNs have brought artificial intelligence from machine learning to deep learning. In Chapter 8, Solving the XOR Problem with a Feedforward Neural Network, we saw that f(x, w) is the building block of any neural network. A function f will transform an input x with weights w to produce an output. This output can be used as such or fed into another layer. In this chapter, we will generalize this principle and introduce several layers. At the same time, we will use datasets with images. We will have a dataset for training and a dataset for validation to confirm that our model works. A CNN relies on two basic tools of linear algebra: kernels and functions, applying them to convolutions as described in this chapter. These tools have been used in mathematics for decades. However, it took the incredible imagination of Yann LeCun, Yoshua Bengio, and others—who built a mathematical model of several layers—to solve reallife problems with CNNs.
[ 205 ]
Abstract Image Classification with Convolutional Neural Networks (CNNs)
This chapter describes the marvels of CNNs, one of the pillars of artificial neural networks (ANNs). A CNN will be built from scratch, trained, and saved. The classification model described will detect production failures on a foodprocessing production line. Image detection will go beyond object recognition and produce abstract results in the form of concepts. A Python TensorFlow 2 program will be built layer by layer and trained. Additional sample programs will illustrate key functions. The following topics will be covered in this chapter: • • • • • • • • • • • • • • •
The differences between 1D, 2D, and 3D CNNs Adding layers to a convolutional neural network Kernels and filters Shaping images The ReLU activation function Kernel initialization Pooling Flattening Dense layers Compiling the model The crossentropy loss function The Adam optimizer Training the model Saving the model Visualizing the PNG of a model
We'll begin by introducing CNNs and defining what they are.
Introducing CNNs
This section describes the basic components of a CNN. CNN_SRATEGY_MODEL. py will illustrate the basic CNN components used to build a model for abstract image detection. For machines, as for humans, concepts are the building blocks of cognition. CNNs constitute one of the pillars of deep learning (multiple layers and neurons). In this chapter, TensorFlow 2 with Python will be running using Keras libraries that are now part of TensorFlow. If you do not have Python or do not wish to follow the programming exercises, the chapter is selfcontained, with graphs and explanations. [ 206 ]
Chapter 9
Defining a CNN
A convolutional neural network processes information, such as an image, for example, and makes sense out of it. For example, imagine you have to represent the sun with an ordinary pencil and a piece of paper. It is a sunny day, and the sun is shining very brightly—too brightly. You put on a special pair of very dense sunglasses. Now you can look at the sun for a few seconds. You have just applied a color reduction filter, one of the first operations of a convolutional network. Then, you try to draw the sun. You draw a circle and put some gray in the middle. You have just applied an edge filter. Finally, you go over the circle several times to make it easy to recognize, progressively reducing what you saw into a representation of it. Now, with the circle, some gray in the middle, and a few lines of rays around it, anybody can see you drew the sun. You smile; you did it! You took a color image of the sun and made a mathematical representation of it as a circle, which would probably look something like this:
Figure 9.1: Mathematical representation of a circle
You just went through the basic processes of a convolutional network. The word convolutional means that you transformed the sun you were looking at into a drawing, area by area. But, you did not look at the whole sky at once. You made many eye movements to capture the sun, area by area, and you did the same when drawing. If you made a mathematical representation of the way you transformed each area from your vision to your paper abstraction, it would be a kernel. You can see that the convolutional operation converts an object into a more abstract representation. This is not limited to images but can apply to any type of data (words, sounds and video) we want to draw patterns from. With that concept in mind, the following graph shows the successive mathematical steps to follow in this chapter's model for a machine to process an image just as you did. A convolutional network is a succession of steps that will transform what you see into a classification status. [ 207 ]
Abstract Image Classification with Convolutional Neural Networks (CNNs)
In the graph, each box represents a layer. Each layer has an input that comes from the previous layer. Each layer will then transform the input and then produce an output that will become the input of the next layer. At each layer, the key features that are necessary to classify the images will be isolated. In your example, it would serve to find out whether your drawing represents the sun or not. This falls under a binary classification model (yes or no, or 1 or 0).
Figure 9.2: Architecture of a CNN
[ 208 ]
Chapter 9
Notice that the size of the outputs diminishes progressively until the outputs reach 1, the binary classification status that will return (1 or 0). These successive steps, or layers, represent what you did when you went from observing the sun to drawing it. In the end, if we draw poorly and nobody recognizes the sun, it means that we'll have to go back to step 1 and change some parameters (weights in this case). That way, we train to represent the sun better until somebody says, "Yes, it is a sun!" That is probability = 1. Another person may say that it is not a sun (probability = 0). In that case, more training would be required. If you carry out this experiment of drawing the sun, you will notice that, as a human, you transform one area at a time with your eye and pencil. You repeat the way you do it in each area. The mathematical repetition you perform is your kernel. Using a kernel per area is the fastest way to draw. For us humans, in fact, it is the only way we can draw. A CNN is based on this process. In this section, we looked at some key aspects of a CNN model, using the analogy of representing the sun as a drawing. This is just one way to start a convolutional neural network, and there are hundreds of different ways to do so. However, once you understand one model, you will have the understanding necessary to implement other variations. In the following section, we'll see how to initialize and build our own CNN.
Initializing the CNN
CNN_SRATEGY_MODEL.py builds the CNN using TensorFlow 2. TensorFlow 2 has made tremendous improvements in terms of development. The Keras datasets, layers, and models are now part of the TensorFlow instance: import tensorflow as tf from tensorflow.keras import datasets, layers, models
The CNN only requires two lines of headers to build the layers! In TensorFlow 2, for each layer, we simply have to call layers. and that's it! The model used is a Keras sequential() called from the TensorFlow from tensorflow.keras instance: classifier = models.Sequential()
And that's it. We have just started to build our own CNN in just a few lines of code. TensorFlow 2 has simplified the whole process of creating a CNN, making it an easy, intuitive process, as we will see throughout this chapter. Let's begin to build upon the foundations of our CNN in the following section and add a convolutional layer. [ 209 ]
Abstract Image Classification with Convolutional Neural Networks (CNNs)
Adding a 2D convolution layer
In this chapter, we will be using a twodimensional model as our example. Twodimensional relationships can be reallife images and also many other objects, as described in this chapter. This chapter describes a twodimensional network, although others exist: • A onedimensional CNN mostly describes a temporal mode, for example, a sequence of sounds (phonemes = parts of words), words, numbers, and any other type of sequence. • A volumetric module is a 3D convolution, such as recognizing a cube or a video. For example, for a selfdriving car, it is critical to recognize the difference between a 2D picture of a person in an advertisement near a road and a real 3D image of a pedestrian that is starting to cross the same road! In this chapter, a spatial 2D convolution module will be applied to images of different kinds. The main program, CNN_STRATEGY_MODEL.py, will describe how to build and save a model. classifier.add will add a layer to the model. The name classifier does not represent a function but simply the arbitrary name that was given to this model in this particular program. The model will end up with n layers. Look at the following line of code: classifier.add(layers.Conv2D(32, (3, 3),input_shape = (64, 64, 3), activation = 'relu'))
This line of code contains a lot of information: the filters (applied with kernels), the input shape, and an activation function. The function contains many more options. Once you understand these indepth, you can implement other options one by one, as you deem necessary, for each project you have to work on.
Kernel
Just to get started, intuitively, let's take another everyday model. This model is a bit more mathematical and closer to a CNN's kernel representation. Imagine a floor of very small square tiles in an office building. You would like each floor tile to be converted from dirty to clean, for example.
[ 210 ]
Chapter 9
You can imagine a cleaning machine capable of converting 3×3 small tiles (pixels) one at a time from dirty to clean. You would laugh if you saw somebody come with one enormous cleaning machine to clean all of the 32×32 tiles (pixels) at the same time. You know it would be very bulky, slow, and difficult to use, intuitively. On top of that, you would need one big machine per surface size! Not only is a kernel an efficient way to filter, but a kernel convolution is also a timesaving resource process. The small cleaning machine is the kernel (dirtytoclean filter), which will save you time performing the convolution (going over all of the tiles to clean a 3×3 area), transforming the floor from dirty to clean. In this case, 32 different filters have been added with 3×3 sized kernels: classifier.add(layers.Conv2D(32, (3, 3)...
The use of kernels as filters is the core of a convolutional network. (32, (3,3)) means (number of filters, (size of kernels)).
An intuitive approach
To understand a kernel intuitively, keep the sun and cleaning tiles examples in mind. In this section, a photograph of a cat will show how kernels work. In a model analyzing cats, the initial photograph would look like this:
Figure 9.3: Cat photograph for model analysis
[ 211 ]
Abstract Image Classification with Convolutional Neural Networks (CNNs)
On the first run of this layer, even with no training, an untrained kernel would transform the photograph:
Figure 9.4: Cat photograph transformation
The first layer has already begun isolating the features of the cat. The edges have begun to appear: the cat's body, ears, nose, and eyes. In itself, this first filter (one of 32) with a size 3×3 kernel—in the first layer and with no training—already produces effective results. The size of a kernel can vary according to your needs. A 3×3 kernel will require a larger number of weights than a 1×1 kernel, for example. A 1×1 kernel will have only one weight, which restricts the size of the features to represent. The rule is that the smaller the kernel, the fewer weights we have to find. It will also perform a feature reduction. When the size of the kernel increases, the number of weights and features to find increases as well as the number of features represented. Each subsequent layer will make the features stand out much better, with smaller and smaller matrices and vectors, until the program obtains a clear mathematical representation. Now that we have an intuitive view of how a filter works, let's explore a developer's approach.
The developers' approach
Developers like to see the result first to decide how to approach a problem. Let's take a quick, tangible shortcut to understand kernels through Edge_detection_ Kernel.py with an edge detection kernel: #I.An edge detection kernel kernel_edge_detection = np.array([[0.,1.,0.], [1.,4.,1.], [0.,1.,0.]]) [ 212 ]
Chapter 9
The kernel is a 3×3 matrix, like the cat example. But the values are preset, and not trained with weights. There is no learning here; only a matrix needs to be applied. The major difference with a CNN is that it will learn how to optimize kernels itself through weights and biases. img.bmp is loaded, and the 3×3 matrix is applied to the pixels of the loaded image, area by area: #II.Load image and convolution image=mpimg.imread('img.bmp')[:,:,0] shape = image.shape
The image before the convolution applying the kernel is the letter A (letter recognition):
Figure 9.5: The letter "A"
Now the convolution transforms the image, as shown in the following code: #III.Convolution image_after_kernel = filter.convolve(image,kernel_edge_ detection,mode='constant', cval=0)
The edges of A now appear clearly in white, as shown in the following graph:
Figure 9.6: The white edges of A are visible
[ 213 ]
Abstract Image Classification with Convolutional Neural Networks (CNNs)
The original image on top displayed a very thick A. The preceding graph displays a thin, identifiable A feature through thin edges that a neural network can classify within a few mathematical operations. The first layers of a convolutional network train to find the right weights to generate the right kernel automatically. Now that we have an intuitive and practical developer's view of a filter, let's add some mathematics to our approach.
A mathematical approach
The initial image has a set of values you can display, as follows: #II.Load image image=mpimg.imread('img.bmp')[:,:,0] shape = image.shape print("image shape",shape)
The code will print a numerical output of the image, as follows: image shape (100, 100) image before convolution [[255 255 255 ..., 255 255 255] [255 255 255 ..., 255 255 255] [255 255 255 ..., 255 255 255] ...,
The convolution filter is applied using filter.convolve, a mathematical function, to transform the image and filter it. The convolution filter function uses several variables: • The spatial index for the 3×3 kernel to apply; in this case, it must know how to access the data. This is performed through a spatial index, j, which manages data in grids. Databases also use spatial indexes to access data. The axes of those grids determine the density of a spatial index. Kernels and the image are convolved using j over W, the weights kernel. • W is the weights kernel. • I is the input image. • k is the coordinate of the center of W. The default value is 0 in this case.
[ 214 ]
Chapter 9
These variables then enter the filter.convolve function as represented by the following equation:
𝐶𝐶𝑖𝑖 = ∑ 𝐼𝐼𝑖𝑖=𝑗𝑗−𝑘𝑘 𝑊𝑊𝑗𝑗 𝑗𝑗
A CNN relies on kernels. Take all the time you need to explore convolutions through the three dimensions required to master AI: an intuitive approach, development testing, and mathematical representation. Now that we have a mathematical idea on how a convolutional filter works, let's determine the shape and the activation function to the convolutional layer.
Shape
input_shape defines the size of the image, which is 64×64 pixels (height×width),
as shown here:
classifier.add(...input_shape = (64, 64, 3)...)
3 indicates the number of channels. In this case, 3 indicates the three parameters
of an RGB color. Each channel can have a given value of 0 to 255.
ReLU
Activation functions provide useful ways to influence the transformation of weighted data calculations. Their output will change the course of classification, a prediction, or whatever goal the network was built for. This model applies a rectified linear unit (ReLU), as shown in the following code: classifier.add(..., activation = 'relu'))
ReLU activation functions apply variations of the following function to an input value:
f(x) = max{0, x} The function returns 0 for negative values; it returns positive values as x; it returns 0 for 0 values. Half of the domain of the function will return zeros. This means that when you provide positive values, the derivative will always be 1. ReLU avoids the squashing effect of the logistic sigmoid function, for example. However, the decision to use one activation function rather than another will depend on the goal of each ANN model.
[ 215 ]
Abstract Image Classification with Convolutional Neural Networks (CNNs)
In mathematical terms, a rectified linear unit (ReLU) function will take all the negative values and apply 0 to them. And all the positive values remain unchanged. The ReLU.py program provides some functions, including a NumPy function, to test how ReLU works. You can enter test values or use the ones in the source code: import numpy as np nx=3 px=5
nx expects a negative value, and px expects a positive value for testing purposes for the relu(x) and lrelu(x) functions. Use the f(x) function if you wish to include zeros in your testing session.
The relu(x) function will calculate the ReLU value: def relu(x): if(x0):ReLU=x return ReLU
In this case, the program will return the following result: negative x= 3 positive x= 5 ReLU nx= 0 ReLU px= 5
The result of a negative value becomes 0, and a positive value remains unchanged. The derivative or slope is thus always 1, which is practical in many cases and provides good visibility when debugging a CNN or any other ANN. The NumPy function, defined as follows, will provide the same results: def f(x): vfx=np.maximum(0.1,x) return vfx
Through trial and error, ANN research has come up with several variations of ReLU. One important example occurs when many input values are negative. ReLU will constantly produce zeros, making gradient descent difficult, if not impossible.
[ 216 ]
Chapter 9
A clever solution was found using a leaky ReLU. A leaky ReLU does not return 0 for a negative value but a small value you can choose, 0.1 instead of 0, for example. See the following equation:
f(x) = max{0.1, x} The leaky ReLU fixes the problem of "dying" neurons. Suppose you have a layer that keeps returning negative values when activating neurons. The ReLU activation will always return 0 in this case. That means that these neurons are "dead." They will never be activated. To avoid these "dying" neurons, a leaky ReLU provides the small positive value seen previously (0.1) that makes sure that a neuron does not "die." Now gradient descent will work fine. In the sample code, the function is implemented as follows: def lrelu(x): if(x0):lReLU=x return lReLU
Although many other variations of ReLU exist, with this in mind, you have an idea of what it does. Enter some values of your own, and the program will display the results, as shown here: print("negative x=",nx,"positive x=",px) print("ReLU nx=",relu(nx)) print("ReLU px=",relu(px)) print("Leaky ReLU nx=",lrelu(nx)) print("f(nx) ReLu=",f(nx)) print("f(px) ReLu=",f(px)) print("f(0):",f(0))
The results will display the ReLU results as follows: negative x= 3 positive x= 5 ReLU nx= 0 ReLU px= 5 Leaky ReLU nx= 0.01
We have processed a large representation of the input image. We now need to reduce the size of our representation to obtain a better, more abstract representation. By pooling some of the pixels we will also reduce the calculations of the subsequent layers. [ 217 ]
Abstract Image Classification with Convolutional Neural Networks (CNNs)
Pooling
A CNN contains hidden layers. The input is visible. Then as the layers work to transform the data, "hidden" work goes on. The output layer is visible again. Let's continue to explore the "hidden" layers! Pooling reduces the size of an input representation, in this case, an image. Max pooling consists of applying a max pooling window to a layer of the image: classifier.add(layers.MaxPooling2D(pool_size = (2, 2)))
This pool_size 2×2 window will first find the maximum value of the 2×2 matrix at the top left of the image matrix. This first maximum value is 4. It is thus the first value of the pooling window on the right. Then, the max pooling window hops over 2 squares and finds that 5 is the highest value. 5 is written in the max pooling window. The hop action is called a stride. A stride value of 2 will avoid overlapping, although some CNN models have strides that overlap. It all depends on your goal. Look at the following diagram:
Figure 9.7: Pooling example
The output size has now gone from a 62×62×32 (number of filters) to a 31×31×32, as shown in the following diagram:
Figure 9.8: Output size changes (pooling)
[ 218 ]
Chapter 9
Other pooling methods exist, such as average pooling, which uses the average of the pooling window and not the maximum value. This depends on the model and shows the hard work that needs to be put in to train a model.
Next convolution and pooling layer
The next two layers of the CNN repeat the same method as the first two described previously, and it is implemented as follows in the source code: # Step 3 Adding a second convolutional layer and pooling layer print("Step 3a Convolution") classifier.add(layers.Conv2D(32, (3, 3), activation = 'relu')) print("Step 3b Pooling") classifier.add(layers.MaxPooling2D(pool_size = (2, 2)))
These two layers have drastically downsized the input to 14×14×32, as shown in this diagram:
Figure 9.9: Convolution and pooling layer
It is possible to insert a padding layer on a CNN. As we shrink our image layer by layer, the filters in a convolutional network will impact the center pixels more than the outer pixels. Suppose you start drawing on a piece of paper. You tend to fill the center of the paper and avoid the edges. The edges of the piece of paper contain less information. If you decide to apply padding to the edges, the image will be more complete. In a neural network, padding has the same function. It makes sure the edges are taken into account by adding values. Padding can be implemented before or after pooling, for example. We will implement an example of padding in Chapter 13, Visualizing Networks with TensorFlow 2.x and TensorBoard. The next layer can apply flattening to the output of the pooling of this section, as we'll see in the next section. [ 219 ]
Abstract Image Classification with Convolutional Neural Networks (CNNs)
Flattening
The flattening layer takes the output of the max pooling layer and transforms the vector of size x * y * z into a flattened vector, as shown in the following code: # Step 4 – Flattening print("Step 4 Flattening") classifier.add(layers.Flatten())
In this case, the layer vector will be 14 × 14 × 32 = 6,272, as shown in the following diagram:
Figure 9.10: Flattening layer
This operation creates a standard layer with 6,272 very practical connections for the dense operations that follow. After flattening has been carried out, a fully connected dense network can be implemented.
Dense layers
Dense layers are fully connected. Full connections are possible through the size reductions calculated so far, as shown before. The successive layers in this sequential model have brought the size of the image down enough to use dense layers to finish the job. dense_1 comes first, as shown here:
Figure 9.11: Dense layer
The flattening layer produced a 14×14×32 size 6,272 layer with a weight for each input. If it had not gone through the previous layers, the flattening would have produced a much larger layer, slowing feature extractions down. The result would produce nothing effective. [ 220 ]
Chapter 9
With the main features extracted by the filters through successive layers and size reduction, the dense operations will lead directly to a prediction using ReLU on the dense operation and then the logistic sigmoid function to produce the final result: print("Step 5 Dense") classifier.add(layers.Dense(units = 128, activation = 'relu')) classifier.add(layers.Dense(units = 1, activation = 'sigmoid'))
Now that we have the dense layer, let's explore the dense layer's activation functions.
Dense activation functions
The ReLU activation function can be applied to a dense layer as in other layers. The domain of the ReLU activation function is applied to the result of the first dense operation. The ReLU activation function will output the initial input for values >=0 and will output 0 for values 100 ep=3 #25>2
While the training runs, measurements are displayed: loss, accuracy, epochs, information on the structure of the layers, and the steps calculated by the algorithm. Here is an example of the loss and accuracy data displayed: Epoch 1/2  23s  loss: 0.1437  acc: 0.9400  val_loss: 0.4083  val_acc: 0.5000 Epoch 2/2  21s  loss: 1.9443e06  acc: 1.0000  val_loss: 0.3464  val_acc: 0.5500
Now that we have built and trained the model, we need to save it. Saving the model will avoid having to train the model each time we wish to use it.
Saving the model
By saving the model, we will not have to train it again every time to use it. We will only go back to training when it's required to finetune it. TensorFlow 2 provides a method to save the structure of the model and the weights in a single line of code and a single serialized file: model3.h5 saved in the following code, contains serialized data with the model
structure and weights. It contains the parameters and options of each layer. This information is very useful to finetune the model: print("Step 10: Saving the model") classifier.save(directory+"model/model3.h5")
The model has been built, trained, and saved.
Next steps
The model has been built and trained. In Chapter 10, Conceptual Representation Learning, we will explore how to load and run it with no training.
[ 230 ]
Chapter 9
Summary
Building and training a CNN will only succeed with hard work, choosing the model, the right datasets, and hyperparameters. The model must contain convolutions, pooling, flattening, dense layers, activation functions, and optimizing parameters (weights and biases) to form solid building blocks to train and use a model. Training a CNN to solve a reallife problem can help sell AI to a manager or a sales prospect. In this case, using the model to help a foodprocessing factory solve a conveyor belt productivity problem takes AI a step further into everyday corporate life. A CNN that recognizes abstract concepts within an image takes deep learning one step closer to powerful machine thinking. A machine that can detect objects in an image and extract concepts from the results represents the true final level of AI. Once the training is over, saving the model provides a practical way to use it by loading it and applying it to new images to classify them. This chapter concluded after we had trained and saved the model. Chapter 10, Conceptual Representation Learning, will dive deeper into how to design symbolic neural networks.
Questions 1. 2. 3. 4. 5. 6. 7. 8. 9.
A CNN can only process images. (Yes  No) A kernel is a preset matrix used for convolutions. (Yes  No) Does pooling have a pooling matrix, or is it random? The size of the dataset always has to be large. (Yes  No) Finding a dataset is not a problem with all the available image banks on the web. (Yes  No) Once a CNN is built, training it does not take much time. (Yes  No) A trained CNN model applies to only one type of image. (Yes  No) A quadratic loss function is not very efficient compared to a crossentropy function. (Yes  No) The performance of a deep learning CNN does not present a real issue with modern CPUs and GPUs. (Yes  No)
Further reading and references • TensorFlow 2: https://www.tensorflow.org/beta [ 231 ]
10 Conceptual Representation Learning Understanding cuttingedge machine learning and deep learning theory only marks the beginning of your adventure. The knowledge you have acquired should help you become an AI visionary. Take everything you see as opportunities and see how AI can fit into your projects. Reach the limits and skydive beyond them. This chapter focuses on decisionmaking through visual representations and explains the motivation that led to conceptual representation learning (CRL) and metamodels (MM), which form CRLMMs. Concept learning is our human ability to partition the world from chaos to categories, classes, sets, and subsets. As a child and young adult, we acquire many classes of things and concepts. For example, once we understand what a "hole" is, we can apply it to anything we see that is somewhat empty: a black hole, a hole in the wall, a hole in a bank account if money is missing or overspent, and hundreds of other cases. By performing concept learning, we humans do not have to learn the same concept over and over again for each case. For example, a hole is a hole. So when we see a new situation such as a crater, we know it's just a "big" hole. I first registered a word2vector patent early in my career. Then I rapidly applied it to concept2vector algorithms. I then designed and developed the CRLMM method successfully for automatic planning and scheduling (APS) software, cognitive chatbots, and more, as we will see in the following chapters. The metamodel term means that I applied one single model to many different domains, just as we humans do. [ 233 ]
Conceptual Representation Learning
Conceptual representations also provide visual images of concepts. To plan, humans need to visualize necessary information (events, locations, and so on) and more critical visual dimensions such as image concepts. A human being thinks in mental images. When we think, mental images flow through our minds with numbers, sounds, odors, and sensations, transforming our environment into fantastic multidimensional representations similar to video clips. The following topics will be covered in this chapter: • An approach to CRLMM in three steps: °
Transfer learning to avoid developing a new program for each variation of a similar case
°
Domain learning to avoid developing a new program each time the domain changes
°
The motivation for using CRLMM
Over the years, I've successfully implemented CRL in C++, Java, and logic programming (Prolog) in various forms on corporate sites. In this chapter, I'll use Python to illustrate the approach with TensorFlow 2.x with the convolutional neural network (CNN) built in Chapter 9, Abstract Image Classification with Convolutional Neural Networks (CNNs). • Transfer learning using a CNN model trained to generalize image recognition • Domain learning to extend image recognition trained in one field to another field We'll begin this chapter by looking at the benefits of transfer learning and how concept learning can boost this process.
Generating profit with transfer learning
Transfer learning means that we can use a model we designed and trained in another similar case. This will make the model very profitable since we do not have to design a new model and write a new program for every new case. You will thus generate profit for your company or customer by lowering the cost of new implementations of your trained model. Think of a good AI model as a reusable tool when applied to similar cases. This is why concept learning, being more general and abstract, is profitable. That is how we humans adapt. When it comes to reasoning and thinking in general, we use mental images with some words. Our thoughts contain concepts, on which we build solutions.
[ 234 ]
Chapter 10
The trained model from Chapter 9, Abstract Image Classification with Convolutional Neural Networks (CNNs), can now classify images of a certain type. In this section, the trained model will be loaded and then generalized through transfer learning to classify similar images. You will notice that I did not use many images for the example. My goal was to explain the process, not to go into building large datasets, which is a task in itself. The primary goal is to understand CNNs and conceptual learning representations.
The motivation behind transfer learning
Transfer learning provides a costeffective way of using trained models for other purposes within the same company, such as the food processing company described in Chapter 9, Abstract Image Classification with Convolutional Neural Networks (CNNs). This chapter describes how the food processing company used the model for other similar purposes. The company that succeeds in doing this will progressively generalize the use of the solution. By doing so, inductive abstraction will take place and lead to other AI projects, which will prove gratifying to the management of a corporation and the teams providing the solutions.
Inductive thinking
Induction uses inferences to reach a conclusion. For example, a food processing conveyor belt with missing products will lead to packaging productivity problems. If an insufficient amount of products reaches the packaging section, this will slow down the whole production process. By observing similar problems in other areas of the company, inferences from managers will come up, such as if insufficient amounts of products flow through the process, production will slow down.
Inductive abstraction
The project team in charge of improving efficiency in any company needs to find an abstract representation of a problem to implement a solution through organization or software. This book deals with the AI side of solving problems. Organizational processes need to define how AI will fit in, with several onsite meetings.
[ 235 ]
Conceptual Representation Learning
The problem AI needs to solve
In this particular example, each section of the factory has an optimal production rate (OPR) defined per hour or per day, for example. The equation of an OPR per hour can be summed up as follows:
OPR : min(p(s)) {}'.format( repr(text[:13]), text_as_int[:13]))
In this example, the result is: 'First Citizen'  characters mapped to int  > [18 47 56 57 58 15 47 58 47 64 43 52]
1
The RNN will run through numerical sequences, integer segments, or windows of the text to train and then make predictions. To do this, the program creates examples and targets as for all neural networks that have training batches.
Building the model
Building neural networks with TensorFlow 2 has become so simple to write in a few lines that you can even miss seeing them in the example programs! Let's clarify some basic concepts before getting to those few lines: • A sequential model contains a pile or stack of layers. • Embedding takes the number of each character and stores it in a vector. • GRU stands for gated recurrent unit. A GRU contains gates that manage hidden units, keeping some information and forgetting other information. An RNN GRU can sometimes get confused when the sequences become long and thus mismanage the gradient, which then disappears. The more efficient LSTM units are part of a recurrent network unit as well with feedback connections with a cell, an input gate, an output gate, and a forget gate. But in the end the choice of the types units will always be yours depending on the context of your project. In any case, the key concept to keep in mind is that recurrent networks manage sequences of data, keeping the past in mind while forgetting some information.
[ 427 ]
Improving the Emotional Intelligence Deficiencies of Chatbots
• A dense layer, in this case, is the output layer. • A timestep is a predefined sequence length. In another model, it could be actual time if we are working on timedependent data. A sequential model is built in three layers only: def build_model(vocab_size, embedding_dim, rnn_units, batch_size): model = tf.keras.Sequential([ tf.keras.layers.Embedding(vocab_size, embedding_dim, batch_input_shape=[batch_size, None]), tf.keras.layers.GRU(rnn_units, return_sequences=True, stateful=True, recurrent_initializer='glorot_uniform'), tf.keras.layers.Dense(vocab_size) ]) return model
And that's it! You can replace the basic rnn_units with an LSTM layer if the model requires it during the training phase. Once the model is built, the model: • Looks an embedding up, as in a "dictionary." • Runs the GRU for a timestep. • The dense layer will then generate logits (see Chapter 2, Building a Reward Matrix – Designing Your Datasets) to produce a prediction using a likelihood function, a probability distribution. The following figure of the TensorFlow author's program sums the process up:
[ 428 ]
Chapter 16
Figure 16.17: TensorFlow model
Generating text
After trying and training the model, the program will generate text automatically, for example: print(generate_text(model, start_string=u"ROMEO: "))
[ 429 ]
Improving the Emotional Intelligence Deficiencies of Chatbots
As you'll notice, ROMEO: has been set up as the starting string. It then shows that the following predictions come from the initial text written by Shakespeare and are loaded at the beginning of the program: ROMEO: Isick a tranch It wast points for a sisten of resold thee, testement. Petch doth my sweety beits are so of my sister. KING RICHARD III: Thou forget, How did you burzenty day, 'tis oatly; heaven, for a womanous dear! This is thy for mercy to the Kanging; He that from the brothers of Gloucestersherding blame, Thisble York, se me?
You can go back to the beginning of the program and change the URL. Instead of loading Shakespeare, change it to your own text: path_to_file = tf.keras.utils.get_file('', '')
Before running the program, go to Runtime > Change runtime type:
Figure 16.18: Runtime type
[ 430 ]
Chapter 16
Click on Change runtime type:
Figure 16.19: Notebook settings
I recommend using the GPU. Also, verify that Omit code cell output when saving this notebook is not checked if you want to save your notebook with the results produced when you run the program. You are now ready to explore and do your own research to contribute to the future of automatic text generation!
Summary
Emotional polysemy makes human relationships rich and excitingly unpredictable. However, chatbots remain machines and do not have the ability to manage wide ranges of possible interpretations of a user's phrases. Presentday technology requires hard work to get a cognitive NPL CUI chatbot up and running. Small talk will make the conversation smoother. It goes beyond being a minor feature; courtesy and pleasant emotional reactions are what make a conversation go well. We can reduce the limits of presentday technology by creating emotions in the users through a meaningful dialog that creates a warmer experience. Customer satisfaction constitutes the core of an efficient chatbot. One way to achieve this goal is to implement cognitive functions based on data logging. We saw that when a user answers "no" when we expect "yes," the chatbot needs to adapt, exactly the way we humans do. [ 431 ]
Improving the Emotional Intelligence Deficiencies of Chatbots
Cognitive data logging can be achieved through the preparation we explored in Chapter 14, Preparing the Input of Chatbots with Restricted Boltzmann Machines (RBMs) and Principal Component Analysis (PCA), the cognitive dialog of Chapter 15, Setting Up a Cognitive NLP UI/CUI Chatbot, and the adaptive dialog built in this chapter. In our example, the viewer changed market segments, and the chatbot logged the new profile. Dialogflowfulfillment scripts can manage the whole adaptive process, though that is beyond the scope of this book. We looked at the study of sequences of data through RNNs eventually leading to automatic dialogs. Chatbots, using cognitive approaches such as the RBMPCA and the adaptive data logging inferences of this chapter, will one day build their own dialogs. The following chapters will explore ways to achieve higher levels of artificial intelligence through genes, biological neurons, and qubits. The next chapter explores genetic algorithms and then implements them into a hybrid neural network.
Questions
1. When a chatbot fails to provide a correct response, a hotline with actual humans needs to take over the conversation. (Yes  No) 2. Small talk serves no purpose in everyday life or with chatbots. It is best to just get to the point. (Yes  No) 3. Data logging can be used to improve speech recognition. (Yes  No) 4. The history of a chatbot agent's conversations will contain valuable information. (Yes  No) 5. Presentday technology cannot make use of the data logging of a user's dialogs. (Yes  No) 6. An RNN uses sequences of data to make predictions. (Yes  No) 7. An RNN can generate the dialog flow of a chatbot automatically for all applications. (Yes  No)
Further reading
• Information on RNNs: https://www.tensorflow.org/tutorials/ recurrent
• More on text generation: https://www.tensorflow.org/tutorials/text/ text_generation
[ 432 ]
17 Genetic Algorithms in Hybrid Neural Networks In this chapter and the following two chapters, we will explore the world inside us. First, in this chapter, we will use the model of our genes as an optimizing tool. In Chapter 18, Neuromorphic Computing, we will enter our biological brain activity and create neuromorphic networks. Finally, in Chapter 19, Quantum Computing, we will go even deeper and use the quantum material in us to build quantum mechanic models for quantum computing. A slight change in any of these tiny entities (genes, neurons, qubits) within us can modify our whole existence. In this chapter, we will discover how to enter our chromosome, find our genes, and understand how our reproduction process works. From there, we will begin to implement an evolutionary algorithm in Python, a genetic algorithm (GA). Charles Darwin offered "survival of the fittest" as a model to represent evolution. In some ways, the model is controversial. In 21st century societies, we tend to provide support to those who are not the fittest, as best as possible. However, in mathematics, we do not have this ethical problem. In AI, we need to provide an accurate solution. If we generate several solutions, we can apply the "survival of the fittest" to abstract numbers. In some cases, GAs dramatically reduce the number of combinations required to find the optimal solution to a problem. By generating mathematical offspring, choosing the fittest, and producing new stronger abstract generations, the system often reaches a more optimal solution than propagating permutations. [ 433 ]
Genetic Algorithms in Hybrid Neural Networks
A welldesigned GA can optimize the architecture of a neural network, thereby producing a hybrid neural network. The following topics will be covered in this chapter: • • • •
Evolutionary algorithms; genetic algorithms Extending the genes of genetic algorithms to optimizing tools Hybrid neural networks Using a genetic algorithm to optimize an LSTM
Let's begin by first working to understand what evolutionary algorithms are.
Understanding evolutionary algorithms In this section, we will drill down from our heredity down to our genes to understand the process that we will then represent while building our Python program.
Successive generations of humans activate some genes and not others, producing the wonderful diversity of humanity. A human lifetime is an episode in a long line of thousands of generations of humans. We all have two parents, four grandparents, and eight greatgrandparents, which amounts to 23 ascendants. Suppose that we extend this line of reasoning to four generations per century and then over about 12,000 years when the last glacial period ended and the planet started warming up. We obtain: • 4 * 1 century * 10 centuries = 1,000 years and 40 generations • 40 generations * 12= 480 • Adding up to 2480 mathematical ascendants to anybody living today on the planet! Even if we limit ourselves to 1,000 years, 240, that adds up to 1,099,511,627,776 ascendants a thousand years ago. But there is a problem. This figure is impossible! Today, we have reached the height of the human population, which is only 7,500,000,000. So, this means that our ascendants had many children who married their cousins of all degrees, making humanity one large extended family, no matter what our skin color or hair color is!
Heredity in humans
First, men fertilize women when a male cell unites with a female cell. The fertilized egg grows and, after quite an adventure, is born and becomes one of us writing or reading this book. [ 434 ]
Chapter 17
To grow and live from being a fertilized egg to human adults, we must gather much of the outside world, absorb, and transform it. This transformation of food and materials by us until we grow into something that is more or less like our ancestors is called heredity. If life were calm and quiet, nothing noticeable would occur. However, our environment has exerted relentless pressure on us for millions of years, back to when we were just some kind of bacteria floating around in an ocean somewhere. That pressure brought about continuous natural selection; what worked would live on, whilst what didn't work would die out. That pressure continues up to present day, forcing us, humans, to adapt genetically or disappear. Those humans who failed to adapt to face the pressure of their environment died out. We who live today have survived. Evolution can be defined as a state of constant conflict. On one hand we have our relentless, often hostile environment. On the other, our equally relentless genes; many of which die out, but others morph, adapt, and continue on through heredity— indeed, they're doing that even now, and who knows what is going to happen next?
Our cells
In our cells, the nucleus contains personal biological data in the form of chromosomes. Our cells contain 46 chromosomes per cell, which, in turn, are formed by 23 pairs of chromosomes. One of these pairs is a sex cell to determine our sex. Inside the chromosomes, we have genes, especially the mitochondrial genome, which is the DNA in tiny cells that take our food and transform it into fuel for our cells. Each cell is a microscopic busy factory containing thousands of genes! The human genome describes an incredible set of sequences for our building blocks contained in the 23 chromosomes of our cells.
How heredity works
Except for the sex cells, we inherit twentythree of our mother's fortysix chromosomes and twentythree of our father's chromosomes. In turn, our parents' cells contain the chromosomes of their parents—our grandparents—and so on in various proportions. Let's take an example to view the complexity of what we are facing. We will take one of our parents, either our mother or father. P represents the set of chromosomes of that parent. [ 435 ]
Genetic Algorithms in Hybrid Neural Networks
Letters with primes will represent their fathers'—our grandfathers'—chromosomes, and letters with double primes will represent their mothers'—our grandmothers'— chromosomes. We can represent this as follows for your father in his sex cell: 𝑃𝑃 = { 𝐴𝐴′ , 𝐵𝐵′ , 𝐶𝐶 ′ , 𝐷𝐷 ′ , 𝐸𝐸 ′ , 𝐹𝐹 ′ , 𝐺𝐺 ′ , 𝐻𝐻′ , 𝐼𝐼′ , 𝐽𝐽′ , 𝐾𝐾 ′ , 𝐿𝐿′ , 𝑀𝑀′ , 𝑁𝑁 ′ , 𝑂𝑂′ , 𝑃𝑃′ , 𝑄𝑄′ , 𝑅𝑅 ′ , 𝑆𝑆 ′ , 𝑇𝑇 ′ , 𝑈𝑈 ′ , 𝑉𝑉 ′ 𝑌𝑌 ′ 𝐴𝐴′′ , 𝐵𝐵′′ , 𝐶𝐶 ′′ , 𝐷𝐷 ′′ , 𝐸𝐸 ′′ , 𝐹𝐹 ′′ , 𝐺𝐺 ′′ , 𝐻𝐻′′ , 𝐼𝐼′′ , 𝐽𝐽′′ , 𝐾𝐾 ′′ , 𝐿𝐿′′ , 𝑀𝑀′′ , 𝑁𝑁 ′′ , 𝑂𝑂′′ , 𝑃𝑃′′ , 𝑄𝑄′′ , 𝑅𝑅 ′′ , 𝑆𝑆 ′′ , 𝑇𝑇 ′′ , 𝑈𝑈 ′′ , 𝑉𝑉 ′′ 𝑋𝑋 ′′}
For your mother, the last chromosome of the first set would be an X in her sex cell:
𝑃𝑃 = { 𝐴𝐴′ , 𝐵𝐵′ , 𝐶𝐶 ′ , 𝐷𝐷 ′ , 𝐸𝐸 ′ , 𝐹𝐹 ′ , 𝐺𝐺 ′ , 𝐻𝐻′ , 𝐼𝐼′ , 𝐽𝐽′ , 𝐾𝐾 ′ , 𝐿𝐿′ , 𝑀𝑀′ , 𝑁𝑁 ′ , 𝑂𝑂′ , 𝑃𝑃′ , 𝑄𝑄′ , 𝑅𝑅 ′ , 𝑆𝑆 ′ , 𝑇𝑇 ′ , 𝑈𝑈 ′ , 𝑉𝑉 ′ 𝑋𝑋 ′ 𝐴𝐴′′ , 𝐵𝐵′′ , 𝐶𝐶 ′′ , 𝐷𝐷 ′′ , 𝐸𝐸 ′′ , 𝐹𝐹 ′′ , 𝐺𝐺 ′′ , 𝐻𝐻′′ , 𝐼𝐼′′ , 𝐽𝐽′′ , 𝐾𝐾 ′′ , 𝐿𝐿′′ , 𝑀𝑀′′ , 𝑁𝑁 ′′ , 𝑂𝑂′′ , 𝑃𝑃′′ , 𝑄𝑄′′ , 𝑅𝑅 ′′ , 𝑆𝑆 ′′ , 𝑇𝑇 ′′ , 𝑈𝑈 ′′ , 𝑉𝑉 ′′ 𝑋𝑋 ′′}
Women are XX and men XY. Imagine the possibilities!
If we only take A, B, and C in only one of our parent's cells, we already obtain the following set, C, of the eight combinations we would inherit:
𝐶𝐶 = {𝐴𝐴′ 𝐵𝐵′ 𝐶𝐶 ′ , 𝐴𝐴′′ 𝐵𝐵′′ 𝐶𝐶 ′′ , 𝐴𝐴′ 𝐵𝐵′′ 𝐶𝐶 ′′ , 𝐴𝐴′′ 𝐵𝐵′ 𝐶𝐶 ′ , 𝐴𝐴′ 𝐵𝐵′ 𝐶𝐶 ′′ , 𝐴𝐴′′ 𝐵𝐵′′ 𝐶𝐶 ′ , 𝐴𝐴′ 𝐵𝐵′′ 𝐶𝐶 ′ , 𝐴𝐴′′ 𝐵𝐵′ 𝐶𝐶 ′′ }
If we extend this to the twentythree chromosomes, the distribution climbs up to 223, or 8,388,608 possibilities. Our evolutionary process contains the right potential for evolutionary algorithms.
Evolutionary algorithms
In this section, we will drill down further into evolutionary algorithms, getting closer to our Python programs. Evolutionary algorithms can be used in any field in which combinations are useful: scheduling, medical research on DNA, weather forecasting, neural network architecture optimizing and a limitless number of domains. Evolutionary computation is a set of algorithms that apply trialanderror techniques to reproduce an abstract mathematical version of biological evolution. This mathematical model does not contend with having solved the difficult task of explaining evolution, which naturally cannot be reduced to a few equations. However, our biological environment produces frameworks that, though the fruits of our imagination, enable us to create efficient abstract algorithms. [ 436 ]
Chapter 17
Evolutionary algorithms enter the category of evolutionary computation. An evolutionary algorithm contains processes such as mutation, crossover, and selection. Many models can achieve the goals set for an evolutionary process. A GA introduces the category of an evolutionary algorithm. We will first define the concepts involved in a GA, which are the building blocks of a Python program.
Going from a biological model to an algorithm
There are many ways of creating a GA model. You can reproduce the exact description of the human model described in the preceding section, you can simplify it, or you can create another view. Our model will contain a set of genes in a chromosome and a population to interact with:
Figure 17.1: Chromosomes in genes
Our model is an abstract model for algorithms, not the actual representation of human chromosomes that come in pairs, for example. A gene in an individual's chromosome will interact with the gene set of the population. This process will be defined in the following sections.
Basic concepts
Let's first describe the concepts of the building blocks of our Python program: • Target: Target defines the length and properties of the child we wish to obtain after n generations. In our model, the target can be specified or unspecified.
[ 437 ]
Genetic Algorithms in Hybrid Neural Networks
A specified target contains a length and its value: target = "Algorithm" # No space unless specified as a character in the gene set
In this case, our environment requires an exact gene set to remain fit in that environment. See the following fitness function. An unspecified target contains a length, but not its actual value, which must be found by the GA after n generations: target="AAAA" #unspecified target
In this case, our environment does not require an exact gene, but rather a gene with the features required to remain fit in that environment. See the following fitness function. • Population: Population first defines the selection of an individual at random we will call parent that will contain a certain length of its string of genes: def gen_parent(length)
Population also defines the potential individuals the parent can interact with to produce a child. In our genetic simulation, the size of the population is represented by the gene set of the population (see the following point). • Gene set of the parent: The gene set of the parent will first be a random set of genes. • Gene set of the population: This parent will then randomly encounter another person in the population with random gene choices. This population is represented by a gene set that we will draw genes from randomly: geneSet = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!."
Since a given population can possess any or several of these genes, our population is very large from the start. • Selection: We will select genes randomly from our gene set to simulate the meeting of our parent with another parent: index=random.randrange(0,len(parent))
• Crossover: We will then form a child from the parent and the random gene taken from our population represented by the gene set. In our model, once the random selection of the gene has been made, the gene will be replaced by a random gene taken from the population represented by the gene set of the population: if(newGene!=oldGene):childGenes[index]=newGene;
[ 438 ]
Chapter 17
Notice that the new gene must be different from the old gene during the crossover in our algorithm. Our algorithm thus avoids getting stuck in some local combination during the reproduction phase. Child after child, we will produce new generations. • Mutation: We will not accept the same gene from a parent to form a child. If we detect this, we will randomly change a gene to make sure each generation is different. As described in the Crossover paragraph, we will not accept a child with the same genes as a parent. As seen in the heredity section of this chapter, it is unlikely that a child would inherit exactly all of the genes of a given parent. Diversity is the key to producing generation after generation of children that will adapt to their environment. In our model, a diversity rule forcing mutation has been introduced: if(newGene==oldGene):childGenes[index]=alternate;
We thus introduce an alternate gene, as we will see while building the Python program. • Child: Child defines a set of genes that contains the same number of genes as the parent but with new genes. Generations of children of child gene strings will be produced and then complete the selection process with a fitness function. • Fitness: Fitness defines the value of the child as defined in a given model. The fittest will then be selected to be the parent for the next generation. In our model, we have two fitness functions that we define in a scenario variable. If scenario=1, then a specified target scenario will be activated. The target will be specified to fit the surrounding environment. Polar bears became white, for example, to blend in with the surrounding snow and ice of their environment. In other areas, bears are often brown, for example, to blend in with the surrounding vegetation. The fitness function thus has a target to reach. For example, in one instance in the program: target="FBDC"
This target could mean many things, as we will see. In the case of the bears, maybe these genes in one of their chromosomes trigger off their color: white or brown.
[ 439 ]
Genetic Algorithms in Hybrid Neural Networks
If our program simulating nature does not produce the right gene in a given generation, this means that the bear is not mutating correctly and will not survive. The fitness function of the Python program simulates nature by only keeping the strings of genes that make the child evolve in the right direction. The specified target, in this case, is an identified string of genes that will make a lifeanddeath difference. def get_fitness(this_choice,scenario): if(scenario==1):
As we will see later, scenario==1 will make sure that the exact gene set required to survive is reached. If scenario=0, then an unspecified target will be activated. The length of the target is not specified. However, a feature set will define the value of the gene set of the population. This feature contains numerical values that open the door to any optimization you wish to solve, as we will see in the Python program example. This numerical set is called the KPI set because the values are key performance indicators of the system we will explore in this model: KPIset ="0123456772012345674701234569980923456767012345671001234 " #KPI set
The KPIset feature set matches the size of the gene set of the population.
Building a genetic algorithm in Python We will now build a GA from scratch using GA.ipynb.
You can use the .py version. The code is the same, although the line numbers in this chapter refer to the Jupyter notebook cells of the .ipynb version. At all times, you can go back to the previous section, Basic concepts, to consult the definitions used to describe the Python program in the following sections.
Importing the libraries
This program is built from scratch with no higherlevel library to get the feel of a GA. Three lines are enough to get everything working: import math import random import datetime [ 440 ]
Chapter 17
Calling the algorithm
In this program, we will be exploring three scenarios. Two scenarios generate specified targets, and one generates an unspecified target. We will begin with a specified target and then move to the more advanced unspecified target that will prepare us for hybrid networks using a GA. We will first go to the Calling the Algorithm cell of the program. The first task is to define which scenario and type of fitness function we will use on line 3: scenario=0 optimizer
# 1=target provided at start, 0=no target, genetic
If scenario=1, the program generates the correct exact genes for a child from several generations, starting with a random seed parent. If scenario=0, the program generates the best features of a type of genes of a child from several generations, starting with a random seed parent. In line 4, GA=2 defines which target we are dealing with. If GA=1, the gene set of the population and the target are defined. The main function is called: if(GA==1): geneSet = "abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!." # target with no space unless specified as a character in the geneSet target = "Algorithm" # No space unless specified as a character in the geneSet print("geneSet:",geneSet,"\n","target:",target) ga_main()
At this point, the gene set of the population is printed along with the target: geneSet: abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!.target: Algorithm
The last line calls ga_main(), the main function.
The main function
The code is in the ga_main() cell of the program. The ga_main() function is divided into three parts: parent generation, child generation, and a summary. [ 441 ]
Genetic Algorithms in Hybrid Neural Networks
The parent generation process The parent generation runs from line 2 to line 7:
startTime=datetime.datetime.now() print("starttime",startTime) alphaParent=gen_parent(len(target)) bestFitness=get_fitness(alphaParent,scenario) display(alphaParent,bestFitness,bestFitness,startTime) #no childFitness for generation 1 so default value sent
• startTime indicates the start time, which is printed: starttime 20191012 10:32:28.294943
• alphaParent is the first parent who will be created by the gen_parent function, which will be described in the following Jupyter notebook cell. • bestFitness is the fitness estimated by the get_fitness function, which will be described in this chapter. • display is the function that describes the result of this process. We now have a process that creates a parent: gen_parent, get_fitness, and display. We will now explore the parent generation function before resuming the main function.
Generating a parent
The parent generation function cell starts with zero genes and the length of the target: def gen_parent(length):
length = the length of the target. The target is "Algorithm", so length=9.
At the beginning of the process, the parent has no genes since the goal of this function is to produce a parent at random that contains a string of genes that is equal to the length of the target. The string of genes to produce a parent is as follows: genes=[]
#genes array
Now, a while loop starts on line 3 of the cell to fill genes[] until it reaches the target's length: while len(genes)=childFitness:
°
If the condition is True, then the evolution must continue until a child is fitter than its parent. This sends the process back to the top of the while loop.
°
If the condition is False, this means that the child is fitter than the parent, and then the code will go beyond the condition.
3. The code beyond the condition and continue method: °
The code displays the child, the bestFitness becomes the childFitness, and the bestParent is now the child: display(child,bestFitness,childFitness,startTime) bestFitness=childFitness bestParent=child
°
The goal of the two scenarios of our model is defined. The goal of scenario==1 is to reach the length of the target with the right genes. The goal of scenario==0 will be to reach a threshold we will define in the next section: if scenario==1: goal=len(alphaParent); if scenario==0: goal=threshold;
4. The break condition of the loop: The evolutionary process will stop when the fittest child has been created after several generations containing the genes that meet the target: if childFitness>=goal: break
The output of the n generations will appear as follows: Genetic Algorithm geneSet: abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!.target: Algorithm starttime 20191012 20:47:03.232931 Selection: Xe!bMSRzV Fittest: 0 This generation Fitness: 0 Time Difference: 0:00:00.000953 Selection: Xe!bMSRhV Fittest: 0 This generation Fitness: 1 Time Difference: 0:00:00.002404 Selection: Xl!bMSRhV Fittest: 1 This generation Fitness: 2 Time Difference: 0:00:00.004391 [ 449 ]
Genetic Algorithms in Hybrid Neural Networks Selection: XlgbMSRhV Fittest: 2 This generation Fitness: 3 Time Difference: 0:00:00.006860 Selection: XlgoMSRhV Fittest: 3 This generation Fitness: 4 Time Difference: 0:00:00.009525 Selection: AlgoMSRhV Fittest: 4 This generation Fitness: 5 Time Difference: 0:00:00.011954 Selection: AlgorSRhV Fittest: 5 This generation Fitness: 6 Time Difference: 0:00:00.013745 Selection: AlgorSthV Fittest: 6 This generation Fitness: 7 Time Difference: 0:00:00.016339 Selection: AlgorithV Fittest: 7 This generation Fitness: 8 Time Difference: 0:00:00.019031 Selection: Algorithm Fittest: 8 This generation Fitness: 9 Time Difference: 0:00:00.022239
We can see the display of all of the generations as described in the display function in the Display parent section of this chapter.
Summary code
Once the evolutionary process is over, a summary code takes over: #III. SUMMARY print("Summa ry") endTime=datetime.datetime.now() print("endtime",endTime) print("geneSet:",geneSet);print("target:",target) print("geneSet length:",len(geneSet)) print("target length:",len(target)) print("generations:",g) print("Note: the process is stochastic so the number of generations will vary")
The output for the example displayed for the evolutionary loop is: Summaryendtime 20191012 20:47:03.257112 geneSet: abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!.target: Algorithm geneSet length: 55 target length: 9 generations: 782 Note: the process is stochastic so the number of generations will vary [ 450 ]
Chapter 17
Before we move on, it is important to note that using permutations for 9 out of 55 elements, it would take 10 ** 15.36 calculations to reach the goal instead of the 782 generations in this example. GAs are thus a productive way of generating permutations.
We have now explored the core of the evolutionary process and the Python code. We will now build the unspecified target and optimizing code that will lead us to hybrid neural networks.
Unspecified target to optimize the architecture of a neural network with a genetic algorithm In this section, we are laying the grounds and motivation to optimize the architecture of neural networks through a hybrid neural network. The architecture of a physical neural network will be optimized by a GA. We will study a physical neural network and then see how to optimize its architecture with our GA.
A physical neural network
We will begin with a physical network named SFNN, a feedforward neural network (FNN). Please look very closely at the following figure and take as much time as necessary to understand its architecture:
Figure 17.4: Architecture of a neural network
[ 451 ]
Genetic Algorithms in Hybrid Neural Networks
This physical network has some important specific features: • This is an FNN • There are three layers. • The neurons in each layer are not fully connected to the neurons of the next layer. The connections between the neurons of the three layers are not based on all of the possibilities. The connections between the layers are based upon statistics of the best connections in the past when running this network. The connections are the result of many runs of this network to determine which ones are the best. • The connections represented are the most productive ones, leading to an output of 1 and not 0. An output of 1 is a success, while an output of 0 is a failure. • The input is a random value of the dataset that must be classified as 1 if it is a success and 0 if it is a failure. • After a careful study of past runs of this network, it has been found that the productivity of the network fully relies, in this example, on the number and quality of the neurons in layer 2. Let's use what we have learned and apply it to the architecture of a soccer team.
What is the nature of this mysterious SFNN?
SFNN is a soccer team's architecture! It is the representation of a 343 disposition of a soccer team before a given game. Like abstract neural networks, the number and quality of neurons we choose per layer are critical. In our abstract world of artificial neural networks (ANNs), we have many problems to solve. How many should we keep? How many should we eliminate through pooling layers? How many should we abandon through dropout layers? How do we really know? How long will trial and error take with large datasets and networks? Now, let's go back to the figure, read the following explanation, and start finding a method to solve the architectural complexity of a neural network: • Input represents the goal of best possible choices to input the ball in the game based on past game statistics. • Layer 1 is the defense layer of three players represented by three neurons.
[ 452 ]
Chapter 17
• Layer 2 is the middle field, the result of the transformation of layer 1's activity, the statistics of past games showing the fittest, the connections. The initial ball of the input has now been flowing through these layers game after game. • Layer 3 is the attack layer, the one that will lead to the output classification; 1 for a success meaning a goal has been scored, or 0 for failure. The connections between layer 2 and layer 3 show the fittest results found game after game. The problem to solve: Layer 2, in our example, has been identified as the critical layer in this team. For years now, it has been a weak point. Let's see how our GA will help find the fittest players for this layer. We could run scenario==1 (specified target) with target ga==3 as follows: if(scenario==1 and GA==3): target="FBDC" # No space unless specified as a character in the geneSet print("geneSet:",geneSet,"\n","target:",target) ga_main()
The geneSet is the population of available players on the soccer market and target is the string of genes we need for layer 2 of our physical network: Genetic Algorithm geneSet: abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!.target: FBDC
The target is found after 851 generations in this run. However, we would like the GA to find the players by itself based on the features with an unspecified target. Let's explore this step by step, cell by cell.
Calling the algorithm cell
From lines 7 to 12, we defined the parameters of the target architecture we want for layer 2 of our network: • geneSet is the set of all the available players for layer 2 of our network, whether they are in the team or on the market: geneSet="abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!." #gene set
[ 453 ]
Genetic Algorithms in Hybrid Neural Networks
• KPIset, or the key performance indicator set, is a performance score per player on the market for middle field. Each integer of the set contains a score between 0 and 9 based on the statistics of a player when playing as a midfielder: KPIset ="0123456772012345674701234569980923456767012345671001234 " #KPI set
The KPI set matches each member of the gene set. • threshold is the sum of the midfielders' performance we need to attain in order to have a strong midfield in the team. threshold=35
Our goal is to find a string of genes based on their features, their KPI properties. The goal is to reach the threshold. We call our evolutionary function on line 23: if(scenario==0 and GA==2): target="AAAA" #unspecified target print("geneSet:",geneSet,"\n","target:",target) ga_main()
As you can see, target is set to a meaningless value that will rapidly evolve. The GA will have to find a fit sequence. We can skip the intermediate fitness functions we have described in the previous section and focus on the scenarios within the fitness cell.
Fitness cell
We have already described the fitness function of the fitness cell in the previous sections. We will focus on the code of the unspecified target, scenario==0. The first part of the scenario calculates the sum of the performance of each gene (potential midfielder): if(scenario==0): cc=list(this_choice) # cc= this choice gs=list(geneSet) # gene set cv=list(KPIset) # value of each KPI in the set fitness=0 for op1 in range(0,len(geneSet)): #2.first find parent gene in gene set for op in range(0,len(target)): if cc[op]==gs[op1]: #3.gene identified in gene [ 454 ]
Chapter 17 set vc=int(cv[op1])
#4.value of critical path
constraint fitness+=vc
The collective fitness of the sequence of genes (midfielders) is contained in the fitness variable. However, we cannot accept the same gene twice, which would mean we have a clone of the midfielder on the field! So, we add some safety code to set fitness to 0 in that case: for op in range(0,len(target)): for op1 in range(0,len(target)): if op!=op1 and cc[op]==cc[op1]: fitness=0 # no repetitions allowed, mutation enforcement
Now, we can go back to ga_main() and complete our process.
ga_main() cell
We have already described the fitness cell in the previous sections. Now, we will focus on the code of the unspecified target, scenario==0. In the ga_main() cell, we simply need to examine lines 22 to 24: if scenario==0: goal=threshold; if childFitness>=goal: break
If the scenario==0, childFitness must be >=goal (sum of KPIs). We have found our midfielders! We will now display the result: Genetic Algorithm geneSet: abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!.target: AAAA starttime 20191013 09:08:21.043754 Selection: PNVx Fittest: 18 This generation Fitness: 18 Time Difference: 0:00:00.001252 Selection: LNVx Fittest: 18 This generation Fitness: 24 Time Difference: 0:00:00.004130 Selection: LNVq Fittest: 24 This generation Fitness: 27 Time Difference: 0:00:00.004708 [ 455 ]
Genetic Algorithms in Hybrid Neural Networks Selection: LNFq Fittest: 27 This generation Fitness: 29 Time Difference: 0:00:00.010362 Selection: LBFq Fittest: 29 This generation Fitness: 31 Time Difference: 0:00:00.015797 Selection: CBFq Fittest: 31 This generation Fitness: 33 Time Difference: 0:00:00.024712 Selection: CBFt Fittest: 33 This generation Fitness: 34 Time Difference: 0:00:00.032680 Selection: CBFD Fittest: 34 This generation Fitness: 35 Time Difference: 0:00:00.048524 Summaryendtime 20191013 09:08:21.094005 geneSet: abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ!.target: AAAA geneSet length: 55 target length: 4 generations: 154 Note: the process is stochastic so the number of generations will vary
In this case, the GA chose midfielders C, B, F, and D. We now have all of the ingredients, concepts, and knowledge required to understand an artificial hybrid neural network.
Artificial hybrid neural networks
In the previous section, we used a GA to optimize a physical neural network. In this section, we will extend the concept of hybrid we have just explored to ANNs. The principle is the same, so it will be relatively easy, with the concepts you now have in mind, to intuitively grasp the RNN we will optimize in this section. The future of AI in society lies in the collective intelligence of humans (diversity), machines (AI and IoT), and nature (sustainable projects for our survival). In AI, this diversity lies in ensemble algorithms, metaalgorithms and hybrid systems. Deep learning has proven its point. We can create a neural network with TensorFlow 2.x in a few lines. However, more often than not, it takes days, weeks, and often months to finetune ANN models that rely on large amounts of data to provide a reliable model. And that's where hybrid neural networks are necessary.
[ 456 ]
Chapter 17
A deep learning network can use any form of another type of algorithm to become a more efficient system. In our case, we have chosen evolutionary algorithms that could be used in deep learning: • To improve the input by extracting sequences of data that fit the patterns we are looking for using features of the data to find the fittest data points • To optimize the weights of the layers of a network to boost the speed and power of traditional optimizers • To improve the classification phase of the output of a network by selecting the fittest solutions processed as a sequence of genes • To improve the tedious task of defining the architecture of a network In this section, we will focus on an example of optimizing the architecture of a network with a GA by: • Creating the RNN • Introducing a GA that will optimize the architecture of the network Note: For information on genetic algorithm representations, please go back to the previous sections at all times to refresh the process of an evolutionary algorithm. Please also refer back to the previous chapters to consult the structure of a neural network, in particular, Chapter 16, Improving the Emotional Intelligence Deficiencies of Chatbots, which describes an RNN.
Building the LSTM
Open Genetic_Algorithm_LSTM.ipynb in Google Colaboratory or Jupyter on your machine or genetic_algorithm_lstm.py. The model of this education example is an LSTM. The goal will be to use the window size generated by the GA and run the LSTM with this window size. Rootmeansquare error (RMSE) will be used to measure the fitness of the configuration. In short, RMSE first calculates the square of the difference between the training data and the testing data, like many similar formulas. Then, the root of the result is calculated. Just keep in mind that the RMSE will compare what we expect to get with what we actually get and produce a value.
[ 457 ]
Genetic Algorithms in Hybrid Neural Networks
Let's now explore the main cells of the program: • Importing required packages cell: Genetic_Algorithm_LSTM.ipynb starts by installing DEAP, an evolutionary computation framework: !pip install deap
We built a GA from scratch in the previous sections. We won't need to begin from nothing this time, since this program uses a framework. Then, the program installs bitstring, which helps to process binary data. • Loading the data cell: The data wind power forecast data in train.csv comes from https://www.kaggle.com/c/GEF2012windforecasting/ data. The wp1 to wp7 columns provide normalized data collected from measurements of the wind power of seven wind farms. The goal of the LSTM will be to take the sequence of data and make wind power forecasts. • Defining the basic functions cell: This cell prepares the dataset and trains the model in a standard process. We will focus on line 14: window_size = window_size_bits.uint
• Evolutionary model cell: The model uses the DEAP framework function, but we easily recognize the concepts we explored in the previous sections and that are initialized as follows: population_size = 4 num_generations = 2 gene_length = 10
The code is all set with a readytouse GA to optimize the window size for our network.
The goal of the model
As in our previous section, the goal is to find the best window size for this network, just as we were looking for the best layer 2 in an earlier section. The model has done the following tasks: • Installed the packages, loaded the data, and found the window size of the LSTM • Then, it ran the GA model to test the possible window size of the LSTM • An RMSE measurement is provided with each generation and production of epochs [ 458 ]
Chapter 17
The program takes some time to run, but the results I ran are saved in the notebook so that you can view them. Here is one result to view how the system works: Epoch 1/5 17200/17200 [==============================]  207s 12ms/sample  loss: 0.0096 Epoch 2/5 17200/17200 [==============================]  202s 12ms/sample  loss: 0.0059 Epoch 3/5 17200/17200 [==============================]  202s 12ms/sample  loss: 0.0058 Epoch 4/5 17200/17200 [==============================]  200s 12ms/sample  loss: 0.0057 Epoch 5/5 17200/17200 [==============================]  200s 12ms/sample  loss: 0.0057 Test RMSE: 0.0926447226146452
As in the previous section, a GA optimized a section of an ANN. You can use a GA for other components of an ANN. The sky is the limit! In fact, there is no limit. The potential of hybrid neural networks using GA or other algorithms to optimize their architecture or process takes your projects to another level!
Summary
Evolutionary algorithms bring new light to AI's optimizing potential. In this chapter, we studied how heredity deeply affects population distribution. The impact of our environment can be measured through genetic mutations. Drilling down further, we focused on a class of GAs implementing a simulation of genetic transformations through many generations. We explored how a parent will transmit some genes, but how the selection of diverse genes from the generation population of genes will produce variations. A chromosome will inherit some genes but not others. The pressure of nature and our environment will take over. A fitness function evaluates a string of genes. Only the fittest will survive. The fittest genetic material will produce a crossover and mutation of the child, making it fitter for its environment.
[ 459 ]
Genetic Algorithms in Hybrid Neural Networks
GAs can be used to represent strings of any type of data and also features of that data. The optimizing process can be applied to warehouses, transportation, and neural networks. Hybrid networks will no doubt expand in the years to come, taking DL to the next level. Using a genetic algorithm to optimize the architecture of an RNN paves the way to optimize the architecture of any DL, ML or AutoML architecture. For example, a hybrid neural network can use a genetic algorithm to optimize inputs with feature reduction or as the weight optimizing function of the network. Nature has provided us with invaluable tools to apply to our artificial network models. This chapter dealt with the invisible building blocks inside us. In the next chapter, Neuromorphic Computing, we will explore other components that enable us to adapt to our environment: neurons. We will explore how neural networks using biological models can solve complex problems.
Questions
1. A cell contains 42 chromosomes. (Yes  No) 2. A genetic algorithm is deterministic, not random. (Yes  No) 3. An evolutionary algorithm means that the program code evolves. (Yes  No) 4. It is best for a child to have the same genes as one of the parents even after many generations. (Yes  No) 5. Diversity makes the gene sets weaker. (Yes  No) 6. Building a neural network only takes a few lines, and the architecture always works. (Yes  No) 7. Building a neural network with a genetic algorithm can help optimize the architecture of the layers. (Yes  No) 8. Hybrid neural networks are useless since deep learning will constantly progress. (Yes  No) 9. Would you trust a genetic algorithm to make decisions for you? (Yes  No) 10. Would you trust a hybrid neural network to optimize the architecture of your network? (Yes  No)
Further reading
• https://github.com/DEAP/deap • https://pypi.org/project/bitstring/ [ 460 ]
18 Neuromorphic Computing Our brain activates thousands or even billions of neurons when necessary, getting our body battleready to face any situation. As we saw in Chapter 17, Genetic Algorithms in Hybrid Neural Networks, evolution has finedtuned biological capacities over thousands of generations and millions of years. In this chapter, we will take a deeper look into the cognitive power inside our bodies. We will go from the chromosomes of the previous chapter to biological neurons that make us intelligent creatures. The neurons interact in billions of ways producing cognitive patterns leading to mind structures. Neuromorphic computing taps into the tremendous optimized power of our brain, which surprisingly consumes very little energy. On average, we consume a few watts, less than a lightbulb, to solve very complex problems. In itself, this shows that the neuronal structure of our brain has a unique architecture that we have yet to reproduce physically. To bring neuromorphic computing into the real world requires hardware and software, as in all computer science models. In this chapter, we will focus on the software, though it is important to mention the hardware associated with the neuromorphic research Intel is conducting. That hardware takes the form of a chip named Loihi, after the emerging Hawaiian underwater volcano that will hit the surface one day. Loihi contains thousands upon thousands of neurons with their synapses, dendrites, and axons reproducing our brain activity. IBM and other corporations have been conducting research in this area. We have around 100 billion neurons. These chips are only reaching hundreds of thousands of neurons. However, by connecting thousands of those chips in physical networks, neuromorphic computing will be an area we all will have to take into account in the near future. [ 461 ]
Neuromorphic Computing
We will first define what neuromorphic computing is and then explore Nengo, a unique neuromorphic framework with solid tutorials and documentation. Nengo is one among many other approaches that go beyond the scope of this book. This chapter is not a neuromorphic course but rather an overview, inviting you to tap into the wonderful power of our brain structures to solve complex problems. The problems we will explore will bring us closer to understanding how our brain works. The following topics will be covered in this chapter: • What neuromorphic computing is • Getting started with Nengo • Basic Nengo concepts • Exploring the Nengo tutorial and interface • The difference between Nengo and classical AI • Applying Nengo's unique Semantic Pointer Architecture (SPA) model to critical research areas Let's start with the basics—what is neuromorphic computing?
Neuromorphic computing
Let's go directly to the core of our thought process to understand neuromorphic computing. For AI experts, I would like to summarize the voyage from our classical models to cuttingedge neuromorphic models in a single phrase:
from mind to brain If we take this further, M is the set of all of our mental representations and B is the world of physical reactions that lead to thinking patterns. In this sense, M is a set of everything we have explored up to this point in this book:
M = {rule based systems, machine learning, deep learning, evolutionary algorithms … m} m is any mathematical mental representation of the world surrounding us. In deep learning, for example, an artificial neural network will try to make sense of the chaos of an image by searching the patterns it can find in an image through lower dimensions and higher levels of abstraction. However, a mental construction, no matter how efficient it seems, remains a representation, not a physical reality. [ 462 ]
Chapter 18
Now, let's observe B = brain constructions: B = phenomena/events (inside us or in the outer world) > physical stimuli > physical neural activity in the brain > higher activity in the target zones > physical electric learning reactions > a human action The architecture of B takes us much closer to reality! Mental representations are minimized, thus reducing the distortion of artificial constructions regardless of their efficiency. Classical AI is about building mental representations of our cognitive activity. Neuromorphic computing is about building a brain that can encode reality, process it like a human brain, and decode the result. Bear in mind that corporations such as Intel are providing the chips to accomplish wonderful things with neuromorphic computing, as we will discover in this chapter. Now that we have some idea of what neuromorphic computing is, let's take a look at the neuromorphic framework, Nengo.
Getting started with Nengo
In a nutshell, Nengo builds brains, not mental representations, as in classical machine learning and deep learning. Nengo stands for Neural Engineering Object. It has both scripting capability with Nengo and a graphical capacity with Nengo GUI. We will be using NEF, which is Nengo's Neural Engineering Framework (NEF). Nengo was created by the Centre for Theoretical Neuroscience at the University of Waterloo (Ontario, Canada). Chris Eliasmith has played an important role in this project. We have explored many ways to approach cognitive modeling in the previous chapters. Nengo uses an NEF to implement an SPA. A semantic pointer is a neural representation in a biological system that carries structures that will lead to higherlevel cognitive representations. The term pointer refers to pointers as we know in C++, for example, because they can access data they do not contain.
[ 463 ]
Neuromorphic Computing
The term semantic refers to the fact that they are not just mathematical tools as in C++ because they contain virtual representations through the distances between them. If we put the two concepts together, this leads to the mindblowing concept of meaning being generated through biological pointer activity located at various distances and states from each other in our brain. Let's dive into neuromorphic computing by first installing Nengo.
Installing Nengo and Nengo GUI
For this chapter, I used the Python interfaces with NumPy and Matplotlib libraries, as we have since the beginning of this book. All that is then required is to install Nengo with pip, for example: pip install nengo
You can install it using other approaches. For more information, go to https:// github.com/nengo/nengo. Let's install the nice HTML 5 visualizer and interact with Nengo through this GUI I installed with pip: pip install nengogui
You can install it using other approaches, too. For more information, go to: https:// github.com/nengo/nengogui/. Once both programs are installed, if you encounter any problems then consult the links, which contain a lot of information, or Nengo's support team, who provide excellent feedback. There are many ways to use Nengo. This chapter describes a quick start method with Python. Once you are all set, open your browser, Chrome, for example, and then type nengo in a commandline console and it should open the Nengo GUI interface in your browser, opening a default.py Python program:
Figure 18.1: Nengo Python code
[ 464 ]
Chapter 18
Click on the folder icon in the top left and click on builtin examples:
Figure 18.2: Nengo examples
A list will appear. Click on tutorial:
Figure 18.3: List of Nengo examples
A list of fascinating educational examples will appear:
Figure 18.4: Examples in the tutorial section
The tutorial programs are in the directory the installer wrote them in. You can see them in the URL of a Python example of the tutorial in the address bar of your browser. [ 465 ]
Neuromorphic Computing
Creating a Python program
Now, let's create a Python program and save the file through a few steps: • Open an empty Python file but do not save it yet • Write the following code to import the nengo library and create a model: import nengo model = nengo.Network() with model: #
Now, save the Python file in the path of the other programs of the tutorial. You can see this path when you open a Nengo example. Just navigate to that path to save your program. We will name it 00myintro.py to fit in nicely with the list of programs in the tutorial:
Figure 18.5: Saving a Python program
We will go back to the tutorial directory as we did previously, and we will open it to add basic Nengo objects. Nengo objects are the building blocks of a Nengo model to create populations of neurons, connecting them to stimulation functions, and managing the outputs. For our Python program, we will use some key Nengo objects, which are detailed in the following sections.
A Nengo ensemble
A Nengo ensemble is a group of neurons. It can be considered as a population of neurons that contain real numbers. An ensemble is an object created with: nengo.Ensemble [ 466 ]
Chapter 18
An ensemble can be created in one line: ensemblex = nengo.Ensemble(n_neurons=5, dimensions=1)
As soon as we add a line to 00myintro.py, a representation of the ensemble appears on the lefthand pane of the Nengo interface:
Figure 18.6: Neuron ensemble
While writing your Python code in the code editor on the right, you can visualize the visual flow of the model in the visual interface pane. An ensemble can contain a population of one individual, a single neuron. This neuron is a representation of our biological neurons: • Postsynaptic currents (PSCs) flow through our dendrites • The current then reaches the core of the neuron (cell body) • If the current exceeds a threshold at the axon's initial segment (axon hillock), then a spike is generated • With ion channels open, the PSCs are produced in the receiving cell Let's take a more detailed look at Nengo's neuron types.
Nengo neuron types
We created 50 neurons in our ensemble: ens = nengo.Ensemble(n_neurons=50...
[ 467 ]
Neuromorphic Computing
There are various neuron types. However, in our example, we will be using the default neuron, a leaky integrateandfire (LIF) neuron. Neuronal dynamics are based on a summation process called integration. This integration is associated with a mechanism that will fire (trigger) up above a critical voltage. A linear differential equation combined with a threshold that will trigger file spiking are the final components that make up the default LIF neuron we will be using, unless specified otherwise. For more on Nengo neuron types, see https://www.nengo.ai/nengoextras/ neurons.html.
Nengo neuron dimensions
In our example, dimensions is set to 1; this means that the ensemble is represented by one number (or dimension): ens = nengo.Ensemble(..., dimensions=1)
A Nengo node
Now that we have defined our ensemble of neurons and their output dimension, we will define the output: node_number = nengo.Node(output=0.5)
The stimulation will be a constant and will be displayed on the slider as such:
Figure 18.7: Nengo slider
[ 468 ]
Chapter 18
An output with a number might not suffice in some cases. This number can be replaced by a function importing NumPy. A sine wave function can be used, for example: node_function = nengo.Node(output=np.sin)
As soon as we enter our node function, it appears on the interface in addition to the previous information displayed:
Figure 18.8: The node function on the interface
node_function provides a way to send nonneural inputs to Nengo objects.
We will explore such an implementation in the 15lorenz.py example in this chapter that is in the tutorial section Nengo's examples. If you rightclick on the node_function image and choose a value, you will see a curve representing the realtime value of the sine wave stimulation:
Figure 18.9: Node function
[ 469 ]
Neuromorphic Computing
For more on Nengo objects, go to https://www.nengo.ai/nengo/getting_ started.html#creatingnengoobjects.
Connecting Nengo objects
We now need to connect the ensemble and the node to make our system work. In this manner, the ensemble will have a function. At this point, we have an ensemble and a node function, as shown in the following figure:
Figure 18.10: Ensemble and node function
To connect them, we will add a Nengo connection: nengo.Connection(node_function, ens)
For more Nengo frontend API definitions, see https://www.nengo.ai/nengo/
frontend_api.html.
Now, let's explore the exciting visual interface.
Visualizing data
The first step is to click on the play button in the bottomright corner of the screen:
Figure 18.11: Play button
[ 470 ]
Chapter 18
This will set time in motion, simulating our brain activity. Time is one of the unique features of neuromorphic computing. We do not pile layer upon layer of static mathematics into a mental representation. With Nengo, we simulate brain activity step by step, second by second! Once you click on the play button, it feels like we are looking inside our brain! We saw how to visualize the slider's activity in the previous section, which produces the stimulations. We will focus on our ensemble in this section. If we rightclick on the ensemble visualization, several options appear: Value, Spikes, Voltages, Firing pattern, and Details...:
Figure 18.12: Ensemble options
• Value: The value of our ensemble will be displayed, from –1 to 1, for example:
Figure 18.13: Values of an ensemble
[ 471 ]
Neuromorphic Computing
• Spikes: The spiking activity produces nice colors that show how our neurons are reacting to sine wave stimulation:
Figure 18.14: Spiking patterns
You will notice that each neuron has its own spiking channel. This property of ensembles produces a wide variety of responses. • Voltages: The voltages provide interesting information on the current that is flowing through our neurons under stimulation from other neurons and, in turn, from yet more neurons that are gathering information from the world outside us. In the following screenshot, the ensemble contained a population of five neurons with color channels providing yet more information:
[ 472 ]
Chapter 18
Figure 18.15: Neuron activity with color patterns
• Firing pattern: The firing pattern of the ensemble in the following image was generated with 50 neurons. The firing pattern is necessarily directly linked to the stimulations and connections:
Figure 18.16: Firing pattern
[ 473 ]
Neuromorphic Computing
I have been observing many patterns through the experiments I carried out. One of the areas of research I'm doing is to feed the thousands of frames of firing patterns of a given function to an ANN and a stochastic ChurchTuring algorithm and generate "thought" patterns. For example, running the channels of the frames could produce sequences of new data. It's worth experimenting with. The following image represents the pattern of 500 neurons bringing complexity to the potential of running deep learning on thousands of frames of these patterns:
Figure 18.17: Firing patterns
• Details...: In the plots section, we can visualize input current and tuning curves. The tuning curve shows how the neurons are reacting to the input current. If there is no pattern, then there must be an explanation or a configuration problem. The visual control of tuning curves is a time saver by showing how they converge!
[ 474 ]
Chapter 18
Figure 18.18: Tuning curves
We have covered some of the main visual tools Nengo provides. Now, we will see how to retrieve data from our system with probes.
Probes
We can retrieve information with probes, either to visualize them or to process output data. In this section, I enhanced 00myintro.py to produce numerical output as well as visual output using the information on the following page that you can get ideas from for your projects: https://www.nengo.ai/nengo/examples/basic/single_neuron.html
The program I created is nengo_probing.py and is available in the GitHub repository of this book. The program is a standalone program that does not run in Nengo's GUI. You cannot use simulation commands as shown in the following in Nengo's GUI. Run this program in your Python interface. It shows yet another way to run the rich Nengo software.
[ 475 ]
Neuromorphic Computing
The program contains additional headers for Matplotlib and distributions for data displaying and processing purposes: import matplotlib.pyplot as plt from nengo.utils.matplotlib import rasterplot from nengo.dists import Uniform
The program contains the same architecture as 00myintro.py. It creates an ensemble, adds a function, and then connects the objects: model = nengo.Network("Probing") with model: ens = nengo.Ensemble(n_neurons=50, dimensions=1) #node_number = nengo.Node(output=0.5) node_function=nengo.Node(output=np.sin) nengo.Connection(node_function, ens) print(ens.probeable) with model: # Connect the input signal to the neuron nengo.Connection(node_function, ens)
We will now add a probing function using nengo.Probe: # The original input function_probe = nengo.Probe(node_function) # The raw spikes from the neuron spikes = nengo.Probe(ens.neurons) # Subthreshold soma voltage of the neuron voltage = nengo.Probe(ens.neurons, 'voltage') # Spikes filtered by a 10ms postsynaptic filter filtered = nengo.Probe(ens, synapse=0.01)
To obtain some data, let's run the simulator for 5 seconds: with nengo.Simulator(model) as sim: sim.run(5)
# Create the simulator
The simulator runs the calculation before displaying the outputs. Then, we can probe the data and display it in numerical format. That way, we can retrieve output data from our system for further use, visualizing or chaining neuromorphic models to other algorithms in a few lines of code:
[ 476 ]
Chapter 18
• Decoded output: The decoded output can be filtered (see the preceding filter): print("Decoded output of the ensemble") print(sim.trange(), sim.data[filtered])
The output data is then displayed or can be processed: [1.000e03 2.000e03 3.000e03 ... 4.998e+00 4.999e+00 5.000e+00] [[ 0. ] [ 0.
]
[0.03324582] ... [1.26366121] [1.22083471] [1.18750863]]
Nengo can produce a chart with Matplotlib: # Plot the decoded output of the ensemble plt.figure() plt.plot(sim.trange(), sim.data[filtered]) #plt.plot(sim.trange(), sim.data[node_function]) plt.xlim(0, 1) plt.suptitle('Filter decoded output', fontsize=16)
The output of the preceding code is plotted as follows:
Figure 18.19: Decoded input
[ 477 ]
Neuromorphic Computing
• Spikes: Spikes are retrieved in a single line of code: print("Spikes") print(sim.trange(),sim.data[spikes])
The output produces sequences of spikes: [1.000e03 2.000e03 3.000e03 ... 4.998e+00 4.999e+00 5.000e+00] [[ 0. 0. 0. ... 0. 0. 0.] [
0.
0.
0. ...
0.
0.
0.]
[
0.
0.
0. ... 1000.
0.
0.]
[
0.
0. 1000. ...
0.
0.
0.]
[
0.
0.
0.
0.
0.]
... 0. ...
The program produces a figure for spikes that matches the data:
Figure 18.20: Spiking output
We can check the visual display with valuable raw data. • Voltage: The simulation provides voltage data: print("Voltage") print((sim.trange(), sim.data[voltage][:, 0]))
The data is stored in a readytouse array: (array([1.000e03, 2.000e03, 3.000e03, ..., 4.998e+00, 4.999e+00, 5.000e+00]), array([0., 0., 0., ..., 0., 0., 0.]))
The program produces a figure for the voltage of the ensemble as well: [ 478 ]
Chapter 18
Figure 18.21: Neuron voltage
At this point, we have covered the main features of Nengo. We will now see how Nengo can boost research in critical AI research areas.
Applying Nengo's unique approach to critical AI research areas
It is useless to apply the power of brain neuromorphic models to simple arithmetic or classical neural networks that do not require any more than TensorFlow 2.x, for example. But it is also a waste of time to try to solve problems with classical networks that neuromorphic computing can solve better with organic brain models. For example: • Deep learning, TensorFlow 2. Convolutional models use a unique activation function such as ReLU (see Chapter 9, Abstract Image Classification with Convolutional Neural Networks (CNNs)). Neuromorphic neurons have a variety of reactions when stimulated. • Neuromorphic models integrate time versus more static DL algorithms. When we run neuromorphic models, we are closer to the reality of our timedriven biological models. • The Human Brain Project, https://www.humanbrainproject.eu/en/, provides wide research and examples of how neuromorphic computing provides additional insights to classical computing. [ 479 ]
Neuromorphic Computing
I recommend testing a given problem with several AI tools and choosing the most efficient one. The SPA examples demonstrate the efficiency of Nengo in several areas. All of the examples in the tutorial section are well documented and run well. You can apply the visualizing functions we explored in this chapter to them and also modify the code, experiment with them in your Python environment, and more. I wish to highlight two domains: linguistics and weather representations: • Linguistics: 25spaparse.py processes neuronal signals and produces words. When you run it, the performances look magical, thanks to the SPA. The code is well documented. The program contains the mindblowing thalamus module that can simulate the subcortical nuclei in our brains (forebrain and midbrain). This part of our brain, the basal ganglia, has highdensity connections. This class can reduce or even eliminate low responses and intensify high responses to the stimulations. The program is worth running and exploring! The Nengo GUI makes it intuitive to understand:
Figure 18.22: Nengo GUI options
• Weather representations: 15lorenz.py is not an SPA program. It is only a few lines long. It displays the graphs of the three basic Lorenz equations that represent temperatures and variations in the atmosphere. Nengo modified the code for educational purposes as explained in a publication at http://compneuro.uwaterloo.ca/publications/eliasmith2005b. html. The Nengo GUI displays an exciting representation: [ 480 ]
Chapter 18
Figure 18.23: Lorenz equations
The code of 15lorenz.py is short, and the program looks simple. But weather forecasting is one of the toughest fields to represent events with AI models! The potential of neuromorphic computing can be a real gamechanger. Let's conduct a mind experiment. Imagine that: • A hurricane is like a living organism • That its center is connected to everything around it • That it "feeds" on the heat and waters of our oceans • That everything that is in it can be considered as small hurricane particles Let's now continue the experiment by: • Feeding the billions of particles in the neuromorphic model in a stream • Using the power of a network of neuromorphic chips • Using the calculation power of quantum computing (see Chapter 19, Quantum Computing) to perform computations with the input/output of the chip • Applying SPA to the hurricane particle representations as if they were neurons and running predictions [ 481 ]
Neuromorphic Computing
I think the future of weather forecasting is in physical, neuromorphic models, that will take billions of parameters into account. The result: we will be able to predict the course and level of a hurricane a few hours to a few days more in advance. This could save lives. In a nutshell, neuromorphic computing has only just begun to demonstrate its worth. When neuromorphic chips hit the market, neuromorphic computing will grow exponentially.
Summary
In this chapter, we built neuromorphic Python programs from scratch. Populations of neurons, in Nengo ensembles, are made up of neurons. The system then has stimulation functions, connections, and probing objects. Nengo offers many other examples you can explore. The NEF was designed to implement neuromorphic computing models. The novel concept of SPA shows that our brains have enhanced pointers that have a meaning and are linked to our physical data. Neuromorphic computing opens tremendous horizons for a complex program that classical machine learning and deep learning cannot solve. Weather forecasting, with the power of the neuromorphic chips that are reaching the market, can tap into the complexity and variety of a machine brain. A machine brain can produce unique calculations by firing hundreds of thousands of neurons with both individual and collective behavior. We have covered many algorithms and frameworks in this book. We have access to the most powerful intelligent algorithms in the history of humanity. From MDP to GA algorithms, and from KMC, KNN, PCA, NLP, and CUI algorithms to CNN, RBM, RNN, and LSTM networks, we have explored many AI tools. But there may be ways to build unified models. We will see in the years to come. In the meantime, neuromorphic computing makes our toolbox incredibly intelligent. Google has TPU hardware to optimize TensorFlow, neuromorphic computing can rely on Intel chips, and many corporations are working to produce more innovative hardware. The future will no doubt rely on hybrid architectures in which some or all of the AI tools will be built into metaAI systems. In Chapter 19, Quantum Computing, we will explore the exponential hardware available for quantum mechanics. Quantum computers, having no memory, rely on other systems to provide inputs and process outputs. Imagine a neuromorphic system chained to a quantum computer! [ 482 ]
Chapter 18
Questions
1. Neuromorphic computing reproduces our mental activity. (Yes  No) 2. Neuromorphic computing reproduces our brain activity. (Yes  No) 3. Semantic Pointer Architecture (SPA) is a hardware architecture. (Yes  No) 4. NEF stands for Neural Engineering Framework. (Yes  No) 5. Loihi is a classical chip. (Yes  No) 6. Reproducing our brain's neural activity cannot solve an equation. (Yes  No) 7. An ensemble in Nengo contains algorithms. (Yes  No) 8. Spiking blocks neuronal activity. (Yes  No) 9. Firing patterns can be used to analyze brain activity. (Yes  No) 10. Machine learning and deep learning are only metaphors of our brain's activity. (Yes  No)
References
Reference programs used for this chapter can be found at https://www.nengo.ai, https://www.nengo.ai/examples/.
Further reading
• Research – How to Build a Brain, Chris Eliasmith: This book provides the theoretical background for neuromorphic computing. Chris Eliasmith is also one of the designers of Nengo.
• Software – Nengo (https://www.nengo.ai/): Nengo is based on solid research, documentation, and an excellent community. • Hardware – Intel: Intel is working hard to produce a neuromorphic chip (https://www.intel.fr/content/www/fr/fr/research/neuromorphiccomputing.html).
[ 483 ]
19 Quantum Computing IBM has begun to build quantum computers for research and business purposes. In a few years, quantum computing will become disruptive and will provide exponential computing power. Google, Xanadu, DWave, Rigetti, and others devote research budgets to quantum computing. This unique computer capacity in the history of humanity opens the doors to obtaining results that would be impossible to obtain with classical computers. In 1994, Peter Shor showed that a quantum algorithm could perform better than a classical one for prime factors of an integer and the discrete logarithm problem. Then in 1995, Lov Grover added the unstructured search problem. With the rise of quantum computers, research can go much further. Quantum computers in themselves will not provide revolutionary algorithms. Any algorithm can be broken down into components that run basic classical machines. Supercomputers can still run artificial intelligence algorithms much easier than implementing them with quantum computers. Quantum computers have no memory, for example, so they rely heavily on classical computers for the input and interpreting the output. Though quantum computers have limits, in time, quantum computing power will take presentday algorithms beyond the limits of our imagination. Many corporations have invested in quantum computing research in the fields of banking, healthcare, cybersecurity and more. This chapter explains why quantum computers are superior to classical computers, what a quantum bit is, how to use it, and how the quantum mind experiment could lead to a quantum thinking machine. The example provided is simply a recreative way to approach quantum computing. [ 485 ]
Quantum Computing
We will be taking results from a classical machine, feed the results to quantum computer and then interpret the results provided by the quantum computing algorithm. Some go further with hybrid quantumclassical algorithms, which is beyond the scope of this chapter. In Chapter 18, Neuromorphic Computing, we explored how to use our brain to create neuromorphic models. In this chapter, we will create a higher level of representation with a research project to create a mind. Quantum mind is based on CRLMM, which I have been successfully applying to many corporate sites. In this research project, a quantum mind named MindX represents the mind of a random person. We will explore MindX in an exciting experiment. The following topics will be covered in this chapter: • Why quantum computers are more powerful than other classical computers • What a quantum bit is • The Bloch sphere • Quantum computing • How to build MindX, a thinking quantum computer research project First, let's discuss some basics behind quantum computers, and what makes them so powerful. Note: This chapter is selfcontained with screenshots of quantum circuits for those who do not wish to install anything before reading the chapter. IBM Q and Quirk can be used online without installing anything locally.
The rising power of quantum computers
Before we begin discussing MindX, an exciting research project to create a thinking quantum computer, we should start with the fundamentals. This section describes: • Why quantum computers are faster • What a qubit is • How a qubit is measured (its position) • How to create a quantum score (program) with quantum gates • How to run a quantum score and use its results for a cognitive NLP chatbot
[ 486 ]
Chapter 19
The goal of this chapter is not to go too deeply into the details of quantum computing, but rather to teach you enough to know how to build a thinking quantum computer. This chapter is simply meant to show how a quantum computer works and open ourselves to new ways to use computers.
Quantum computer speed
A standard computer bit has a 0 or a 1 state. A classical computer will manage 0 or 1, as the system chooses, but it remains limited to choosing 1 or (XOR) 0. It cannot manage both states at the same time. A quantum computer is not constrained by an XOR state. It is an AND state. It can manage values between 0 and 1 at the same time until it is measured! A quantum state is unknown until it's observed, which means that a quantum program can use values between 0 and 1 at the same time. Once observed, the qubit will take a value of 0 or 1 because of the physical instability of a quantum state. Just observing a quantum state makes the state break down. In quantum computing, this is called decoherence. It is not magic. Qubits are unstable. When observed, the quantum state breaks down.
This means quantum computing is memoryless once it is measured. Storage does not exist in a quantum computer. The input is made by a classical computer, and the output goes back to a classical computer, to be stored through the following process: 1. A classical computer provides an input 2. A quantum computer processes the input and produces an output 3. A classical computer interprets the output That being said, the computing power of a quantum computer fully justifies this architectural constraint. Until a qubit is observed, it can have a 0 or 1 state or a probability in between, such as 0.1, 0.7, or 0.9. Observing the situation is called measuring the state. When measured, only 0 or (XOR) 1 will become the result. Until the state is measured, a large number of probabilities are possible. If a qubit is added to the system, we now have two qubits and four elementary combinations all at the same time.
[ 487 ]
Quantum Computing
Unlike standard computer logic, all four of the states can be used to compute algorithms at the same time in a parallel process. The volume of the possible states of a given algorithm will thus expand with the number of qubits involved. An estimation of the volume of the states can be made with the following number, in which q is the number of qubits:
2q Looking at this tiny equation does not seem awesome at all. Now, let's see what it looks like in a loop that runs up to 100 qubits with the number, nb, of possible states: import numpy as np for q in range(101): v=(2**q) print("Size of nb to describe",q," qubits:","{:,}".format(v))
The program does not appear fearsome either. However, the following output is awesome: Size of nb to describe 0 qubits: 1 Size of nb to describe 1 qubits: 2 Size of nb to describe 2 qubits: 4 Size of nb to describe 3 qubits: 8 Size of nb to describe 4 qubits: 16 Size of nb to describe 5 qubits: 32 Size of nb to describe 6 qubits: 64 ... Size of nb to describe 10 qubits: 1,024 ... Size of nb to describe 50 qubits: 1,125,899,906,842,624 ... Size of nb to describe 97 qubits: 158,456,325,028,528,675,187,087,900,672 Size of nb to describe 98 qubits: 316,912,650,057,057,350,374,175,801,344 Size of nb to describe 99 qubits: 633,825,300,114,114,700,748,351,602,688 Size of nb to describe 100 qubits: 1,267,650,600,228,229,401,496,703,205,376
[ 488 ]
Chapter 19
Presently, big data is often calculated in petabytes. A petabyte=1015 or about 250 bytes. Facebook stores data for over 2 billion accounts. Imagine Facebook managing 500 petabytes on a given day. Let's see what 500 petabytes approximately add up to in the following code: print("Facebook in the near future:") s=(2**50)*500 print("{:,}".format(v))
The output is quite surprising because it is about the size of data a 100qubit quantum computer can compute in one run: A segment of Facebook data : 1,267,650,600,228,229,401,496,703,205,376
This means that a single quantum computer with 100 qubits can run a calculation of the size of all the data that over 2,000,000,000 Facebook accounts might represent in the near future. A quantum computer would not actually contain that volume of data at all, but it shows that one could produce a calculation with that volume of computation information. More importantly, this also means that a single quantum computer can run a minddataset of a single mind (see the next section) and calculate associations. This thinking process can generate an exponential volume of connections. A classical nbit computer manages n bits whereas a quantum computer will manage 2n bits or 2q bits. Compared to quantum computers' 2q exponential power, soon, classical computers will seem like relics of the past for scientific calculations. Classical computers will still be in use, but quantum computers will be the tools with which to explore the world beyond the present limits of artificial intelligence. Visualize all the AI solutions you have seen in this book. They will already seem to have some dust of the past on them once you get your hands on quantum computing.
Quantum computers will beat any other computer in many fields in the years to come. In one parallel computation, a quantum computer will do in one run something that it would take a classical computer years to calculate.
[ 489 ]
Quantum Computing
Now think about what a network of many quantum computers could do! Often, we try to compare large volumes with the number of stars in the universe and we say, "That's more than the number of stars in our universe." We must now look in the opposite direction, at very small numbers. The lesson is clear: the future lies in nano models. Quantum computing represents both a challenge and an opportunity.
Defining a qubit
A qubit, a quantum bit, has a physical counterpart. For example, a quantum state can be encoded in oscillating currents with superconductor loops. Google and IBM have experimented with this approach. Another way is to encode a qubit in an ion trapped in an electromagnetic field in a vacuum trap. Photons, electrons, the state of light, and other technologies have emerged. Whatever the approach, calculations done by over 50qubit quantum computers will outrun classical supercomputers. The competition is fierce because the market will rapidly become huge. Get ready now to face the disruption that's coming!
Representing a qubit
The mathematical representation of a qubit is: • 0⟩ for a 0 value
• 1⟩ for a 1 value
• 𝛼𝛼0⟩ where alpha is a probability parameter
• 𝛽𝛽1⟩ where beta is a probability parameter
These brackets are called bracket or braket notation. This linear representation is called superposition. In itself, it explains most of the power of quantum computing. The superposition of 0 and 1 in a quantum state can thus be expressed as follows in kets such as 1⟩ and not bras such as in ⟨1:
quantum state = 𝛼𝛼1⟩ + 𝛽𝛽0⟩ [ 490 ]
Chapter 19
The alpha and beta probabilities look like weights, and the total probabilities of those probable states of qubit must add up to 1. We use partition functions, softmax, and other techniques to make sure to keep the sum of probabilities equal to 1. This is no surprise since computer geeks like us designed the way to program quantum computers. Translated into mathematics, this means that the probabilities of 𝛼𝛼 and 𝛽𝛽 must add up to 1. In the case of qubit probabilities, the values are squared, leading to the following constraint:
𝛼𝛼 2 + 𝛽𝛽2  = 1
To describe the probable state of a qubit, we thus need three numbers: the 0 and 1 possible states and a number to determine the value of the probabilities (the other is implicit since the total must add up to 1).
Since qubits interact in their respective states, an interaction is described as an entanglement. An entanglement designates at least two interacting qubits. They cannot be described without taking all the states of all the qubits into account. This has been reproduced physically, which means that this entanglement seems strange because their quantum entanglement (relation) can occur at a distance. One qubit can influence a qubit that is physically far away. It's an odd way to think, and one that was not readily accepted at first. Albert Einstein is oftquoted as referring to entanglement derisively as "spooky action at a distance."
The position of a qubit
One of the main ways to represent the state of a qubit is by using the Bloch sphere. It shows how a qubit spins and can be used to describe qubit states. To properly grasp this, the following section will first provide a refresher on some properties of a circle.
[ 491 ]
Quantum Computing
Radians, degrees, and rotations
The radius is the distance between the center of a circle and its circumference, as shown in the following diagram:
Figure 19.1: Radius of a circle
The radius of a circle is half of the circle's diameter. The relation between the radius r and the circumference C is (where 𝜋𝜋 = 3.14):
𝐶𝐶 = 2𝜋𝜋𝜋𝜋
If the length of the radius is wrapped around the circumference of a circle, that arc forms a radian, as shown in the following diagram:
Figure 19.2: A radian
The angle formed by a radian is equal to about 57.29° (degrees). The properties of the radian can be used for rotations: • 3.14 × 57.29° = about 180° • Thus 𝜋𝜋 radians = 180° [ 492 ]
Chapter 19
Rotations are often described by radians expressed in 𝜋𝜋, as displayed in the following table: Degrees
Radians
30°
𝜋𝜋/6
45° 60° 90° 180° 270° 360°
𝜋𝜋/4
𝜋𝜋/3
𝜋𝜋/2
𝜋𝜋 3𝜋𝜋/2 2𝜋𝜋
Now that we've had that recap, let's explore the Bloch sphere.
The Bloch sphere
The radian table shown just now is a practical way to describe rotations. The Bloch sphere, shown in the following figure, provides a visual representation of the position and rotation of a qubit:
Figure 19.3: Bloch sphere
[ 493 ]
Quantum Computing
The North and South Pole (polar coordinates) represent the basic states of a qubit:
1 "North" pole = 1⟩ = ( ) 0 0 "South" pole = 0⟩ = ( ) 1
A qubit can take any value on the sphere.
Composing a quantum score
Composing a quantum score consists of positioning gates on a stave (or circuit) and adding a measurement. The input comes from a classical computer. After the measurement phase, the output goes back to a classical computer. The reason is that quantum computers have no memory and thus cannot store their intermediate states because of their instability. This section uses Quirk, a very educative quantum circuit simulator, to present quantum gates and a quantum composer. You can access Quirk online at this link: https://algassert.com/quirk.
Quantum gates with Quirk
Qubits are represented by lines, and they start on the left, as shown in the following quantum gate programming interface:
Figure 19.4: Quantum gate programming interface
The gates are logic gates that will transform the state of the qubits.
[ 494 ]
Chapter 19
NOT gate
A NOT gate will transform a ketzero 0⟩ into a ketone 1⟩. It will transform a ketone 1⟩ into a ketzero 0⟩. In the circuit description, On is the ketone state and Off is the ketzero state, as shown in the following quantum score:
Figure 19.5: Circuit description of a quantum score
You can see that: • A NOT gate symbol is a circle with a vertical and horizontal line inside it • The On status means that the state is 0⟩
• The Bloch sphere representation is at 𝜋𝜋 (starting from the top of the Bloch sphere, as it should)
H gate
An H gate or Hadamard gate will perform the following transformation:
Figure 19.6: Hadamard gate transformation
[ 495 ]
Quantum Computing
The following 50% chance will be displayed in a rectangle and the position on the Bloch sphere will be displayed as well:
Figure 19.7: 50% chance in a rectangle and on the Bloch sphere
The fundamental role of a gate is to rotate the qubits on a Bloch sphere and produce a probable condition if measured. There are many possible gates to be explored and used, as shown in this design menu diagram:
Figure 19.8: Gates design menu
These gates are more than enough to build many algorithms.
A quantum computer score with Quirk
The goal here is to play around with the interface to intuitively see how circuits run. Building a quantum score (or circuit) with Quirk means the following: • Dragging and dropping gates that will make a qubit turn in a specific direction and produce probable outcomes • Adding another qubit, doing the same, and so on • Being able to perform an intermediate measurement, although this is impossible for a real physical quantum computer (the observations make the system collapse) A score is represented as follows, for example:
[ 496 ]
Chapter 19
Figure 19.9: Quantum computer score representation
There are two qubits with a gate on each line to start. Then the intermediate result is shown, making the simulator very educational. Then two more gates are added. Finally, the following measurement probe is added at the end:
Figure 19.10: Measurement probe
Once the measurement is made, the final result is displayed on the right of the measurement symbols.
A quantum computer score with IBM Q
IBM Q provides a cloud platform to run a real physical quantum computer.
[ 497 ]
Quantum Computing
Create a free account and access the IBM quantum computing composer. Just as with Quirk, quantum gates are dragged on the following score, as shown in this diagram:
Figure 19.11: Quantum gates dragged
The score can be run on a simulator like Quirk or a real quantum computer, as shown in this interface diagram:
Figure 19.12: Score interface
Click on Simulate, which will run a simulator. Run launches a calculation on IBM's physical quantum computer. It is an exhilarating experience! The future is at the tip of your fingers.
The following output is interesting. It is a bit different from Quirk for the same score, but the probabilities add up to 1 as expected:
[ 498 ]
Chapter 19
Figure 19.13: Quantum computer score output
IBM also possesses a source code version (QASM) of the score, as shown in the following code: include "qelib1.inc"; qreg q[5]; creg c[5]; h q[0]; z q[1]; y q[0]; x q[1]; measure q[0] > c[0]; measure q[1] > c[1];
This language is an Open Quantum Assembly Language. It can be written in an editor like any other language on IBM's Q platform. A development kit can also be downloaded, and the APIs are functional. IBM provides extensive and detailed documentation on all the aspects of this approach. All of this being said, let's find out how quantum computing can boost an AI project. It is now time to define the thinking quantum computer project.
[ 499 ]
Quantum Computing
A thinking quantum computer
A thinking quantum computer is not a reproduction of brain function, but rather a representation of the mind of a person. Neuromorphic computing is one approach that represents our neurons and how brain uses spiking neurons to think. Quantum computing can provide an exciting way to imitate our mind's capacity with mathematical algorithms that work with qubits, not neurons. We've just begun to explore the potential of these approaches, which in fact will most probably merge into an ensemble of hybrid software and hardware solutions. The endeavor of the quantum MindX experiment is to build a personal mind named MindX, with memories of past events, conversations, chats, and photographs stored on a classical computer. The program will then transform subsets of the data into quantum circuits to see what happens, how our quantum circuit behaves when left to produce millions of possibilities. This section describes how to build MindX, a thinking computer that is just a research project. It must be noted that this approach is experimental. It could be viewed as an advanced mind experiment. I've been doing research on this subject for many years. The power of quantum computation will no doubt boost research in this field.
Representing our mind's concepts
The input consists of encoding a state of mind of a PCA CRLMM representation, as we built in Chapter 14, Preparing the Input of Chatbots with Restricted Boltzmann Machines (RBMs) and Principal Component Analysis (PCA). The CRLMM representation is not a general dictionary or encyclopedia dataset but a minddataset of actual personal data that is collected, classified, and transformed into a PCA.
Expanding MindX's conceptual representations
MindX's mind dataset will grow continuously if implemented beyond this research experiment. It will have sensors to process body temperature for emotion classification, facial recognition for expression detection, body language detectors, and more. All of this technology is already available. The New York Stock Exchange already has artificial intelligence IoT agents that gather information from outer sources to make decisions. These AI agents have replaced a large amount of human decisionmaking. Imagine what is going to happen when quantum computing becomes disruptive!
[ 500 ]
Chapter 19
The MindX experiment
The aim of the quantum MindX experiment is to build a mind and let it think with the power of a quantum computer. This section will show how to run a 16qubit mind. The size of the numbers needed to describe a 16qubit quantum simulation is 65,536. This section first describes how to do the following: • Prepare the data • Create and run a quantum score • Use the output Let's get on with preparing the data.
Preparing the data
To prepare data for higherdimension calculations, I have been using a concept encoding method for corporate projects for over 30 years to provide embedded data to the algorithms I have developed. This guarantees highlevel abstraction to solve problems. It is a very profitable way to implement transfer learning and domain learning. This way, you can apply the same model to many different fields. The method consists of embedding data with the methods described in this book. The goal always remains the same—to transform the data points into higher dimensions to visualize features. For quantum computing, the method remains the same for the MindX experiment.
Transformation functions – the situation function Two functions need to be applied before creating and running a score: a state of mind function and a quantum transformation function.
The situation function consists of building a vector of features in PCA dimensions. We accomplished this in Chapter 14, Preparing the Input of Chatbots with Restricted Boltzmann Machines (RBMs) and Principal Component Analysis (PCA), and applied it to a chatbot in Chapter 15, Setting Up a Cognitive NLP UI/CUI Chatbot. The chatbot was able to use person X's minddataset to build a meaningful dialog. In this chapter, the quantum dimensions will provide the chatbot with a nascent personal mind, not a general mind that can recognize, classify, and predict like a machine. [ 501 ]
Quantum Computing
MindX has a mind that is biased by its way of thinking like humans. MindX has an exceptionally open mind to adapt to everyone, which gives it empathy.
MindX can doubt. Thus it can learn better than dogmatic machines.
A situation function will create a situation matrix for a movie suggestion by the MindX bot, which will communicate with consumers. In a situation where a movie suggestion is made by the MindX bot, for example, it could be something as follows for 16 qubits: Qubit
Concept
Image
Initial polarity
1
cities
parking
0.146
2
darkness
dark forest
0.5
3
nostalgia
autumn leaves
0.5
4
worrying
dark background
0.146
5
job
sad face
0.5
…
..
..
15
consider movie "Lost"
0.38
16
decision to suggest "Lost"
0.0
MindX is not analyzing person X anymore. It has now loaded an empathy matrix from its minddataset, which contains data and sentiment analysis polarity. Values close to 1 are positive. Values close to 0 or negative are negative. MindX loads its mind plus the mind of another person. Empathy colors your thoughts and feelings with the thoughts and feelings of another person.
The 16qubit matrix shown just now contains four columns, as shown in the preceding table: • Qubit: The line for this qubit on the quantum composer [ 502 ]
Chapter 19
• Concepts and mental representation: The concept loaded in MindX's situation dataset derived through the process described in Chapter 14, Preparing the Input Chatbots with Restricted Boltzmann Machines (RBM) and Principal Component Analysis (PCA), in which we ran an RBM to generate features we represented through a PCA. The mental representation appears as shown in the following PCA:
Figure 19.14: PCA representation of a "mind"
These features, drawn from the RBM on the movie preferences of a particular age segment, can now be used as input in a quantum circuit to generate random mental activity with information and noise. This movie example is for explaining how to create a thinking, empathetic chatbot. This approach can be applied to other commercial market segments or any situation in which a chatbot is required to think beyond preset answers. Many fields, such as healthcare, pharmaceutical research, and security, will benefit from mindopening quantum algorithms.
[ 503 ]
Quantum Computing
Transformation functions – the quantum function
The algorithms have produced a qubit line number with labels (concepts and images). Each line also possesses a sentiment analysis polarity expressed in normalized values from 0 to 1 in probabilistic format. We will consider negative and positive views of the mind reacting to an object. Keep in mind that "negative" is close to 0, "positive" is close to 1, and the intermediate values give a more detailed approximation. 0.4 to 0.6 is a turning point. This first transformation into another dimension is the first step to initialize the quantum transformation function. The quantum transformation function consists of initializing the first column of all 16 qubits with a quantum gate. Just like datasets were transformed into principal component features, the polarity transformation function enables the quantum transformation function to bring the dataset into a quantum universe. The function will automatically find the quantum gates that represent the normalized sentiment analysis polarity of the data points of the following situation matrix: Initial polarity
Quantum gate
0.146
X1/4
0.5
X1/2
0.5
X1/2
0.146
X1/4
0.5
X1/2
.. 0.38
X1/8
0.0
Z1/8
The state of mind of MindX, a random person, is now in a matrix, and its concepts have now been transformed into a quantum dimension, in which the data status can be traced throughout the creation of the quantum score.
Creating and running the score
A thought process of MindX is described in a quantum score. There are two ways to build this score: • Manually, just like a musician writing a music score. This requires thinking about the previous logic gate and the effect of the next one, taking a decision process into account. It is like writing any artificial intelligence program. [ 504 ]
Chapter 19
• Automatically, by building a function that reads a rule base of MindX's way of thinking and applies the quantum gates through that rule base. You can apply this to machine learning and deep learning as well to test the performance of a quantum computer versus a classical one. In any case, it requires writing an algorithm. This algorithm is a decisionmaking algorithm that takes emotions into account. Doubt, for example, is what keeps many humans from making bad decisions. Too much doubt, for example, will make a person back out of a situation. Here is the quantum transcription of an algorithm that takes MindX's concepts into account and represents how they interact. This requires very precise thinking and cognitive science programming. The following is an experimental quantum score I built with Quirk:
Figure 19.15: Quantum score built with Quirk
Once the quantum score has been run and the measurement has been done, the green rectangles on the right provide the output.
[ 505 ]
Quantum Computing
Using the output
The output of the quantum score is now added as a column to the situation matrix. If you want to implement such solutions, just bear in mind that it will take some time to write the functions with some sweat and tea to make it through the nights. MindX could be used to enhance a chatbot with unplanned responses. MindX has proven it has imagination potential.
Qubit
Concept
Image
Initial normalized polarity expressed in quantum gate form
Quantum output directly interpreted as sentiment analysis polarity
1
cities
parking
0.146
0.677
2
darkness
dark forest
0.5
0.691
3
nostalgia
autumn leaves
0.5
0.5
4
worrying
dark background
0.146
0.48
5
job
sad face
0.5
0.36
…
..
..
15
consider movie "Lost"
0.38
0.82
16
decision to suggest "Lost"
0.0
0.75
MindX has given a 65,536 quantum state description of its thoughts about suggesting a given movie to person X. Lines 15 and 16 show that the normalized polarity value has risen over 0.5 to a positive feeling about the movie. The reasoning is that the first lines show that MindX feels person X's doubts about life at that moment: • That person X will identify with the "Lost" movie • The movie has a happy ending (MindX knows that through the features of the movie) • That person X's spirits will be most probably lifted after watching the movie You can experiment with building quantum scores. You can use Quirk without installing anything and exploring the list of circuits available in many areas: Grover's search, Shor's periodfinding, quantum Fourier transform, and more.
[ 506 ]
Chapter 19
Summary
Quantum computers have opened the door to scientific experiments that could never have been carried out with classical computers. Within a few years, quantum computers will have become mainstream, unavoidable, and a key asset for businesses and research labs. The race has begun to conquer the market. CRLMM applied to quantum computers could make MindX one of the most powerful thinking minds on earth—human or machine. With an unlimited minddataset and a 2q quantum computer starting at 250, a 50qubit machine, MindX could gain the thinking power and experience of a human who has lived for 1,000 years. MindX's thinking power and an exponential amount of realtime memory of past experiences, loaded through transformation functions, could help solve many medical, logistic, and other decisionmaking problems. Quantum thinking has just begun to change the perception of the world. Conceptual AI models such as CRLMM will no doubt be the starting point for the next generation of AI solutions. These CRLMM models will be much more powerful because they will be gifted with empathy and complex minds. Hopefully, this overview of quantum computing will open the doors of your imagination to the new world awaiting you! Artificial intelligence has only just begun its long journey into our lives. Always trust innovations. Never trust a solution that solves a problem without opening the door to a universe of questions and ideas!
Questions
1. Beyond the hype, no quantum computer exists. (Yes  No) 2. A quantum computer can store data. (Yes  No) 3. The effect of quantum gates on qubits can be viewed with the Bloch sphere. (Yes  No) 4. A mind that thinks with past experiences, images, words, and other bits of everyday information, like stored memories, will find deeper solutions to problems that mathematics alone cannot solve. (Yes  No) 5. A quantum computer will solve medical research problems that cannot be solved today. (Yes  No)
[ 507 ]
Quantum Computing
6. A quantum computer can solve mathematical problems exponentially faster than classical computers. (Yes  No) 7. Classical computers will soon disappear and smartphone processors too. (Yes  No) 8. A quantum score cannot be written in source code format but only with a visual interface. (Yes  No) 9. Quantum simulators can run as fast as quantum computers. (Yes  No) 10. Quantum computers produce intermediate results when they are running calculations. (Yes  No)
Further reading
• Theory: Quantum Computation and Quantum Information: 10th Anniversary Edition, Michael Nielson, Isaac L. Chuang • Explore IBM Q and discover how you can implement quantum scores: https://www.ibm.com/quantumcomputing/
• Use Quirk, an intuitive quantum score designing tool: http://algassert. com/2016/05/22/quirk.html
[ 508 ]
Answers to the Questions Chapter 1 – Getting Started with NextGeneration Artificial Intelligence through Reinforcement Learning 1. Is reinforcement learning memoryless? (Yes  No)
The answer is yes. Reinforcement learning is memoryless. The agent calculates the next state without looking into the past. This is significantly different from humans. Humans rely heavily on memory. A CPUbased reinforcement learning system finds solutions without experience. Human intelligence merely proves that intelligence can solve a problem. No more, no less. An adaptive thinker can then imagine new forms of machine intelligence. It must be noted that exceptions exist in some cases, but the general rule is a memoryless system. 2. Does reinforcement learning use stochastic (random) functions? (Yes  No) The answer is yes. In the particular Markov decision process model, the choices are random. In just two questions, you can see that the Bellman equation is memoryless and makes random decisions. No human reasons like that. Being an adaptive thinker is a leap of faith. You have to leave who you were behind and begin to think in terms of equations. 3. Is the MDP based on a rule base? (Yes  No) The answer is no. Human rulebased experience is useless in this process. Human thinking is often based on rules of cause and effect, for example. Furthermore, the MDP provides efficient alternatives to long consulting times with future users who cannot clearly express their problem.
[ 509 ]
Answers to the Questions
4. Is the Q function based on the MDP? (Yes  No) The answer is yes. The use of the expression Q appeared around the time the Bellman equation, based on the MDP, came into fashion. It is more trendy to say you are using a Q function than to speak about Bellman, who put all of this together in 1957. The truth is that Andrey Markov was Russian and applied this method in 1913 using a dataset of 20,000 letters to predict the future use of letters in a novel. He then extended that to a dataset of 100,000 letters. This means that the theory was there 100 years ago. Q fits our new world of impersonal and powerful CPUs. 5. Is mathematics essential to AI? (Yes  No) The answer is yes. If you master the basics of linear algebra and probability, you will be on top of all the technology that is coming. It is worth spending a few months' worth of evenings on the subject or taking a MOOC. Otherwise, you will depend on others to explain things to you. 6. Can the BellmanMDP process in this chapter apply to many problems? (Yes  No) The answer is yes. You can use this for robotics, market analysis, IoT, linguistics, and scores of other problems. 7. Is it impossible for a machine learning program to create another program by itself? (Yes No) The answer is no. It is not impossible. It has already been done by DeepCode: https://www.deepcode.ai/. Do not be surprised. Now that you have become an adaptive thinker and know that these systems rely on equations, not humans, you can easily understand the fact that mathematical systems are not that difficult to reproduce. 8. Is a consultant required to enter business rules in a reinforcement learning program? (Yes No) The answer is no. It is only an option. Reinforcement learning in the MDP process is memoryless and random. Consultants are there to manage, explain, and train in these projects. 9. Is reinforcement learning supervised or unsupervised? (Supervised  Unsupervised) The answer is unsupervised. The whole point is to learn from unlabeled data. If the data is labeled, then we enter the world of supervised learning; that will be searching for patterns and learning them. At this point, you can easily see you are at sea in an adventure—a memoryless, random, and unlabeled world for you to discover. [ 510 ]
Appendix
10. Can Qlearning run without a reward matrix? (Yes  No) The answer is no. A smart developer could always find a way around this, of course. The system requires a starting point. You will see in the second chapter that it is quite a task to find the right reward matrix in reallife projects.
Chapter 2 – Building a Reward Matrix – Designing Your Datasets 1. Raw data can be the input to a neuron and transformed with weights. (Yes  No)
The answer is yes if the data is in numerical format. If it is in a proper numerical format, the input can be multiplied by the weights and biases. If the data is not in a numerical format, then it requires a numerical encoding phase. 2. Does a McCullochPitts neuron require a threshold? (Yes  No) The answer is yes. Adding up weights does not mean much if you do not have something to measure the value. It took months of work for McCulloch and Pitt to put this together. At first, time was in the equation, just like it is in our brain. But then, like Joseph Fourier (17681830), they found cycles that repeated themselves—periods that did not require much more than that neuron. Warren McCulloch and Walter Pitts invented the first neuron and published a paper in 1943. Legend has it that at age 12 years in 1935, Walter Pitts, a poor child living in a bad neighborhood, was chased by bullies and sought refuge in a library. There, he discovered Principia Mathematica, by Bertrand Russell and Alfred Whitehead. Anyway, not only did he find mistakes in the reasoning, but he also sent a letter to Bertrand Russell! From then on, Walter was noted for his genius in mathematics. With Warren McCulloch, another genius, they invented the first neuron. It seems simple. But it's the result of a number of sleepless nights. Just as the invention of the wheel appears simple, nothing better has been found to this day. This concept of a neuron is the wheel of AI. 3. A logistic sigmoid activation function makes the sum of the weights larger. (Yes  No) The answer is no. The whole point of a sigmoid function is to reduce the sums when necessary to have comparable numbers to work with.
[ 511 ]
Answers to the Questions
4. A McCullochPitts neuron sums the weights of its inputs. (Yes  No) The answer is yes. It's only when you sum the weights that they make sense. 5. A logistic sigmoid function is a log10 operation. (Yes  No) The answer is no. The sigmoid function is based on Euler's number, e, a constant that is equal to 2.71828. This number produces a natural logarithm. Leonhard Euler (17071783) discovered this in the 18th century with a quill—no scientific calculator or computer! Did you notice that the main mathematical functions used in AI run far back in history? This aspect of the hype around what we think we have found now but has existed for decades, and sometimes centuries, will be dealt with in the following chapters. 6. A logistic softmax is not necessary if a logistic sigmoid function is applied to a vector. (Yes  No) The answer is no. Calculating the sum of several numbers of a vector and then dividing each number by that sum gives a view of the proportions involved. It is a precious tool to keep in mind. 7. A probability is a value between –1 and 1. (Yes  No) The answer is no. Probabilities lie between 0 and 1.
Chapter 3 – Machine Intelligence – Evaluation Functions and Numerical Convergence 1. Can a human beat a chess engine? (Yes  No)
The answer is no. Today, the highestlevel chess tournaments are not between humans but between chess engines. Each chess engine software editor prepares for these competitions by making their algorithms faster and requiring less CPU. Today, a top chess engine running on a smartphone can beat humans. In humantohuman chess competitions, the level of chess has reached very high limits of complexity. Humans now mostly train against machines. 2. Humans can estimate decisions better than machines with intuition when it comes to large volumes of data. (Yes  No) The answer is no. The sheer CPU power of an average machine or even a smartphone can generate better results than humans with the proper algorithms. [ 512 ]
Appendix
3. Building a reinforcement learning program with a Q function is a feat in itself. Using the results afterward is useless. (Yes  No) The answer is no. While learning AI, just verifying that the results are correct is enough. In reallife applications, the results are used in databases or as input to other systems. 4. Supervised learning decision tree functions can be used to verify that the result of the unsupervised learning process will produce reliable, predictable results. (Yes  No) The answer is yes. Decision tree functions are very efficient in many cases. When large volumes are involved, decision tree functions can be used to analyze the results of the machine learning process and contribute to a prediction process. 5. The results of a reinforcement learning program can be used as input to a scheduling system by providing priorities. (Yes  No) The answer is yes. The output of reinforcement learning Q functions can be injected as input into another Q function. Several results can be consolidated in phase 1 and become the reward matrix of a phase 2 reinforcement learning session. 6. Can artificial intelligence software think like humans? (Yes  No) The answer is yes, and no. In the early days, this was attempted with neurosciencebased models. However, applying mathematical models is presently far more efficient. Who knows what will happen in future research with neuromorphic computing, for example? But for the time being, deep learning, the main trend, is based on mathematical functions.
Chapter 4 – Optimizing Your Solutions with KMeans Clustering
1. Can a prototype be built with random data in corporate environments? (Yes  No) The answer is yes, and no. To start developing a prototype, using random data can help make sure that the basic algorithm works as planned. However, once the prototype is advanced, it will be more reliable to use a welldesigned dataset. Then, once the training has been accomplished, random data can again help to see how your system behaves in all situations.
[ 513 ]
Answers to the Questions
2. Do design matrices contain one example per matrix? (Yes  No) The answer is no. A good design matrix contains one example in each row or each column depending on the shape you want it to have. But be careful; a design matrix that contains data that is too efficient might overfit. That means the learning algorithm will be efficient with that data but not adapt to new data. On the other hand, if the dataset contains too many errors, then the algorithm might underfit, meaning it won't learn correctly. A good design matrix should contain reliable data, some imprecise data, and some noise (some data that can influence the algorithm in unreliable ways). 3. Automated guided vehicles (AGVs) can never be widespread. (Yes  No) The answer is no. The sentence is not a correct assertion. AGVs will expand endlessly from now on: drones, cars, planes, warehouse vehicles, industrial vehicles, and more. AGVs, added to AI and IoT, constitute the fourth industrial revolution. 4. Can kmeans clustering be applied to drone traffic? (Yes  No) The answer is yes. Seeing where traffic builds up will prevent drone jams (drones circling and waiting). 5. Can kmeans clustering be applied to forecasting? (Yes  No) The answer is yes. Kmeans clustering can be used for predictions. 6. Lloyd's algorithm is a twostep approach. (Yes  No) Yes, Lloyd's algorithm first classifies each data point in the best cluster. Then, once that is done, it calculates the geometric center or centroid of that center. When no data point changes the cluster anymore, the algorithm has been trained. 7. Do hyperparameters control the behavior of the algorithm? (Yes  No) The answer is yes. Hyperparameters determine the course of the computation: the number of clusters, features, batch sizes, and more. 8. Once a program works, the way it is presented does not matter. (Yes  No) The answer is no. Without a clear presentation of the results, the whole training process is confusing at best and useless at worst. 9. Kmeans clustering is only a classification algorithm. It's not a prediction algorithm. (Yes  No) The answer is no. Kmeans clustering can be used as a prediction algorithm as well.
[ 514 ]
Appendix
Chapter 5 – How to Use Decision Trees to Enhance KMeans Clustering The questions will focus on the hyperparameters.
1. The number of k clusters is not that important. (Yes  No) The answer is no. The number of clusters requires careful selection, possibly a trialanderror approach. Each project will lead to different clusters. 2. Minibatches and batches contain the same amount of data. (Yes  No) The answer is no. "Batch" generally refers to the dataset, and "minibatch" represents a "subset" of data. 3. Kmeans can run without minibatches. (Yes  No) The answer is yes, and no. If the volume of data remains small, then the training epochs can run on the whole dataset. If the data volume exceeds a reasonable amount of computer power (CPU or GPU), minibatches must be created to optimize training computation. 4. Must centroids be optimized for result acceptance? (Yes  No) The answer is yes, and no. Suppose you want to put a key in a keyhole. The keyhole represents the centroid of your visual cluster. You must be precise. If you are simply throwing a piece of paper in your garbage can, you do not need to aim at the perfect center (centroid) of the cluster (marked by the rim of the garbage can) to attain that goal. Centroid precision depends on what is asked of the algorithm. 5. It does not take long to optimize hyperparameters. (Yes  No) The answer is yes, and no. If it's a simple project, it will not take long. If you are facing a large dataset, it will take some time to find the optimal hyperparameters. 6. It sometimes takes weeks to train a large dataset. (Yes  No) The answer is yes. Media hype and hard work are two different worlds. Machine learning and deep learning are still tough projects to implement. 7. Decision trees and random forests are unsupervised algorithms. (Yes  No) The answer is yes, and no. Decision trees can both be used for supervised or unsupervised learning. Decision trees can start with a target value, which makes them supervised. Random forests can be used in the same way.
[ 515 ]
Answers to the Questions
Chapter 6 – Innovating AI with Google Translate
1. Is it better to wait until you have a topquality product before putting it on the market? (Yes  No) The answer is yes, and no. Context is everything. In the early 21st century, Airbus was struggling to complete the A380, the largest ever passenger airplane. Their engineers worked on hundreds of improvements before transporting commercial passengers. We would expect no less! In the case of Google Translate, it is a massive no. By putting Google Translate online and providing an API, Google encouraged thousands of AI developers, linguists, consultants, and users to provide feedback and improvements. Furthermore, Google, once again, occupies a large share of the web market. 2. Considering the investment made, a new product should always be priced high to reach the top segment of the market. (Yes  No) The answer is yes and no. Again, context determines the answer. When Ferrari puts a new car on the market, the price has to be high for two reasons; the quality of the car and the cost of production make it necessary to do so to make the innovation profitable. Also, Ferrari avoids mass production to keep its quality at high levels. When Amazon Web Services put machine learning on the market with SageMaker, it put a "payasyougo" policy in place, starting at a very low end of the market. The product had, and has, its limits, but Amazon now receives tremendous volumes of feedback and is continuously improving the product. 3. Inventing a new solution will make it known in itself. (Yes  No) Yes. An invention society is ready to accept will make its way on its own no matter what. You might be surprised to know that saving a camera's projected image by drawing it dates so far back in history that nobody knows for sure when it was first used. Nobody knows if it was invented or discovered. In any case, the first camera obscura (the first "cameras") was revolutionary. It is now proven that famous painters used the technique. The picture was projected on a paper or canvas. The "printer" was manual. The painter was the "printer." However, cameras, as we know, only became disruptive in the 20th century.
[ 516 ]
Appendix
4. AI can solve most problems without using standard nonlearning algorithms. (Yes  No) The answer is no. Nonlearning classical algorithms are necessary to create datasets, for example. Furthermore, AI relies on cloud servers, architectures, standard algorithms in all languages (C++, Java, Python, and others), and Apache servers. Even on a selfdriving car, the sensors installed require standard hard work to get them working and interpreting information before AI comes in to solve some of the problems. AI is like our brain. Without a body, it cannot function. 5. Google Translate can satisfactorily translate all languages. (Yes  No) After reading this chapter, you might be surprised to have a yes and no answer. If you are using Google Translate to say "hello," "how are you?", "thanks for the message," and similar friendly phrases on your favorite social network or in an email, it is good enough. But when dealing with more detailed phrases and sentences, Google Translate provides random satisfactory results. From a user's perspective, this is bad news. For a developer, it is a goldmine! 6. If you are not creative, it is no use trying to innovate. (Yes  No) The answer is a massive no. You certainly do not need to be either imaginative or creative to innovate. Do not let anybody convince you of such nonsense. If you are designing a solution and find a missing component, look for some alternative components on the web, talk about it, and find people that can help. Then get it done through teamwork. This works every time! Even the great Bill Gates was smart enough to ask Tim Patterson for help to develop MSDOS, and he went on to become a billionaire. 7. If you are not a linguist, it is no use bothering with trying to improve Google Translate. (Yes  No) The answer is no! Once again, never let somebody convince you of such nonsense. Innovating is teamwork. If you like Google Translate and you understand this chapter and have ideas, team up with a linguist around you or through a social network. The world is yours to improve! 8. Translating is too complicated to understand. (Yes  No) No. The way some explain it is too complicated. If you speak a language, you are an expert in translating your thoughts into words. With work, you can get into the translation business.
[ 517 ]
Answers to the Questions
9. AI has already reached its limits. (Yes  No) Certainly not! We have just scratched the surface of both theory and applications.
Chapter 7 – Optimizing Blockchains with Naive Bayes 1. Cryptocurrency is the only use of blockchains today. (Yes  No)
No. IBM HyperLedger, for example, uses blockchains to organize secure transactions in a supply chain environment. 2. Mining blockchains can be lucrative. (Yes  No) Yes. But it is a risk, like any other mining operation or any speculative endeavor. Some companies have huge resources to mine cryptocurrency, meaning that they can beat smaller competitors in creating a block. 3. Blockchains for companies cannot be applied to sales. (Yes  No) No. Blockchain cloud platforms provide smart contracts and a secure way of managing transactions during a sales process. 4. Smart contracts for blockchains are more accessible to write than standard offline contracts. (Yes  No) Yes, if they are standard contracts, this speeds the transaction up. On the other hand, no. If the transaction is complex and requires customization, a lawyer will have to write the contract, which can then only be used on a blockchain cloud platform. 5. Once a block is in a blockchain network, everyone in the network can read the content. (Yes  No) Yes, if no privacy rule has been enforced. No, if a privacy rule has been enforced. IBM Hyperledger, for example, provides privacy functions. 6. A block in a blockchain guarantees that absolutely no fraud is possible. (Yes  No) Yes and no. A block in a blockchain can never be changed again, thereby avoiding fraud. Nobody can tamper with the data. However, if the transaction is illegal in the first place, then the block will be fraudulent as well.
[ 518 ]
Appendix
7. There is only one way of applying Bayes' theorem. (Yes  No) No. There are many variations of Bayes' theorem. Using naive Bayes, for example, avoids the conditional probability constraint. But another approach could use conditional probability. 8. Training a naive Bayes dataset requires a standard function. (Yes  No) No. Gaussian functions, for example, can be used to calculate naive Bayes algorithms, among others. 9. Machine learning algorithms will not change the intrinsic nature of the corporate business. (Yes  No) No. Welldesigned machine learning will disrupt every area of business as algorithms spread through the company, optimizing processes.
Chapter 8 – Solving the XOR Problem with a Feedforward Neural Network 1. Can the perceptron alone solve the XOR problem? (Yes  No)
Yes. The answer would have been no in 1969. A neural network, or some other mathematical process, is necessary to solve this problem. For the record, this is a common problem for electric circuits that function with "feedforward" electricity, and was solved long ago. 2. Is the XOR function linearly nonseparable? (Yes  No) The answer is no if you use a single neuron, and yes if you use a hidden layer with at least two neurons. That is a major problem to address in deep learning. If you cannot separate the features of a face, for example, in a picture, recognizing that face will prove difficult. Imagine a picture with one half of the face in shadow and the other half in bright sunlight. Since the eye and features of one half are in shadow, a poor deep learning program might only capture half of the face, separating the face in the wrong place with a poor edge detection function. Linear separability is thus a key aspect of machine learning. 3. One of the main goals of layers in a neural network is classification. (Yes  No) The answer is yes. Once the data is identifiable with a given neural network architecture, predictions and many other functions become possible. The key to deep learning is to be able to transform data into pieces of information that will make sense through the abstraction obtained over the layers. [ 519 ]
Answers to the Questions
4. Is deep learning the only way to classify data? (Yes  No) The answer is no. You can classify data with a SQL query, spreadsheets, AI, machine learning, and standard source code. Deep learning becomes vital when many dimensions of classification are involved: first finding the edges of objects in a picture, then forms, and then determining what the object represents. To do this with millions of pictures is beyond the scope of standard programming or early AI and machine learning programs. 5. A cost function shows the increase in the cost of a neural network. (Yes  No) The answer is no. A cost function determines how much the training costs you. Running 100,000 epochs is more expensive than running 50,000 epochs. So, at each epoch, the cost of training (how far the system is from its goal) must be estimated. Thus, a good cost function will decrease the cost of running a neural network. 6. Can simple arithmetic be enough to optimize a cost function? (Yes  No) The answer is yes. As long as you know to what extent your cost function is increasing or decreasing, anything that works is fine. 7. A feedforward network requires inputs, layers, and an output. (Yes  No) The answer is yes. Without layers, there is no network. 8. A feedforward network always requires training with backpropagation. (Yes  No) The answer is often yes in changing environments. Since the field is new, we tend to think that once the training is done, the work is done. If the datasets are very stable in a repetitive environment, such as recognizing the difference between various constant products in a shop, warehouse, or factory, then the neural network will do the classification it is designed for. If new products are introduced, then training can be initiated again. 9. In reallife applications, solutions are only found by following existing theories. (Yes  No) The answer is no. Without academic research, deep learning would not even exist. Without universities, the ideas used would be so simple that they would never work well. On the other hand, researchers need reallife feedback. If we find new ways of doing things they recommend, we should publish them to help global research. It's a twoway street.
[ 520 ]
Appendix
Chapter 9 – Abstract Image Classification with Convolutional Neural Networks (CNNs) 1. A convolutional neural network (CNN) can only process images. (Yes  No)
The answer is no. CNNs can process words, sounds, or video sequences, to classify and predict. 2. A kernel is a preset matrix used for convolutions. (Yes  No) The answer is yes, and no. There are many preset matrices used to process images, such as the one used in Edge_detection_Kernel.py in this chapter. However, in this chapter, kernels were created randomly, and then the network trained their weights to fit the target images. 3. Does pooling have a pooling matrix, or is it random? In some cases, a pooling matrix has a size that is an option when the pooling layer is added to the model, such as a 2×2 pooling window. However, in AutoML neural networks, for example, we can try to run optimizing algorithms that will test various sizes to see which one produces the best performance. 4. The size of the dataset always has to be large. (Yes  No) No. A dataset does not have a standard size. It depends on the training model. If the target images, for example, do not contain complex features, the dataset will be smaller than a complex feature dataset. Furthermore, the ImageDataGenerator function will expand the data by distorting it with the options provided. 5. Finding a dataset is not a problem with all the available image banks on the web. (Yes  No) The answer is yes, and no. Yes, because if the model remains a standard academic one, then the available images (CIFAR, MNIST, or others) will suffice. No, because in reallife corporate situations, you will have to build your dataset and add images containing noise. Noise requires more finetuning of the model to become reliable and generalized. 6. Once a CNN is built, training it does not take much time. (Yes  No) The answer is no. Whatever the model is, training will remain timeconsuming if you want it to be reliable. As seen in this chapter, a model requires a lot of options and mathematical thinking. [ 521 ]
Answers to the Questions
7. A trained CNN model applies to only one type of image. (Yes  No) Yes and no. There are three main types of overfitting: °
Overfitting a model for a certain type of image with absolutely no consequence of implementation. In this case, the model classifies and predicts enough to satisfy the goals set. Suppose you are using a type of image with a very high definition. The CNN will detect the small details we may be trying to detect. However, when the application is in production, lowerquality images might make the model fail to identify what it had accurately detected with highquality images.
°
Overfitting a model that creates implementation problems because it cannot adapt to different images at the same time. The model will then go through more training.
°
Overfitting a model that trains a certain type of image quite well but does not fit similar types of images when needed.
Each situation has its constraints. As long as the model works, no general rules apply. It is up to you to decide. 8. A quadratic loss function is not very efficient compared to a crossentropy function. (Yes  No) The answer is no. Each model has its constraints. Quadratic loss functions work fine on some models and do not provide good results on others. This represents the main problems of training a model. No general rules will help you. You have to use your neurons or write a program that modifies the model automatically. 9. The performance of a deep learning CNN does not present a real issue with modern CPUs and GPUs. (Yes  No) The answer is yes, and no. If the model runs quickly enough for your needs, then performance will not limit the outcome of your project. However, in many cases, it remains a problem. Reducing features to focus on the best ones is one of the reasons that the layers bring the size to analyze down, layer by layer.
Chapter 10 – Conceptual Representation Learning
1. The curse of dimensionality leads to reducing dimensions and features in machine learning algorithms. (Yes  No) [ 522 ]
Appendix
Yes. The volume of data and features makes it necessary to extract the main features of an observed event (an image, sound, and words) to make sense of it. Overfitting and underfitting apply to dimensionality reduction as well. Reducing the features until the system works in a lab (overfitting) might lead to nowhere once the application faces reallife data. Trying to use all the features might lead to underfitting because the application solves no problem at all. Regularization applies not just to data but to every aspect of a project. 2. Transfer learning determines the profitability of a project. (Yes  No) Yes, if an application of an AI model in itself was unprofitable the first time, but could generate profit if used for a similar type of learning. Reusing some functions would generate profit, no doubt. No, if the first application was extremely profitable but "overfitted" to meet the specifications of a given project. 3. Reading model.h5 does not provide much information. (Yes  No) No. No in this case means that the assertion of the sentence is wrong. The saved model does provide useful information. Saving the weights of a TensorFlow model, for example, is vital during the training process to control the values. Furthermore, trained models often use HDF files (.h5) to load the trained weights. A Hierarchical Data Format (HDF) contains multidimensional arrays of scientific data. 4. Numbers without meaning are enough to replace humans. (Yes  No) Yes, in the sense that in many cases, mathematics provides enough tools to replace humans for many tasks (games, optimization algorithms, and image recognition). No, in cases where mathematics cannot solve problems that require concepts, such as many aspects of NLP. 5. Chatbots prove that body language doesn't mean that much. (Yes  No) Yes, in the sense that in many applications, body language does not provide additional information. If only a yes or no answer is required, body language will not add much to the conversation. No, in the sense that if emotional intelligence is required to understand the tone of the user, a webcam detecting body language could provide useful information.
[ 523 ]
Answers to the Questions
6. Presentday artificial neural networks (ANNs) provide enough theory to solve all AI requests. (Yes  No) No. Artificial neural networks (ANNs) cannot solve thousands of problems, for example, translating poetry novels or recognizing images with forms that constantly vary. 7. Chatbots can now replace humans in all situations. (Yes  No) No. Concepts need to be added. The market provides all the necessary tools. It will take some years to be able to speak effectively with chatbots. 8. Selfdriving cars have been approved and do not need conceptual training. (Yes  No) Yes and no; there is some debate in this area. On the one hand, sensors and mathematics (linear algebra, probabilities) might succeed in effectively navigating roads in most driving conditions within a few years. However, certain problems will require concepts (and more robotics) in critical situations. If a selfdriving car encounters a wounded person lying on the road, what is the best approach? The choices are to call for help, find another person if the help arrives too late, pick up the victim, drive them to a hospital (robotics), and much more. 9. Industries can implement AI algorithms for all of their needs. (Yes  No) Yes and no. In many cases, all of the necessary tools are there to be used. If the right team decides to solve a problem with AI and robotics, it can be done. However, some tools are missing when it comes to addressing particular circumstances; for example, realtime management decision tools when faced with unplanned events. If a system breaks down, humans can still adapt faster to find alternative solutions to continue production.
Chapter 11 – Combining Reinforcement Learning and Deep Learning 1. A CNN can be trained to understand an abstract concept? (Yes  No)
Yes. A CNN can classify images and make predictions. But CNNs can analyze any type of object or representation. An image, for example, can be linked to a word or phrase. The image thus becomes a message in itself.
[ 524 ]
Appendix
2. Is it better to avoid concepts and only use reallife images? (Yes  No) No. Images provide many practical applications, but at some point, more is required to solve planning problems, for example. Planning requires much more than this type of dataset. 3. Do planning and scheduling mean the same thing? (Yes  No) No. Planning describes the tasks that must be carried out. Scheduling adds a time factor. Planning tells us what to do, and scheduling tells us when. 4. Is Amazon's manufacturing patent a revolution? (Yes  No) No. Manufacturing clothing has been mastered by factories around the world. It would be a revolution if something in the process was entirely new. However, automation in the apparel industry has been around for 20+ years. The patent is a specialized case of elements that exist, like a new brand of car. Yes. With such a worldwide distribution, Amazon has come very close to the end user. The end user can choose a new garment, and it will be manufactured directly on demand. This connectivity will change the apparel manufacturing processes and force its competitors to find new ways of making and selling garments. 5. Learning how warehouses function is not useful. (Yes  No) No. Online shopping requires more and more warehouse space and processes. The number of warehouses will now increase faster than shops. There are many opportunities for AI applications in warehouses. 6. Online marketing does not require AI. (Yes  No) No. On the contrary, AI is used by applications for online marketing every day, and this will continue for decades.
Chapter 12 – AI and the Internet of Things 1. Driving quickly to a location is better than safety in any situation. (Yes  No) Yes and no. Selfdriving cars face the same difficulties as humandriven cars: getting to a location on time, respecting speed limits, or driving as safely as possible. Selfdriving cars, like humans, are constantly improving their driving abilities through experience.
[ 525 ]
Answers to the Questions
Yes. Sometimes, a selfdriving car will perform better on a highway with little traffic. No. Sometimes, if the highways are dangerous (owing to weather conditions and heavy traffic), a selfdriving car should take a safer road defined by slow speed and little to no traffic. This way, if difficulties occur, the selfdriving car can slow down and even stop more easily than on a highway. 2. Selfdriving cars will never really replace human drivers. (Yes  No) Nobody can answer that question. As selfdriving cars build their abilities and experience, they might well end up driving better than humans. In very unpredictable situations, humans can go off the road to avoid another car and back off a bit, for example. It will take more work to get a selfdriving car to do that. One thing is certain, though. If a human is driving all night and falls asleep, the selfdriving car will detect the head slumping movement, take over, and save lives. The selfdriving car can also save lives if the human has a medical problem while driving. 3. Will a selfdriving fire truck with robots be able to put out a fire one day? (Yes  No) Yes. Combining selfdriving fire trucks with robots will certainly save many lives when a fire department faces difficult fires to extinguish. Those saved lives include firefighters who risk their own lives. It might help firefighters focus on helping people while robots do tougher jobs. This robothuman team will no doubt save thousands of lives in the future. 4. Do major cities need to invest in selfdriving cars or avoid them? (Invest  Avoid) Invest. With slow but safe selfdriving cars, commuters could share public, free, or very cheap electric selfdriving cars instead of having to drive. It would be like having a personal chauffeur. 5. Would you trust a selfdriving bus to take children to school and back? (Yes  No) This answer will change with time, as technology continues to evolve. No. Not in the present state of selfdriving vehicles. You should not fully trust an autonomous vehicle 100%! Yes, when selfdriving cars, buses, and trucks prove that they can outperform humans. Selfdriving vehicles will not make the same mistakes as humans: using smartphones while driving, talking to passengers without looking at the road, and many others. [ 526 ]
Appendix
6. Would you be able to sleep in a selfdriving car on a highway? (Yes  No) No, not in the present state of selfdriving vehicle technology. Yes, when reliability replaces doubts. 7. Would you like to develop a selfdriving program for a project for a city? (Yes  No) That one is for you to think about! You can also apply the technology to warehouses for AGVs by contacting the companies or AGV manufacturers directly.
Chapter 13 – Visualizing Networks with TensorFlow 2.x and TensorBoard 1. A CNN always has the same number of layers. (Yes  No)
No. A CNN does not have the same number of layers or even the same type of layers. The number of layers is part of the work to optimize an artificial neural network. 2. ReLU is the best activation function. (Yes  No) No. ReLU is an efficient activation function, but there are others such as leaky ReLU, softmax, sigmoid, and tanh. 3. It is not necessary to compile a sequential classifier. (Yes  No) No. The assertion should be yes – it is necessary. 4. The output of a layer is best viewed without running a prediction. (Yes  No) No. The output of a layer and a prediction are unrelated. The output of the layer can be the transformation of a layer (convolutional, pooling, dropout, flattening, other) or a prediction. 5. The names of the layers mean nothing when viewing their outputs. (Yes  No) No. The layers mean everything! A pooling layer and a dropout layer are quite different. 6. TensorFlow 2.x does not include Keras. (Yes  No) No. TensorFlow has now integrated Keras, which helps to build seamless neural networks.
[ 527 ]
Answers to the Questions
7. Google Colaboratory is just a repository, like GitHub. (Yes  No) No. Google Colaboratory provides a small but free server to create and run programs online. 8. Google Colaboratory cannot run notebooks. (Yes  No) No. It can run notebooks. 9. It is possible to run TensorBoard in Google Colaboratory notebooks (Yes  No) Yes. This is a useful feature. 10. Accuracy is displayed in TensorBoard (Yes  No) Yes. It is an efficient way to evaluate the efficiency of ANNs, for example.
Chapter 14 – Preparing the Input of Chatbots with Restricted Boltzmann Machines (RBMs) and Principal Component Analysis (PCA) 1. RBMs are based on directed graphs. (Yes  No)
No. RBM graphs are undirected, unsupervised, and memoryless, and the decisionmaking is based on random calculations. 2. The hidden units of an RBM are generally connected to one another. (Yes  No) No. The hidden units of an RBM are not generally connected to each other. 3. Random sampling is not used in an RBM. (Yes  No) No. False. Gibbs random sampling is frequently applied to RBMs. 4. PCA transforms data into higher dimensions. (Yes  No) Yes. The whole point of PCA is to transform data into a lower number of dimensions in higher abstraction dimensions (key dimensions isolated) to find the principal component (highest eigenvalue of a covariance matrix), then the second highest, down to the lowest values. 5. In a covariance matrix, the eigenvector shows the direction of the vector representing that matrix, and the eigenvalue shows the size of that vector. (Yes  No) Yes. Eigenvalues indicate how important a feature is, and eigenvectors provide a direction. [ 528 ]
Appendix
6. It is impossible to represent a human mind in a machine. (Yes  No) No. It is possible to a certain extent with sensors and in a limited perimeter. 7. A machine cannot learn concepts, which is why classical applied mathematics is enough to make efficient AI programs for every field. (Yes  No) No. Never believe that. Progress is being made and will never stop until mind machines become mainstream.
Chapter 15 – Setting Up a Cognitive NLP UI/CUI Chatbot 1. Can a chatbot communicate like a human? (Yes  No)
No. Communicating like a human means being human: having a body with body language, sensations, odors, fear hormones, and much more. Chatbots only emulate these behaviors. 2. Are chatbots necessarily AI programs? (Yes  No) No. Many call centers use the "press 1, press 2 … press n" method, which requires careful organization but no AI. 3. Chatbots only need words to communicate. (Yes  No) The answer is not a clearcut one. Yes. Simple chatbots can communicate with words in a controlled situation. No. When polysemy (several meanings for the same word or situation) is involved, pictograms and more add more efficient dimensions. 4. Do humans only chat with words? (Yes  No) No. Humans express themselves through the tone of their voice, body language, or music, for example. 5. Humans only think in words and numbers. (Yes  No) No. Certainly not. Humans think in images, sounds, odors, and feelings. 6. Careful machine learning preparation is necessary to build a cognitive chatbot. (Yes  No) The answer depends on the context. No. In limited "press 1 or press 2 " situations, chatbots can perform well with limited cognitive capacities. Yes. To engage in a real conversation with a human, mental images are the key to providing an empathetic exchange. [ 529 ]
Answers to the Questions
7. For a chatbot to function, a dialog flow needs to be planned. (Yes  No) This depends upon what you want your chatbot to do. Yes. It will provide better results in a business environment. No. If you want the chatbot to talk freely, you need to free it a bit. This still requires planning of the dialog, but it is more flexible. 8. A chatbot possesses general AI, so no prior development is required. (Yes  No) No. This is presently impossible. Only narrow (specific to one or a few fields) AI exists in real life, contrary to science fiction movies and media hype. 9. A chatbot translates fine without any function other than a translation API. (Yes  No) No. At this point in the history of translation bots, the translations are not quite reliable without additional customization. 10. Chatbots can already chat like humans in most cases. (Yes  No) No. Interpreting a language will take quite some more challenging work and contributions.
Chapter 16 – Improving the Emotional Intelligence Deficiencies of Chatbots
1. When a chatbot fails to provide a correct response, a hotline with actual humans needs to take over the conversation. (Yes  No) This comes down to context and practicality. Yes. That is what the best solution would be. To have an interactive chat service kick in. No. In many cases, it would be too expensive. A nice support screen could do the job and send an email to the support team. 2. Small talk serves no purpose in everyday life or with chatbots. It is best to just get to the point. (Yes  No) Again, this is a matter of context. Yes. If it's an emergency bot, of course! No. If the chatbot is there to perform a tedious administrative function, some degree of small talk will make the system more bearable.
[ 530 ]
Appendix
3. Data logging can be used to improve speech recognition. (Yes  No) Yes. Absolutely. More data means better machine learning training. 4. The history of a chatbot agent's conversations will contain valuable information. (Yes  No) Yes. Absolutely. More feedback means more machine learning progress. 5. Presentday technology cannot make use of the data logging of a user's dialogs. (Yes  No) No. We can, of course, parse data logging and extract valuable information. 6. An RNN uses sequences of data to make predictions. (Yes  No) Yes, it does. 7. An RNN can generate the dialog flow of a chatbot automatically for all applications. (Yes  No) Yes and no. It can, but the quality is sometimes still terrible at this point in the history of automatic dialog flows!
Chapter 17 – Genetic Algorithms in Hybrid Neural Networks 1. A cell contains 42 chromosomes. (Yes  No) No. There are 46 chromosomes in a cell. 2. A genetic algorithm is deterministic, not random. (Yes  No) No. A genetic algorithm is random, which makes it more efficient than many deterministic algorithms. 3. An evolutionary algorithm means that program code evolves. (Yes  No) There is not a single clearcut answer. No. The program runs like any other program. Yes. In some ways, when it is used to optimize a neural network in a hybrid neural network, it changes the parameters of the system. Also, it is possible to use a genetic algorithm to add or take layers out of a CNN, for example. 4. It is best for a child to have the same genes as one of the parents even after many generations. (Yes  No) No. Certainly not! We need diverse genes to remain a fit genetic group. [ 531 ]
Answers to the Questions
5. Diversity makes the gene sets weaker. (Yes  No) No. The greater the diversity, the richer the genetic pool is to adapt and remain fit. 6. Building a neural network only takes a few lines, and the architecture always works. (Yes  No) This depends on what you mean by "work." Yes. Building a neural network only takes a few lines with TensorFlow 2.x, for example, and it will work. No. The neural network will work, but it will most probably not be efficient until its architecture and hyperparameters are finetuned. 7. Building a neural network with a genetic algorithm can help optimize the architecture of the layers. (Yes  No) Yes. It is possible to extend a genetic algorithm to layer optimizing. Each layer can be a gene, and then the various alternatives can be run to check their accuracy. 8. Hybrid neural networks are useless since deep learning will constantly progress. (Yes  No) No. Deep learning will reach an asymptote, as do all systems. At that point, new dimensions can be added to deep learning architecture, such as genetic algorithms. 9. Would you trust a genetic algorithm to make decisions for you? (Yes  No) Yes, if the system is properly designed. No if you don't trust genetic algorithms! 10. Would you trust a hybrid neural network to optimize the architecture of your network? (Yes  No) Yes, if the system is properly designed. No, if it is unreliable or biased.
Chapter 18 – Neuromorphic Computing 1. Neuromorphic computing reproduces our mental activity. (Yes  No)
No. That is the point. Neuromorphic begins with subsymbolic lowlevel neuronal brain activity stimulations that do not form highlevel mental activity yet. Our mental activity already uses symbols and contains representations in the form of words, images, numbers, and all kinds of constructions in general.
[ 532 ]
Appendix
However, we can say YES if we are referring to the output results translated into mental activity. My point is that yes and no answers limit our views of AI. 2. Neuromorphic computing reproduces our brain activity. (Yes  No) Yes. That is the point! The core concept is to go from brain activity to structures formed by neuron spikes. 3. Semantic Pointer Architecture (SPA) is a hardware architecture. (Yes  No) No. Semantic pointers are like computer program pointers such as C++ pointers. The difference is that they carry a partial meaning of representation to come, hence the word "semantic." 4. NEF stands for Neural Engineering Framework. (Yes  No) Yes, this is true. 5. Loihi is a classical chip. (Yes  No) No. Not at all! Loihi is an Intel neurocomputing chip containing a massive number of neurons. A human brain contains around 100 billion neurons. Imagine that, soon, you'll have a network of neurocomputing chips (Intel or other) that attain that number. Then imagine what can be done with semantic pointers through neurocomputing. 6. Reproducing our brain's neural activity cannot solve an equation. (Yes  No) No. Of course, we can use neurocomputing to solve equations through the SPA approach. 7. An ensemble in Nengo contains algorithms. (Yes  No) No. But the question was tricky! Basically, Nengo uses a nonsymbolic approach as discussed at length previously. However, it contains Python tutorials with many algorithms solved through neurocomputing, forming a complete problemsolving package. 8. Spiking blocks neuronal activity. (Yes  No) No. But again, this is a tricky question. Spiking is reflected in the level of activity in a neuron. So, in that respect, the answer is no. But a spiking neuron can inhibit another neuron, thereby blocking it indirectly. 9. Firing patterns can be used to analyze brain activity. (Yes  No) Yes. Firing patterns change in time in neurocomputing, providing useful information on how the neurons reach a given state.
[ 533 ]
Answers to the Questions
10. Machine learning and deep learning are only metaphors of our brain's activity. (Yes  No) Yes. That is the core problem associated with deep learning. They are highlevel representations of our brain's neuronal activity.
Chapter 19 – Quantum Computing 1. Beyond the hype, no quantum computer exists. (Yes  No)
No. False. You can already run a quantum computer on IBM Q's cloud platform: https://www.research.ibm.com/ibmq/. The following screenshot is the result of one of the real quantum computer (IBM) calculations I ran on a quantum score explained in the chapter:
Figure A.1: IBM quantum computer calculation
[ 534 ]
Appendix
2. A quantum computer can store data. (Yes  No) No. Instability prevents any form of storage at this point. 3. The effect of quantum gates on qubits can be viewed with the Bloch sphere. (Yes  No) Yes. A Bloch sphere will display the state of a qubit. 4. A mind that thinks with past experiences, images, words, and other bits of everyday information, like stored memories, will find deeper solutions to problems that mathematics alone cannot solve. (Yes  No) There is no single generally accepted answer for this. Just as qubits, the answer is somewhere between yes (1) and no (0)! No. False. Many researchers believe that mathematics alone can solve all human problems. Yes. True. Mathematics alone cannot replace deep thinking. Even if computers have incredible power and can beat human players at chess, for example, they still cannot adapt to new situations without going through a design and training process. Concepts need to be added and experienced (memory as well). I bet that machine mind concepts will become progressively more mainstream to solve deep thinking problems. 5. A quantum computer will solve medical research problems that cannot be solved today. (Yes  No) Yes. There is no doubt about that. The sheer computing power of a quantum computer can provide exponential DNA sequencing programs for epigenetic research. 6. A quantum computer can solve mathematical problems exponentially faster than classical computers. (Yes  No) Yes. Classical computers function at 2 × n (number of bits), and quantum computers run at 2n (n being the number of qubits)! 7. Classical computers will soon disappear and smartphone processors too. (Yes  No) No. Quantum computers require such a large amount of space and physical stability that this will not happen soon. Furthermore, classical computers and smartphones can store data; quantum computers cannot.
[ 535 ]
Answers to the Questions
8. A quantum score cannot be written in source code format but only with a visual interface. (Yes  No) No. False. IBM, for example, can swap the quantum from score to a QASM interface or display both, as shown here:
Figure A.2: QASM interface
9. Quantum simulators can run as fast as quantum computers. (Yes  No) No. Certainly not! A simulator shows how a quantum score would behave on a real quantum computer. Although the simulator can help build the score, a quantum computer will run exponentially faster than the simulator. 10. Quantum computers produce intermediate results when they are running calculations. (Yes  No) No. This is not possible. The qubits are too unstable. Observing them makes the system collapse. However, simulators such as Quirk come in handy. Since they are not real, intermediate results can be displayed to design a quantum score.
[ 536 ]
Other Books You May Enjoy If you enjoyed this book, you may be interested in these other books by Packt:
Artificial Intelligence with Python Alberto Artasanchez, Prateek Joshi ISBN: 9781839219535 ●
Understand what artificial intelligence, machine learning, and data science are
●
Explore the most common artificial intelligence use cases
●
Learn how to build a machine learning pipeline
●
Assimilate the basics of feature selection and feature engineering
●
Identify the differences between supervised and unsupervised learning [ 537 ]
Other Books You May Enjoy ●
Discover the most recent advances and tools offered for AI development in the cloud
●
Develop automatic speech recognition systems and chatbots
●
Apply AI algorithms to time series data
[ 538 ]
Other Books You May Enjoy
AI Crash Course Hadelin de Ponteves ISBN: 9781838645359 ●
Master the key skills of deep learning, reinforcement learning, and deep reinforcement learning
●
Understand Qlearning and deep Qlearning
●
Learn from friendly, plain English explanations and practical activities
●
Build fun projects, including a virtualselfdriving car
●
Use AI to solve realworld business problems and win classic video games
●
Build an intelligent, virtual robot warehouse worker
[ 539 ]
Other Books You May Enjoy
Leave a review  let other readers know what you think
Please share your thoughts on this book with others by leaving a review on the site that you bought it from. If you purchased the book from Amazon, please leave us an honest review on this book's Amazon page. This is vital so that other potential readers can see and use your unbiased opinion to make purchasing decisions, we can understand what our customers think about our products, and our authors can see your feedback on the title that they have worked with Packt to create. It will only take a few minutes of your time, but is valuable to other potential customers, our authors, and Packt. Thank you!
[ 540 ]
Index Symbols 2D convolution layer adding 210 kernel 210, 211 shape 215
A absolute error 117 activation functions 215, 340, 341 Adam optimizer 225, 226 adaptive moment estimation (Adam) 225 adaptive thinker becoming 4 affirmation example 410 agent about 4 creating, to learn Dialogflow 377, 378 fulfillment functionality, adding to 392, 393 AI as frontier 135 disruption 123 emergency, in recent years 125 innovation 123 mathematical theories 124 public awareness 126 algorithm cell 453, 454 apparel conveyor belt processing 271 apparel manufacturing process 267, 268 artificial hybrid neural networks about 456 LSTM, building 457, 458 model, goal 458 artificial neural networks (ANNs) 206
automated apparel manufacturing process CRLMM, applying to 266 Automatic Text to Speech function 399 Auto Speech Adaptation 399
B backpropagation used, for implementing vintage XOR solution in Python 189191 backtranslation 145147 Bellman equation about 11, 13 mathematical representation, building of 10 binary crossentropy 224, 225 bitcoin mining 159, 160 blob 305 Bloch sphere 493, 494 block creating 165 exploring 166 blockchain anticipation novelty 169, 170 blockchain data used, for optimizing storage levels 170 blockchains background 158, 159 used, for sharing information in supply chain 161164 using, in supply chain network 164 Boltzmann equation and constant 3 bot running, on website 397 bot virtual clusters as solutions 86, 87
[ 541 ]
C cells 435 centroid 83, 96 chatbot agents 376 machine learning, using in 398 child 439 classifier about 210 using, for training 229, 230 CLT 96 clusters 83 CNNCRLMM implementing, to detect gaps 276 CNN model compiling 223 data, loading 227, 228 goal 222, 223 saving 230 training 221 cogfilmdr agent enhancing, with fulfillment webhook 394396 conceptual representation learning metamodels blessing of dimensionality 255 curse of dimensionality 254 motivation 254 conditional probability 167 conditioning management 75, 76 context 138, 387391 continuous cycle of kmeans clustering chaining, to decision tree 110114 contrastive divergence 354 convergence about 9, 48 implicit convergence 49 numerically controlled gradient descent convergence 4955 Conversational User Interfaces (CUI) 386 convolutional 207 convolutional layer activation functions 328, 329 higherlevel representations, through layers 329331 convolutional neural networks (CNNs) 2D convolution layer, adding 210
about 124, 205, 206, 297 convolution layer 219 defining 207209 dense layers 220 flattening layer 220 initializing 209 layers, building of 319322 pooling 218, 219 pooling layer 219 cost function 191194 covariance 365 CRLMM applying, to automated apparel manufacturing process 266 model, applying of 297, 298 parking lots, classifying 301 running 307 trained model, using 300 training 269 crossover 438 cryptocurrency using 160 curse of dimensionality 124
D data augmentation about 227 on testing dataset 228 data logging 415418 data points 83 dataset about 298300 deriving 367 designing 28, 29, 69 dataset optimization 68 decision line 304 decision tree training 106109 decoherence 487 deeper translation with phrasebased translations 147150 degrees 492 dense activation functions 221 dense layer 428 derivative 50 design matrix
[ 542 ]
approval 70, 71 approval, obtaining on format 71, 72 Dialogflow learning 377, 378 difficulty of problem identifying 94 dimensionality reduction 7274 disruption in AI 123 disruptive solutions versus revolutionary solutions 127 domain learning about 247 gap concept 249252 gap datasets 253 trained model program 248 trained models 248 doublechecking function 116 dropout layers for higherlevel representations 333
E eigenvalues 365, 366 eigenvectors 366 embedding 427 emotional polysemy problems, solving of 408 emotions creating 418422 energy function 354 Enhanced Speech Models 399 entities about 378 creating 379 saving 381 episodes evaluating, of learning sessions 4648 epoch accuracy 339 epochs running 355357 error function 354 evolutionary algorithms 434437 evolutionary computation 436
F facial analysis fallacy about 411 frown 411 smile 411 feature entity creating 382 features 29, 167 feature vector creating 366 feedforward neural network (FNN) about 179 building 184 defining 184 used, for implementing vintage XOR solution in Python 189191 fitness 439, 440 fitness cell 454, 455 FNN XOR function applying, to optimize subsets of data 196202 food conveyor belt processing 270 frequentist error probability function adding 151, 152 fulfillment defining 393, 394 fulfillment functionality adding, to agent 392, 393 fulfillment webhook cogfilmdr agent, enhancing 394396 full training dataset training 98
G ga_main() cell 455, 456 gamma 13 GA model creating 437 gap concept 237, 238 gap conceptual dataset 253, 254 gaps abstract notions 272, 273 Gaussian naive Bayes implementing 173, 174 gene set of parent 438 gene set of population 438 [ 543 ]
genetic algorithm, building in Python about 440 algorithm, calling 441 crossover function 445 display parent 444, 445 fitness 443, 444 generations, producing of children 447450 libraries, importing 440 main function 441 mutation function 446, 447 parent generation 442, 443 parent generation process 442 summary code 450 geometric center 83 Gibbs sampling 352 Gini impurity 64 Google Colaboratory about 334336 URL 334 Google's developers' API client library page reference link 128 Google's translation service implementing 129, 130 Google Translate about 136 customizing, with Python program 137, 138 from linguist's perspective 130 header 128 linguistic assessment 131 program 128 using 128 working with 131 Google Translate customized experiment conclusions 152 Google_Translate_Customized.py KNN function, implementing in 144 gradient 50 gradient descent 191194 graphics processing unit (GPU) 93 greetings problem example 409 grouping 196 GRU 427
H heredity in humans 434, 435 working 435, 436 H gate 495, 496 hidden units computing, in training function 351, 352 hub 169 human analytic capacity evaluating 5659 hyperparameters 81
I IBM Q quantum computer score 497499 implicit convergence 49 inductive abstraction 235 inductive thinking 235 innovation in AI 123 integration 468 intent about 382 creating 384, 385 inventions versus innovations 126 isotropic distribution 305
J Jargon 132
K kernel about 210 developers' approach 212214 intuitive approach 211, 212 mathematical approach 214 kmeans clustering mathematical definition 78, 79 unsupervised learning 92, 93 kmeans clustering algorithm limits of implementation 87 kmeans clustering program 77, 78 kmeans clustering solution [ 544 ]
data 75 implementing 74 model, loading 84, 85 model, saving 84, 85 results, analyzing 85, 86 vision 74, 75 KNN algorithm about 138 implementing 139142 KNN function implementing, in Google_Translate_ Customized.py 144 knn_polysemy.py program 142144
L labels 304 layers building, of CNN 319322 leaf 64 lexical field 132, 135 linearly separable model about 181, 182 disadvantages 182, 183 linear separability 194, 195 linguistic assessment of Google Translate 131 LLN using 96 Lloyd's algorithm 80 load keyword 34 logistic activation functions 35 logistic classifier 36, 37 logistic function 37, 38 loss function about 223 quadratic loss function 223, 224 LSTM 425
M machine learning using, in chatbot 398 versus traditional applications 23 machine learning agents about 398 speechtotext function 398 spelling correction 401, 402
texttospeech function 399, 400 machine learning algorithms significance 403, 404 machine thinking adapting 4 macrostates 4 mapping environment 8 margin 304 Markov chain 12 Markov decision process (MDP) about 1, 124, 297 in natural language 7, 8 mathematical representation, building of 10 Markov property 12 mathematical description 28 mathematical model building 10 McCullochPitts neuron about 3134 using 2931 MDP agent 8, 9 MDP function standard output 278, 279 target, finding for 284286 MDP input 278 MDP output 278, 279 MDP output matrix graph interpretation 280 MDP result matrix 281, 282 metrics 226 microstates 4 MindX conceptual representations 500 MindX experiment about 501 data, preparing 501 quantum function 504 situation function 501503 miner 160 minibatches random sampling, implementing 95 ML algorithms 103 ML/DL model selecting 69, 70 ML projects key standard corporate guidelines, avoiding 76, 77 [ 545 ]
model applying, of CRLMM 297, 298 defining 336 training 336 Monte Carlo estimator using 97 mutation 439
N naive Bayes example 167169 implementing, in Python 173 limits 174 Python program 174176 supply chain, optimizing with 167 natural language 28 natural language processing (NLP) 376 natural language understanding (NLU) 376 Nengo about 463 data, visualizing 470474 examples 465 information, retrieving with probes 475478 installing 464 neuron dimensions 468 node 468, 469 Python program, creating 466 reference link 464 unique approach, applying to critical AI research areas 479482 Nengo ensemble 466, 467 Nengo frontend API definitions reference link 470 Nengo GUI installing 464 reference link 464 Nengo neuron types about 467 reference link 468 Nengo objects connecting 470 reference link 470 Neural Engineering Framework (NEF) 463 neural machine translation (NMT) 144 neural networks 124 neuromorphic computing 462, 463
nodes 160 NOT gate 495 NPhard 94, 95 numerically controlled gradient descent 49,55
O optimal production rate (OPR) 236 optimizer about 281 as regulator 281 original perceptron 180 output of layers, CNN exploring, with TensorFlow 318, 319 overfitting 247
P parking lot, finding about 310, 311 itinerary graph 313, 314 support vector machine 311, 312 weight vector 314 parking space finding 307310 partition function 354 PCA about 362, 363 analyzing, to obtain entry point for chatbot 370372 representing, with TensorBoard Projector 367370 weights of RBM, using as feature vectors 357362 phrasebased machine translation (PBMT) 144 physical neural network about 451 features 452 Pickle 84 plan 264 pointer 463 policy 8, 12 polysemy 136 pooling 218 pooling layer for higherlevel representations 332 [ 546 ]
poor conditioning 75 population 438 posterior probability 167 postsynaptic currents (PSCs) 467 prediction 84 prediction program running 274 probabilistic distributions 3 problem describing, to solve 7, 8 profit generating, with transfer learning 234, 235 proof of concept (POC) 93 public service project 294 purchase plan about 260 example 260 Python naive Bayes, implementing 173 restricted Boltzmann machine (RBM), building in 350 vintage XOR solution, implementing with backpropagation 189191 vintage XOR solution, implementing with FNN 189191 Python program about 80 Google Translate, customizing 137, 138 hyperparameters 81 kmeans clustering algorithm 82 result labels, defining 82 results, displaying 83 training dataset 81 PythonTensorFlow architecture 35
Q Qlearning 4, 277 quadratic loss function 223, 224 quantum computers about 486 speed 487, 489 quantum computer score with IBM Q 497499 with Quirk 496, 497 quantum gates, with Quirk about 494
H gate 495, 496 NOT gate 495 quantum score composing 494 creating 504, 505 output 506 running 505 qubit defining 490 position 491 representing 490, 491 Quirk quantum computer score 496, 497 URL 494
R radians 492 radius 492 random forests 114116 randomness 13 random sample training, of training dataset 98, 100 random sampling implementing, with minibatches 95 of hidden units 352 raw data preprocessing 103 RBM class creating 350 training function, creating in 350, 351 reallife issues, overcoming with threestep approach about 5 mathematical model, building 10 problem, describing 7, 8 source code, writing 1416 realtime manufacturing process 262 realtime manufacturing revolution 263266 reconstruction 353 rectified linear unit (ReLU) 215217 recurrent neural network (RNN) 144 regularization 247 reinforcement learning model use cases 2023 reinforcement learning (RL) about 1, 12 [ 547 ]
Boltzmann equation and constant 3 Boltzmann, with optimal transport 3 concepts 2 lessons 16, 17 optimal transport 2 outputs, using 1820 probabilistic distributions 3 restricted Boltzmann machine (RBM) about 345 architecture 346, 347 building 345 building, in Python 350 energybased model 347349 structure 350 revolutionary solutions versus disruptive solutions 127 reward 12 reward matrix 41 RLDLCRLMM building 274, 275 circular model 286290 circular process 275 components 274, 275 RLDLCRLMM model setting up 295297 RNN text generation 426 using 424, 425 RNN research for future automatic dialog generation 423 rootmeansquare deviation (RMSprop) 226 rotations 492, 493
S safety rank 314 scheduling 264 scripts pipeline 103 selection 438 semantic 464 semantic pointer 463 sequential model 427 services expanding, to face competition 262 SFNN 452, 453 shuffling 101, 102
small talk about 412 courtesy 412414 emotions 415 softmax function 3840 software implementation 28 speech recognition fallacy 410 speech recognition functions, settings Auto Speech Adaptation 399 Enhanced Speech Models 399 speechtotext function 398 spelling 401, 402 standardization 29 state transition function 10 stochastic process 12 stock keep units (SKUs) 168 storage levels, optimizing with blockchain data about 170 dataset, defining 170, 171 frequency, calculating 171, 172 likelihood, calculating 172 naive Bayes equation, applying 172, 173 stride 218 subject matter expert (SME) 6 subsets of data optimizing, by applying FNN XOR function 196202 supervised learning chaining, to verify unsupervised learning 102 used, for evaluating result that surpasses human analytic capacity 6064 supply chain blockchains, using to share information 161164 optimizing, with naive Bayes 167 supply chain management (SCM) 169 supply chain network blockchains, using in 164 support points 304 support vector machine (SVM) about 303 example 303 Python function 305, 306 used, for increasing safety levels 302 support vectors 304 SVM function adding 301 [ 548 ]
T target 437 TensorBoard accuracy, analyzing of CNN 334 representation of features 383 running 337 TensorBoard Projector used, for representing PCA 367370 TensorFlow installing 337 output of layers, exploring of CNN 318, 319 test dataset 84 testing data loading 228 testing dataset about 228 data augmentation 228 text generating 429431 vectorizing 426, 427 text dialog 386 text generation with RNN 426 texttospeech about 399 settings, configuring 400 thinking quantum computer 500 time squashing 268 timestep 428 traditional applications versus machine learning 23 trained TensorFlow 2.x model applying 246 displaying 239241 loading 238 loading, for usage 242244 profitable, making 246 strategy, defining 245 training dataset about 226 random sample, training of 98100 volume 74 training function creating, in RBM class 350, 351 hidden units, computing in 351, 352 training phrases 382
transfer learning motivation 235 profit, generating with 234, 235 using 245 transition 12 translating 132 translation checking 134
U underfitting 247 unit training dataset generalizing 269 unsupervised learning about 12 with kmeans clustering 92, 93 unsupervised learning algorithm 81 unsupervised ML algorithm data, exporting from 106 data, training from 105, 106
V vanishing gradients 425 variance 364 vertex weights vector 282284 vintage XOR solution implementing, in Python with backpropagation 189191 implementing, in Python with FNN 189191 visible binary units 351 visual output of CNN layers analyzing 327 processing 323327
W webhook 393
X X dataset translating, line by line from English to French 145 XOR function 180, 181 XOR problem solving, examples 185189 [ 549 ]