ABSTRACT. Technically, the brain presents the "end-to-end" system (ETES), which monitors the state of the external world using numerous receptors and acts back to it via multiple effectors to impose the world's states, comforting the brain's (the host organism's) needs/goals. In ETES, the concrete aspects of work of particular parts and elements of the system do not matter per se, as long as the system's needs are reasonably satisfied. Taking into consideration the "kluge principles" of the brain construction (Marcus, 2009) it seems quite improbable that inside brain there are well mathematically-treatable schemes. Most probably, the processes there are well "entangled" (in not quantum mechanical way). Examples of real language words entanglement are recently found in deep learning based computer translation systems. The likely aspects we should be ready to see in brain. Along with that, the simple examples of data formats are also prone to be found in natural neural systems. Especially, if they have precious qualities and are easy to get in simple neural networks (neural models). Examples of such data formats will be presented in the talk. The work was supported by the Russian Foundation of Basic Research grant # 16-07-01059.

ABSTRACT. Over the last decade the deep neural networks are the powerful tool in the domain of machine learning. In the general case a deep neural network consists of multiple layers of neural units and can accomplish a deep hierarchical representation of their input data. The first layer extracts low-level features; the second layer detects higher level features, and as a result the deep neural network performs deep non-linear transformation of input data into more abstract level of representation. The important problem is training of deep neural network, because learning of such a network is much complicated compared to shallow neural networks. This is due to the vanishing gradient problem, poor local minima and unstable gradient problem. Therefore a lot of deep learning techniques were developed that permit us to overcome some limitations of conventional training approaches. Since 2006, the unsuper-vised pre-training of deep neural network was proposed. In this case the training of deep neural networks consists of two stages: pre-training of the neural network using a greedy layer-wise approach and fine-tuning all the parameters of the neural network using back-propagation or the wake sleep algorithm. The pre-training of deep neural network is based on either the restricted Boltzmann machine (RBM) or auto-encoder approach. At present, the stochastic gradient descent (SGD) with rectified linear unit (ReLU) activation function is used for training of deep neural networks in supervised manner. There exist different types of deep neural networks, namely deep belief neural networks, deep convolutional neural networks, deep recurrent neural networks, deep autoencoder and so on. Deep neural networks currently provide the best performance to many problems in images, video, speech recognition, and natural language processing. This lecture provides an overview of deep neural networks and deep learning. Different deep learning techniques, including well-known and new approaches are discussed. So, for instance, a new technique called “REBA” for the training of deep neural networks, based on the restricted Boltzmann machine is demonstrated. In contrast to Hinton’s traditional approach to the training of restricted Boltzmann machines, which is based on linear training rules, the proposed technique is founded on nonlinear training rules. Experiments demonstrate a high potential of deep neural networks in real applications.

ABSTRACT. At present, the cortical columns attract an attention of the scientists and developers in various areas of neuroscience and neuroengineering. In the present report, the results obtained in experimental and model studies have been considered The problems of the columnar organization that are not solved up to now, have been determined, namely: (i) what are appropriate criteria for the identification of functional columns; (ii) may the cortical functional columns to be considered as relatively discrete units; (iii) what are the principles of intrinsic functional structure of the columns; (iv) what are cooperative functions of a column as compare with single neuron level; (v) what are the dynamic operations within the columns. An approach to the study of the internal functional structure of cortical columns has been considered.

ABSTRACT. How the brain forms a representation of space, called a "cognitive map", for use in navigation and memory, has been an intensive topic of investigation for several decades. It began with the discovery by John O'Keefe of place cells, which fire at high rates when an animal ventures into particular parts of the environment. Subsequent decades saw the discovery of additional types of spatially encoding neurons including head direction cells and grid cells. Study of these neurons has been very informative in understanding how rats, and we think other mammals including humans, form cognitive maps. More recently attention is turning to how these neurons function in the complex real world, which has multiple compartments and in which animals are not always constrained to a horizontal surface. This talk will review the spatially encoding neurons and discuss recent research into the encoding of complex space.