Analytics and Visualization of Big Data: An Introduction to Artificial Neural Networks

Introduction

Artificial Neural Network (ANN) is mathematical model inspired by the way biological nervous systems, such as the brain process information with large number of highly interconnected processing elements.

ANN process information in a similar way the human brain does. In the human brain, there are approximately 10 billion neurons, each of which is connected to about 10,000 other neurons which communicate through electro-chemical signals. The neurons are connected through junctions called synapses. Each neutron receives thousands of connections with other neutrons, constantly receiving incoming signals to reach the cell body. If the resulting sum of the signals surpasses a certain threshold, a response is sent through the axon. The ANN attempts to recreate the computational mirror of the biological neural network, although it is not comparable since the number and complexity of neutrons in a biological neural network is many times more than those in an artificial neutral network.

The ANN is composed of a large number of highly interconnected processing elements (neurons) working in parallel to solve a specific problem. ANNs learn by example, like people. They cannot be programmed to perform a specific task. The examples must be selected carefully otherwise useful time is wasted or even worse the network might be functioning incorrectly. ANNs are typically organized in layers. Layers are made up of a number of interconnected 'nodes' which contain an 'activation function'. Patterns are presented to the network via the 'input layer', which communicates to one or more 'hidden layers' where the actual processing is done via a system of weighted 'connections'. The hidden layers then link to an 'output layer' where the answer is output as shown in the graphic.

ANNs, with their remarkable ability to derive meaning from complicated or imprecise data, can be used to extract patterns and detect trends that are too complex to be noticed by either humans or other computer techniques. There are different ways in which information can be processed by a neuron, and different ways of connecting the neurons to one another. Different neural network structures can be constructed by using different processing elements and by the specific manner in which they are connected.

Different types of ANN

ANNs come in many different shapes and sizes. There are mainly two types of ANNs: Feed Forward Neural Networks (FFNNs) and Recurrent Neural Networks (RNNs). In FFNN there are no feedback loops. The flow of signals/information is only in the forward direction. FFNN may have a single layer of weights, where the inputs are directly connected to the outputs, or multiple layers with intervening sets of hidden units. In RNN there are feedback loops (essentially FFNN with output fed back to input). Different types of neural network architectures are briefly mentioned below:

Single-layer feed forward networks
Multi-layer feed forward networks
Recurrent neural networks
Lattice networks

Neural Network used frequently to solve four different types of the problems:

Classification
Prediction
Clustering
Pattern recognition

Learning

In general, an ANN structure is trained with known samples of data. As an example, if a particular pattern is to be recognized, then the ANN is first trained with the known pattern/information. Then the ANN is ready to recognize a similar pattern when it is presented to the network. There are three main learning paradigms:

Supervised Learning with a Teacher: The network is provided with a set of inputs and the appropriate outputs for those inputs.
Supervised Learning with Reinforcement: The network is provided with a evaluation of its output given the input and alters the weights to try to increase the reinforcement it receives.
Unsupervised Learning: The network receives no external feedback but has an internal criterion that it tries to fulfill given the inputs that it faces.

Application of ANNs

Signal processing: suppress line noise, with adaptive echo canceling, blind source separation
Control: e.g. backing up a truck: cab position, rear position, and match with the dock get converted to steering instructions. Manufacturing plants for controlling automated machines.
Robotics - navigation, vision recognition
Pattern recognition, i.e. recognizing handwritten characters, e.g. the current version of Apple's Newton uses a neural net
Medicine, i.e. storing medical records based on case information
Speech production: reading text aloud
Speech recognition
Vision: face recognition , edge detection, visual search engines
Business, e.g.. rules for mortgage decisions are extracted from past decisions made by experienced evaluators, resulting in a network that has a high level of agreement with human experts.
Financial Applications: time series analysis, stock market prediction
Data Compression: speech signal, image, e.g. faces
Game Playing: backgammon, chess, go,