Artificial Neural Network Models
In this article from PythonGeeks, we are going to scrutinize one of the most important aspects of using ANNs, the models that are used in building the Artificial Neural Network. We will learn about the various models that ANNs use in order to replicate a biological neural network.
At the first stage, we will discuss two main Supervised Learning Models, namely, Multilayer Perceptron Network and then Radial Basis Function Network. In the next section, we will discuss one Unsupervised Learning Model- Kohonen Model. We will conclude this article by learning about the difference between the Multilayer Perceptron Model and Radial Basis Function Network. Let us look at these models in brief.
Artificial Neural Networks Models
1. Multilayer Perceptron Model
As we have discussed in the introduction, the Multilayer Perceptron Model is a Supervised Learning Model. This model works on the principle of a Feedforward artificial neural network. This model maps the set of inputs we feed to the algorithm to the corresponding set of suitable outputs. As we know, feedforward neural networks are unidirectional i.e., data can flow in one direction only.
As the name suggests, the model has multiple layers of artificial neurons (nodes) which are connected through a directed graph. Each layer of the nodes is connected to its consequent next layer. Each neuron of these layers forms a linear equation as that of in linear regression as below:
yi= w0+ w1*x1 +w2*x2 + ………. + wn*xn
This function is known as the transfer function in a neural network. We adjust the weights of the connections between the neurons in such a way that at some point they can attain a threshold value. As a consequence of this, the value of output at these nodes is either 1 or 0.
Multilayer Perceptron Network is a famous ANN model which is predominantly used for regression problems. Each node in the MLP apart from the input neuron consists of a nonlinear activation function. MLP makes use of the backpropagation technique to implement Supervised Learning. It proves to be highly reliable when we have to deal with supervised learning as well as research into computational neural science and parallel distributed processing. MLP is used primarily in problems like image recognition, machine translation, and speech recognition.
2. Radial Basis Function Network
Similar to the Multilayer Perceptron Network, Radial Basis Function Network is a Supervised Learning Model. It resembles the MLP in many ways. However, the key difference lies in the layering of the networks. Multilayer Perceptron Network has only a single linear separability because it is composed of only inputs and outputs layers with some hidden layers in MLP. Whereas, Radial Basis Function tends to find a nonlinear separation.
In order to derive a non-linearity separation, the network needs to have at least one hidden layer. Radial Basis Function does exactly this job, it transforms the signal that we provide as input into another form. We can then feed this transformed input to the network to attain Linear stability. Radial Basis Function is structurally similar to Multilayer Perceptron Network.
We structurally restrict the network to have exactly one hidden layer. This layer is known as the feature vector. Apart from this, its structure has input and output layers. RBFN tends to increase the dimension of the feature vector. The network attains this by calculating the value of each entity of the hidden layer for observation. Unlike MLP, RBFN does not consider the sum of the weighted values of units of the previous level.
In RBFN, the neurons of the hidden are virtually independent of each other. This leads to RBFN emerging as a faster learning model. The response surface of an entity of the hidden layer is a hypersphere. Radial Functions are used for problems dealing with regression or function approximation. The input layer of the RBFN has a strong connection with the hidden layer. The network uses Gaussian Distribution in order to achieve the desired results.
Now that you know about the Supervised Learning Models, let us move towards the Unsupervised Learning Model, The Kohonen Network.
3. Kohonen Network
Self-Organizing Map or Kohonen Map is a type of Artificial Neural Network which is trained using the Unsupervised Learning Algorithm. It trains the network by following a competitive learning algorithm. The main use of SOM is to perform clustering and mapping (dimensionality reduction). For simplicity in interpretation, the network tends to map multidimensional data onto lower-dimension. SOM is structurally quite basic having only a single input and a single output layer.
Let us understand the working of SOM in a little more detail. Consider that we have input data of size (m, n). Here, m is the count of examples for training while n is the count of features for each example. The network first initializes the weights of size (n, C). Here, C is the number of clusters. The network then iterates over the input and updates the winning vector (weight vector having the shortest distance) of each training example. The rule that updates the weight of the examples is
wij= wij (old) + α(t) * (xik – wij (old))
In the above example, α is the learning rate at time t, j is the winning vector, i indicates the ith feature of the training example and k is kth training example. After we train the network, we use the trained weights for clustering new examples. The network follows the below algorithm:
- Initialize the weights
- Iterate over 1 to N number of epochs
- Choose a training example
- Calculate the winning vector
- Update the winning vector according to the given equation
- Repeat the above steps for all training examples
- Cluster the test sample
Now that we are familiar with the basics of the models used in Artificial Neural Networks, let us look at the difference between the two main Supervised Learning Models, Multilayer Perceptron Networks and Radial Basis Function Networks.
Difference Between Multilayer Perceptron Network and Radial Basis Function Networks
In the field of neural networks, the approximation models that link the inputs and the outputs are known as “black box” models. They provide a very slight peek of their working algorithm. The user who tends to deploy the neural network model has to consider several assumptions before training the model.
For example, the user has to assume prior to modeling, the number of hidden layers in the network model and the number of entities present in each model. Thus, one may require a considerable amount of experience in order to determine the most accurate representation. Adding more complexity to it, if we do not specify the correct form of the backpropagation, then the neural model can turn out to be quite slow.
As we have seen in the above discussion, MLP and RBNF are similar in many ways. They are the most common examples of feedforward Artificial Neural Network. Structurally both the networks have a very similar architecture. The key difference however lies in the way in which the networks handle the hidden unit values from the preceding layer. While MLP uses an inner product, RBF uses Euclidean Distance. Both the models also differ when it comes to training the networks using customary methods.
Conclusion
At the end of the discussion, we hope that you are now fully aware of the various models of training Artificial Neural Networks. The arriving times will certainly see an exponential increase in the use of Artificial Neural Networks. The development of probabilistic modeling is possible by including ANN-like models for providing techniques that will incorporate both explanatory models as well as data-driven approaches.