Understanding Neural Networks

Neural Networks is one of the most powerful and widely used algorithms in machine learning. A neural network works similarly to the human brain’s neural network.

Neural networks are set layers of highly interconnected processing elements (neurons) that make a series of transformations on the data to generate its own understanding of it(what we commonly call features).

  • Input layer: It is used to pass in our input(an image, text or any suitable type of data for NN).
  • Hidden Layer: These are the layers in between the input and output layers. These layers are responsible for learning the mapping between input and output.
  • Output Layer: This layer is responsible for giving us the output of the Neural Network given our inputs.
  1. Information is fed into the input layer which transfers it to the hidden layer
  2. The interconnections between the two layers assign weights to each input randomly
  3. A bias added to every input after weights are multiplied with them individually
  4. The weighted sum is transferred to the activation function
  5. The activation function determines which nodes it should fire for feature extraction
  6. The model applies an application function to the output layer to deliver the output
  7. Weights are adjusted, and the output is back-propagated to minimize error

The model uses a cost function to reduce the error rate. You will have to change the weights with different training models. The model compares the output with the original result. It repeats the process to improve accuracy.

The learning process works forward and backward, which is calles Forward and Backward propagation.

Forward Propagation: The input data is fed in the forward direction through the network. Each hidden layer accepts the input data, processes it as per the activation function and passes to the successive layer.

Backward Propagation: It is the method of fine-tuning the weights of a neural net based on the error rate obtained in the previous iteration.

We use activation functions to propagate the output of a neutron forward. This output is received by the neurons of the next layer to which this neuron is connected. As we have said, the activation function serves to introduce non-linearity in the modeling capabilities of the network. Here are some of the Activation functions.

Linear: A straight line function where activation is proportional to input (which is the weighted sum from neuron).

Sigmoid: A sigmoid function is a bounded, differentiable, real function that is defined for all real input values and has a non-negative derivative at each point and exactly one inflection point.

Tanh: The tanh represents the relationship between the hyperbolic sine and the hyperbolic cosine: tanh(x)=sinh(x)/cosh(x).

ReLU: The rectified linear activation function or ReLU for short is a piecewise linear function that will output the input directly if it is positive, otherwise, it will output zero. It has become the default activation function for many types of neural networks because a model that uses it is easier to train and often achieves better performance.

Below are the three main types of neural networks.

Artificial neural networks, or ANNs, are like the neural networks in the image above, which is composed of a collection of connected nodes that takes an input or a set of inputs and returns an output. This is the most fundamental type of neural network.

A convolutional neural network (CNN) is a type of neural network that uses a mathematical operation called convolution. Convolution is mathematical operation on two functions that produces a third function expressing how the shape of one is modified by the other. Thus, CNN's use convolution instead of general matrix multiplication in at least one of their layers.

Recurrent neural networks (RNNs) are a type of ANNs where connections between the nodes form a digraph along a temporal sequence, allowing them to use their internal memory to process variable-length sequences of inputs. RNNs are great at text recognition or audio recognition.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store