Types of RNN
The main reason that the recurrent nets are more exciting is that they allow us to operate over sequences of vectors: Sequence in the input, the output, or in the most general case, both. A few examples may this more concrete:
Each rectangle in the above image represents vectors, and arrows represent functions. Input vectors are Red, output vectors are blue, and green holds RNN’s state.
One-to-one:
This is also called Plain Neural networks. It deals with a fixed size of the input to the fixed size of output, where they are independent of previous information/output.
Example: Image classification.
One-to-Many:
It deals with a fixed size of information as input that gives a sequence of data as output.
Example: Image Captioning takes the image as input and outputs a sentence of words.
Many-to-One:
It takes a sequence of information as input and outputs a fixed size of the output.
Example: sentiment analysis where any sentence is classified as expressing the positive or negative sentiment.
Many-to-Many:
It takes a Sequence of information as input and processes the recurrently outputs as a Sequence of data.
Example: Machine Translation, where the RNN reads any sentence in English and then outputs the sentence in French.
Bidirectional Many-to-Many:
Synced sequence input and output. Notice that in every case are no pre-specified constraints on the lengths sequences because the recurrent transformation (green) is fixed and can be applied as many times as we like.
Example: Video classification where we wish to label every frame of the video.
Advantages of Recurrent Neural Network
- RNN can model a sequence of data so that each sample can be assumed to be dependent on previous ones.
- A recurrent neural network is even used with convolutional layers to extend the active pixel neighborhood.
Disadvantages of Recurrent Neural Network
- Gradient vanishing and exploding problems.
- Training an RNN is a complicated task.
- It could not process very long sequences if it were using tanh or relu like an activation function.