A recurrent neural network (RNN) introduces recurrence to the [[neural network]] with feedback loops. RNNs allow for varying input lengths. RNNs are useful for sequential data like [[time-series]] data. However, vanilla RNN suffers from vanishing or exploding gradients. Vanilla RNNs have been improved by the [[LSTM]] and [[GRU]] architectures. ## bidirectional neural network Bidirectional recurrent neural networks (Bi-RNN) improve upon vanilla recurrent neural networks for long sequences. A forward RNN processes the sequence from start to end, and a backward RNN processes the sequence from end to start. The final embedding is a concatenation of the two. For sequence classification, only the last hidden states are concatenated; for sequence labeling all hidden states are concatenated. ## hierarchical recurrent neural network The hierarchical recurrent neural network improve upon vanilla recurrent neural networks and are used for part of speech tagging. > [!Tip]- Additional Resources > - [Recurrent Neural Networks (RNNs), Clearly Explained!!!](https://youtu.be/AsNTP8Kwu80?si=ZUJaZ-Aw8EH2YhSr) | StatQuest with Josh Starmer