This course is a part of "Deep Learning for NLP" Series. In this course, Iwill introduce basic deep learning concepts like multi-layered perceptrons, word embeddings and recurrent neural networks. These concepts form the base for good understanding of advanced deep learning models for Natural Language Processing.

The course consists of three sections.

In the first section, I will talk about Basic concepts in artificial neural networks like activation functions (like ramp, step, sigmoid, tanh, relu, leaky relu), integration functions, perceptron and back-propagation algorithms. I also talk about what is deep learning, how is it related to machine learning and artificial intelligence? Finally, I will talk about how to handle overfittting in neural network training using methods like regularization, early stopping and dropouts.

In the second section, I will talk about various kinds of word embedding methods. I will start with basic methods like Onehot encoding and Singular Value Decomposition (SVD). Next I will talk about the popular word2vec model including both the CBOW and Skipgram methods. Further, I will talk about multiple methods to make the softmax computation efficient. This will be followed by discussion on GloVe. As special word embedding topics I will cover Cross-lingual embeddings. Finally, I will also talk about sub-word embeddings like BPE (Byte Pair Encoding), wordPiece, SentencePiece which are popularly used for Transformer based models.

In the third session, I will start with general discussion on ngram models. Next I will briefly introduce the neural network language model (NNLM). Then we will spend quite some time understanding how RNNs work. We will also talk about RNN variants like BiRNNs, Deep BiRNNs. Then I will discuss the vanishing and exploding gradients problem. This will be followed by details of the LSTMs and GRUs architectures.