Deep learning

[L1] Course modalities and DL and AI in modern times [Slides][Video]

[L2] Quick review of ML - Part 1 [Notes][Slides][Video]

[L3] Quick review of ML - Part 2, Perceptron and Fully connected network (shallow) [Notes][Slides][Video]

[L4] Fully connected network and backpropagation (continued) [Notes][Slides][Video]

[L5] Backpropagation

[L6] Backpropagation and optimization in DL - P1

[L7] Backpropagation and optimization in DL - P2

[L8] Backpropagation and optimization in DL - P3

[L9] Generalization - P1

[L10] Generalization - P2

[L11] Convolutional neural network (CNN) - part 1

[L12] Convolutional neural network (CNN) - part 2

[L13] Convolutional neural network (CNN) - part 3

[L14] Convolutional neural network (CNN) - part 4

[L15] Sequence learning - part 1

[L16] Sequence learning - part 2

[L17] Sequence learning - part 3

[L18] Sequence learning - part 4

[L19] Sequence learning - part 5

[L20] Building generative language model - part 1

[L21] Building generative language model - part 2

[L22] Building generative language model - part 3

[L23] Graph machine learning - part 1

[L24] Graph machine learning - part 2

[L25] Graph machine learning - part 3

[L26] Reinforcement and Deep reinforcement learning - part 1

[L27] Reinforcement and Deep reinforcement learning - part 2

[L28] Conclusion and way forward.

• A quick recap of linear algebra, probability, and numerical computations

• Introduction to machine learning including linear regression, classification, and single layer perceptron.

• Deep feed-forward network – learning XOR, Gradient based learning, Hidden units and network architecture, regularization, optimization in deep learning

• Convolutional neural network – the convolution operator, pooling, Variants of the Basic Convolution Function, structured outputs, The Neuroscientific Basis for Convolutional Networks

• Recurrent neural network, unfolding computational graph, bidirectional RNN, Deep Recurrent Networks, The Challenge of Long-Term Dependencies, The Long Short-Term Memory

• Attention, Transformers, Generative Language Model

• Graph Machine Learning

• Deep Reinforcmeent Learning

• Concluding remarks. Challenges and way ahead

Lecture notes and references will be provided on the course web site. The following books are recommended:

• Goodfellow, Y. Bengio, A. Courville “Deep Learning”, Vol. 1. Cambridge: MIT press, 2016.

• Bengio, Y. “Learning deep architectures for AI”. Now Publishers Inc, 2009.

• Nielsen, M. A. “Neural networks and deep learning”. Vol. 2018. San Francisco, CA: Determination press, 2015.

• Zhang, A., Lipton, Z. C., Li, M. and Smola, A. J. “Dive into Deep Learning”.

• Murphy, K.P. “Machine learning: A Probabilistic Perspective”, MIT press, 2022.

• Homework-1 [QP]

• Homework-2 [QP]

• Homework-3 [QP]

• Practical-1 [QP]

• Practical-2 [QP]

• Practical-3 [QP]

• Practical-4 [QP]

• Practical-5 [QP]

• Practical-6 [QP]

Credit: 4 Units (3-0-2)

Lectures and practicals/tutorial: MTh (9:30 - 11:00 AM) , Practical: TBD

Instructor: Dr. Souvik Chakraborty

Teaching Assistants: Head TAs: Sawan Kumar, Subhankar Sarkar, and Rakesh Mandal.

Course Objective: The objective of this course offered by the Department of Applied Mechanics is to introduce the concepts of Deep Learning (DL) algorithms to the students. The course will dive into the fundamental concepts of DL and its application in solving scientific and engineering problems. Of particular interest are multi-layer perceptron, CNN, RNN, LSTM, Attention, Transformer, and Graph Learning. The course will emphasize on the mathematical learning of these concepts along with applications. The course is particularly designed for PG, Ph.D., and senior UG students.

Intended audience: Senior UG, PG, and Ph.D. students