Deep learning For mechanics

MODULE 0: REVIEW OF FUNDAMENTALS

[Week-1]Course modalities, what is AI, potential of DL in mechanics, and quick review of ML [lecture video][lecture notes][lecture notes (annotated)]

MODULE 1: FCNN AND DL BASICS

[Week-2]Fully connected neural network and backpropagation [lecture video][lecture notes][lecture notes (annotated)]

[L7]Fully connected neural network (continued) and backpropagation [lecture video][lecture notes][lecture notes (annotated)]

[L8]Backpropagation [lecture video][lecture notes][lecture noted (annotated)]

[L9]Backpropagation and optimization in DL - P1 [lecture video][lecture notes][lecture noted (annotated)]

[L10]Optimization in DL - P2 [lecture video][lecture notes][lecture noted (annotated)]

[L11]Optimization in DL - P3 [lecture video][lecture notes][lecture noted (annotated)]

[L12]Optimization in DL - P4 [lecture video][lecture notes][lecture notes (annotated)]

[L13]Optimization in DL - P5 [lecture video][lecture notes][lecture notes (annotated)]

[L14]Generalization - P1 [lecture video][lecture notes][lecture notes (annotated)]

[L15]Generalization - P2 [lecture video][lecture notes][lecture notes (annotated)]

[L16]Generalization - P3 [lecture video][lecture notes][lecture notes (annotated)]

MODULE 2: CNN (IMAGE LEARNING)

[L17]Convolutional neural network (CNN) - part 1 [lecture video][lecture notes][lecture notes (annotated)]

[L18]Convolutional neural network (CNN) - part 2 [lecture video][lecture notes][lecture notes (annotated)]

[L19]Convolutional neural network (CNN) - part 3 [lecture video][lecture notes][lecture notes (annotated)]

[L20]Convolutional neural network (CNN) - part 4 [lecture video][lecture notes][lecture notes (annotated)]

[L21]Convolutional neural network (CNN) - part 5 [lecture video][lecture notes][lecture notes (annotated)]

[L22]Convolutional neural network (CNN) - part 6 [lecture video][lecture notes][lecture notes (annotated)]

MODULE 3: SEQUENCE LEARNING

[L23]Sequence learning - part 1 [lecture video][lecture notes][lecture notes (annotated)]

[L24]Sequence learning - part 2 [lecture video][lecture notes][lecture notes (annotated)]

[L25]Sequence learning - part 3 [lecture video][lecture notes][lecture notes (annotated)]

[L26]Sequence learning - part 4 [lecture video][lecture notes][lecture notes (annotated)]

[L27]Sequence learning - part 5 [lecture video][lecture notes][lecture notes (annotated)]

[L28]Sequence learning - part 6 [lecture video][lecture notes][lecture notes (annotated)]

[L29]Sequence learning - part 7 [lecture video][lecture notes][lecture notes (annotated)]

MODULE 4: GENERATIVE DEEP LEARNING

[L30]Invertible Neural Network- part 1 [lecture video][lecture notes][lecture notes (annotated)]

[L31]Invertible Neural Network - part 2 [lecture video][lecture notes][lecture notes (annotated)]

[L32]Invertible Neural Network - part 3 [lecture video][lecture notes][lecture notes (annotated)]

[L33] Diffusion models - part 1 [lecture video][lecture notes][lecture notes (annotated)]

[L34] Diffusion models - part 2 [lecture video][lecture notes][lecture notes (annotated)]

MODULE 5: PHYSICS-INFORMED LEARNING

[L35] Physics-informed deep learning - Part 1 [lecture video][lecture notes][lecture notes (annotated)]

[L36] Physics-informed deep learning - Part 2 [lecture video][lecture notes]

MODULE 6: OPERATOR LEARNING

[L37] Operator learning - part 1 [lecture video][lecture notes][lecture notes (annotated)]

[L38] Operator learning - part 2 [lecture video][lecture notes][lecture notes (annotated)]

[L39] Operator learning - part 3 [lecture video][lecture notes][lecture notes (annotated)]

[L40] Operator learning - part 4 [lecture video][lecture notes]

CONCLUSION

[L41] The way forward [lecture video][lecture notes][lecture notes (annotated)]

MISCELLANEOUS MODULE (SELF STUDY/RECORDED VIDEO)

[M1] Neural ODE [lecture video][lecture notes]

[M2] Liquid Neural Network [lecture video][lecture notes]

[M3] GAN [lecture video][lecture notes]

[M4] VAE [lecture video][lecture notes]

• A quick recap of linear algebra, probability, and numerical computations

• Introduction to machine learning including linear regression, classification, and single layer perceptron.

• Deep feed-forward network – learning XOR, Gradient based learning, Hidden units and network architecture, regularization, optimization in deep learning

• Convolutional neural network – the convolution operator, pooling, Variants of the Basic Convolution Function, structured outputs, The Neuroscientific Basis for Convolutional Networks

• Recurrent neural network, unfolding computational graph, bidirectional RNN, Deep Recurrent Networks, The Challenge of Long-Term Dependencies, The Long Short-Term Memory

• Some advanced topics in deep learning – Attention, Transformers, Flow-based models, diffusion-based models

• From data-driven to physics-informed deep learning, the need for physics informed deep learning, challenges in physics-informed deep learning

• Operator learning

• Concluding remarks. Challenges and way ahead

Lecture notes and references will be provided on the course web site. The following books are recommended:

• Goodfellow, Y. Bengio, A. Courville “Deep Learning”, Vol. 1. Cambridge: MIT press, 2016.

• Bengio, Y. “Learning deep architectures for AI”. Now Publishers Inc, 2009.

• Nielsen, M. A. “Neural networks and deep learning”. Vol. 2018. San Francisco, CA: Determination press, 2015.

• Zhang, A., Lipton, Z. C., Li, M. and Smola, A. J. “Dive into Deep Learning”.

• Murphy, K.P. “Machine learning: A Probabilistic Perspective”, MIT press, 2022.

• Homework-1 [QP]

• Homework-2 [QP]

• Homework-3 [QP]

• Homework-4 [QP]

• Practical-1 [QP]

• Practical-2 [QP]

• Practical-3 [QP]

• Practical-4 [QP]

• Practical-5 [QP]

• Practical-6 [QP]

• Practical-7 [QP]

• Practical-8 [QP]

• Introduction to Pytorch [slides][video(p1)][video p(2)][notebook-1][notebook-2]

• Neural Network on Pytorch [slides][video][notebook]

Credit: 4 Units (3-0-2)

Lectures and practicals/tutorial: TWF (10:00 - 11:00 AM) 

Instructor: Dr. Souvik Chakraborty

Teaching Assistants: Shailesh Garg, Tushar, and Navaneeth.

Course Objective: The objective of this course offered by the Department of Applied Mechanics is to introduce the concepts of Deep Learning (DL) algorithms to the students. The course will dive into the fundamental concepts of DL and its application in solving scientific and engineering problems. Data-driven and physics-informed deep learning algorithms will be covered in this course. Of particular interest are multi-layer perceptron, CNN, RNN, LSTM, Attention, Transformer, GAN, and VAE. The course will emphasize on the mathematical learning of these concepts along with applications. The course is particularly designed for PG, Ph.D., and senior UG students.

Intended audience: Senior UG, PG, and Ph.D. students