Probabilistic Machine learning For mechanics

1. Introduction to ML [lecture video][lecture notes][lecture notes annotated]

2. Review of Bayesian Statistics - Part 1 [lecture video][lecture notes][lecture notes annotated]

3. Review of Bayesian Statistics - Part 2 [lecture video - P1][lecture video - P2][lecture notes][lecture notes annotated]

4. Review of Bayesian Statistics - Part 3 [lecture video - P1][lecture video - P2][lecture notes][lecture notes annotated]

5. Review of Bayesian Statistics (concl) and Prior modeling [lecture video][lecture notes][lecture notes annotated]

6. Conjugate prior and Exponential family [lecture video][lecture notes][lecture notes annotated]

7. Review of Linear Regression, MLE, and MAP [lecture video][lecture notes][lecture notes annotated]

8. Bayesian linear regression - Part 1 [lecture video][lecture video (correction)][lecture notes][lecture notes annotated] 

9. Bayesian linear regression - Part 2 [lecture video][lecture notes][lecture notes (annotated)]

10. Bayesian linear regression (contd), Bayesian inference using sampling methods - Part 1 [lecture video][lecture notes][lecture notes (annotated)]

11. Bayesian inference using sampling methods - Part 2 [lecture video][lecture notes][lecture notes (annotated)]

12. Bayesian inference using sampling methods - Part 3 [lecture video][lecture notes][lecture notes (annotated)]

13. Bayesian inference using sampling methods - Part 4 [lecture video][lecture notes][lecture notes (annotated)]

14. Bayesian inference using sampling methods - Part 5 [lecture video][lecture notes][lecture notes (annotated)]

15. Bayesian inference using sampling methods - Part 6 [lecture video][lecture notes][lecture notes (annotated)]

16. Approximate methods for Bayesian inference - Part 1 [lecture video][lecture notes][lecture notes (annotated)]

17. Approximate methods for Bayesian inference - Part 2 [lecture video][lecture notes][lecture notes (annotated)]

18. Robust and sparse linear regression and RVM[lecture video][lecture notes]

19. Gaussian process - Part 1 [lecture video][lecture notes]

20. Gaussian process - Part 2 [lecture video][lecture notes][lecture notes (annotated)]

21. Sparse Gaussian Process - Part 1 [lecture video (Part 1)][lecture video (Part 2)][lecture notes][lecture notes (annotated)]

22. Sparse Gaussian Process - Part 2 [lecture video][lecture notes]

23. Factor analysis, Probabilistic PCA, Duel Probabilistic PCA, and GP-LVM [lecture video][lecture notes][lecture notes (marked)]

• Introduction to Statistical Computing and Probability and Statistics

• Sum and Product Rules, Conditional Probability, Independence, PDF and CDF, Bernoulli, Categorical and Multinomial Distributions, Poisson, Student’s T, Laplace, Gamma, Beta and Pareto distribution.

• Generative Models; Bayesian concept learning, Likelihood, Prior, Posterior, Posterior predictive distribution, Plug-in Approximation

• Bayesian Model Selection (continued) and Prior Models, Hierarchical Bayes, Empirical Bayes

• Bayesian linear regression

• Introduction to Monte Carlo Methods, Sampling from Discrete and Continuum Distributions, Reverse Sampling, Transformation Methods, Composition Methods, Accept-Reject Methods, Stratified/Systematic Sampling

• Importance sampling, Gibbs sampling, MCMC, Metropolis Hasting algorithm 

• Sequential importance sampling, Sequential Monte Carlo

• Latent variable model, probabilistic PCA, Expectation maximization

• Gaussian process and variational inference

• Some advanced topics in probabilistic ML: Bayesian neural network, GAN, VAE, Flow-based model, Diffusion model

Lecture notes and references will be provided on the course web site. The following books are recommended:

• Bishop, C.M. Pattern recognition and Machine learning, Springer, 2007.

• Murphy, K.P. “Machine learning: A Probabilistic Perspective”, MIT press, 2022.

• Rasmussen, Carl Edward. Gaussian processes in machine learning, In Summer school on machine learning, pp. 63-71. Springer, Berlin, Heidelberg, 2003

• Homework-1: Review of probability and statistics [homework]

• Homework-2: MLE MAP, Bayesian statistics, Linear regression [homework]

• Homework-3 and 4 (combined): Bayesian linear regression, Sampling method [homework]

• Homework-5: Approximate methods for Bayesian inference [homework]

• Homework-6: Sparse and robust linear regression, relevance vector machine [homework]

• Homework 7: Gaussian processes [homework]

• Practical-1: Introduction to statistical computing [QP][material]

• Practical-2: MLE, MAP, linear regression [QP]

• Practical-3: Constitutive modeling using ML [QP]

• Practical-4: Reliability analysis and UQ using ML [QP][material]

• Practical-5 and 6: Equation discovery and parameter estimation using ML [QP][material]

• Practical-7: Force identification in dynamical systems [QP][material-1][material-2]

• Each student will have to complete a term project as part of this course

Credit: 4 Units (3-0-2)

Timing: Lecture - Monday and Thursday (9:30 am - 11:00 am), Practical - Thursday (3:30 pm - 5:30 pm)

Venue: IV LT 4

Instructor: Dr. Souvik Chakraborty 

Teaching Assistants: Navaneeth. N, Shailesh Garg, Tapas Tripura

Course Objective: In this course, the students will be introduced to the fundamentals of probabilistic machine learning and its application in computational mechanics. Students are expected to learn different probabilistic machine learning algorithms and applications in solving mechanics problems The course will emphasize on the mathematical learning of these concepts along with applications. The course is particularly designed for PG, Ph.D., and senior UG students.

Intended audience: Senior UG, PG, and Ph.D. students