# Probabilistic Machine learning For mechanics

## Lecture notes

Introduction to ML [lecture video][lecture notes][lecture notes annotated]

Review of Bayesian Statistics - Part 1 [lecture video][lecture notes][lecture notes annotated]

Review of Bayesian Statistics - Part 2 [lecture video - P1][lecture video - P2][lecture notes][lecture notes annotated]

Review of Bayesian Statistics - Part 3 [lecture video - P1][lecture video - P2][lecture notes][lecture notes annotated]

Review of Bayesian Statistics (concl) and Prior modeling [lecture video][lecture notes][lecture notes annotated]

Conjugate prior and Exponential family [lecture video][lecture notes][lecture notes annotated]

Review of Linear Regression, MLE, and MAP [lecture video][lecture notes][lecture notes annotated]

Bayesian linear regression - Part 1 [lecture video][lecture video (correction)][lecture notes][lecture notes annotated]

Bayesian linear regression - Part 2 [lecture video][lecture notes][lecture notes (annotated)]

Bayesian linear regression (contd), Bayesian inference using sampling methods - Part 1 [lecture video][lecture notes][lecture notes (annotated)]

Bayesian inference using sampling methods - Part 2 [lecture video][lecture notes][lecture notes (annotated)]

Bayesian inference using sampling methods - Part 3 [lecture video][lecture notes][lecture notes (annotated)]

Bayesian inference using sampling methods - Part 4 [lecture video][lecture notes][lecture notes (annotated)]

Bayesian inference using sampling methods - Part 5 [lecture video][lecture notes][lecture notes (annotated)]

Bayesian inference using sampling methods - Part 6 [lecture video][lecture notes][lecture notes (annotated)]

Approximate methods for Bayesian inference - Part 1 [lecture video][lecture notes][lecture notes (annotated)]

Approximate methods for Bayesian inference - Part 2 [lecture video][lecture notes][lecture notes (annotated)]

Robust and sparse linear regression and RVM[lecture video][lecture notes]

Gaussian process - Part 1 [lecture video][lecture notes]

Gaussian process - Part 2 [lecture video][lecture notes][lecture notes (annotated)]

Sparse Gaussian Process - Part 1 [lecture video (Part 1)][lecture video (Part 2)][lecture notes][lecture notes (annotated)]

Sparse Gaussian Process - Part 2 [lecture video][lecture notes]

Factor analysis, Probabilistic PCA, Duel Probabilistic PCA, and GP-LVM [lecture video][lecture notes][lecture notes (marked)]

## Syllabus

Introduction to Statistical Computing and Probability and Statistics

Sum and Product Rules, Conditional Probability, Independence, PDF and CDF, Bernoulli, Categorical and Multinomial Distributions, Poisson, Student’s T, Laplace, Gamma, Beta and Pareto distribution.

Generative Models; Bayesian concept learning, Likelihood, Prior, Posterior, Posterior predictive distribution, Plug-in Approximation

Bayesian Model Selection (continued) and Prior Models, Hierarchical Bayes, Empirical Bayes

Bayesian linear regression

Introduction to Monte Carlo Methods, Sampling from Discrete and Continuum Distributions, Reverse Sampling, Transformation Methods, Composition Methods, Accept-Reject Methods, Stratified/Systematic Sampling

Importance sampling, Gibbs sampling, MCMC, Metropolis Hasting algorithm

Sequential importance sampling, Sequential Monte Carlo

Latent variable model, probabilistic PCA, Expectation maximization

Gaussian process and variational inference

Some advanced topics in probabilistic ML: Bayesian neural network, GAN, VAE, Flow-based model, Diffusion model

## References

Lecture notes and references will be provided on the course web site. The following books are recommended:

Bishop, C.M. Pattern recognition and Machine learning, Springer, 2007.

Murphy, K.P. “Machine learning: A Probabilistic Perspective”, MIT press, 2022.

Rasmussen, Carl Edward. Gaussian processes in machine learning, In Summer school on machine learning, pp. 63-71. Springer, Berlin, Heidelberg, 2003

## Homework

Homework-1: Review of probability and statistics [homework]

Homework-2: MLE MAP, Bayesian statistics, Linear regression [homework]

Homework-3 and 4 (combined): Bayesian linear regression, Sampling method [homework]

Homework-5: Approximate methods for Bayesian inference [homework]

Homework-6: Sparse and robust linear regression, relevance vector machine [homework]

Homework 7: Gaussian processes [homework]

## Practical

Practical-1: Introduction to statistical computing [QP][material]

Practical-2: MLE, MAP, linear regression [QP]

Practical-3: Constitutive modeling using ML [QP]

Practical-4: Reliability analysis and UQ using ML [QP][material]

Practical-5 and 6: Equation discovery and parameter estimation using ML [QP][material]

Practical-7: Force identification in dynamical systems [QP][material-1][material-2]

## Projects

Each student will have to complete a term project as part of this course

## Course info

Credit: 4 Units (3-0-2)

Timing: Lecture - Monday and Thursday (9:30 am - 11:00 am), Practical - Thursday (3:30 pm - 5:30 pm)

Venue: IV LT 4

Instructor: Dr. Souvik Chakraborty

Teaching Assistants: Navaneeth. N, Shailesh Garg, Tapas Tripura

Course Objective: In this course, the students will be introduced to the fundamentals of probabilistic machine learning and its application in computational mechanics. Students are expected to learn different probabilistic machine learning algorithms and applications in solving mechanics problems The course will emphasize on the mathematical learning of these concepts along with applications. The course is particularly designed for PG, Ph.D., and senior UG students.

Intended audience: Senior UG, PG, and Ph.D. students