Introduction to the Project: Pen and Paper Exercises in Machine Learning
Pen and Paper Exercises in Machine Learning is an educational project that focuses on providing a wide variety of hands-on exercises related to key concepts in machine learning. Curated by Michael U. Gutmann from the University of Edinburgh, this collection aims to enhance learning through traditional pen and paper exercises. Each exercise is paired with a comprehensive solution to reinforce understanding and facilitate self-paced learning.
Topics Covered
The exercises in this collection span several fundamental topics in machine learning:
- Linear Algebra: The foundation of many machine learning algorithms, including vector and matrix operations.
- Optimisation: Techniques to find the best solution, often relating to minimizing error or maximizing efficiency in models.
- Directed Graphical Models: Understanding probabilistic models that capture relationships with directed edges.
- Undirected Graphical Models: These involve network models with undirected edges, focusing on how data points relate.
- Expressive Power of Graphical Models: Evaluating the capability of models to represent complex relationships.
- Factor Graphs and Message Passing: In-depth exploration of graph-based models and algorithms.
- Inference for Hidden Markov Models: Techniques for making predictions in systems that follow the Markov process.
- Model-Based Learning (Including ICA and Unnormalised Models): Insights into constructing models to understand data.
- Sampling and Monte Carlo Integration: Computational techniques for estimating and approximating functions.
- Variational Inference: Estimating the probabilities in large, complex datasets.
Access and Compilation
The compiled exercises can be accessed as a PDF on arXiv. For Linux users, the collection can be compiled using the command make. Solutions to each exercise are included by default, but users can compile the document without solutions by modifying main.tex accordingly.
Contribution and Collaboration
The project encourages community involvement, allowing users to report mistakes or suggest improvements via GitHub issues. Community contributions are welcomed, particularly for adding exercises that include detailed solutions. Interested contributors are encouraged to contact Michael Gutmann for further discussion.
Acknowledgements and Resources
This project benefits from a variety of resources:
- TikZ settings and macros generously provided by David Barber, used partly in his book on Bayesian reasoning and machine learning.
- The
ethuebungpackage by Philippe Faist, tailored to support multiple chapter compilations and exercise inclusion in a table of contents. - Parts of the exercises have been developed for courses like Unsupervised Machine Learning at the University of Helsinki and Probabilistic Modelling and Reasoning at the University of Edinburgh.
Overall, Pen and Paper Exercises in Machine Learning serves as an invaluable toolkit for students and educators looking to deepen their understanding of machine learning through practical exercises and detailed discussions.