ML-from-scratch-seminar: An Insightful Exploration
The "ML-from-scratch-seminar" is an innovative initiative hosted by the Department of Neurobiology at Harvard Medical School. This seminar brings together a group of enthusiastic graduate students and postdocs who are passionate about machine learning (ML). Their mission? To create simple, yet effective, Python implementations of well-known ML models. The primary aim is to illustrate the learning dynamics, strengths, and weaknesses of different algorithms, all while maintaining computational simplicity.
Seminar Structure
The seminar is thoughtfully designed to unfold over two evenings per topic:
-
Evening 1: Theoretical Insights
The first evening is all about theory. Participants are introduced to the fundamental concepts behind each machine learning model. These sessions are concise, ideally taking less than an hour, with explanations aided by whiteboards or slides. The emphasis is on understanding how and why each algorithm works.
-
Evening 2: Hands-on Coding
The second evening shifts focus to practical application. Participants dive into coding, crafting key elements of the model within a Python 3 notebook. The session is structured to ensure participants not only grasp the workings of the model but also manage to code essential parts of it within a few hours. The idea of "ML from scratch" here is to demystify each computational step without necessarily creating a universal tool. Typically, training times are optimized to remain under five minutes on a standard desktop, without relying on GPUs.
Role of Session Chairs
Each session is led by one or two session chairs, who play a critical role in steering the learning experience:
- They kick off the session with a short theoretical introduction.
- Design and oversee a coding task that synthesizes theoretical insights with practical experience.
- Offer suggested readings and serve as guides for any theory or implementation queries.
- Communicate any non-standard software requirements beforehand.
Session chairs are also responsible for uploading all materials—be it notes, readings, or reference code—to the seminar's GitHub repository, enriching the learning resource pool.
Participation and Organization
While the seminar encourages participation without the need to lead a session, regular attendees are often inspired to host future sessions. John Vastola, the seminar organizer, facilitates scheduling and provides guidance to session chairs on theory and coding strategies. He is also responsible for maintaining the platform for sharing documents and code, along with ensuring food is part of the experience.
Sessions and Suggested Topics
The seminar has explored a diverse array of topics over the years, from Variational Auto-Encoders and Hidden Markov Models to Reinforcement Learning and Transformers. Each session is an opportunity to delve into the intricacies of machine learning through both theoretical and practical lenses.
Looking to the future, potential topics for exploration might include Switching Linear Dynamical Systems, LSTM/GRU, ODE nets, Graph Neural Networks, Meta-learning, among others. These topics promise to continue the tradition of deep, insightful inquiry combined with practical implementation skills.
The "ML-from-scratch-seminar" stands as a unique platform blending theory with practice, making complex machine learning concepts accessible and engaging for all participants.