mlfortherestofus

Machine Learning (ML) for the Rest of Us (Lectures by Dr. Keshav Pingali)

This website hosts the materials for the ML for the Rest of Us course, presented by Dr. Keshav Pingali at BOOST’24 (Bologna Orthogonal Summer Term) in Bologna, Italy in August 2024.

Schedule

There are 5 lectures encompassing three lectures on mathematical background for machine learning (ML) and neural networks, and two lectures on reinforcement learning (RL).

Lectures

• Lecture 1: Mathematical Background
  • Multivariate calculus, Parameter optimization, Gradient descent
  • Slides
• Lecture 2: Neural Networks
  • Multilayer Perceptrons (MLPs) and Deep Neural Networks (DNNs)
  • Slides
• Lecture 3: Neural Networks, Continued
  • Convolutional Neural Networks (CNNs)
  • Recurrent Neural Networks (RNNs)
  • Slides
• Lecture 4: Model-Based Reinforcement Learning
  • Markov Decision Processes (MDPs)
  • Value and Policy Iterations
  • Slides
• Lecture 5: Sampling-Based Methods for Reinforcement Learning
  • Temporal Difference (TD) Learning: TD(0), TD(n), SARSA, Q-Learning, TD(λ)
  • Monte Carlo Methods
  • Slides: Lecture 4 slides, continued
• Optional - Lecture 6: Deep Reinforcement Learning
  • Function approximation
  • Policy gradient methods for policy optimization
  • Slides: Lecture 4 slides, continued

Demos

Slides on the demos are available here

• Lecture 1:

  • Optional: Multi-Layer Perceptron for Diabetes Prediction

    dataset

    code

• Lecture 2:

  • Multi-Layer Perceptron for Stair Classification

    code

  • Convolutional Neural Network for CIFAR-10 Classification

    code

• Lecture 3:

  • Recurrent Neural Network for Weather Prediction

    dataset

    code

• Lecture 4:

  • GridWorld with Value and Policy Iteration

    instructions

    code

• Lecture 5:

  • N-state Random Walk for Model-free Methods: MC, TD(N) and TD(λ)

    instructions

    code

Prerequisites

• Free Colab account to run the demos for Lecture 1, 2 and 3
• Any IDE (such as VS Code) to run Python for the demos for Lecture 4 and 5

Additional Resources

• Barto and Sutton’s Reinforcement Learning Textbook

  pdf

• David Silver’s Reinforcement Learning Course

  website

• DeepRL Bootcamp

  website

• Pieter Abbeel’s Foundations of DeepRL series

  playlist

• Github repo with tutorials on RL with code and demos from Tim Miller, University of Queensland

  website

Help

For any help running the code or for additional explanations on the materials through email or Zoom meetings, feel free to reach out to zsm@utexas.edu.