Madhavan Mukund

Data Mining and Machine Learning

Jan-Apr, 2025

Administrative details

• Teaching assistants: Alok Dhar Dubey, Ankan Kar, Rohit Roy

• Evaluation:

  • Assignments 30-40%, quizzes and midsemester exam 20-30%, final exam 40%

  • Copying is fatal

• Course outline (tentative)

  • Supervised learning: Association rules, regression, decision trees, naive Bayes, SVM, classifier evaluation, expectation maximization, ensemble classifiers.

  • Unsupervised learning: Clustering, outlier detection, dimensionality reduction.

  • Text mining: Basic ideas from information retrieval, TF/IDF model, Page Rank, HITS

  • Other topics (if time permits): Probabilistic graphical models, Bayesian networks, Markov models, neural networks, ranking and social choice, …

• Text and reference books:

  • Web Data Mining by Bing Liu, 2nd edition, Springer (2011).

  • Foundations of Data Science by Avrim Blum, John Hopcroft and Ravi Kannan

  • Machine Learning by Tom Mitchell.

  • C4.5: Programs for Machine Learning by Ross Quinlan.

  • Artificial Intelligence: A Modern Approach by Stuart J Russell and Peter Norvig, 3rd edition, Pearson (2016).

  • Hands-On Machine Learning with Scikit-Learn, Keras and Tensorflow by Aurélien Géron, 3rd edition, O'Reilly (2022)

    Github repository with code from the book by Aurélien Géron

  • Reinforcement Learning: An Introduction, by Richard S. Sutton and Andrew G. Barto, MIT Press, 2nd ed (2018)

Assignments

• Practice Assignment 1, 3 Feb 2025

• Assignment 1, 8 Feb 2025, due 21 Feb 23 Feb 2025

• Assignment 2, 17 Feb 2025, due 2 Mar 2025

Lecture summary

• Lecture 1: 7 Jan 2025
  (Class Notes (pdf))

  Introduction to supervised and unsupervised learning

• Lecture 2: 16 Jan 2025
  (Class Notes (pdf))

  Market-basket analysis, frequent itemsets, Apriori algorithm

  • Liu, Chapter 2.1, 2.2, 2.2.1

• Lecture 3: 21 Jan 2025
  (Class Notes (pdf), Lecture Slides (pdf))

  Association rules, class association rules

  • Liu, Chapter 2.2.1, 2.2.2

  Supervised learning, decision trees, impurity

  • Liu, Chapter 3.1, 3.2, 3.2.1
  • Mitchell, Chapter 3
  • Quinlan, Chapters 1,2
  • Supplementary reading (optional):
    Laurent Hyafil and Ronald Rivest: Constructing optimal binary decision trees is NP-complete

• Lecture 4: 23 Jan 2025
  (Class Notes (pdf), Lecture Slides (pdf))

  Decision trees: impurity measures (entropy, Gini index), information gain ratio, handling numeric attributes

  • Liu, Chapter 3.2.2., 3.2.3
  • Mitchell, Chapter 3.7.2
  • Quinlan, Chapter 2.4

  Evaluating classifiers: training/test sets, confusion matrix

  • Liu, Chapter 3.3.1

• Lecture 5: 28 Jan 2025
  (Class Notes (pdf))

  Decision Trees in Python

  • Decision Trees, Géron, Chapter 6,
    (Jupyter notebook, pdf )

• Lecture 6: 30 Jan 2025
  (Class Notes (pdf), Lecture Slides (pdf))

  Linear Regression: loss functions, normal equation, gradient descent

  • Lecture notes by Andrew Ng, Sections 1, 2
  • Géron, Chapter 4

• Lecture 7: 4 Feb 2025
  (Class Notes (pdf), Slides (pdf))

  Linear Regression: gradient descent, probabilistic justification for SSE loss

  • Lecture notes by Andrew Ng, Sections 1, 2, 3

  Polynomial regression, regularization, the non-polynomial case

  • Géron, Chapter 4

  Quiz 1, Solutions.

• Lecture 8: 06 Feb 2025
  (Class Notes (pdf), Slides (pdf))

  Regression using decision trees

  • Géron, Chapter 6

  Handling overfitting in decision trees

  • Liu, Chapter 3.2.4
  • Mitchell, Chapter 3.7.1
  • Quinlan, Chapter 4

  Regression for Classification

  • Lecture notes by Andrew Ng, Section 5
  • Neural Networks and Deep Learning by Michael Nielsen, Chapter 3, Improving the way neural networks learn
    

• Lecture 9: 11 Feb 2025
  (Class Notes (pdf))

  Scaling regression inputs

  Linear, polynomial and logistic regression in Python

  • Training Models, Géron, Chapter 4,
    (Jupyter notebook, pdf )

  Regression Trees in Python

  • Decision Trees, Géron, Chapter 6,
    (Jupyter notebook, pdf )

• Lecture 10: 13 Feb 2025
  (Class Notes (pdf), Slides (pdf))

  Naïve Bayesian classifiers

  • Liu, Chapter 3.6

  Naïve Bayes text classification

  • Liu, Chapter 3.7