Machine Learning 2 (Summer Term 2020)

Overview

• Online course consisting of self-study and exercises (2/2/0). No audio/video lectures.
• Modules: CMS-CE-EL1, CMS-CE-EL2, CMS-CLS-ELG, CMS-VC-ELV1, CMS-VC-ELV2, INF-04-FG-IS, INF-B-520, INF-B-540, INF-BAS-2, INF-BAS-7, INF-E-3, INF-LE-MA, INF-PM-FOR, INF-VERT-2, INF-VERT-7, INF-VMI-8, MATH-MA-INFGDV, MCL-AI
• Lecturer: Bjoern Andres
• Assistant: Holger Heidrich
• Online registration is possible via SELMA (CMS), jExam or OPAL
• Forum and exam scheduling (OPAL)

Lecture

Material for self-study is made available here as the course progresses.

• Machine Learning II (lecture notes)
  • Logistic regression • Machine Learning I recap
  • Supervised structured learning
    • Conditional graphical models
      • Learning and inference problem
      • Learning algorithm (message passing)
      • Inference algorithm (ICM, message passing)
  • Clustering of graphs
    • Learning and inference problem
    • Inference algorithms

Examination

• Format
  • Individual remote oral examinations about the contents of this course are offered for all participating students of all degree programmes.
  • There will not be any written examination on the contents of this course in the Summer Term 2020.
• Creditability
  • Regulations regarding the creditability of the results of the examination depend on the degree programme and module. Regulations we are aware of are summarized below. Students in doubt are asked to check with the coordinator of their degree programme.
  • CMS-*. In the degree programme Computational Modeling and Simulation (CMS), the individual remote oral examination replaces the written examination.
  • INF-BAS-*, INF-E-*, INF-PM-ANW, INF-PM-FOR, INF-VERT-*, INF-VMI-*, MCL-AI. In these modules, there is an additional, exceptional opportunity in the Summer Term 2020 to take separate examinations on the contents of each course creditable toward the module, instead of taking one complex examination on the contents of multiple courses. A prerequisite, however, is that all these examinations are taken this term. In the future, remote oral examinations on the contents of multiple courses, possibly involving multiple examiners, can be scheduled individually as usual, as described here.
• Scheduling open from 2020-05-28 through 2020-06-28
  • Appointments for individual remote oral examinations are available now and can be scheduled exclusively through OPAL.
• Registration open through 2020-07-03 (all forms need to be submitted by then)
  • Only scheduled examinations can be registered.
  • Students enrolled in the degree programme Computational Modeling and Simulation (CMS) need to
    • register their scheduled examination via SELMA
    • send the filled in and signed form Declaration of consent to an alternative form of an immaterial/oral examination to mlcv-exams@tu-dresden.de.
  • Students enrolled in other degree programmes of the Faculty of Computer Science
    • who wish to credit the course toward one of the modules INF-BAS-*, INF-VERT-*, INF-VMI-* or INF-E-* need to
      • follow these instructions by the examination office of the faculty until 2020-07-03
      • send the filled in and signed form Declaration of consent to an alternative form of an immaterial/oral examination to mlcv-exams@tu-dresden.de.
    • who wish to credit the course toward any other module need to send to mlcv-exams@tu-dresden.de:
      • The filled in and signed registration form of the module toward which the course is to be credited. Respective forms are provided by the Examination Office and by the Service Center for International Students
      • The filled in and signed form Declaration of consent to an alternative form of an immaterial/oral examination
  • Students enrolled in degree programmes of other faculties of TU Dresden need to email to mlcv-exams@tu-dresden.de:
    • Registration documents issued by the faculty of their degree programme, in case such documents exist
    • In case such documents do not exist: An informal request of a course certificate, including full name, matriculation number, degree programme, course title and email address to be used for correspondence
    • The filled in and signed form Declaration of consent to an alternative form of an immaterial/oral examination.

Preparation

The following questions are similar in style to questions that are asked during the examination. They might be of help when preparing for the examination. We emphasize, however, that the examination is in no way constrained by, let alone limited to these questions.

• Supervised learning
  • What do you know about supervised learning?
  • What is the purpose of a loss function? Give an example!
  • What is the purpose of a regularizer? Give an example!
  • What is the effect of making the regularization parameter larger?
  • Suppose you have solved a supervised learning problem. How do you infer decisions for unlabeled data?
• Semi-supervised learning
  • What do you know about semi-supervised learning?
  • What is the learning and inference problem?
  • Explain how the supervised learning problem is a special case of the learning and inference problem!
  • How is the inference problem in the context of semi-supervised learning different from the inference problem in the context of supervised learning?
  • Give a (made-up or real-world) example of a semi-supervised learning problem!
• Classification
  • What do you know about classification?
  • The lecture has introduced classification as a special case of the learning and inference problem. Explain how!
  • What do you know about the learning problem for classification?
  • What do you know about the inference problem for classification?
  • How can you solve the inference problem for classification?
• Supervised structured learning
  • What is a factor graph?
  • What does it mean for a function to factorize with respect to a factor graph?
  • What is structured data?
  • What is a conditional graphical model?
  • What is the energy function of a conditional graphical model?
  • What is supervised structured learning?
  • How is supervised structured learning different from supervised learning?
  • What do you know about the supervised structured learning problem (Section 6.3.4)?
  • How can you solve the supervised structured learning problem (Section 6.3.4)?
  • What do you know about the (structured) inference problem (Section 6.3.5)?
  • How can you solve the (structured) inference problem (Section 6.3.5)?
• Clustering
  • What do you know about clustering?
  • What is a partition of a set/a decomposition of a graph/a multicut of a graph?
  • How can you encode any decomposition of a given graph in a binary vector of fixed dimension?
  • What is the difference between decision problems and clustering problems?
  • What do you know about the learning problem in the context of clustering?
  • What do you know about the inference problem in the context of clustering?
  • How can you solve the inference problem in the context of clustering?
  • How does greedy joining work?
  • How does greedy moving work?
  • Describe the technique of Kernighan and Lin!

Exercises

Assignments are published here as the course progresses.

For video conf discussions see post in OPAL.

• Logistic Regression and corresponding notebook
  • Logistic Regression Solution and corresponding notebook
• Supervised structured learning - Factor Graphs
  There was additional explanation for this exercise in the VideoConf on Monday, 2020-06-08, 4.DS, 13:00.
  
• There is no video conf by Prof.Andres at lecture time planned currently.
  From next week on I will use the lecture time slot (Monday, 3.DS, 11:10).
  
  The solution for the last exercise (factor graphs) is here: pdf
• Here is the link for the VideoConf for Monday, 2020-06-22, 3.DS, 11:10 (lecture time slot).
  Prepare your questions.
  
  The solution for the last exercise (factor graphs) is here: pdf