Mathematics of Lifelong Sequential Learning 
(4541.676 Artificial Neural Networks, 132.531 Cognitive Processes and AI,
3391.506 Computational Neuroscience, 2013)

 

 

 

  • School of Computer Science and Engineering, Seoul National University
  • Instructor: Prof. Byoung-Tak Zhang
  • TA: Min-Oh Heo,  (Room : 302-314-1, Tel : 02-880-1847)
  • Classroom: 302-105 (Lab for Practice: 302-311-1)
  • Time: Tue 14:00-15:15 and Thu 14:00-15:15
  • Course Description

This course aims to understand the principles and mechanisms of cognitive agents that learn continuously and sequentially over lifetime in dynamic environments. In this lifelong learning setting, the target distribution of the learning is typically non-stationary and changing. This is in sharp contrast to the usual assumptions made in most supervised and unsupervised learning algorithms. That is, lifelong sequential learning violates the assumption of the examples being i.i.d (independent and identically distributed) and the assumption of availability of all the training data at the outset of learning. However, lifelong learning research is still in its infancy and there is no single universal algorithm that addresses all these issues. Thus, in this course we discuss three specific aspects of lifelong learning problems, i.e. sequence learning, sequential decision making, and moving target tracking. We review the existing machine learning algorithms that best deal with these problems so far, namely hidden Markov models, reinforcement learning, and particle filters. Based on this knowledge, we will discuss how to develop a humanlike intelligent agent that that can learn actively, sequentially, and lifelong in real-time from real-life sensing data in real-worlds.

 


 

  Lecture Schedule

 

Week

Topics

Materials

Week 1
(9/3, 9/5)

  • Why lifelong learning agents?
  • Problem 1: Sequence Learning

 

Week2
(9/10, 9/12)

  • Hidden Markov model (HMM) practice 1
  • Hidden Markov model (HMM) practice 2

Term project announced (First report due 10/10)

 

Week3
(9/17, 9/19)

  • HMM Theory 1
  • No Class (Thanksgiving holiday)

LSL_Note_1

LSL_Note_2

Week4
(9/24, 9/26)

  • HMM Theory 2
  • HMM Theory 3

LSL_Note_3

LSL_Note_4

LSL_Note_5

LSL_Note_6

LSL_Note_7

Text [4]

Week5
(10/1, 10/3)

  • Problem 2: Sequential decision making
  • No Class (Holiday)

 

Week6
(10/8, 10/10)

  • Reinforcement Learning 1
  • Reinforcement Learning 2

Due of project report 1

LSL_Note_8

LSL_Note_9

LSL_Note_10

Week7
(10/15, 10/17)

  • Reinforcement learning 3
  • Problem 3: Moving target tracking

LSL_Note_11

Week8
(10/22, 10/24)

  • Particle filter theory 1
  • Particle filter theory 2

LSL_Note_12

LSL_Note_13

LSL_Note_14

Week9
(10/29, 10/31)

  • Particle filter practice 1
  • Particle filter practice 2

 

Week10
(11/5, 11/7)

  • Particle filter practice 3
  • Exam (open book)

 

Week11
(11/12, 11/14)

  • Project meeting 1
  • Project meeting 2

 

Week12
(11/19, 11/21)

  • Lifelong learning agents 1
  • Lifelong learning agents 2

 

Week13
(11/26, 11/28)

  • Project poster presentation 1
  • Project poster presentation 2

 

Week14
(12/3, 12/5)

  • Due of project report 2

 

 


 

This page is maintained by Min-Oh Heo 
Last update: Nov. 21, 2013.