Self-learning Neural Algorithms (a.k.a Artificial Neural Networks)

2017 Fall Graduate Course in Computer Science and Engineering and Brain Science
- 컴퓨터공학부: Artificial Neural Networks
- 뇌과학협동과정: Computational Neuroscience
Instructor: Byoung-Tak Zhang
Main TA: Jiseob Kim (jkim@bi.snu.ac.kr)
Sub TA: Je-Hwan Ryu (jhryu@bi.snu.ac.kr)
Class Room: ~~302-209~~302-107
Time: Tue & Thu, 14:00-15:15h
Textbooks:
- Haykin, S. (2009). Neural Networks and Learning Machines, 3rd Ed., Pearson.
- 장병탁 (2017). 딥러닝, 홍릉출판사
Evaluation:
- Two exams (80%)
- Homework (10%)
- Participation and discussion (10%)
Announcement:
- (07/25) Welcome to the class!
- (09/13) If you cannot come to the class due to attending a conference, please email the TA (both of us) at least a week prior to the date with the documentative reference (e.g., invoice of the registration).
- (09/13) The schedule is updated. Please check it.
- (09/19) Mid-term Exam date is fixed to Oct. 26.
- (09/19) No classes in Week 4 (09/26, 09/28). Makeup class will be on Oct. 12 at 19:30 at the same class room.
- (09/28) Newer version of Ch 11. slides have been uploaded.
- (10/09) Homework #1 is announced. Please look inside this file for the details.
- (10/09) There will be a Problem Solving Session by TA on Oct. 19. There will be a lecture (as usual) on Oct. 17.
- (10/17) Late submissions of HW#1 is accepted at the beginning of the class on Oct. 19. This submission will get 80% of the credit.
- (10/17) The scope of the Mid-term Exam is up to Ch. 11.
- (10/19) Solution for HW#1 is uploaded. Check it.
- (10/27) The class on the next Thursday (Nov. 2) will be replaced by PACS-2017 conference. Students are required to attend the conference session 14:00-15:30 The Neural Code at the Base of Perception, Action, and Cognition, Christoph von der Malsburg. Registration is not needed to enter this session (this is agreed with the staffs). There is going to be the attendance check in the lobby of the conference venue (Engineering House).
- (11/01) Students can attend the whole PACS-2017 conference (both Thu and Fri) without registration (Go to the registration desk and tell them you are taking Prof. Byoung-Tak Zhang's class).
- (11/13) Thursday (Nov. 16)'s lecture will be given in a different building. Please come to Mogam Hall, Building #500 Campus Map.
- (11/14) Final Exam date is fixed to Nov. 30. Also, there will be a makeup class on Nov. 21 at 19:00 at the same class room due to schedule adjustment.
- (11/22) Homework #2 is out. Please look inside this file for the details.
- (11/28) Solution for HW#2 is uploaded. Check it.
- (12/15) Attendance, Mid-term, Final Exam Score is uploaded. You can claim your score visiting TA office (Room 417, Building 138) on 18(Mon), 19(Tue) from 1PM to 4PM. Note that homework scores are not yet included, but we will not run the claim hours for the homework scores.

Course Description:

We study “self-learning” networks, i.e. models that learn in an unsupervised and “self-supervised” way without the help of an explicit teacher. These models are neuro-biologically inspired and, usually, self-organizing, dynamic, recurrent, and auto-encoding networks. We examine the principles of neural learning algorithms from the historical models, such as Willshaw-von der Malsburg feature maps, Linsker models, Kohonen’s self-organizing maps, Grossberg models, recurrent networks, Anderson’s brain-state-in-a-box, actor-critic networks, Hopfield’s associative memory, Boltzmann machines, and deep belief networks. We study mathematical tools for approximation and optimization of the neural learning models. These include information-theoretic algorithms, such as maximum entropy, mutual information, and KL divergence as well as the statistical-mechanical methods, such as Markov chains, Metropolis algorithms, Gibbs sampling, and simulated annealing. We also examine the neurodynamic models of self-supervised, end-to-end learning to solve the challenging problems, such as time series prediction and reconstruction. These include Markov decision processes, approximate dynamic programming, reinforcement learning, sequential Bayesian estimation, Kalman filtering, particle filtering, real-time recurrent learning, dynamic reconstruction of a chaotic process.

Schedule

Week	Topics	Slides
`Week 1 (9/5, 9/7)`	Learning in Neurodynamic Self-organizing Systems Neural Networks, Unsupervised / Self-supervised Learning Mathematics for Neural Learning Principal-Components Analysis (Ch. 8) Principal Component Analysis Hebbian-Based Maximum Eigenfilter Hebbian-Based PCA (Ch. 8) Generalized Hebbian Algorithm Kernel PCA
`Week 2` (9/12, 9/14)	Self-organizing Maps (Ch. 9) Willshaw-von der Malsburg Model Kohonen’s SOM Model
`Week 3` (9/19, 9/21)	Information-Theoretic Learning Models (Ch. 10) Maximum Entropy, Kullback-Leibler Divergence Mutual Information, Infomax, ICA
`Week 4` (9/26, 9/28) `_No Class_`	Statistical-Mechanical Learning Methods (Ch. 11) Statistical Mechanics, Markov Chains Metropolis, Gibbs Sampling Simulated Annealing
`Week 5` (10/3, 10/5)	Korean Thanksgiving Holiday
`Week 6` (10/10, 10/12) MakeUp Class	Deep Neural Networks (Ch. 11) Boltzmann Machines Deep Belief Networks	(Same as Week 4)
`Week 7` (10/17, 10/19)	Dynamic Programming (Ch. 12) Problem Solving Session by TA
`Week 8` (10/24, 10/26)	Summary (10/24) Mid-term Exam (10/26)
`Week 9` (10/31, 11/2)	Dynamic Programming (Ch. 12) Markov Decision Process, DP, Bellman Equation ADP, Reinforcement Learning, TD, Q (11/2) PACS-2017. Please see the announcement.	(Same as Week 7)
`Week 10` (11/7, 11/9)	Neurodynamic Models (Ch. 13) Dynamic Systems, Attractors, Chaos Hopfield Models, Dynamic Reconstruction
`Week 11` (11/14, 11/16) Classroom Change	Bayesian Filtering (Ch. 14) State Space Models Kalman Filters, EKF, CKF
`Week 12` (11/21, 11/23) MakeUp Class	Particle Filters (Ch. 14) Approximate Bayesian Filtering Particle Filters, SIR Algorithm Dynamic Recurrent Networks (Ch. 15) Recurrent Network Architectures Backpropagation through Time
`Week 13` (11/28, 11/30)	Real-Time Recurrent Learning (Ch. 15) RTRL Algorithm, Vanishing Gradients EKF Algorithm for Training RMLP Final Exam (11/30)
`Week 14` (12/5, 12/7)	Review and discussion

  Week 1

(9/5, 9/7)

Learning in Neurodynamic Self-organizing Systems

Neural Networks, Unsupervised / Self-supervised Learning
Mathematics for Neural Learning

Principal-Components Analysis (Ch. 8)

Principal Component Analysis
Hebbian-Based Maximum Eigenfilter

Hebbian-Based PCA (Ch. 8)

Generalized Hebbian Algorithm
Kernel PCA

  Week 2

(9/12, 9/14)

Self-organizing Maps (Ch. 9)

Willshaw-von der Malsburg Model
Kohonen’s SOM Model

  Week 3

(9/19, 9/21)

Information-Theoretic Learning Models (Ch. 10)

Maximum Entropy, Kullback-Leibler Divergence
Mutual Information, Infomax, ICA

  Week 4

(9/26, 9/28)

_No Class_

Statistical-Mechanical Learning Methods (Ch. 11)

Statistical Mechanics, Markov Chains
Metropolis, Gibbs Sampling Simulated Annealing

  Week 5

(10/3, 10/5)

Korean Thanksgiving Holiday

  Week 6

(10/10, 10/12)

MakeUp Class

Deep Neural Networks (Ch. 11)

Boltzmann Machines
Deep Belief Networks

(Same as Week 4)

  Week 7

(10/17, 10/19)

Dynamic Programming (Ch. 12)

Problem Solving Session by TA

  Week 8

(10/24, 10/26)

Summary (10/24)

Mid-term Exam (10/26)

  Week 9

(10/31, 11/2)

Dynamic Programming (Ch. 12)

Markov Decision Process, DP, Bellman Equation
ADP, Reinforcement Learning, TD, Q
(11/2) PACS-2017. Please see the announcement.

(Same as Week 7)

  Week 10

(11/7, 11/9)

Neurodynamic Models (Ch. 13)

Dynamic Systems, Attractors, Chaos
Hopfield Models, Dynamic Reconstruction

  Week 11

(11/14, 11/16)

Classroom Change

Bayesian Filtering (Ch. 14)

State Space Models
Kalman Filters, EKF, CKF

  Week 12

(11/21, 11/23)

MakeUp Class

Particle Filters (Ch. 14)

Approximate Bayesian Filtering
Particle Filters, SIR Algorithm

Dynamic Recurrent Networks (Ch. 15)

Recurrent Network Architectures
Backpropagation through Time

  Week 13

(11/28, 11/30)

Real-Time Recurrent Learning (Ch. 15)

RTRL Algorithm, Vanishing Gradients
EKF Algorithm for Training RMLP

Final Exam (11/30)

  Week 14

(12/5, 12/7)

Review and discussion