Artificial Neural Networks

Week Topics Slides
  Week 1

(9/1, 9/3)

Learning in Neurodynamic Self-organizing Systems

  • Neural Networks, Unsupervised / Self-supervised Learning
  • Mathematics for Neural Learning

Principal-Components Analysis (Ch. 8)

  • Principal Component Analysis
  • Hebbian-Based Maximum Eigenfilter

Hebbian-Based PCA (Ch. 8)

  • Generalized Hebbian Algorithm
  • Kernel PCA

pdf

  Week 2

(9/15, 9/17)

Self-organizing Maps (Ch. 9)

  • Willshaw-von der Malsburg Model
  • Kohonen’s SOM Model

pdf

  Week 3

(9/22, 9/24)

Information-Theoretic Learning Models (Ch. 10)

  • Maximum Entropy, Kullback-Leibler Divergence
  • Mutual Information, Infomax, ICA

pdf

  Week 4

(9/29, 10/1)

Statistical-Mechanical Learning Methods (Ch. 11)

  • Statistical Mechanics, Markov Chains
  • Metropolis, Gibbs Sampling Simulated Annealing

Korean Thanksgiving Holiday

pdf

  Week 5

(10/6, 10/8)

Deep Neural Networks (Ch. 11)

  • Boltzmann Machines
  • Deep Belief Networks

pdf

  Week 6

(10/13, 10/15)

Deep Neural Networks (Ch. 11)

  • Boltzmann Machines
  • Deep Belief Networks

Dynamic Programming(Ch. 12)

pdf

(Same as Week 5)

  Week 7

(10/20, 10/22)

Summary (10/15)

Mid-term Exam (10/17)

  Week 8

(10/27, 10/29)

Dynamic Programming (Ch. 12)

  • Markov Decision Process, DP, Bellman Equation

pdf

  Week 9

(11/3, 11/5)

Dynamic Programming (Ch. 12)

  • ADP, Reinforcement Learning, TD, Q

pdf

(Same as Week 8)

  Week 10

(11/10, 11/12)

Neurodynamic Models (Ch. 13)

  • Dynamic Systems, Attractors, Chaos
  • Hopfield Models, Dynamic Reconstruction

pdf

  Week 11

(11/17, 11/19)

Bayesian Filtering (Ch. 14)

  • State Space Models
  • Kalman Filters, EKF, CKF

pdf

  Week 12

(11/24, 11/26)

Particle Filters (Ch. 14)

  • Approximate Bayesian Filtering
  • Particle Filters, SIR Algorithm

Dynamic Recurrent Networks (Ch. 15)

  • Recurrent Network Architectures
  • Backpropagation through Time

pdf

pdf

  Week 13

(12/1, 12/3)

Real-Time Recurrent Learning (Ch. 15)

  • RTRL Algorithm, Vanishing Gradients
  • EKF Algorithm for Training RMLP

Final Exam (12/3)

pdf

  Week 14

(12/8)

Review and discussion