Artificial Neural Networks

Week Topics Slides
  Week 1

(9/1)

Learning in Neurodynamic Self-organizing Systems

  • Neural Networks, Unsupervised / Self-supervised Learning
  • Mathematics for Neural Learning

pdf

 Week 2

(9/6, 9/8)

Principal-Components Analysis (Ch. 8)

  • Principal Component Analysis
  • Hebbian-Based Maximum Eigenfilter

Hebbian-Based PCA (Ch. 8)

  • Generalized Hebbian Algorithm
  • Kernel PCA

pdf

(Same as Week 1)

  Week 3

(9/13, 9/15)

Self-organizing Maps (Ch. 9)

  • Willshaw-von der Malsburg Model
  • Kohonen’s SOM Model

pdf

  Week 4

(9/20, 9/22)

Information-Theoretic Learning Models (Ch. 10)

  • Maximum Entropy, Kullback-Leibler Divergence
  • Mutual Information, Infomax, ICA

pdf

  Week 5

(9/27, 9/29)

Statistical-Mechanical Learning Methods (Ch. 11)

  • Statistical Mechanics, Markov Chains
  • Metropolis, Gibbs Sampling Simulated Annealing

pdf

  Week 6

(10/4, 10/6)

Deep Neural Networks (Ch. 11)

  • Boltzmann Machines
  • Deep Belief Networks

pdf

  Week 7

(10/11, 10/13)

Deep Neural Networks (Ch. 11)

  • Boltzmann Machines
  • Deep Belief Networks

Dynamic Programming(Ch. 12)

pdf

(Same as Week 6)

  Week 8

(10/18, 10/20)

Summary (10/18)

Mid-term Exam (10/20)

  Week 9

(10/25, 10/27)

Dynamic Programming (Ch. 12)

  • Markov Decision Process, DP, Bellman Equation

pdf

  Week 10

(11/1, 11/3)

Dynamic Programming (Ch. 12)

  • ADP, Reinforcement Learning, TD, Q

pdf

(Same as Week 9)

  Week 11

(11/8, 11/10)

Neurodynamic Models (Ch. 13)

  • Dynamic Systems, Attractors, Chaos
  • Hopfield Models, Dynamic Reconstruction

pdf

  Week 12

(11/15, 11/17)

Bayesian Filtering (Ch. 14)

  • State Space Models
  • Kalman Filters, EKF, CKF

pdf

  Week 13

(11/22, 11/24)

Particle Filters (Ch. 14)

  • Approximate Bayesian Filtering
  • Particle Filters, SIR Algorithm

Dynamic Recurrent Networks (Ch. 15)

  • Recurrent Network Architectures
  • Backpropagation through Time

pdf

pdf

  Week 14

(12/29, 12/1)

Real-Time Recurrent Learning (Ch. 15)

  • RTRL Algorithm, Vanishing Gradients
  • EKF Algorithm for Training RMLP

Final Exam (12/1)

pdf

  Week 15

(12/6)

Review and discussion