Course 4541.676
Hypernetwork Models of Learning and Memory

(Artificial Neural Networks)

 

 

School of Computer Science and Engineering,
Seoul National University

Lecturer

    Prof. Byoung-Tak Zhang

TA

    Ho-Sik Seok (hsseok at bi.snu.ac.kr, Tel: 02-880-1847, Room: 301-419)

Classroom

    302-106

Time

    Tue, 13:00 ~ 14:15 and Thur, 13:00 ~ 14:15

Objectives

- To review the cognitive neuroscience of human memory and learning

- To understand the biological mechanisms underlying memory and learning

- To learn the mathematics of information theory and machine learning

- To understand the recently-developed hypernetwork models of memory

- To simulate cognitive learning and memory models by the hypernetworks

- To design improved learning algorithms for hypernetworks based on information theory

Textbook

Information Theory, Inference, and Learning Algorithms
David, J. C. MacKay, Cambridge University Press, 2004 [M-04].

References

Books:

[B-73] Berge, C., Graphs and Hypergraphs, North-Holland, 1973.

[B-85] Bollobas, B., Random Graphs, Academic Press, 1985.

[DHS-01] Duda, R.O., Hart, P.E., and Stork, D.G., Pattern Classification, Wiley, 2001.

[GIM-02] [CNS] Gazzaniga, M.S., Ivry, R.B., and Mangun, G.R., Cognitive Neuroscience: The Biology of the Mind, 2nd Ed., Norton, 2002.

[Ch. 2][Ch. 5][Ch.8]-Black and white version[Ch. 2bw][Ch. 5bw][Ch. 8bw]

[K-06] Kandel, E.R., In Search of Memory: The Emergence of a New Science of Mind, Norton, 2006.

[SC-04] Shawe-Taylor, J. and Cristianini, N., Kernel Methods for Pattern Analysis, Cambridge University Press, 2004.

Articles:

[Z-06] Zhang, B.-T., Random Hypergraph Models of Learning and Memory in Biomolecular Networks: Shorter-Term Adaptivity vs. Longer-Term Persistency, The First IEEE Symposium on Foundations of Computational Intelligence (FOCI'07), April 1-5,  2007 (submitted).

[ZK-06] Zhang, B.-T. and Kim, J.-K., DNA Hypernetworks for Information Storage and Retrieval, DNA12, 2006.

[Z-05-Hyper] Zhang, B.-T., Hyperinteractionism: Computationalist Investigations into Artificial Minds beyond Symbolism, Connectionism, and Dynamicism (in Korean), Natural Sciences, 18, 2005.

[Z-05-NBIC] Zhang, B.-T., Molecular Nanobiointelligence Computers: Computer Science Meets Nanotechnology, Biotechnology, and Cognitive Brain Science (in Korean), Communications of the Korea Information Science Society, 23(5): 41-56, 2005.

[ZJ-04] Zhang, B.-T. and Jang, H.-Y., A Bayesian Algorithm for In Vitro Molecular Evolution of Pattern Classifiers, LNCS 3384: 458-467, 2005.

[Z-03-BMC] Zhang, B.-T., Bio-Molecular Computer Technology (in Korean), Physics and High-Technology, 12(5): 13-19, 2003.

[ZM-93] Zhang, B.-T. and Muehlenbein, H., Evolving Optimal Neural Networks Using Genetic Algorithms with Occams Razor, Complex Systems, 7(3): 199-220, 1993.

Evaluation

- One term project [Document]

Data sets for the term project [Data1][Data2](Option)

Due to Dec. 12

- Presentation of a paper

- List of papers to choose

- One exam

- Dec. 5

- Score

 

Notice

New class material

Project materials

Homework & A supplementary lecture

Presentation

 

 

 

  • Course Schedule

    Topic

    References

    Learning and Memory: Human and Machine

[Slides]

   Cognitive Science of Human Memory: Some Facts

[GIM-02, Ch. 8]

    Neurobiological Basis for Learning and Memory

[GIM-02, Ch.2][K-06]

    Learning and Memory in Computers

[Z-05-Hyper][Z-05-NBIC]

    Information Theory of Learning and Memory

[Slides]

    Probability, Entropy, and Inference

[M-04, Ch. 2]

    Data Compression

[M-04, Ch. 4]

    Noisy-Channel Coding

[M-04, Chs. 8 & 9]

Assignment: Paper Reading and Presentation

Hypernetwork Models of Memory

[Hypernet memory1]

[Hypernet memory2]

[Hypernet memory3]

 Graphs, Random Graphs, Hypergraphs

[B-73][B-85]

     Hypernetworks: Formal Definition

[Z-05-Hyper][ZK-06]

Hypernetworks and Graphical Models

 

Hypernetworks and Kernel Machines

[Z-06]

  Molecular Computation of Hypernetworks

[ZK-06]

[Ch. 5 molecular computation]

    DNA Encoding of Hypernetworks

 

    Molecular Operations for Library Manipulation

 

    Information Storage and Retrieval

 

    Term Project (Stage 1): Pattern Recognition with Hypernetworks
[Program1][Program2]

    Localist Memory Models and Their Learning

[Localist Memory Models1]

[Localist Memory Models2]

[Localist Memory Models3]

    Parzen Windows

 

    K-NN and K-Means Methods

[M-04, Ch. 20]

    Kernel Machines   

[M-04, Ch. 45]       

    Globalist Memory Models and Their Learning

 

Hopfield Networks

[M-04, Chs. 31 & 42]

Boltzmann Machines

[M-04, Ch. 43]

Bayesian Networks

[M-04, Ch. 21]

Exam

Term Project (Stage 2): Pattern Recall with Hypernetworks

Learning Algorithms for Hypernetworks

Random Graph Processes

[B-85]

Bayesian Algorithms for Learning and Evolution

[M-04, Ch. 22][ZM-93]

Occams Razor

(IT Ch. 28, [ZM-93])

Project Presentations

    Variants, Extensions, and Refinements

Unsupervised Learning Hypernets

 

Supervised Learning Hypernets

 

Comparison to Other Learning Algorithms

[M-04, Chs. 38-46]

 


This page is maintained by Ho-Sik Seok
Last update: September 5, 2006.