Course 4541.676
Hypernetwork Models of Learning and Memory

(Artificial Neural Networks)

	School of Computer Science and Engineering, Seoul National University
Lecturer	Prof. Byoung-Tak Zhang
TA	Ho-Sik Seok (hsseok at bi.snu.ac.kr, Tel: 02-880-1847, Room: 301-419)
Classroom	302-106
Time	Tue, 13:00 ~ 14:15 and Thur, 13:00 ~ 14:15
Objectives	- To review the cognitive neuroscience of human memory and learning - To understand the biological mechanisms underlying memory and learning - To learn the mathematics of information theory and machine learning - To understand the recently-developed hypernetwork models of memory - To simulate cognitive learning and memory models by the hypernetworks - To design improved learning algorithms for hypernetworks based on information theory
Textbook	Information Theory, Inference, and Learning Algorithms David, J. C. MacKay, Cambridge University Press, 2004 [M-04].
References	Books: [B-73] Berge, C., Graphs and Hypergraphs, North-Holland, 1973. [B-85] Bollobas, B., Random Graphs, Academic Press, 1985. [DHS-01] Duda, R.O., Hart, P.E., and Stork, D.G., Pattern Classification, Wiley, 2001. [GIM-02] [CNS] Gazzaniga, M.S., Ivry, R.B., and Mangun, G.R., Cognitive Neuroscience: The Biology of the Mind, 2nd Ed., Norton, 2002. [Ch. 2][Ch. 5][Ch.8]-Black and white version[Ch. 2bw][Ch. 5bw][Ch. 8bw] [K-06] Kandel, E.R., In Search of Memory: The Emergence of a New Science of Mind, Norton, 2006. [SC-04] Shawe-Taylor, J. and Cristianini, N., Kernel Methods for Pattern Analysis, Cambridge University Press, 2004. Articles: [Z-06] Zhang, B.-T., Random Hypergraph Models of Learning and Memory in Biomolecular Networks: Shorter-Term Adaptivity vs. Longer-Term Persistency, The First IEEE Symposium on Foundations of Computational Intelligence (FOCI'07), April 1-5,  2007 (submitted). [ZK-06] Zhang, B.-T. and Kim, J.-K., DNA Hypernetworks for Information Storage and Retrieval, DNA12, 2006. [Z-05-Hyper] Zhang, B.-T., Hyperinteractionism: Computationalist Investigations into Artificial Minds beyond Symbolism, Connectionism, and Dynamicism (in Korean), Natural Sciences, 18, 2005. [Z-05-NBIC] Zhang, B.-T., Molecular Nanobiointelligence Computers: Computer Science Meets Nanotechnology, Biotechnology, and Cognitive Brain Science (in Korean), Communications of the Korea Information Science Society, 23(5): 41-56, 2005. [ZJ-04] Zhang, B.-T. and Jang, H.-Y., A Bayesian Algorithm for In Vitro Molecular Evolution of Pattern Classifiers, LNCS 3384: 458-467, 2005. [Z-03-BMC] Zhang, B.-T., Bio-Molecular Computer Technology (in Korean), Physics and High-Technology, 12(5): 13-19, 2003. [ZM-93] Zhang, B.-T. and Muehlenbein, H., Evolving Optimal Neural Networks Using Genetic Algorithms with Occam′s Razor, Complex Systems, 7(3): 199-220, 1993.
Evaluation	- One term project [Document] Data sets for the term project [Data1][Data2](Option) Due to Dec. 12 - Presentation of a paper - List of papers to choose - One exam - Dec. 5 - Score

Notice

New class material

Project materials

Homework & A supplementary lecture

Presentation

• Course Schedule

Topic	References
Learning and Memory: Human and Machine	[Slides]
Cognitive Science of Human Memory: Some Facts	[GIM-02, Ch. 8]
Neurobiological Basis for Learning and Memory	[GIM-02, Ch.2][K-06]
Learning and Memory in Computers	[Z-05-Hyper][Z-05-NBIC]
Information Theory of Learning and Memory	[Slides]
Probability, Entropy, and Inference	[M-04, Ch. 2]
Data Compression	[M-04, Ch. 4]
Noisy-Channel Coding	[M-04, Chs. 8 & 9]
Assignment: Paper Reading and Presentation Hypernetwork Models of Memory	[Hypernet memory1] [Hypernet memory2] [Hypernet memory3]
Graphs, Random Graphs, Hypergraphs	[B-73][B-85]
Hypernetworks: Formal Definition	[Z-05-Hyper][ZK-06]
Hypernetworks and Graphical Models
Hypernetworks and Kernel Machines	[Z-06]
Molecular Computation of Hypernetworks	[ZK-06] [Ch. 5 molecular computation]
DNA Encoding of Hypernetworks
Molecular Operations for Library Manipulation
Information Storage and Retrieval
Term Project (Stage 1): Pattern Recognition with Hypernetworks [Program1][Program2]
Localist Memory Models and Their Learning		[Localist Memory Models1] [Localist Memory Models2] [Localist Memory Models3]
Parzen Windows
K-NN and K-Means Methods	[M-04, Ch. 20]
Kernel Machines	[M-04, Ch. 45]
Globalist Memory Models and Their Learning
Hopfield Networks	[M-04, Chs. 31 & 42]
Boltzmann Machines	[M-04, Ch. 43]
Bayesian Networks	[M-04, Ch. 21]
Exam Term Project (Stage 2): Pattern Recall with Hypernetworks
Learning Algorithms for Hypernetworks
Random Graph Processes	[B-85]
Bayesian Algorithms for Learning and Evolution	[M-04, Ch. 22][ZM-93]
Occam′s Razor	(IT Ch. 28, [ZM-93])
Project Presentations
Variants, Extensions, and Refinements
Unsupervised Learning Hypernets
Supervised Learning Hypernets
Comparison to Other Learning Algorithms	[M-04, Chs. 38-46]

---

This page is maintained by Ho-Sik Seok
Last update: September 5, 2006.