-
To study the information theory and statistical physics of computational
learning
- To understand the architectures and principles of probabilistic
graphical models
- To study the algorithms for learning and inference in graphical models
- To learn how to use graphical models for neural and cognitive modeling
- To practice the use of the hypernetwork models for language and vision
computing
[4] Mackay, D. J. C.,
Information Theory, Inference, and Learning Algorithms, Cambridge Univ.
(Chs. 42 & 43).
[5] Zhang, B.-T.,
Hypernetworks: A Molecular Evolutionary Architecture for Cognitive
Learning and Memory, IEEE Computational Intelligence Magazine,
3(3):49-63, 2008 [PDF]
References
References
[1] Geman, S. and Geman, D.,
Stochastic relaxation, Gibbs distribution, and the Bayesian restoration of
images, IEEE Trans. on Patt. Anal. and Mach. Int., 6(1): 721-741, 1984 [PDF].
[2] Hinton, G. E., Dayan, P.,
Frey, B. J., and Neal, R. M., The wake-sleep algorithm for unsupervised
neural networks, Science 268(5214): 1158-1161, 1995 [PDF].
[3] Hopfield, J. J., Neural
networks and physical systems with emergent collective computational
abilities, Proc. Natl., Acad. Sci. USA 79: 2554-2556, 1982 [PDF].
[4] Kirkpatrick, S., Gelatt Jr, C.
D., and Vecchi, M. P., Optimization by simulated annealing, Science, 220:
671-680 [PDF].
[5] Solomonoff, R. J., A formal
theory of inductive inference, Information and Control, 7: 1-22, 1964 [PDF1][PDF2].
[6] Valiant, L. G., A theory of
the learnable, Comm. ACM, 27, 1134-1142, 1984 [PDF].
Evaluation
- Two exams (60%) ->70%
- Termproject (20%)->10% (mini report on software practice)
- Essay (10%)
- Participation in discussion (10%)
2. Project report (max: 3 pages) due: Dec. 13th (23:59).
- Delay: Dec. 14th (13:00) Report files received after 13:00 will be ignored. So please send your report as soon as possible.
- Email to TA
- Supplementary materials.
a. Zhang, B.-T., Hypernetworks: A Molecular Evolutionary Architecture for Cognitive Learning and
Memory, IEEE Computational Intelligence Magazine, 3(3):49-63, 2008[PDF]-New!
b. Hypernetworks for Human-level Machine Learning [PDF]-New!
c. Evolving Hypernetworks for Language Modeling [PDF]-New!
d. Unsubmitted report: 20457
[Evaluation criteria change]
?Course
Schedule
Date
Topic
Lecture Notes
Week 1
Computational
Learning Theory
- Learning as a lifelong
process
- Inductive inference
- Statistical learning theory
and the VC dimension