Byoung-Tak Zhang's Home Page

Byoung-Tak Zhang

Professor
School of Computer Science and Engineering &
Cognitive Science, Brain Science, and Bioinformatics
Seoul National University

Seoul 151-744, Korea
Office: Room 302-323 (New Engineering Building)

Phone: +82-2-880-1833 (office),

880-1847 (secretary)

Fax: +82-2-875-2240

E-mail: btzhang@bi.snu.ac.kr

URL: https://bi.snu.ac.kr/~btzhang/

Research Overview
How does the biological mind work? How does the brain build a model of the world to instantly and robustly act on it? What kinds of processing and organizational mechanisms allow the brain to learn so rapidly, flexibly, and continuously in a noisy, dynamic, and changing environment? How can we build smart machines that perceive, think, act, and learn like the human brain and mind? In the Biointelligence Lab and the Cognitive Robotics and ArtificialIntelligence Center (CRAIC) we pursue these questions using diverse methods of cognitive modeling, computational neuroscience, robotics, and computer science.

Deep Learning Textbook (2017)

Winning the 2017 RoboCup@Home

We build robotic surrogates by "cloning" the embodied and situated mind using wearable sensors, such as smart glasses, watches, brain scanners, body sensors, and mobile devices. We collect human activity data by long-term life-tracking in the real world. Based on these observational data, we reverse-engineer the brain architecture by exploring the plausible neurocognitive models that best explain the mind in working. We also use videos and cartoons to study how the cognitive brain constructs episodic memory and mental imagery from a long sequence of context-rich events while watching dramas and movies.
Our current efforts center around the deep, recurrent, and sparse hypernetwork architectures and learning algorithms that self-organize their structures instantly, incrementally, and continuously in a self-supervised way by perception-action cycle. The ultimate goal is to discover a large-scale, neurocognitive computational model of the brain that autonomously develops or evolves into human-level machine intelligence in lifelong interactions with the environment. For the ongoing and completed projects, visit BI research. For general information, visit the Biointelligence Laboratory home.

Theory: Hypernetwork Models of Learning and Memory
Applications: Data Mining in Life Sciences and Life Blogs
Technology: Molecular Evolutionary Computation in Silico and in Vitro

Journal Editorship

IEEE Transactions on Evolutionary Computation, Associate Editor (1997-2010)
Applied Intelligence, Review Board Member
BioSystems, Associate Editor
Journal of Cognitive Science, Editorial Board Member
Genetic Programming and Evolvable Machines, Editorial Board Member
Advances in Natural Computation (Book Series), Associate Editor

Graduate Courses Taught

Self-learning Neural Algorithms (2017)
Autonomous Machine Learning (2016)
Robots Learning from Humans (2015)
Mathematics of Lifelong Sequential Learning (2013)
Cognitive Neural Computation (2013)
Action-Perception-Learning Cycles (2012)
Multisensory Predictive Learning (2011)
Dynamic Learning (2010)
Bayesian Cognition (2010)
Probabilistic Graphical Models (2009)
Cognitive Machine Learning (2008)
Hypernetwork Models of Learning and Memory (2006)
Information Theory of Learning (2005)
Molecular Machine Learning (2004)
Biomolecular Computation (2003)
Molecular Intelligence (2002)
Bioinformatics and Machine Learning (2001)
Intelligent Learning Agents (2000)
Statistical Learning Theory (1999)
Machine Learning (1998, 1997)

Talks and Tutorials_International

Talks and Tutorials_Domestic

Position Papers & Monographs [Korean]

Meetings Organized

Perception, Action, and Cognitive Systems (PACS-2017, PACS-2016, PACS-2015)
Cognitive ICT Forum
Brain and Artificial Intelligence (BESK-2013)
Winter School on Pattern Recognition and Machine Learning (PRML-2013)
Cognition and Artificial Intelligence Symposium (CIS-2012)
AI Expo 2012: The Next Big Thing in AI
Cognitive Machines: Convergence of Biological and Physical Intelligence (GRF-2009)

Selected Publications

Attend What You Need: Motion-Appearance Synergistic Networks for Video Question Answering, A.-J. Seo, G.-C. Kang, J.-H. Park, B.-T. Zhang, Proceedings of the Joint Conference of the 59th Annual Meeting of the Association for Computational Linguistics and the 11th International Joint Conference on Natural Language Processing (ACL-IJCNLP 2021), August 2021.
Message Passing Adaptive Resonance Theory for Online Active Semi-supervised Learning, T. Kim, I. Hwang, H.-D. Lee, H. Kim, W.-S. Choi, J. Lim, B.-T. Zhang, Proceedings of the 38th International Conference on Machine Learning (ICML 2021), July 2021.
Co-attentional Transformers for Story-Based Video Understanding, B. Bebensee, B.-T. Zhang, Proceedings of the 46th International Conference on Acoustics, Speech, and Signal Processing (ICASSP 2020), June 2021. [PDF]
DramaQA: Character-Centered Video Story Understanding with Hierarchical QA, S.-H. Choi, K.-W. On, Y.-J Heo, A.-J. Seo, Y.-W. Jang, M.-S. Lee and B.-T. Zhang, Proceedings of the Thirty-Fifth AAAI Conference on Artificial Intelligence (AAAI 2021), February 2021. [PDF]
Hypergraph Attention Networks for Multimodal Learning, E.-S. Kim, W.-Y. Kang, K.-W. On, Y.-J. Heo and B.-T. Zhang, The IEEE Conference on Computer Vision and Pattern Recognition (CVPR, 2020), 2020. [PDF]
Cut-Based Graph Learning networks to Discover Compositional Structure of Sequential Video Data, K.-W. On, E.-S. Kim, Y.-J Heo and B.-T. Zhang, Proceedings of the Thirty-Fourth AAAI Conference on Artificial Intelligence (AAAI 2020), 2020.(oral presentation, accept ratio=5.8%) [PDF]
Answerer in questioner's mind: Information theoretic approach to goal-oriented visual dialog, S.-W. Lee, Y.-J. Heo, B.-T. Zhang, The 32nd Annual Conference on Neural Information Processing Systems (NIPS 2018), 2018. (Spotlight) [PDF]
Bilinear attention networks, J.-H. Kim, J. Jun, B.-T. Zhang, The 32nd Annual Conference on Neural Information Processing Systems (NIPS 2018), 2018. [PDF]
Multimodal dual attention memory for video story question answering, K. Kim, S.-H. Choi, J.-H. Kim, B.-T. Zhang, The 15th European Conference on Computer Vision (ECCV 2018), 2018. [PDF]
Robust human following by deep Bayesian trajectory prediction for home service robots, B.-J. Lee, J. Choi, C. Baek, B.-T. Zhang, 2018 IEEE International Conference on Robotics and Automation (ICRA 2018), 2018. [PDF]
Human Intelligence and Machine Intelligence: Cognitive Artificial Intelligence (in Korean), Zhang, B.-T., Communications of the KIISE, 36(1): 27-36, 2018. [PDF]
Overcoming catastrophic forgetting by incremental moment matching, S.-W. Lee, J.-H. Kim, J. Jun, J.-W. Ha, and B.-T. Zhang, The 31st Annual Conference on Neural Information Processing Systems (NIPS 2017), 2017. (Spotlight) [PDF]
Dual-memory neural networks for modeling cognitive activities of humans via wearable sensors, S.-W. Lee, C.-Y. Lee, D.-H. Kwak, J.-W. Ha, J. Kim, and B.-T. Zhang, Neural Networks , 92:17-28, 2017. [PDF]
DeepStory: Video story QA by deep embedded memory networks, K.-M. Kim, M.-O. Heo, S.-H. Choi, and B.-T. Zhang, The 26th International Joint Conference on Artificial Intelligence (IJCAI 2017), 2017. [PDF]
Hadamard product for low-rank bilinear pooling, J.-H. Kim, K.-W. On, W. Lim, J. Kim, J.-W. Ha, B.-T. Zhang, The 5th International Conference on Learning Representations (ICLR 2017), 2017. [PDF]
Humans and machines in the evolution of AI in Korea, AI Magazine, 37(2):108-112, 2016. [PDF]
Multimodal residual learning for visual QA, J.-H. Kim, S.-W. Lee, D.-H. Kwak, M.-O. Heo, J. Kim, J.-W. Ha, and B.-T. Zhang, In The 30th Annual Conference on Neural Information Processing Systems (NIPS 2016) [PDF]
DeepSchema: Automatic schema acquisition from wearable sensor data in restaurant situations, E.-S. Kim, K.-W. On, and B.-T. Zhang, In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI 2016), pp. 834-840, 2016. [PDF]
Dual-memory deep learning architectures for lifelong learning of everyday human behaviors, S.-W. Lee, C.-Y. Lee, D. H. Kwak, J. Kim, J. Kim, and B.-T. Zhang, In Proceedings of the International Joint Conference on Artificial Intelligence (IJCAI 2016), 1669-1675, 2016. [PDF]
Deep hypernetwork models (in Korean), Communications of the Korean Institute of Information Scientists and Engineers, 33(8):11-24, 2015. [PDF]
Team THOR's entry in the DARPA robotics challenge trials 2013, Journal of Field Robotics, 32(3):315-335, 2015. [PDF]
Consensus analysis and modeling of visual aesthetic perception, IEEE Transactions on Affective Computing, 6(3):272-285. [PDF]
Automated construction of visual-linguistic knowledge via concept learning from cartoon videos, J.-W. Ha, K.-M. Kim, and B.-T. Zhang, In Proceedings of the Twenty-Ninth AAAI Conference on Artificial Intelligence (AAAI 2015), 2015. [PDF]
Ontogenesis of agency in machines: A multidisciplinary review, B.-T. Zhang, AAAI 2014 Fall Symposium on The Nature of Humans and Machines: A Multidisciplinary Discourse, Arlington, VA, 2014. [PDF]
Communication as moving target tracking: Dynamic Bayesian inference with an action-perception-learning cycle, B.-T. Zhang, Alignment in Communication. Towards a New Theory of Communication, Chapter 7, pp. 133-147, 2013. [PDF]
Information-theoretic objective functions for lifelong learning, B.-T. Zhang, AAAI 2013 Spring Symposium on Lifelong Machine Learning, Stanford University, March 25-27, 2013. [PDF]
Sparse population code models of word learningin concept drift, B.-T. Zhang, J.-W. Ha, and M. Kang, In Proceedings of the 34th Annual Meeting of the Cognitive Science Society (CogSci 2012), 2012. [PDF]
A DNA assembly model of sentence generation, J.-H. Lee, S. H. Lee, W.-H. Chung, E. S. Lee, T. H. Park, R. Deaton, and B.-T. Zhang, BioSystems, 106:51-55, 2011. [PDF]
Hypernetworks: A molecular evolutionary architecture for cognitive learning and memory, B.-T. Zhang, IEEE Computational Intelligence Magazine, 3(3):49-63, August 2008.[PDF][LINK]
Molecular basis for the recognition of primary microRNAs by the Drosha-DGCR8 complex, J. Han, Lee Y, K.H. Yeom, J.-W. Nam, I. Heo, J.-K. Rhee, S. Y. Sohn, Y. Cho, B.-T. Zhang and V.N. Kim, Cell, 125:887-901, 2006. [PDF]
Multi-objective evolutionary optimization of DNA sequences for reliable DNA computing, S.-Y. Shin, I.-H. Lee, D. Kim, and B.-T. Zhang, IEEE Transactions on Evolutionary Computation, 9(2):143-158, 2005. [PDF]
Bayesian model averaging of Bayesian network classifiers over multiple node-orders: application to sparse datasets, K.-B. Hwang and B.-T. Zhang, IEEE Transactions on Systems, Man, and Cybernetics Part B: Cybernetics, 35(6):1302-1310, 2005. [PDF]
Self-organizing latent lattice models for temporal gene expression profiling, Zhang, B.-T, Yang, J. and Chi, S. W., Machine Learning, 52(1/2):67-89, 2003. [PDF]
System identification using evolutionary Markov chain monte carlo, Zhang, B.-T. and Cho, D.-Y., Journal of Systems Architecture, 47(7):587-599, 2001. [PDF]
Bayesian methods for efficient genetic programming, Zhang, B.-T., Genetic Programming and Evolvable Machines, 1(3):217-242, 2000. [PDF]
Evolutionary induction of sparse neural trees, Zhang, B.-T., Ohm, P., and M?hlenbein, H., Evolutionary Computation, 5(2):213-236, 1997. [PDF]
Balancing accuracy and parsimony in genetic programming, Zhang, B.-T. and M?hlenbein, H., Evolutionary Computation, 3(1):17-38, 1995. [PDF]
Accelerated learning by active example selection, Zhang, B.-T., International Journal of Neural Systems, 5(1):67-75, 1994. [PDF]
Evolving optimal neural networks using genetic algorithms with Occam's razor, Zhang, B.-T. and M?hlenbein, H., Complex Systems, 7(3):199-220, 1993. [PDF] [LINK]

[more]

Biointelligence Lab

School of Computer Sci. & Eng.

SNU