Xtran:

Crossmodal Translation of Language and Vision


Ƽ̵ Ϳ Ե ̹ ؽƮ ޸Ƽ(cross modality) ϴ ̸ ޸Ƽ ȣ ǥ Ѵ.

  • 1: ðܾ ܾ ̿ .
  • 2: ̹ ׸Ʈ ̿ ΰ, ұ Cross Modal Information Generation (CMIG) ý

̸ ұ Ʒ ִ.

  • Ƽ̵ Visual Word
  • Ƽ̵ Ȯ ׷ н
  • Image-Text , Image to Text / Text to Image ˻
  • ҵ յ Cross Modality Information Generation (CMIG)
  • CMIG ý Intelligent Documentary Dictionary (IDD)







DRW000008208fa6

Ư¡ ȭ, SIFT Visual words




UNI00000f9c71db

ť͸ 󿡼 ̹ ĸ нϿ ۳Ʈũ



Text words query ־ ܾ ϴ






EMB000008208f84

ó ý Intelligent Documentary Dictionary (IDD)






Publications

  • ۳Ʈũ ̿ - ũν , ο, , 庴Ź, HCI2009 мȸ, pp. 278-284, 2009.02.
  • ̿ ̹-ؽƮ н, ̻, , , б, 2009.02.
  • ۸ ޸ н , , , 庴Ź, Korea Computer Congress 2009, 2009.07.
  • Ʈũ 𵨿 ȣۿ , ۰, ȣ, 庴Ź, Korea Computer Congress 2009, 2009.07.
  • ȭ ۳Ʈũ ̿ н ũν , , , 庴Ź, Korea Computer Congress 2009, 2009.07.
  • ġ Ư Ʈ н, ȣ, 庴Ź, Korea Computer Congress 2009, 2009.07.
  • Sparsity Eliminated Collaborative Filtering Using Cluster-based Weighting, ſ, , 庴Ź, Korea Computer Congress 2009, 2009.07.
  • ۳Ʈũ ̿ ȭ ֽĵ , ̼, 庴Ź, Korea Computer Congress 2009, 2009.07.
  • Latent Semantic Analysis ̿ Naïve Bayes ؽƮ з, , ȫ, 庴Ź, Korea Computer Congress 2009, 2009.07.
  • Teaching an agent by playing a multimodal memory game: challenges for machine learners and human teachers, B.-T Zhang, AAAI 2009 Spring Symposium: Agents that Learn from Human Teachers, p.134, 2009.03.
  • Evolving hypernetwork models of binary time series for forecasting price movements on stock markets, E. Bautu, S. Kim, A. Bautu, H. Luchian, and B.-T. Zhang, IEEE Congress on Evolutionary Computation (CEC 2009), 2009.05
  • A Genetic Algorithm with Local Map for Path Planning in Dynamic Environments, Ivan Koryakovskiy, Nguyen Xuan Hoai and Kyoung Mu Lee, Genetic and Evolutionary Computation Conference 2009 (GECCO2009), 2009.07
  • Evolutionary Hypernetwork Classifiers for Protein-Protein Interaction Sentence Filtering, J. Bootkrajang, S. Kim, and B.-T. Zhang, Genetic and Evolutionary Computation Conference 2009 (GECCO2009), 2009.07
  • Gender Classification with Cortical Thickness Measurement from Magnetic Resonance Imaging by Using Feature Selection Method Based on Evolving Hypernetwork, J.-W. Ha, J. H. Jang, D.-H. Kang, W. H. Jung, J. S. Kwon, and B.-T. Zhang, 2009 IEEE International Conference on Fuzzy Systems, 2009.08.
  • Evolutionary Hypernetworks for Learning to Generate Music from Examples, H.-W. Kim, B.-H. Kim, and B.-T. Zhang, 2009 IEEE International Conference on Fuzzy Systems, 2009.08.

 


Project Title

Ƽ̵ ڵ ± Ȯ ׷ Cross Modality (Probabilistic Graphical Model-Based Crossmodal Information Extraction and Generation Technologies for Auto-Annotation of Multimedia Data)

Sponsor

KRF (Korea Research Foundation)

Principal Investigator

Prof.Byoung-Tak Zhang, Prof. Kyung Moo Lee

Researchers

Min-Oh Heo

Jung-Woo Ha

Minsu Cho

Jungmin Lee

Sang-Yoon Yi

Young-Kil Ko

Moon Bin Lee

Duration

2008.07 – 2010.06


Contact

 Min-Oh Heo

E-Mail

moheo (a) bi.snu.ac.kr

Phone

 +82-2-880-1835

Fax

 +82-2-875-2240