Deep NLP 2 (2019.3-5)

기계학습 기반 자연어 처리(Natural Language Processing) 방법론 스터디입니다. 널리 사용되는 Word2Vec (2012)부터, 2019년도 연구 동향까지 다룹니다. 자세한 일정, 인원, 장소 및 내용은 아래와 같습니다.

• 일정: 2019.3.9~ (토) / 매주 토요일 오후 2시-5시 (주 1회, 3시간)
• 인원: 10명 내외 (현재 6명 참여)
• 장소 : 스테이지나인 삼성점 네이버 지도
• 참여가능한 배경지식 : 기계학습 기초 모두를 위한 딥러닝: 기본

Part 1 : Natural Language Processing Basics (Week 1-4)

Week 1 : Introduction / Overview

• Welcome!
• Introduction to Machine Learning & Deep Learning
• Introduction to Natural Language Processing
• Course Overview / Logistics

Week 2 : Word, Sentence, and Document Embedding

• Word2Vec Distributed Representations of Words and Phrases and their Compositionality (NIPS 2012) Link
• Word2Vec 2 Efficient Estimation of Word Representations in Vector Space (ICLR 2013) Link
• Glove GloVe: Global Vectors for Word Representation (EMNLP 2014) Link
• FastText Enriching Word Vectors with Subword Information (TACL 2017) Link
• FastText for Korean Subword-level Word Vector Representations for Korean (ACL 2018) Link
• Skip-thought Skip-Thought Vectors (NIPS 2015) Link
• Doc2Vec Distributed Representations of Sentences and Documents (ICML 2014) Link

Week 3 : Document Classification

• Document Classification Basics
• Recursive Neural Networks Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank (EMNLP 2013) Link
• CNN Convolutional Neural Networks for Sentence Classification (EMNLP 2014) Link
• CNN 2 Character-level Convolutional Networks for Text Classification (NIPS 2015) Link
• Attention Hierarchical Attention Networks for Document Classification (NAACL 2016) Link

Week 4 : Language Modeling & Transfer Learning

• Language Modeling Basics A Neural Probabilistic Language Model (JMLR 2003) Link
• CNN Character-Aware Neural Language Models (AAAI 2016) Link
• RNN Regularizing and Optimizing LSTM Language Models (ICLR 2018) Link
• ELMo Deep Contextualized Word Representations (NAACL 2018) Link
• ULMFiT Universal Language Model Fine-tuning for Text Classification (ACL 2018) Link

Part 2 : Advanced Topics in Natural Language Processing (Week 5-8)

Week 5 : Machine Translation

• Seq2Seq Basics
• GRU Learning Phrase Representations using RNN Encoder–Decoder for Statistical Machine Translation (EMNLP 2014) Link
• CNN Convolutional Sequence to Sequence Learning (ICML 2017) Link
• Transformer Attention Is All You Need (NIPS 2017) Link
• BERT BERT: Pre-training of Deep Bidirectional Transformers for Language Understanding (EMNLP 2018) Link
• Unsupervised MT Word Translation Without Parallel Data (ICLR 2018) Link

Week 6 : Conversation Modeling & Response Generation

• HRED Building End-To-End Dialogue Systems Using Generative Hierarchical Neural Network Models (AAAI 2016) Link
• Variational AutoEncoder (VAE) Basics Auto-Encoding Variational Bayes Link(tutorial)
• VHRED A Hierarchical Latent Variable Encoder-Decoder Model for Generating Dialogues (AAAI 2017) Link
• VHCR A Hierarchical Latent Structure for Variational Conversation Modeling (NAACL 2018) Link
• Generative Adversarial Network (GAN) Basics Generative Adversarial Nets (NIPS 2014) Link
• DialogueWAE DialogWAE: Multimodal Response Generation with Conditional Wasserstein Auto-Encoder (ICLR 2019) Link

Week 7 : Question & Answering / Summarization

• Memory Networks Memory Networks (ICLR 2015) Link
• End-To-End Memory Networks End-To-End Memory Networks (NIPS 2015) Link
• SQUAD 2.0 Know What You Don’t Know: Unanswerable Questions for SQuAD (ACL 2018) Link
• Pointer Networks Get To The Point: Summarization with Pointer-Generator Networks (ACL 2017) Link

Week 8 : Recent (2019) Trends in NLP / Other topics

• Hyperbolic Embeddings Poincaré GloVe: Hyperbolic Word Embeddings (ICLR 2019, accepted) Link
• SOTA Language Models: GPT-2 Language Models are Unsupervised Multitask Learners Link
• Computational Psychotherapy Large-scale Analysis of Counseling Conversations: An Application of Natural Language Processing to Mental Health (TACL 2016) Link
• Computational Psychotherapy 2 Conversation Model Fine-Tuning for Classifying Client Utterances in Counseling Dialogues Link
• Closing Remarks

Part 3 : Auto-correcting of Sentences via Out-of-vocubulary Generation for Korean Documents (Collaborative Project)

• TBD