Deep Learning & Natural Language Processing Reading Group (DL-NLP) | University of British Columbia (UBC)

If you are interested in receiving the meeting information and attending, please subscribe to the mail: [email protected]

Winter 2022

Time: Thursday 10:00 - 11:30 (Vancouver time)

Place: ICCS 104 & Zoom

Time	Topic/Paper	Presenter	Slides
Nov 17	Diffusion-LM Improves Controllable Text Generation FastSpeech 2: Fast and High-Quality End-to-End Text to Speech	Chiyu Zhang Abdellah EL Mekki	slides slides
Nov 10	Diffusion-LM Improves Controllable Text Generation	Chiyu Zhang	slides
Oct 13	Robust Speech Recognition via Large-Scale Weak Supervision	Peter Sullivan	slides
Oct 6	Cross-lingual Retrieval for Iterative Self-Supervised Training Bilingual Lexicon Induction via Unsupervised Bitext Construction and Word Alignment	Abdellah EL Mekki	slides
Sep 29	Research Discussion	All members	NA
Sep 22	Research Discussion	All members	NA
Sep 15	SEQZERO: Few-shot Compositional Semantic Parsing with Sequential Prompts and Zero-shot Models The Power of Scale for Parameter-Efficient Prompt Tuning	Wei-Rui Chen	slides

Summer 2022

Time: Monday & Wednesday 10:30am - 12:00pm (Vancouver time)

Place: ICCS 104 & Zoom

Time	Topic/Paper	Presenter	Slides
Aug 3	Research Discussion	All members	NA
Jul 18	Research Discussion	All members	NA
Jul 6	A Contrastive Framework for Neural Text Generation	Tawkat Islam Khondaker	Slides
Jul 4	Hands-on Tutorial for SpeechT5	Gagan Bhatia	Notebook
Jun 29	SpeechT5	Bashar Talafha	pending
Jun 27	Research Discussion	All members	NA
Jun 15	Engineering Tutorial for Contrastive Learning	Chiyu Zhang	Notebook
Jun 13	Language Model Prompting: Recent Research Work	Wei-Rui Chen	Slides
Jun 8	Introduction to Language Model Prompting	Wei-Rui Chen	Slides
Jun 3	Discussion of ACL-2022 papers	Chiyu Zhang, Tawkat Islam Khondaker	NA
Jun 1	Discussion of ACL-2022 papers	Ganesh Jawahar, Ife Adebara, Peter Sullivan	NA
May 16	Contrastive Learning with Adversarial Perturbations for Conditional Text Generation	Tawkat Islam Khondaker	Slides
May 11	Research Discussion	All members	NA
May 09	UCTopic: Unsupervised Contrastive Learning for Phrase Representations and Topic Mining	Tawkat Islam Khondaker	Slides
May 04	AutoBERT-Zero: Evolving BERT Backbone from Scratch	Ganesh Jawahar	Slides

Winter II 2022

Time: Monday & Wednesday 1:00pm - 2:30pm (Vancouver time)

Place: ICCS 104 & Zoom

Time	Topic/Paper	Presenter	Slides
Apr 25	Diacritization	Wei-Rui Chen	Slides
Apr 11	Gradient Matching for Domain Generalization	Tawkat Islam Khondaker	Slides
Mar 14	GreaseLM: Graph REASoning Enhanced Language Models for Question Answering	Tawkat Islam Khondaker	Slides
Mar. 9	data2vec: A General Framework for Self-supervised Learning in Speech, Vision and Language	Gagan Bhatia	Slides
Mar. 7	Diverse decoding methods for NLP (cont'd)	Farhan Samir, Ganesh Jawahar	Slides
Mar. 2	W2V-BERT and Conformer (Cont'd)	Peter Sullivan, Bashar Talafha	Slides1, Slides2
Feb. 28	Diverse decoding methods for NLP	Farhan Samir, Ganesh Jawahar	Slides
Feb. 23	W2V-BERT and Conformer	Peter Sullivan, Bashar Talafha	Slides1, Slides2
Feb. 14	Contrastive Learning in NLP	Chiyu Zhang	Slides1, Slides2
Feb. 2	Speech Features: Fundamental representations of spoken audio	Peter Sullivan, Bashar Talafha, Gagan Bhatia	Slides
Jan. 26	Speech Features: Fundamental representations of spoken audio	Peter Sullivan, Bashar Talafha, Gagan Bhatia	Slides
Jan. 31	Graph Representation Learning	Ganesh Jawahar, Weirui Chen, Tawkat Islam Khondaker	Slides
Jan. 24	Graph Representation Learning	Ganesh Jawahar, Weirui Chen, Tawkat Islam Khondaker	Slides

Winter I 2021

Time: Thursday 4:00pm - 5:30pm (Vancouver time)

Place: Totem Fields Studios 121 & Zoom

Time	Topic/Paper	Presenter	Slides
Sep 22	SimCSE: Simple Contrastive Learning of Sentence Embeddings	Chiyu Zhang	Slides
Oct 7	Machine Translation into Low-resource Language Varieties	Farhan Samir	Slides
Oct. 14	Micro-services and Docker Images	Bashar Talafha	Slides
Oct. 21	Graph Attention Networks	Tawkat Islam Khondaker	Slides
Oct. 28	Machine Translation on Low-resource/Indigenous Languages	Garrett Nicolai	--
Nov. 4	Graph Neural Networks for Natural Language Processing	Ganesh Jawahar	Slides
Nov. 18	Textless NLP: Generating expressive speech from raw audio	Peter Sullivan	Slides
Nov. 15	Graph Neural Networks for Natural Language Processing	Ganesh Jawahar	Slides
Dec. 2	Towards Trusted AI for Healthcare: from Theory to Practice	Xiaoxiao Li	Slides
Dec. 9	Contrastive Learning in NLP	Chiyu Zhang, Ganesh Jawahar	Slides1, Slides2

Summer I & II 2021

Time: Wenesday 4:15pm - 5:30pm (Vancouver time)

Place: Zoom

Time	Topic/Paper	Presenter	Slides
May 19	Arabic Speech Processing	Ahmed Ali
June 2	Arabic Speech Processing	Ahmed Ali
June 16	NAACL-2021 follow-up	Chiyu Zhang	Slides
June 23	NAACL-2021 follow-up	Ganesh Jawahar	Slides
June 28	Multi-Style Transfer with Discriminative Feedback on Disjoint Corpus	Chiyu Zhang	Slides
June 30	Unsupervised Speech Recognition	Peter Sulliva	Slides
July 7	Adapter Modules: [Parameter-Efficient Transfer Learning for NLP] [Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks]	Chiyu Zhang	Slides
July 14	DExperts: Decoding-Time Controlled Text Generation with Experts and Anti-Experts	Alisa Liu	Slides
July 21	Re-visit Adapter Modules: [Parameter-Efficient Transfer Learning for NLP] [Parameter-efficient Multi-task Fine-tuning for Transformers via Shared Hypernetworks]	Chiyu Zhang	Slides
Aug 11	[Hubert: How Much Can a Bad Teacher Benefit ASR Pre-Training?]	Peter Sullivan	Slides
Aug 18	[Charformer: Fast Character Transformers via Gradient-based Subword Tokenization]	Ganesh Jawahar	Slides

---

Winter II 2021

Time	Topic/Paper	Presenter	Slides
Jan. 13	K-BERT: Enabling Language Representation with Knowledge Graph, DALL·E	Chiyu Zhang, Muhammad Abdul-Mageed	[Slides]
Jan. 20	Visualizing Linguistic Diversity in Vancouver	Roger Yu-Hsiang Lo	[Slides]
Feb. 10	Train No Evil: Selective Masking for Task-Guided Pre-Training	Ganesh Jawahar	[Slides]
Feb. 17	As good as new. How to successfully recycle English GPT-2 to make models for other languages	Ganesh Jawahar	[Slides]
Feb. 24	CSP: Code-Switching Pre-training for Neural Machine Translation	Ife Adebara	[Slides]
Mar. 10	Language Model Prior for Low-Resource Neural Machine Translation	Ife Adebara	[Slides]
Mar. 17	chrF: character n-gram F-score for automatic MT evaluation CHRF ++: words helping character n-grams	Weirui Chen	[Slides]
Mar. 24	Controllable Text Simplification with Explicit Paraphrasing	Ganesh Jawahar	[Slides]
Mar. 31	Reformulating Unsupervised Style Transfer as Paraphrase Generation	Chiyu Zhang	[Slides]
Apr. 7	[If Beam Search is the Answer, What was the Question?] [Analyzing Multi-Head Self-Attention: Specialized Heads Do the Heavy Lifting, the Rest Can Be Pruned]	Dujian Ding	[Slides]
Apr. 14	Challenges of language technologies for the indigenous languages of the Americas Indigenous language technologies in Canada: Assessment, challenges, and successes	Weirui Chen	[Slides]
Apr. 21	[Reusing a Pretrained Language Model on Languages with Limited Corpora for Unsupervised NMT ] [Improving the Lexical Ability of Pretrained Language Models for Unsupervised Neural Machine Translation]	Ife Adebara	[Slides]
Apr. 28	EACL-2021 Paper Review		[ACL Anthology]
May 5	Multilingual Speech Translation with Efficient Finetuning of Pretrained Models	Peter Sulliva	[Slides]

---

Winter I 2020

Time	Topic/Paper	Presenter	Slides
Sep. 16	IARPA Babel corpora and implications for multilingual grapheme-to-phoneme and phone detection models	Peter Sullivan	[Slides]
Sep. 23	Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer	Muhammad Abdul-Mageed	[Slides]
Sep. 30	Exploring the Limits of Transfer Learning with a Unified Text-to-Text Transformer	Muhammad Abdul-Mageed	[Slides]
Oct. 7	Leveraging multi-parallelism in the Bible for cross-lingual learning	Garrett Nicolai	[Video]
Oct. 14	Uncertainty-aware Self-training for Few-shot Text Classification	Ganesh Jawahar	[Slides]
Oct. 21	ELECTRA: Pre-training Text Encoders as Discriminators Rather Than Generators	Chiyu Zhang	[Slides]
Oct. 28	Context-Aware Neural Machine Translation Decoding, Improving Context-aware Neural Machine Translation with Target-side Context	Ife Adebara	[Slides]
Nov. 4	Context-Aware Monolingual Repair for Neural Machine Translation	Ife Adebara	[Slides]
Nov. 4	HuggingFace Datasets tutorial	Ganesh Jawahar	[Notebook]
Nov. 11	Remembrance Day
Nov. 18	Self-Training for End-to-End Speech Translation, Self-training and Pre-training are Complementary for Speech Recognition	Peter Sullivan	[Slides]
Nov. 25	CAIDA "BC AI showcase" event	Ganesh, Abdul, Moatez and Chiyu	[Posters]
Dec. 2	EMNLP-2020 Paper Review	Ganesh Jawahar	[Slides]
Dec. 9	EMNLP-2020 Paper Review	Ganesh Jawahar, Chiyu Zhang	[Slides]
Dec. 16	Bilingual Acoustic Voice Variation is Similarly Structured Across Languages	Khia Johnson	[Slides]
Dec. 23	Twitter User Ideology Detection via BERT and Social Network	Chiyu Zhang	[Slides]
Dec. 30	Automatic Detection of Manipulated Text using Factual Knowledge	Ganesh Jawahar	[Slides]

---

Summer I & II 2020

Time	Topic/Paper	Presenter	Slides
May. 6	[Normalizing Flows: An Introduction and Review of Current Methods]	Sylvester Li	[slides]
May. 13	[Neural Text Generation with Unlikelihood Training]	Ganesh Jawahar	[slides]
June. 17	[Language Models are Few-Shot Learners]	Muhammad Abdul-Mageed	[slides]
July. 15	[Introduction to Automatic Speech Recognition and Speech Translation]	Peter Sullivan	[slides]
July. 22	[ACL 2020 Paper Review]	Peter Sullivan, Ganesh Jawahar	[Peter's slides], [Ganesh's slides]
July. 29	[ACL 2020 Paper Review]	Ganesh Jawahar	[Ganesh's slides]
Aug. 5	[ACL 2020 Paper Review]	Chiyu Zhang	[Chiyu's slides]
Aug. 26	Introducing the speech processing toolkit ESPnet	Peter Sullivan	[ESPnet documentation]

---

Spring / Winter II 2020

Time	Topic/Paper	Presenter	Slides
Jan. 9	[Programming tutorial of Pytorch]	Ganesh Jawahar	[slides]
Jan. 22	[WAV2VEC: UNSUPERVISED PRE-TRAINING FOR SPEECH RECOGNITION]	Muhammad M. Abdul-Mageed	[slides]
Jan. 29	[Massively Multilingual Sentence Embeddings for Zero-Shot Cross-Lingual Transfer and Beyond]	Ife Adebara	[slides]
Feb. 5	[Controllable Unsupervised Text Attribute Transfer via Editing Entangled Latent Representation Comprehension]	Chiyu Zhang	[slides]
Feb. 12	[Intro to Speech Processing]	Arian Shamei	[Notebook]
Feb. 19	Reading Break
Feb. 12	[Intro to Speech Processing]	Arian Shamei	[slides]
Mar. 4	[Effectiveness of self-supervised pre-training for speech recognition]	Muhammad M. Abdul-Mageed	[slides]
Mar. 11	[Compressive Transformers for Long-Range Sequence Modelling]	Arun Rajendran	[slides]
Mar. 18	Canceled
Mar. 15	[Analysis Methods in Neural Language Processing: A Survey]	Ganesh Jawahar	[slides]
Apr. 8	dialogue act recognition	Rudra Saha, Ife Adebara, and Chiyu Zhang	[slides]
Apr. 15	[GLTR: Statistical Detection and Visualization of Generated Text], [Human and Automatic Detection of Generated Text]	Ganesh Jawahar	[slides]
Apr. 22	[Developing Yorùbá-English Machine Translation Models and Assessing What They Learn About Yorùbá Bare Nouns]	Ife Adebara	[slides]

---

Fall / Winter I, 2019

Time	Topic/Paper	Presenter	Slides
Sept. 12	[Defending Against Neural Fake News]	Ganesh Jawahar	[slides]
Sept. 19	[Distilling Task-Specific Knowledge from BERT into Simple Neural Networks]	Chiyu Zhang	[slides]
Sept. 26	[Progressive Self-Supervised Attention Learning for Aspect-Level Sentiment Analysis]	Azadeh Hashemi	[slides]
Oct. 10	[Code-Switched Language Models Using Neural Based Synthetic Data from Parallel Sentences]	Kaili Vesik	[slides]
Oct. 17	[Learning Transferable Feature Representations Using Neural Networks]	Ganesh Jawahar	[slides]
Oct. 24	[Learning Transferable Feature Representations Using Neural Networks]	Ganesh Jawahar	[slides]
Oct. 31	[ALBERT] & [ROBERTA]	Rudra Saha	[slides]
Nov. 7	[Topic-Guided Variational Autoencoders for Text Generation]	Raymond Li	[slides]
Nov. 14	[BART: Denoising Sequence-to-Sequence Pre-training for Natural Language Generation, Translation, and Comprehension]	Arun Rajendran	[slides]

---

Spring / Winter II, 2018

Time	Topic/Paper	Presenter	Slides
Jan. 9	[Generative Adversarial Nets]	Pramit Saha	[slides]
Jan. 16	[Recipe1M: A Dataset for Learning Cross-Modal Embeddings for Cooking Recipes and Food Images]	Borna Ghotbi	[slides]
Jan. 18	[Bert: Pre-training of deep bidirectional transformers for language understanding]	Arun Rajendran	[slides]
Jan. 23	[Bert: Pre-training of deep bidirectional transformers for language understanding]	Arun Rajendran	[slides]
Jan. 25	[Multiple-Attribute Text Style Transfer]	Ife Adebara	[slides]
Feb. 6	[Unsupervised Machine Translation Using Monolingual Corpora Only]	Michael Przystupa	[slides]
Feb. 8	[OpenMT Code Tutorial]	Michael Przystupa	[slides]
Feb. 13	[Graph Convolutional Networks for Text Classification]	Rudra Saha	[slides]
Feb. 15	[Auto-Encoding Variational Bayes]	Mohit Bajaj	[slides]
Feb. 27	[Unsupervised Image Captioning]	Borna Ghotbi	[slides]
March 5	[When Convolutions Meet Reality -- Structured Neural Networks in Vision and Graphics]	Andreas Lehrmann (Facebook AI Research)	[3:00PM - 4:00PM; X836]
March 7	[Artificial Intelligence: From Analog to Digital and Back]	George Dyson	[3:30PM – 4:30PM; Forest Sciences Centre, Room 1221- 2424 Main Mall, Zone 4]
March 8	[Transformer-XL: Attentive Language Models Beyond a Fixed-Length Context]	Chiyu Zhang	[slides]
March 13	[Variational Neural Machine Translation]	Michael Przystupa	[slides]
March 15	[Language Models are Unsupervised Multitask Learners]	Arun Rajendran	[slides]
March 20	[Video and Language]	Jiebo Luo (University of Rochester)	[2:00PM - 3:30PM; ICCS146]
March 22	[Graph Attention Networks]	Rudra Saha	[slides]
April 14	[Adversarial Autoencoders]	Mohit Bajaj	[slides]
June 26	[Normalizing Flows]	Michael Przystupa	[slides]
July 4	[XLNet: Generalized Autoregressive Pretraining for Language Understanding]	Arun Rajendran	[slides]
July 11	[Linguistic Variation]	Ife Adebara	[slides]
July 25	[Analyzing Polarization in Social Media: Method and Application to Tweets on 21 Mass Shootings]	Peter Sullivan	[slides]
August 1	[Analysis and detection of information types of open source software issue discussions]	Arthur Marques	[slides]
August 8	[PyTorch Seq2Seq]	Rudra Saha	[slides]

---

Fall / Winter I, 2018

Time	Topic/Paper	Presenter	Slides
Oct. 3	[A large annotated corpus for learning natural language inference]	Mohamed Elaraby	[slides]
Oct. 10	[GLUE: A Multi-Task Benchmark and Analysis Platformfor Natural Language Understanding]	Ife Adebara	[slides]
Oct. 17	[Towards Automatic Speech Identification from Vocal Tract Shape Dynamics in Real-time MRI]	Pramit Saha	[slides]
Oct. 24	[Evaluating Content Selection in Summarization: The Pyramid Method]	Arthur Marques	[slides]
Oct. 31	[A Stylometric Inquiry into Hyperpartisan and Fake News]	Chiyu Zhang	[slides]
Nov. 7	[Attention Is All You Need]	Arun Rajendran	[slides]
Nov. 14	[PartI: Phrase-Based & Neural Unsupervised Machine Translation]	Michael Przystupa	[slides]
Nov. 21	[Part II: Phrase-Based & Neural Unsupervised Machine Translation]	Michael Przystupa	[slides]
Nov. 28	[XNLI: Evaluating Cross-lingual Sentence Representations]	Ife Adebara	[slides]
Dec. 12	[Word Translation Without Parallel Data]	Rudra Saha	[slides]