sangramga / jsalt19-gluonnlp

JSALT 2019 Montréal: Dive into Deep Learning for Natural Language Processing

JSALT 2019 Montréal: Dive into Deep Learning for Natural Language Processing

Time: Friday, June 14, 2019
Location: Ecole de Technology Superieure in Montréal, Canada

Presenter: Leonard Lausen, Haibin Lin

Abstract

Deep learning has rapidly emerged as the most prevalent approach for training predictive models for large-scale machine learning problems. Advances in the neural networks also push the limits of available hardware, requiring specialized frameworks optimized for GPUs and distributed cloud-based training. Moreover, especially in natural language processing (NLP), models contain a variety of moving parts: character-based encoders, pre-trained word embeddings, long-short term memory (LSTM) cells, transformer layers, and beam search for decoding sequential outputs, among others.

This introductory and hands-on tutorial walks you through the fundamentals of machine learning and deep learning with a focus on NLP. We start off with a crash course on deep learning with Gluon, covering data, automatic differentiation, and various model architectures such as convolutional, recurrent, and attentional neural networks. Then, we dive into how context-free and contexual representations help various NLP domains. Throughout the tutorial, we start off from the basic classification problem, and progress into how it can be structured to solve various NLP problems such as sentiment analysis, question answering, and natural language generation.

Agenda

Time	Title
8:30-9:00	Continental Breakfast
9:00-9:45	Introduction and Setup
9:45-10:30	Neural Networks 101
10:30-10:45	Break
10:45-11:15	Machine Learning Basics
11:15-11:45	Context-free Representations for Language
11:45-12:15	Convolutional Neural Networks
12:15-13:15	Lunch Break
13:15-14:00	Recurrent Neural Networks
14:00-14:45	Attention Mechanism and Transformer
14:45-15:00	Coffee Break
15:00-16:15	Contextual Representations for Language
16:15-17:00	Language Generation