Research Area
Year Published

300 Results

December 8, 2018

From Satellite Imagery to Disaster Insights

AI for Social Good Workshop at NeurIPS 2018

The use of satellite imagery has become increasingly popular for disaster monitoring and response. After a disaster, it is important to prioritize rescue operations, disaster response and coordinate relief efforts. These have to be carried out in a fast and efficient manner since resources are often limited in disaster affected areas and it’s extremely important to identify the areas of maximum damage. However, most of the existing disaster mapping efforts are manual which is time-consuming and often leads to erroneous results.

By: Jigar Doshi, Saikat Basu, Guan Pang

December 8, 2018

SGD Implicitly Regularizes Generalization Error

Integration of Deep Learning Theories Workshop at NeurIPS

We derive a simple and model-independent formula for the change in the generalization gap due to a gradient descent update. We then compare the change in the test error for stochastic gradient descent to the change in test error from an equivalent number of gradient descent updates and show explicitly that stochastic gradient descent acts to regularize generalization error by decorrelating nearby updates.

By: Daniel A. Roberts

December 7, 2018

Stochastic Gradient Push for Distributed Deep Learning

Systems for Machine Learning Workshop at NeurIPS 2018

Large mini-batch parallel SGD is commonly used for distributed training of deep networks. Approaches that use tightly-coupled exact distributed averaging based on AllReduce are sensitive to slow nodes and high-latency communication. In this work we show the applicability of Stochastic Gradient Push (SGP) for distributed training.

By: Mahmoud Assran, Nicolas Loizou, Nicolas Ballas, Mike Rabbat

December 7, 2018

Rethinking floating point for deep learning

Systems for Machine Learning Workshop at NeurIPS 2018

We improve floating point to be more energy efficient than equivalent bit width integer hardware on a 28 nm ASIC process while retaining accuracy in 8 bits with a novel hybrid log multiply/linear add, Kulisch accumulation and tapered encodings from Gustafson’s posit format.

By: Jeff Johnson

December 7, 2018

Causality in Physics and Effective Theories of Agency

Causal Learning Workshop at NeurIPS

We propose to combine reinforcement learning and theoretical physics to describe effective theories of agency. This involves understanding the connection between the physics notion of causality and how intelligent agents can arise as a useful effective description within some environments.

By: Daniel A. Roberts, Max Kleiman-Weiner

December 6, 2018

Fast Approximate Natural Gradient Descent in a Kronecker-factored Eigenbasis

Neural Information Processing Systems (NeurIPS)

Optimization algorithms that leverage gradient covariance information, such as variants of natural gradient descent (Amari, 1998), offer the prospect of yielding more effective descent directions. For models with many parameters, the covariance matrix they are based on becomes gigantic, making them inapplicable in their original form.

By: Thomas George, Cesar Laurent, Xavier Bouthillier, Nicolas Ballas, Pascal Vincent

December 4, 2018

Non-Adversarial Mapping with VAEs

Neural Information Processing Systems (NeurIPS)

The study of cross-domain mapping without supervision has recently attracted much attention. Much of the recent progress was enabled by the use of adversarial training as well as cycle constraints. The practical difficulty of adversarial training motivates research into non-adversarial methods.

By: Yedid Hoshen

December 4, 2018

A2-Nets: Double Attention Networks

Neural Information Processing Systems (NeurIPS)

Learning to capture long-range relations is fundamental to image/video recognition. Existing CNN models generally rely on increasing depth to model such relations which is highly inefficient. In this work, we propose the “double attention block”, a novel component that aggregates and propagates informative global features from the entire spatio-temporal space of input images/videos, enabling subsequent convolution layers to access features from the entire space efficiently.

By: Yunpeng Chen, Yannis Kalantidis, Jianshu Li, Shuicheng Yan, Jiashi Feng

December 4, 2018

Improving Semantic Parsing for Task Oriented Dialog

Conversational AI Workshop at NeurIPS 2018

Semantic parsing using hierarchical representations has recently been proposed for task oriented dialog with promising results. In this paper, we present three different improvements to the model: contextualized embeddings, ensembling, and pairwise re-ranking based on a language model.

By: Arash Einolghozati, Panupong Pasupat, Sonal Gupta, Rushin Shah, Mrinal Mohit, Mike Lewis, Luke Zettlemoyer

December 3, 2018

Explore-Exploit: A Framework for Interactive and Online Learning

Systems for Machine Learning Workshop at NeurIPS 2018

We present Explore-Exploit: a framework designed to collect and utilize user feedback in an interactive and online setting that minimizes regressions in end-user experience. This framework provides a suite of online learning operators for various tasks such as personalization ranking, candidate selection and active learning.

By: Honglei Liu, Anuj Kumar, Wenhai Yang, Benoit Dumoulin