Research Area
Year Published

170 Results

December 2, 2018

The Description Length of Deep Learning Models

Neural Information Processing Systems (NeurIPS)

We demonstrate experimentally the ability of deep neural networks to compress the training data even when accounting for parameter encoding. The compression viewpoint originally motivated the use of variational methods in neural networks (Hinton and Van Camp, 1993; Schmidhuber, 1997).

By: Léonard Blier, Yann Ollivier

November 30, 2018

A Block Coordinate Ascent Algorithm for Mean-Variance Optimization

Neural Information Processing Systems (NeurIPS)

Risk management in dynamic decision problems is a primary concern in many fields, including financial investment, autonomous driving, and healthcare. The mean-variance function is one of the most widely used objective functions in risk management due to its simplicity and interpretability. Existing algorithms for mean-variance optimization are based on multi-time-scale stochastic approximation, whose learning rate schedules are often hard to tune, and have only asymptotic convergence proof. In this paper, we develop a model-free policy search framework for mean-variance optimization with finite-sample error bound analysis (to local optima).

By: Tengyang Xie, Bo Liu, Yangyang Xu, Mohammad Ghavamzadeh, Yinlam Chow, Daoming Lyu, Daesub Yoon

November 30, 2018

A Lyapunov-based Approach to Safe Reinforcement Learning

Neural Information Processing Systems (NeurIPS)

To incorporate safety in RL, we derive algorithms under the framework of constrained Markov decision processes (CMDPs), an extension of the standard Markov decision processes (MDPs) augmented with constraints on expected cumulative costs. Our approach hinges on a novel Lyapunov method.

By: Yinlam Chow, Ofir Nachum, Mohammad Ghavamzadeh, Edgar Duenez-Guzman

November 27, 2018

AnoGen: Deep Anomaly Generator

Outlier Detection De-constructed (ODD) Workshop

Motivated by the continued success of Variational Auto-Encoders (VAE) and Generative Adversarial Networks (GANs) to produce realistic-looking data we provide a platform to generate a realistic time-series with anomalies called AnoGen.

By: Nikolay Laptev

November 24, 2018

On Periodic Functions as Regularizers for Quantization of Neural Networks

ArXiv

Deep learning models have been successfully used in computer vision and many other fields. We propose an unorthodox algorithm for performing quantization of the model parameters.

By: Maxim Naumov, Utku Diril, Jongsoo Park, Benjamin Ray, Jedrzej Jablonski, Andrew Tulloch

November 2, 2018

Jump to better conclusions: SCAN both left and right

Empirical Methods in Natural Language Processing (EMNLP)

Lake and Baroni (2018) recently introduced the SCAN data set, which consists of simple commands paired with action sequences and is intended to test the strong generalization abilities of recurrent sequence-to-sequence models. Their initial experiments suggested that such models may fail because they lack the ability to extract systematic rules. Here, we take a closer look at SCAN and show that it does not always capture the kind of generalization that it was designed for.

By: Joost Bastings, Marco Baroni, Jason Weston, Kyunghyun Cho, Douwe Kiela

November 2, 2018

Dynamic Meta-Embeddings for Improved Sentence Representations

Empirical Methods in Natural Language Processing (EMNLP)

While one of the first steps in many NLP systems is selecting what pre-trained word embeddings to use, we argue that such a step is better left for neural networks to figure out by themselves.

By: Douwe Kiela, Changhan Wang, Kyunghyun Cho

November 2, 2018

Do explanations make VQA models more predictable to a human?

Empirical Methods in Natural Language Processing (EMNLP)

A rich line of research attempts to make deep neural networks more transparent by generating human-interpretable ‘explanations’ of their decision process, especially for interactive tasks like Visual Question Answering (VQA). In this work, we analyze if existing explanations indeed make a VQA model – its responses as well as failures – more predictable to a human.

By: Arjun Chandrasekaran, Viraj Prabhu, Deshraj Yadav, Prithvijit Chattopadhyay, Devi Parikh

November 1, 2018

How agents see things: On visual representations in an emergent language game

Empirical Methods in Natural Language Processing (EMNLP)

There is growing interest in the language developed by agents interacting in emergent-communication settings. Earlier studies have focused on the agents’ symbol usage, rather than on their representation of visual input. In this paper, we consider the referential games of Lazaridou et al. (2017), and investigate the representations the agents develop during their evolving interaction.

By: Diane Bouchacourt, Marco Baroni

November 1, 2018

Non-Adversarial Unsupervised Word Translation

Empirical Methods in Natural Language Processing (EMNLP)

In this paper, we make the observation that two sufficiently similar distributions can be aligned correctly with iterative matching methods.

By: Yedid Hoshen, Lior Wolf