Filter by Research Area
Filter by Research Area
Year Published

557 Results

April 30, 2018

Multi-Scale Dense Networks for Resource Efficient Image Classification

International Conference on Learning Representations (ICLR)

In this paper we investigate image classification with computational resource limits at test time. Two such settings are: 1. anytime classification, where the network’s prediction for a test example is progressively updated, facilitating the output of a prediction at any time; and 2. budgeted batch classification, where a fixed amount of computation is available to classify a set of examples that can be spent unevenly across “easier” and “harder” inputs.

By: Gao Huang, Danlu Chen, Tianhong Li, Felix Wu, Laurens van der Maaten, Kilian Weinberger
April 30, 2018

Enhancing The Reliability of Out-of-distribution Image Detection in Neural Networks

International Conference on Learning Representations (ICLR)

We consider the problem of detecting out-of-distribution images in neural networks. We propose ODIN, a simple and effective method that does…

By: Shiyu Liang, Yixuan Li, R. Srikant
April 30, 2018

An Evaluation of Fisher Approximations Beyond Kronecker Factorization

International Conference on Learning Representations (ICLR)

We study two coarser approximations on top of a Kronecker factorization (K-FAC) of the Fisher Information Matrix, to scale up Natural Gradient to deep and wide Convolutional Neural Networks (CNNs). The first considers the feature maps as spatially uncorrelated while the second considers only correlations among groups of channels

By: Cesar Laurent, Thomas George, Xavier Bouthillier, Nicolas Ballas, Pascal Vincent
April 30, 2018

Can Recurrent Neural Networks Wrap Time?

International Conference on Learning Representations (ICLR)

We prove that learnable gates in a recurrent model formally provide quasiinvariance to general time transformations in the input data. We recover part of the LSTM architecture from a simple axiomatic approach.

By: Corentin Tallec, Yann Ollivier
April 30, 2018

Empirical Analysis of the Hessian of Over-Parametrized Neural Networks

International Conference on Learning Representations (ICLR)

We study the properties of common loss surfaces through their Hessian matrix. In particular, in the context of deep learning, we empirically show that the spectrum of the Hessian is composed of two parts: (1) the bulk centered near zero, (2) and outliers away from the bulk. We present numerical evidence and mathematical justifications to the following conjectures laid out by Sagun et al. (2016): Fixing data, increasing the number of parameters merely scales the bulk of the spectrum; fixing the dimension and changing the data (for instance adding more clusters or making the data less separable) only affects the outliers.

By: Levent Sagun, Utku Evci, V. Ugur Güney, Yann Dauphin, Leon Bottou
April 30, 2018

Parametric Adversarial Divergences are Good Task Losses for Generative Modeling

International Conference on Learning Representations (ICLR)

In this paper, we argue that adversarial learning, pioneered with generative adversarial networks (GANs), provides an interesting framework to implicitly define more meaningful task losses for unsupervised tasks, such as for generating “visually realistic” images. By relating GANs and structured prediction under the framework of statistical decision theory, we put into light links between recent advances in structured prediction theory and the choice of the divergence in GANs.

By: Gabriel Huang, Hugo Berard, Ahmed Touati, Gauthier Gidel, Pascal Vincent, Simon Lacoste-Julien
April 30, 2018

mixup: Beyond Empirical Risk Minimization

International Conference on Learning Representations (ICLR)

In this work, we propose mixup, a simple learning principle to alleviate these issues. In essence, mixup trains a neural network on convex combinations of pairs of examples and their labels. 

By: Hongyi Zhang, Moustapha Cisse, Yann Dauphin, David Lopez-Paz
April 30, 2018

Emergent Translation in Multi-Agent Communication

International Conference on Learning Representations (ICLR)

In this work, we propose a communication game where two agents, native speakers of their own respective languages, jointly learn to solve a visual referential task. We find that the ability to understand and translate a foreign language emerges as a means to achieve shared goals.

By: Jason Lee, Kyunghyun Cho, Jason Weston, Douwe Kiela
April 30, 2018

Consequentialist Conditional Cooperation in Social Dilemmas with Imperfect Information

International Conference on Learning Representations (ICLR)

Social dilemmas, where mutual cooperation can lead to high payoffs but participants face incentives to cheat, are ubiquitous in multi-agent interaction. We wish to construct agents that cooperate with pure cooperators, avoid exploitation by pure defectors, and incentivize cooperation from the rest. We show how to construct such strategies using deep reinforcement learning techniques and demonstrate, both analytically and experimentally, that they are effective in social dilemmas beyond simple matrix games.

By: Alexander Peysakhovich, Adam Lerer
April 30, 2018

Learning One-hidden-layer Neural Networks with Landscape Design

International Conference on Learning Representations (ICLR)

We consider the problem of learning a one-hidden-layer neural network: we assume the input x ∈ Rd is from Gaussian…

By: Rong Ge, Jason D. Lee, Tengyu Ma