All Research Areas
Research Areas
Year Published

198 Results

August 6, 2017

Unsupervised Learning by Predicting Noise

International Conference on Machine Learning (ICML)

Convolutional neural networks provide visual features that perform well in many computer vision applications. However, training these networks requires large amounts of supervision; this paper introduces a generic framework to train such networks, end-to-end, with no supervision. We propose to fix a set of target representations, called Noise As Targets (NAT), and to constrain the deep features to align to them.

By: Piotr Bojanowski, Armand Joulin
August 6, 2017

Efficient Softmax Approximation for GPUs

International Conference on Machine Learning (ICML)

We propose an approximate strategy to efficiently train neural network based language models over very large vocabularies.

By: Edouard Grave, Armand Joulin, Moustapha Cisse, David Grangier, Hervé Jégou
August 6, 2017

Convolutional Sequence to Sequence Learning

International Conference on Machine Learning (ICML)

We introduce an architecture based entirely on convolutional neural networks.

By: Jonas Gehring, Michael Auli, David Grangier, Denis Yarats, Yann N. Dauphin
August 6, 2017

Language Modeling with Gated Convolutional Networks

International Conference on Machine Learning (ICML)

The pre-dominant approach to language modeling to date is based on recurrent neural networks. Their success on this task is often linked to their ability to capture unbounded context. In this paper we develop a finite context approach through stacked convolutions, which can be more efficient since they allow parallelization over sequential tokens.

By: Yann Dauphin, Angela Fan, Michael Auli, David Grangier
August 6, 2017

Wasserstein Generative Adversarial Networks

International Conference on Machine Learning (ICML)

We introduce a new algorithm named WGAN, an alternative to traditional GAN training. In this new model, we show that we can improve the stability of learning, get rid of problems like mode collapse, and provide meaningful learning curves useful for debugging and hyperparameter searches.

By: Martin Arjovsky, Soumith Chintala, Leon Bottou
July 31, 2017

Learning Multilingual Joint Sentence Embeddings with Neural Machine Translation

ACL workshop on Representation Learning for NLP (ACL)

In this paper, we use the framework of neural machine translation to learn joint sentence representations across six very different languages. Our aim is that a representation which is independent of the language, is likely to capture the underlying semantics.

By: Holger Schwenk, Matthijs Douze
July 31, 2017

Enriching Word Vectors with Subword Information

TACL, Association for Computational Linguistics (ACL 2017)

In this paper, we propose a new approach based on the skipgram model, where each word is represented as a bag of character n-grams.

By: Piotr Bojanowski, Edouard Grave, Armand Joulin, Tomas Mikolov
July 30, 2017

Reading Wikipedia to Answer Open-Domain Questions

Association for Computational Linguistics (ACL 2017)

This paper proposes to tackle open- domain question answering using Wikipedia as the unique knowledge source: the answer to any factoid question is a text span in a Wikipedia article.

By: Danqi Chen, Adam Fisch, Jason Weston, Antoine Bordes
July 30, 2017

Automatically Generating Rhythmic Verse with Neural Networks

Association for Computational Linguistics (ACL 2017)

We propose two novel methodologies for the automatic generation of rhythmic poetry in a variety of forms.

By: Jack Hopkins, Douwe Kiela
July 30, 2017

A Convolutional Encoder Model for Neural Machine Translation

Association for Computational Linguistics 2017 (ACL 2017)

The prevalent approach to neural machine translation relies on bi-directional LSTMs to encode the source sentence. We present a faster and simpler architecture based on a succession of convolutional layers.

By: Jonas Gehring, Michael Auli, David Grangier, Yann Dauphin