Filter by Research Area
Filter by Research Area
Year Published

557 Results

May 8, 2018

Optimization Methods for Large-Scale Machine Learning

SIAM Review

This paper provides a review and commentary on the past, present, and future of numerical optimization algorithms in the context of machine learning applications.

By: Leon Bottou, Frank E. Curtis, Jorge Nocedal
May 7, 2018

Advances in Pre-Training Distributed Word Representations

Language Resources and Evaluation Conference (LREC)

In this paper, we show how to train high-quality word vector representations by using a combination of known tricks that are however rarely used together.

By: Tomas Mikolov, Edouard Grave, Piotr Bojanowski, Christian Puhrsch, Armand Joulin
May 7, 2018

A Corpus for Multilingual Document Classification in Eight Languages

Language Resources and Evaluation Conference (LREC)

In this paper, we propose a new subset of the Reuters corpus with balanced class priors for eight languages. By adding Italian, Russian, Japanese and Chinese, we cover languages which are very different with respect to syntax, morphology, etc. We provide strong baselines for all language transfer directions using multilingual word and sentence embeddings respectively. Our goal is to offer a freely available framework to evaluate cross-lingual document classification, and we hope to foster by these means, research in this important area.

By: Holger Schwenk, Xian Li
May 2, 2018

Exploring the Limits of Weakly Supervised Pretraining

ArXiv

In this paper, we present a unique study of transfer learning with large convolutional networks trained to predict hashtags on billions of social media images.

By: Dhruv Mahajan, Ross Girshick, Vignesh Ramanathan, Kaiming He, Manohar Paluri, Yixuan Li, Ashwin Bharambe, Laurens van der Maaten
April 30, 2018

Residual Connections Encourage Iterative Inference

International Conference on Learning Representations (ICLR)

Residual networks (Resnets) have become a prominent architecture in deep learning. However, a comprehensive understanding of Resnets is still a topic of ongoing research. A recent view argues that Resnets perform iterative refinement of features. We attempt to further expose properties of this aspect.

By: Stanislaw Jastrzebski, Devansh Arpit, Nicolas Ballas, Vikas Verma, Tong Che, Yoshua Bengio
April 30, 2018

Building Generalizable Agents with a Realistic and Rich 3D Environment

International Conference on Learning Representations (ICLR)

Teaching an agent to navigate in an unseen 3D environment is a challenging task, even in the event of simulated environments. To generalize to unseen environments, an agent needs to be robust to low-level variations (e.g. color, texture, object changes), and also high-level variations (e.g. layout changes of the environment). To improve overall generalization, all types of variations in the environment have to be taken under consideration via different level of data augmentation steps.

By: Yi Wu, Yuxin Wu, Georgia Gkioxari, Yuandong Tian
April 30, 2018

Word Translation Without Parallel Data

International Conference on Learning Representations (ICLR)

In this work, we show that we can build a bilingual dictionary between two languages without using any parallel corpora, by aligning monolingual word embedding spaces in an unsupervised way.

By: Alexis Conneau, Guillaume Lample, Marc'Aurelio Ranzato, Ludovic Denoyer, Hervé Jégou
April 30, 2018

VoiceLoop: Voice Fitting and Synthesis via a Phonolgoical Loop

International Conference on Learning Representations (ICLR)

We present a new neural text to speech (TTS) method that is able to transform text to speech in voices that are sampled in the wild. Unlike other systems, our solution is able to deal with unconstrained voice samples and without requiring aligned phonemes or linguistic features. The network architecture is simpler than those in the existing literature and is based on a novel shifting buffer working memory. The same buffer is used for estimating the attention, computing the output audio, and for updating the buffer itself.

By: Yaniv Taigman, Lior Wolf, Adam Polyak, Eliya Nachmani
April 30, 2018

Countering Adversarial Images Using Input Transformations

International Conference on Learning Representations (ICLR)

This paper investigates strategies that defend against adversarial-example attacks on image-classification systems by transforming the inputs before feeding them to the system. Specifically, we study applying image transformations such as bit-depth reduction, JPEG compression, total variance minimization, and image quilting before feeding the image to a convolutional network classifier.

By: Chuan Guo, Mayank Rana, Moustapha Cisse, Laurens van der Maaten
April 30, 2018

Intrinsic Motivation and Automatic Curricula via Asymmetric Self-Play

International Conference on Learning Representations (ICLR)

We describe a simple scheme that allows an agent to learn about its environment in an unsupervised manner. Our scheme pits two versions of the same agent, Alice and Bob, against one another.

By: Sainbayar Sukhbaatar, Zeming Lin, Ilya Kostrikov, Gabriel Synnaeve, Arthur Szlam, Rob Fergus