Explore the latest research from Facebook

All Publications

January 9, 2022 Xiaohui Zhang, Frank Zhang, Chunxi Liu, Kjell Schubert, Julian Chan, Pradyot Prakash, Jun Liu, Ching-Feng Yeh, Fuchun Peng, Yatharth Saraf, Geoffrey Zweig
Paper

Benchmarking LF-MMI, CTC and RNN-T Criteria for Streaming ASR

In this work, to measure the accuracy and efficiency for a latency-controlled streaming automatic speech recognition (ASR) application, we perform comprehensive evaluations on three popular training criteria: LF-MMI, CTC and RNN-T.
Paper
December 13, 2021 Vimal Manohar, Tatiana Likhomanenko, Qiantong Xu, Wei-Ning Hsu, Ronan Collobert, Yatharth Saraf, Geoffrey Zweig, Abdelrahman Mohamed
Paper

Kaizen: Continuously Improving Teacher Using Exponential Moving Average For Semi-supervised Speech Recognition

In this paper, we introduce the Kaizen framework that uses a continuously improving teacher to generate pseudo-labels for semi-supervised speech recognition (ASR). The proposed approach uses a teacher model which is updated as the exponential moving average (EMA) of the student model parameters.
Paper
December 9, 2021 Alexei Baevski, Wei-Ning Hsu, Alexis Conneau, Michael Auli
Paper

Unsupervised Speech Recognition

This paper describes wav2vec-U, short for wav2vec Unsupervised, a method to train speech recognition models without any labeled data.
Paper
November 10, 2021 Santosh Gondi, Vineel Pratap
Paper

Performance and Efficiency Evaluation of ASR Inference on the Edge

On-device ASR can also lead to a more sustainable solution by considering the energy vs. accuracy trade-off and choosing right model for specific use cases/applications of the product. Hence, in this paper we evaluate energy-accuracy trade-off of ASR with a typical transformer based speech recognition model on an edge device.
Paper
November 10, 2021 Marzieh Saeidi, Majid Yazdani, Andreas Vlachos
Paper

Cross-Policy Compliance Detection via Question Answering

In this paper we propose to address policy compliance detection via decomposing it into question answering, where questions check whether the conditions stated in the policy apply to the scenario, and an expression tree combines the answers to obtain the label. Despite the initial upfront annotation cost, we demonstrate that this approach results in better accuracy, especially in the cross-policy setup where the policies during testing are unseen in training.
Paper
November 10, 2021 Rahma Chaabouni, Roberto Dessì, Eugene Kharitonov
Paper

Can Transformers Jump Around Right in Natural Language? Assessing Performance Transfer from SCAN

We present several focused modifications of Transformer that greatly improve generalization capabilities on SCAN and select one that remains on par with a vanilla Transformer on a standard machine translation (MT) task. Next, we study its performance in low-resource settings and on a newly introduced distribution-shifted EnglishFrench translation task.
Paper
November 9, 2021 Verna Dankers, Anna Langedijk, Kate McCurdy, Adina Williams, Dieuwke Hupkes
Paper

Generalising to German Plural Noun Classes, from the Perspective of a Recurrent Neural Network

Here, in line with that tradition, we explore how recurrent neural networks acquire the complex German plural system and reflect upon how their strategy compares to human generalisation and rule-based models of this system.
Paper
November 8, 2021 Guillaume Wenzek, Vishrav Chaudhary, Angela Fan, Sahir Gomez, Naman Goyal, Somya Jain, Douwe Kiela, Tristan Thrush, Francisco Guzmán
Paper

Findings of the WMT 2021 Shared Task on Large-Scale Multilingual Machine Translation

We present the results of the first task on Large-Scale Multilingual Machine Translation. The task consists on the many-to-many evaluation of a single model across a variety of source and target languages.
Paper
November 8, 2021 Baptiste Rozière, Marie-Anne Lachaux, Marc Szafraniec, Guillaume Lample
Paper

DOBF: A Deobfuscation Pre-Training Objective for Programming Languages

In this paper, we introduce a new pre-training objective, DOBF, that leverages the structural aspect of programming languages and pre-trains a model to recover the original version of obfuscated source code.
Paper
November 8, 2021 Lucia Specia, Frédéric Blain, Marina Fomicheva, Chrysoula Zerva, Zhenhao Li, Vishrav Chaudhary, André F. T. Martins
Paper

Findings of the WMT 2021 Shared Task on Quality Estimation

We report the results of the WMT 2021 shared task on Quality Estimation, where the challenge is to predict the quality of the output of neural machine translation systems at the word and sentence levels.
Paper