October 31, 2018
Reference-less Quality Estimation of Text Simplification Systems
International Conference on Natural Language Generation (INLG)
In this paper, we compare multiple approaches to reference-less quality estimation of sentence-level text simplification systems, based on the dataset used for the QATS 2016 shared task. We distinguish three different dimensions: grammaticality, meaning preservation and simplicity. We show that n-gram-based MT metrics such as BLEU and METEOR correlate the most with human judgment of grammaticality and meaning preservation, whereas simplicity is best evaluated by basic length-based metrics.
By: Louis Martin, Samuel Humeau, Pierre-Emmanuel Mazaré, Antoine Bordes, Éric de La Clergerie, Benoit Steiner
Facebook AI Research
Natural Language Processing & Speech