ParlAI, is a unified platform, implemented in Python, for training and evaluating AI models on a variety of openly available dialog datasets using open-sourced learning agents.
Its goal is to provide researchers:
- A unified framework for sharing, training and testing dialog models
- Multi-task training over many datasets at once
- Seamless integration of Amazon Mechanical Turk for data collection and human evaluation
Over 20 tasks are currently supported, including popular datasets such as SQuAD, bAbI tasks, MSMARCO, MCTest, [WikiQA, WebQuestions, SimpleQuestions, WikiMovies, QACNN & QADailyMail, CBT, BookTest, bAbI Dialog tasks, Ubuntu Dialog, OpenSubtitles, Cornell Movie, VQA-COCO2014, VisDial and CLEVR. See here for the current complete task list.
Our aim is for the number of tasks and agents that train on them to grow in a community-based way.
ParlAI is described in the following paper: ParlAI: A Dialog Research Software Platform, arXiv:1705.06476.