ReproGen Shared Task

Results Report

The 2022 ReproGen Shared Task on Reproducibility of Evaluations in NLG: Overview and Results
Anya Belz, Anastasia Shimorina, Maja Popovič, and Ehud Reiter

Track A

  • Reproducibility of Exploring Neural Text Simplification Models: A Review
    Mohammad Arvan, Luís Pina, and Natalie Parde

  • A reproduction study of methods for evaluating dialogue system output: Replicating Santhanam and Shaikh (2019)
    Anouck Braggaar, Frédéric Tomas, Peter Blomsma, Saar Hommes, Nadine Braun, Emiel van Miltenburg, Chris van der Lee, Martijn Goudbeek, and Emiel Krahmer

  • Reproducing a Manual Evaluation of Simplicity in Text Simplification System Outputs
    Maja Popovič, Sheila Castilho, Rudali Huidrom, and Anya Belz

Track B

  • Two Reproductions of a Human-Assessed Comparative Evaluation of a Semantic Error Detection System
    Rudali Huidrom, Ondřej Dušek, Zdeněk Kasner, Thiago Castro Ferreira, and Anya Belz

  • The Accuracy Evaluation Shared Task as a Retrospective Reproduction Study
    Craig Thomson and Ehud Reiter