Submission

ReproGen Shared task

What to submit

• 4-page or 8-page report of the reproduction study including:
  - description of how the study was conducted
  - link to where resources used in reproduction study can be obtained
  - overview of respects in which the reproduction was the same, respects in which it was different, and respects in which choices had to made without knowing what was done in the original study (i.e. whether or not it was the same or different is unknown)
  - side by side tabular presentation of the original and the reproduction scores
  - some amount of analysis and comparison between the original and reproduction results
• Completed HEDS datasheet (please fill in here: https://forms.gle/MgWiKVu7i5UHeMNQ9)
• Any supplementary material

Format and author guidelines

Submissions to the ReproGen Shared Task 2022 should follow the same instructions as INLG 2022 submissions, and the ACL Author Guidelines and policies for submission, review and citation.

Please use ACL Rolling Review style files for which LaTeX style files and Microsoft Word templates are available at https://aclrollingreview.org/cfp#paper-submission-and-templates. Alternatively you can use the Overleaf template at https://www.overleaf.com/read/crtcwgxzjskr.

Page limit and content

ReproGen papers have a page limit of 4 or 8 pages plus any number of pages with references. Supplementary materials can be submitted alongside a paper submission. Participants are also required to fill in the Human Evaluation Datasheet for each evaluation. More instructions here.

Please note that the submissions need not be anonymised for reviewing.

Deadline and where to submit

Submissions are due on June 6, 2022 (11.59 pm UTC-12) and should be emailed to reprogen.task@gmail.com.

For more information, please follow the CFP.