[epe-users] entering the EPE 2017 evaluation phase

Stephan Oepen oe at ifi.uio.no
Tue Jun 6 00:43:46 CEST 2017


dear colleagues,

thanks again for your interest in our shared task on Extrinsic Parser
Evaluation (EPE) supported jointly by the DepLing and IWPT conferences
this year.  we have received trial submissions from five teams, but
from recent communication with candidate participants we
optimistically expect a couple more teams in the end.

the task is about to move into its evaluation phase.

due to illness among several of the task co-organizers, we have fallen
a little behind schedule, both in providing feedback to some recent
trial submissions and, more importantly, in releasing the end-to-end
evaluation data.  we have thus moderately adjusted our schedule as
follows:

June 6–June 15, 2017
  Evaluation Period (Held-Out Data)
June 26, 2017
  Official End-to-End Evaluation Results

please watch the task web site during the day tomorrow (tuesday) for a
download link to the evaluation data:

  http://epe.nlpl.eu/index.php?page=4

also, please observe our instructions on how to package and submit
parser outputs for evaluation:

  http://epe.nlpl.eu/index.php?page=5

in case any of the above is unclear or incomplete, please do not
hesitate to get in touch with us at ‘epe-organizers at nlpl.eu’.

—your trial submissions of parser outputs in a broad variety of
dependency representations have been very helpful in generalizing our
downstream systems and general EPE infrastructure, e.g. for conversion
from CoNLL-U or SDP files.  in several cases, we have also been able
to offer feedback on how to make better use of the EPE interchange
format.

thus, we would like to offer those teams who have signed up for the
task recently a last chance to jointly with us calibrate the use of
dependency outputs from your parser in the EPE context.  in case you
would want to make a last-minute trial submission (which will be
evaluated on the development splits), we will do our best to run these
through still until the end of this week.

best wishes, oe (for the EPE 2017 co-organizers)



More information about the epe-users mailing list