[mrp-users] moving into the MRP 2020 evaluation period

[Stephan Oepen]

dear colleagues:

we sincerely hope that your work with the meaning representation
graphs from our MRP 2020 shared task has been fruitful (and ideally
also enjoyable :-)!  this coming monday, july 27, marks the start of
the evaluation period for the shared task.  the competition will
officially close two weeks later, on monday, august 10 (at which point
we will score submissions, share results with all participating teams,
and start to work with you toward the shared task proceedings).

the task web site has been extended with technical information about
the evaluation period, the format of parser inputs, requirements on
parser outputs to be submitted, and the on-line system for team
registration and uploading submissions:

http://mrp.nlpl.eu/2020/index.php?page=16
http://mrp.nlpl.eu/2020/index.php?page=14
http://svn.nlpl.eu/mrp/2020/public/evaluation/README.txt
https://competitions.codalab.org/competitions/25567

we of course hope that this information is clear and sufficient for
all participants.  but in case you are left wondering about anything:
please do not hesitate to email us (at 'mrp-organizers at nlpl.eu')
whatever questions you might have as soon as possible!  the CodaLab
submission site is open already, and we kindly ask all prospective
participants to register and make a trial submission (e.g. using the
MRP 2020 validation graphs).

the shared task comprises two distinct tracks (cross-frameworks and
cross-lingual), multiple meaning representation frameworks, and four
distinct languages.  partial submissions, however, will be very
welcome (even for just one framework and a single language)—with or
without cross-framework transfer learning.  we expect that the parsing
problem, for each framework, defined by the shared task will provide a
stable benchmark for at least a few years to come.

we would also like to use this opportunity to share one minor
correction of the chinese syntactic companion trees with you: one
participant pointed out to us that two of the chinese AMR graphs
lacked a corresponding companion dependency tree.  these trees had
been inadvertently lost during conversion, and we suspect that most
teams (who use the companion parses) may simply have discarded those
two CAMR training graphs.  that is, of course, an okay solution; but
for completeness we attach the two missing companion trees to this
message.

best wishes, oe (for the task co-organizers)
-------------- next part --------------
A non-text attachment was scrubbed...
Name: zho.mrp
Type: application/octet-stream
Size: 3511 bytes
Desc: not available
URL: <http://lists.nlpl.eu/archives/mrp-users/attachments/20200722/8e905eb8/attachment.obj>