[epe-users] Regarding UniMelb with illegitimate training data at EPE 2018 ?

[Dat Quoc Nguyen]

Dear EPE 2018 organisers,

I have just learned that our UniMelb submission for EPE 2018
used illegitimate training data.

I was not aware of the restriction of using only English UD treebanks.

>From your (previous) email, I initially thought that participating teams
could use any treebank available for English (i.e., similar to EPE 2017):

On Fri, Jul 6, 2018 at 11:54 AM, Stephan Oepen <oe at ifi.uio.no> wrote:
>
> >
> > furthermore, it appears that some teams struggle with the unusual
> > ‘tcode’ values in the EPE data set (which does not correspond to one
> > of the english UD treebanks, meaning that
> *teams are free to decide> freely on which of the available treebanks to
> train)*, or maybe also
> > stumble over the missing ‘goldfile’ entries in our ‘metadata.json’.
> > in case any of these throw up your scripts from the core parsing task,
> > please use the public EPE trial data (available on TIRA) to debug, so
> > that you can easily inspect system behavior.
> >
>
>
It was my mistake for not carefully double-checking the EPE website.

I understand that our UniMelb submission scores are likely not officially
ranked. So in this email I would like to ask you that whether our scores
will be mentioned in the EPE overview paper ?
(i.e. will there be any option for unofficial rank in the overview paper?)

I completed camera ready version for our CoNLL 2018 shared task paper (as
in the attachment file), including our EPE scores with a comparison
analysis to the EPE 2017 Stanford-Paris system (for which I followed the
data used by this system).

Your response will help me to decide: either removing the EPE 2018 section
in our shared task paper in case our UniMelb scores not mentioned in the
overview paper, or modifying this section to mention an unofficial rank.

Thank you very much.

Best regards,

Dat Quoc Nguyen
Research Fellow
School of Computing and Information Systems
The University of Melbourne
https://people.eng.unimelb.edu.au/dqnguyen <https://datquocnguyen.github.io>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nlpl.eu/archives/epe-users/attachments/20180803/f5f7f63a/attachment-0001.html>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: CoNLL2018st.pdf
Type: application/pdf
Size: 277938 bytes
Desc: not available
URL: <http://lists.nlpl.eu/archives/epe-users/attachments/20180803/f5f7f63a/attachment-0001.pdf>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: conll18st-epe.png
Type: image/png
Size: 233305 bytes
Desc: not available
URL: <http://lists.nlpl.eu/archives/epe-users/attachments/20180803/f5f7f63a/attachment-0001.png>