[NLPL Task Force (A)] partial mirror of OPUS to abel

[Stephan Oepen]

hi joerg,

our storage on Abel was extended to two terabytes this fall, and
currently we have some 800 gigabytes available.

i feel i (still) know too little about OPUS to say whether a partial
replica on Abel would be beneficial to NLPL users?  could you suggest
a sub-set (below 800 gigabytes) to mirror from Taito, and sketch a
typical use case?  could we sketch the reciple for a user to train
their OpenNMT-py system (more or less) straight from the OPUS
directory?

cheers, oe

On Wed, Dec 19, 2018 at 2:37 PM Tiedemann, Jörg
<jorg.tiedemann at helsinki.fi> wrote:
>
>
> This is especially for Stephan: One of the deliverables for this year in the OPUS activity is to create a partial mirror of OPUS data on abel. So far, I still don’t really know what we would like to make available and what kind of space we have for that on abel. In some sense, it could be enough to have that availability via the NIRD storage that you already fill with OPUS data, right? This also counts on longterm storage I guess. I also have the data in IDA here on CSC.
>
> This is activity G1.4 and i wonder if I have to do something about it:
> http://wiki.nlpl.eu/index.php/Infrastructure/home
>
> All the best,
> Jörg
>
> ********************************************************************************************
> Jörg Tiedemann
> Language Technology https://blogs.helsinki.fi/language-technology/
> University of Helsinki
>