[NLPL Users] migration of the NLPL virtual laboratory

Stephan Oepen oe at ifi.uio.no
Sun Oct 20 23:08:58 CEST 2019


dear colleagues,

many of you received subscription notices for the mailing list
'users at nlpl.eu' earlier today, which is a very low-traffic
announcement channel from the NLPL initiative; please see
'http://nlpl.eu' for general background.

you are currently on this mailing list because you have accounts on
one of the four superclusters that host the NLPL virtual laboratory,
viz. Abel and Saga (in norway), and Puhti and Taito (in finland).
mailing list membership is automatically updated each night, so if you
feel that you really should not be receiving NLPL announcements,
please just send us a quick email at 'infrastructure at nlpl.eu'!

we are writing now because Abel and Taito are going to be
decommissioned sometime near the end of 2019.  this means that the
NLPL software and data installations have to be migrated to the
successor systems (Saga and Puhti, respectively), which above all is
good news: more and faster hardware, including better gpu resources,
as well as an update of much of the NLPL-maintained software.  for
technical details on the new clusters, please see:

https://documentation.sigma2.no/quick/saga.html
https://www.csc.fi/-/supertietokone-puhti-on-avattu-tutkijoiden-kayttoon

in principle, all users at NLPL partner sites and associates can gain
access to these systems.  however, usage conditions beyond december
2019 remain to be formalized for users from outside finland and
norway.  for current procedures, please see:

http://wiki.nlpl.eu/index.php/Infrastructure/resources

for Saga, NLPL continues as a single 'project', in the sense of
sharing cpu and storage allocations among all users associated with
the project; we expect no shortage of allocations just now, though
physical resources (in particular gpu capacity on Saga) may quickly
become a limiting factor.  i will have to enable NLPL users for
access, to Saga; if you see a need for migrating your computation to
the new system, please make contact at 'infrastructure at nlpl.eu'.

for Puhti, the per-site allocation regime continues, meaning that NLPL
member sites each have their own allocation 'project'.  site managers
can enable project access to Puhti in the new on-line management
interface at CSC:

https://my.csc.fi/welcome

several of the NLPL data resources are already being replicated across
all four systems, viz. the corpora and vectors repositories, as well
as the shared parsing and translation data modules.  the much larger
OPUS collection is currently being migrated to Puhti, and over the
next couple of weeks we expect to replicate it to Saga (replication to
Abel remains out of reach, due to insufficient disk space).

regarding NLPL-maintained software, the infrastructure task force is
currently working to compile a prioritized list of modules to be
(re-)installed on Puhti and Saga.  even more so than we have in the
past, we plan to keep the NLPL inventory of software modules as
uniform across the two new systems as possible, i.e. by default we aim
to provide the same selection of packages and version on both sides.

a reasonably up-to-date catalogue of NLPL-maintained software on Abel
and Taito is available on the project wiki:

http://wiki.nlpl.eu/index.php/Infrastructure/software/catalogue

in case you have been using NLPL-maintained software modules in the
past already, or would like to start using some of these or similar
modules, we welcome your input.  in your view, which modules are most
desirable to provide (short-term) on the new systems?  please email us
at 'infrastructure at nlpl.eu'!

best wishes, oe (for the NLPL infrastructure task force)


More information about the users mailing list