[NLPL Users] updates to NLPL software modules on Abel

Stephan Oepen oe at ifi.uio.no
Fri May 10 11:51:56 CEST 2019


dear all,

over the past few weeks, a number of updates to software maintained
under the NLPL umbrella have accumulated on Abel, most notably
upgrades to NumPy, Gensim, and PyTorch, and the addition of CuPy.

i am still unsure about when and how best to communicate software
updates, but i suspect several of you might be affected without you
knowing.

complete NLPL module identifiers (since late 2018) take a form like
'nlpl-pytorch/1.1.0/3.7', where the last two components are the
software and Python versions, respectively.  the 'module load' command
will fill in missing values for you, where the default will be to
select the most recent version.

in other words, if you ordinarily just issue a command like

  module purge; module load nlpl-gensim nlpl-pytorch

then you used to get what were the latest versions until recently,
viz. 'nlpl-gensim/3.7.2/3.7' and 'nlpl-pytorch/1.0.0/3.7'.  as of
today, however, the underspecified command above will get you newer
versions (3.7.3 and 1.1.0, respectively).  such a silent upgrade may
or may not be a welcome change :-).

as a rule of thumb, i recommend to use fully specified module
identifiers at least in all job scripts.  this will guard you against
unexpected version changes and contribute to reproducability over
time.  at the same time, to keep abreast of available versions, you
can inspect the collection of software modules as follows, for
example:

  module avail 2>&1 | grep nlpl

i am afraid there is no running away from multiple available versions.
not all modules are interoperable, of course, and often we receive
requests from users for specific versions, including for specific base
Python versions (e.g. back to 2.7).  while as of 2019, we try to
standardize on Python 3.7, several modules are also available for
Python 3.5 and (in specific cases, by request) for 2.7.

this means that there can be 'gaps' in module coverage (for a specific
version of the base Python), and the 'module load' command is not
intelligent enough to fill in default values that respect
interoperability.  for example, if you are working in a Python 3.5
environment, you need to see to it that you only select software
modules that are compatible with 3.5 (i.e. whose identifier has the
suffix '/3.5').

in case anyone has suggestions for how to make it easier to navigate
the space of available modules and versions, or ideas about how best
to communicate incremental updates along the way, please do not
hesitate to contact us at 'infrastructure at nlpl.eu'.

best wishes, oe


More information about the users mailing list