[NLPL Users] updated and extended collection of Python add-on modules on Abel

Stephan Oepen oe at ifi.uio.no
Fri Feb 1 01:11:44 CET 2019


dear all,

partly in preparation for the annual NLPL winter school next week, i
have just rolled out a new scheme of maintaining project-specific
Python add-ons.  for now, this is only in effect on Abel; but we
anticipate mirroring the same set of modules on Taito relatively soon.

key improvements over how things used to be are increased modularity
and interoperability and support for different base Python versions
(currently 3.5 and 3.7; though if anyone were to ask for 2.7, that
should now be possible).  for general background, please see:

http://wiki.nlpl.eu/index.php/Infrastructure/software/python

in a nutshell, the new scheme is characterized by structured version
identifiers for NLPL modules, where the final component selects a
specific Python dialect:

$ module avail
[...]
nlpl-cython/0.29.1/3.5
nlpl-cython/0.29.3/3.5
nlpl-cython/0.29.3/3.7
nlpl-dozat/201812/3.5
nlpl-gensim/3.6.0
nlpl-gensim/3.7.0/3.5
nlpl-gensim/3.7.0/3.7
nlpl-in5550/201901/3.7
nlpl-nltk/3.3
nlpl-nltk/3.4/3.5
nlpl-nltk/3.4/3.7
nlpl-numpy/1.16.0/3.5
nlpl-numpy/1.16.0/3.7
nlpl-opennmt-py/0.2.1
nlpl-pytorch/0.4.1
nlpl-pytorch/1.0.0/3.5
nlpl-pytorch/1.0.0/3.7
nlpl-scipy/201810/2.7
nlpl-scipy/201810/3.5
nlpl-scipy/201901/3.5
nlpl-scipy/201901/3.7
nlpl-spacy/2.0.12
nlpl-tensorflow/1.11

note that some of the older Python add-ons remain available in the
former scheme and (implicitly) depend on Python 3.5, for example
‘nlpl-gensim/3.6.0’ or ‘nlpl-pytorch/0.4.1’.  where multiple versions
of the same base module exist, the default now will be the most recent
version for Python 3.7, e.g.

$ module purge; module load nlpl-nltk nlpl-gensim nlpl-pytorch
$ python3 --version
Python 3.7.0
$ module list
Currently Loaded Modulefiles:
  1) gcc/4.9.2                4) python3/3.7.0            7)
nlpl-scipy/201901/3.7   10) nlpl-pytorch/1.0.0/3.7
  2) intel/2019.0             5) nlpl-nltk/3.4/3.7        8)
nlpl-gensim/3.7.0/3.7
  3) openssl.intel/1_1_1      6) nlpl-numpy/1.16.0/3.7    9) cuda/9.0
$ python3 -c 'import nltk, gensim, torch; \
  print("{}; {}; {}".format(nltk.__version__, gensim.__version__,
torch.__version__));'
3.4; 3.7.1; 1.0.0

—thank you for reading this far :-).  if anyone wanted to experiment
with ELMo embeddings, that should now be possible.  we would be most
grateful if active users of some of these modules (on Abel) could test
the new versions as soon as possible.  maybe we can jointly debug and
further refine our inventory of Python add-ons already during the
winter school next week.  once we know the new scheme works mostly as
expected on Abel, we will start reproducing these modules on Taito
(which should be a largely automated process from now on).

please direct any feedback or suggestions for improvements you might
have to the ‘infrastructure at nlpl.eu’ task force.

cheers, oe

ps: i have taken the liberty of purging some outdated unofficial
modules, notably ‘nlpl-inf5820’, ‘nlpl-play’, and ‘nlpl-oe’.  their
functionality should be wholly subsumed by the official modules
(listed on the software catalogue on the NLPL wiki).



More information about the users mailing list