[NLPL Task Force (A)] early feedback

Wed Feb 13 15:44:03 UTC 2019

hi asad,

my apologies, i had not realized you were still using that
‘nlpl-play’environment; i had intended it at the time as something
transient.

please see whether the following works for you:

$ module purge; module load nlpl-sayeed
$ module list
Currently Loaded Modulefiles:
  1) intel/2015.3           3) python2/2.7.10         5) cuda/9.0
  2) libffi/3.0.13          4) gcc/4.9.2              6) nlpl-sayeed/1.11/2.7
$ python -c "import tensorflow as tf; print(tf.__version__);"
1.11.0

best wishes, oe

On Wed, Feb 13, 2019 at 1:21 AM Asad Sayeed <asayeed at mbl.ca> wrote:
>
> Hi,
>
> It looks like the nlpl-play thing has disappeared?  I really need the python2.7 back that will let me run the labeller below (whose author has not upgraded from python 2.7, nevertheless it is the best off-the-shelf near-SOTA tool I have for my purposes).  I wonder why it disappeared and how hard it would be to return it...
>
> Yours,
> --Asad.
>
>
> On 2018-10-10 12:08 PM, Stephan Oepen wrote:
>
> hi again, asad,
>
> sorry, this took me a little while to return to!  and thanks again for your feedback; this is useful to us in the larger picture too :-)!
>
> our NLPL python modules are indeed created as virtual environments, but that should of course not prevent users from adding software using ‘pip install --user’.  i believe i have addressed that problem in the ‘nlpl-play’ module now; at least, i was able to install pyhocon on top of it, but into my home directory.
>
> that being said, i then added pyhocon and tensorflow_hub (which we should include by default, i now realize) to ‘nlpl-play’ ... so if you are still game, please give getting your SRL pipeline running on Abel another shot!
>
> best wishe, oe
>
>
>
>
> On Thu, Oct 4, 2018 at 10:41 PM Asad Sayeed <asayeed at mbl.ca> wrote:
>>
>> Hi,
>>
>> Yes, typing python2 or python2.7 works now. Thanks!
>>
>> Unfortunately I cannot install tensorflow_hub because it thinks that I am in a virtualenv and I'm pretty sure I am not. But I need that too. And pyhocon.
>>
>> The software I am trying to run is here, if you're wondering:
>>
>> https://github.com/luheng/lsgn
>>
>> It needs the latest stuff.  But under python2.7.  But it's really very good...
>>
>> Yours,
>> --Asad.
>>
>>
>> On 2018-10-04 10:27 PM, Stephan Oepen wrote:
>>
>> i think you are getting the wrong ’python‘ binary; try ’python2‘ or ’python2.7‘.  use
>>
>> type -all python2
>>
>> to confirm you end up with the binary from ’.../play/1.11/bin/‘.
>>
>> oe
>>
>>
>> On Thu, 4 Oct 2018 at 22:23 Asad Sayeed <asayeed at mbl.ca> wrote:
>>>
>>> Hi,
>>>
>>> I tried it, including the module purge, but then when I start Python 2.7
>>> and try to load tensorflow, I get:
>>>
>>> ***************
>>>
>>> -bash-4.1$ python
>>> Python 2.7.10 (default, Jul  1 2015, 11:02:23)
>>> [GCC Intel(R) C++ gcc 4.4 mode] on linux2
>>> Type "help", "copyright", "credits" or "license" for more information.
>>>  >>> import tensorflow as tf
>>> Traceback (most recent call last):
>>>    File "<stdin>", line 1, in <module>
>>>    File
>>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/__init__.py",
>>> line 22, in <module>
>>>      from tensorflow.python import pywrap_tensorflow  # pylint:
>>> disable=unused-import
>>>    File
>>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/__init__.py",
>>> line 49, in <module>
>>>      from tensorflow.python import pywrap_tensorflow
>>>    File
>>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py",
>>> line 74, in <module>
>>>      raise ImportError(msg)
>>> ImportError: Traceback (most recent call last):
>>>    File
>>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py",
>>> line 58, in <module>
>>>      from tensorflow.python.pywrap_tensorflow_internal import *
>>>    File
>>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py",
>>> line 28, in <module>
>>>      _pywrap_tensorflow_internal = swig_import_helper()
>>>    File
>>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py",
>>> line 24, in swig_import_helper
>>>      _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname,
>>> description)
>>> ImportError:
>>> /cluster/software/VERSIONS/python_packages-2.7_6/lib64/libc.so.6:
>>> version `GLIBC_2.14' not found (required by
>>> /projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
>>>
>>>
>>> Failed to load the native TensorFlow runtime.
>>>
>>> See
>>> https://www.tensorflow.org/install/install_sources#common_installation_problems
>>>
>>> for some common reasons and solutions.  Include the entire stack trace
>>> above this error message when asking for help.
>>>  >>>
>>>
>>> ***************************
>>>
>>> So there is still a glibc discrepancy on the abel login nodes?
>>>
>>> Thanks!
>>>
>>> Yours,
>>> --Asad.
>>>
>>> On 2018-10-04 01:29 PM, Stephan Oepen wrote:
>>> > hi asad,
>>> >
>>> > thanks for your feedback!  i am glad you like the Abel cluster, even
>>> > though it cannot currently run what you would like to do :-).
>>> >
>>> > it seems it did not take very long for an NLPL user to question our
>>> > assumption that folks should be able to make do with just Python 3.x
>>> > these days.  challenge accepted!
>>> >
>>> > i cannot yet promise this will play out in the long run, nor do i want
>>> > to promise we will commit to supporting it.  but it appears i may have
>>> > managed to apply the ‘glibc gymnastics’ to a TensorFlow installation
>>> > that supports both Python 2.7 and 3.5.
>>> >
>>> > i would encourage you to try the following:
>>> >
>>> > module purge
>>> > module use -a /projects/nlpl/software/modulefiles
>>> > module load nlpl-play/1.11/2.7
>>> >
>>> > does the above work for you?  oe
>>> >
>>> > On Thu, Oct 4, 2018 at 12:56 AM Asad Sayeed <asayeed at mbl.ca> wrote:
>>> >> Hi,
>>> >>
>>> >> I've started trying to use the cluster.  It's great but my current task
>>> >> seems to be impossible there, so I am running it for longer on local
>>> >> machines.  The most recent issue is that I wanted to run a neural
>>> >> semantic role labeller that requires a recent tensorflow that runs on
>>> >> python2.7.  I was only able to find a tensorflow that runs on python3,
>>> >> and my attempts to use a singularity container failed on abel even when
>>> >> I tried to package the container on my local system with the same
>>> >> singularity version that is on abel.  Also virtualenv and user-level
>>> >> package install did not work for me because of an incompatible glibc
>>> >> library. Would there be any chance of getting a tensorflow for python2.7
>>> >> working on abel, along with tensorflow_hub?
>>> >>
>>> >> The purpose is to perform SRL on about 10 million sentences, which
>>> >> should take a day if I can run 400 processes with 10GB memory each (to
>>> >> hold glove vectors).  I may in the future have even larger SRL-related
>>> >> tasks.
>>> >>
>>> >> Yours,
>>> >> --Asad.
>>> >>
>>>
>>
>