[NLPL Task Force (A)] early feedback

Asad Sayeed asayeed at mbl.ca
Wed Feb 13 15:54:21 UTC 2019 

Hi,

It works.

Yours,
--Asad.

On 2019-02-13 16:44, Stephan Oepen wrote:
> hi asad,
>
> my apologies, i had not realized you were still using that
> ‘nlpl-play’environment; i had intended it at the time as something
> transient.
>
> please see whether the following works for you:
>
> $ module purge; module load nlpl-sayeed
> $ module list
> Currently Loaded Modulefiles:
>    1) intel/2015.3           3) python2/2.7.10         5) cuda/9.0
>    2) libffi/3.0.13          4) gcc/4.9.2              6) nlpl-sayeed/1.11/2.7
> $ python -c "import tensorflow as tf; print(tf.__version__);"
> 1.11.0
>
> best wishes, oe
>
> On Wed, Feb 13, 2019 at 1:21 AM Asad Sayeed <asayeed at mbl.ca> wrote:
>> Hi,
>>
>> It looks like the nlpl-play thing has disappeared?  I really need the python2.7 back that will let me run the labeller below (whose author has not upgraded from python 2.7, nevertheless it is the best off-the-shelf near-SOTA tool I have for my purposes).  I wonder why it disappeared and how hard it would be to return it...
>>
>> Yours,
>> --Asad.
>>
>>
>> On 2018-10-10 12:08 PM, Stephan Oepen wrote:
>>
>> hi again, asad,
>>
>> sorry, this took me a little while to return to!  and thanks again for your feedback; this is useful to us in the larger picture too :-)!
>>
>> our NLPL python modules are indeed created as virtual environments, but that should of course not prevent users from adding software using ‘pip install --user’.  i believe i have addressed that problem in the ‘nlpl-play’ module now; at least, i was able to install pyhocon on top of it, but into my home directory.
>>
>> that being said, i then added pyhocon and tensorflow_hub (which we should include by default, i now realize) to ‘nlpl-play’ ... so if you are still game, please give getting your SRL pipeline running on Abel another shot!
>>
>> best wishe, oe
>>
>>
>>
>>
>> On Thu, Oct 4, 2018 at 10:41 PM Asad Sayeed <asayeed at mbl.ca> wrote:
>>> Hi,
>>>
>>> Yes, typing python2 or python2.7 works now. Thanks!
>>>
>>> Unfortunately I cannot install tensorflow_hub because it thinks that I am in a virtualenv and I'm pretty sure I am not. But I need that too. And pyhocon.
>>>
>>> The software I am trying to run is here, if you're wondering:
>>>
>>> https://github.com/luheng/lsgn
>>>
>>> It needs the latest stuff.  But under python2.7.  But it's really very good...
>>>
>>> Yours,
>>> --Asad.
>>>
>>>
>>> On 2018-10-04 10:27 PM, Stephan Oepen wrote:
>>>
>>> i think you are getting the wrong ’python‘ binary; try ’python2‘ or ’python2.7‘.  use
>>>
>>> type -all python2
>>>
>>> to confirm you end up with the binary from ’.../play/1.11/bin/‘.
>>>
>>> oe
>>>
>>>
>>> On Thu, 4 Oct 2018 at 22:23 Asad Sayeed <asayeed at mbl.ca> wrote:
>>>> Hi,
>>>>
>>>> I tried it, including the module purge, but then when I start Python 2.7
>>>> and try to load tensorflow, I get:
>>>>
>>>> ***************
>>>>
>>>> -bash-4.1$ python
>>>> Python 2.7.10 (default, Jul  1 2015, 11:02:23)
>>>> [GCC Intel(R) C++ gcc 4.4 mode] on linux2
>>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>   >>> import tensorflow as tf
>>>> Traceback (most recent call last):
>>>>     File "<stdin>", line 1, in <module>
>>>>     File
>>>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/__init__.py",
>>>> line 22, in <module>
>>>>       from tensorflow.python import pywrap_tensorflow  # pylint:
>>>> disable=unused-import
>>>>     File
>>>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/__init__.py",
>>>> line 49, in <module>
>>>>       from tensorflow.python import pywrap_tensorflow
>>>>     File
>>>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py",
>>>> line 74, in <module>
>>>>       raise ImportError(msg)
>>>> ImportError: Traceback (most recent call last):
>>>>     File
>>>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py",
>>>> line 58, in <module>
>>>>       from tensorflow.python.pywrap_tensorflow_internal import *
>>>>     File
>>>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py",
>>>> line 28, in <module>
>>>>       _pywrap_tensorflow_internal = swig_import_helper()
>>>>     File
>>>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py",
>>>> line 24, in swig_import_helper
>>>>       _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname,
>>>> description)
>>>> ImportError:
>>>> /cluster/software/VERSIONS/python_packages-2.7_6/lib64/libc.so.6:
>>>> version `GLIBC_2.14' not found (required by
>>>> /projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
>>>>
>>>>
>>>> Failed to load the native TensorFlow runtime.
>>>>
>>>> See
>>>> https://www.tensorflow.org/install/install_sources#common_installation_problems
>>>>
>>>> for some common reasons and solutions.  Include the entire stack trace
>>>> above this error message when asking for help.
>>>>   >>>
>>>>
>>>> ***************************
>>>>
>>>> So there is still a glibc discrepancy on the abel login nodes?
>>>>
>>>> Thanks!
>>>>
>>>> Yours,
>>>> --Asad.
>>>>
>>>> On 2018-10-04 01:29 PM, Stephan Oepen wrote:
>>>>> hi asad,
>>>>>
>>>>> thanks for your feedback!  i am glad you like the Abel cluster, even
>>>>> though it cannot currently run what you would like to do :-).
>>>>>
>>>>> it seems it did not take very long for an NLPL user to question our
>>>>> assumption that folks should be able to make do with just Python 3.x
>>>>> these days.  challenge accepted!
>>>>>
>>>>> i cannot yet promise this will play out in the long run, nor do i want
>>>>> to promise we will commit to supporting it.  but it appears i may have
>>>>> managed to apply the ‘glibc gymnastics’ to a TensorFlow installation
>>>>> that supports both Python 2.7 and 3.5.
>>>>>
>>>>> i would encourage you to try the following:
>>>>>
>>>>> module purge
>>>>> module use -a /projects/nlpl/software/modulefiles
>>>>> module load nlpl-play/1.11/2.7
>>>>>
>>>>> does the above work for you?  oe
>>>>>
>>>>> On Thu, Oct 4, 2018 at 12:56 AM Asad Sayeed <asayeed at mbl.ca> wrote:
>>>>>> Hi,
>>>>>>
>>>>>> I've started trying to use the cluster.  It's great but my current task
>>>>>> seems to be impossible there, so I am running it for longer on local
>>>>>> machines.  The most recent issue is that I wanted to run a neural
>>>>>> semantic role labeller that requires a recent tensorflow that runs on
>>>>>> python2.7.  I was only able to find a tensorflow that runs on python3,
>>>>>> and my attempts to use a singularity container failed on abel even when
>>>>>> I tried to package the container on my local system with the same
>>>>>> singularity version that is on abel.  Also virtualenv and user-level
>>>>>> package install did not work for me because of an incompatible glibc
>>>>>> library. Would there be any chance of getting a tensorflow for python2.7
>>>>>> working on abel, along with tensorflow_hub?
>>>>>>
>>>>>> The purpose is to perform SRL on about 10 million sentences, which
>>>>>> should take a day if I can run 400 processes with 10GB memory each (to
>>>>>> hold glove vectors).  I may in the future have even larger SRL-related
>>>>>> tasks.
>>>>>>
>>>>>> Yours,
>>>>>> --Asad.
>>>>>>

More information about the infrastructure mailing list