[NLPL Task Force (A)] early feedback

Wed Oct 10 10:08:45 UTC 2018

hi again, asad,

sorry, this took me a little while to return to!  and thanks again for your
feedback; this is useful to us in the larger picture too :-)!

our NLPL python modules are indeed created as virtual environments, but
that should of course not prevent users from adding software using ‘pip
install --user’.  i believe i have addressed that problem in the
‘nlpl-play’ module now; at least, i was able to install pyhocon on top of
it, but into my home directory.

that being said, i then added pyhocon and tensorflow_hub (which we should
include by default, i now realize) to ‘nlpl-play’ ... so if you are still
game, please give getting your SRL pipeline running on Abel another shot!

best wishe, oe

On Thu, Oct 4, 2018 at 10:41 PM Asad Sayeed <asayeed at mbl.ca> wrote:

> Hi,
>
> Yes, typing python2 or python2.7 works now. Thanks!
>
> Unfortunately I cannot install tensorflow_hub because it thinks that I am
> in a virtualenv and I'm pretty sure I am not. But I need that too. And
> pyhocon.
>
> The software I am trying to run is here, if you're wondering:
>
> https://github.com/luheng/lsgn
>
> It needs the latest stuff.  But under python2.7.  But it's really very
> good...
>
> Yours,
> --Asad.
>
> On 2018-10-04 10:27 PM, Stephan Oepen wrote:
>
> i think you are getting the wrong ’python‘ binary; try ’python2‘ or
> ’python2.7‘.  use
>
> type -all python2
>
> to confirm you end up with the binary from ’.../play/1.11/bin/‘.
>
> oe
>
>
> On Thu, 4 Oct 2018 at 22:23 Asad Sayeed <asayeed at mbl.ca> wrote:
>
>> Hi,
>>
>> I tried it, including the module purge, but then when I start Python 2.7
>> and try to load tensorflow, I get:
>>
>> ***************
>>
>> -bash-4.1$ python
>> Python 2.7.10 (default, Jul  1 2015, 11:02:23)
>> [GCC Intel(R) C++ gcc 4.4 mode] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>>  >>> import tensorflow as tf
>> Traceback (most recent call last):
>>    File "<stdin>", line 1, in <module>
>>    File
>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/__init__.py",
>>
>> line 22, in <module>
>>      from tensorflow.python import pywrap_tensorflow  # pylint:
>> disable=unused-import
>>    File
>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/__init__.py",
>>
>> line 49, in <module>
>>      from tensorflow.python import pywrap_tensorflow
>>    File
>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py",
>>
>> line 74, in <module>
>>      raise ImportError(msg)
>> ImportError: Traceback (most recent call last):
>>    File
>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py",
>>
>> line 58, in <module>
>>      from tensorflow.python.pywrap_tensorflow_internal import *
>>    File
>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py",
>>
>> line 28, in <module>
>>      _pywrap_tensorflow_internal = swig_import_helper()
>>    File
>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py",
>>
>> line 24, in swig_import_helper
>>      _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname,
>> description)
>> ImportError:
>> /cluster/software/VERSIONS/python_packages-2.7_6/lib64/libc.so.6:
>> version `GLIBC_2.14' not found (required by
>>
>> /projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
>>
>>
>> Failed to load the native TensorFlow runtime.
>>
>> See
>>
>> https://www.tensorflow.org/install/install_sources#common_installation_problems
>>
>> for some common reasons and solutions.  Include the entire stack trace
>> above this error message when asking for help.
>>  >>>
>>
>> ***************************
>>
>> So there is still a glibc discrepancy on the abel login nodes?
>>
>> Thanks!
>>
>> Yours,
>> --Asad.
>>
>> On 2018-10-04 01:29 PM, Stephan Oepen wrote:
>> > hi asad,
>> >
>> > thanks for your feedback!  i am glad you like the Abel cluster, even
>> > though it cannot currently run what you would like to do :-).
>> >
>> > it seems it did not take very long for an NLPL user to question our
>> > assumption that folks should be able to make do with just Python 3.x
>> > these days.  challenge accepted!
>> >
>> > i cannot yet promise this will play out in the long run, nor do i want
>> > to promise we will commit to supporting it.  but it appears i may have
>> > managed to apply the ‘glibc gymnastics’ to a TensorFlow installation
>> > that supports both Python 2.7 and 3.5.
>> >
>> > i would encourage you to try the following:
>> >
>> > module purge
>> > module use -a /projects/nlpl/software/modulefiles
>> > module load nlpl-play/1.11/2.7
>> >
>> > does the above work for you?  oe
>> >
>> > On Thu, Oct 4, 2018 at 12:56 AM Asad Sayeed <asayeed at mbl.ca> wrote:
>> >> Hi,
>> >>
>> >> I've started trying to use the cluster.  It's great but my current task
>> >> seems to be impossible there, so I am running it for longer on local
>> >> machines.  The most recent issue is that I wanted to run a neural
>> >> semantic role labeller that requires a recent tensorflow that runs on
>> >> python2.7.  I was only able to find a tensorflow that runs on python3,
>> >> and my attempts to use a singularity container failed on abel even when
>> >> I tried to package the container on my local system with the same
>> >> singularity version that is on abel.  Also virtualenv and user-level
>> >> package install did not work for me because of an incompatible glibc
>> >> library. Would there be any chance of getting a tensorflow for
>> python2.7
>> >> working on abel, along with tensorflow_hub?
>> >>
>> >> The purpose is to perform SRL on about 10 million sentences, which
>> >> should take a day if I can run 400 processes with 10GB memory each (to
>> >> hold glove vectors).  I may in the future have even larger SRL-related
>> >> tasks.
>> >>
>> >> Yours,
>> >> --Asad.
>> >>
>>
>>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nlpl.eu/archives/infrastructure/attachments/20181010/890ac82a/attachment.htm>