[NLPL Task Force (A)] Python2?

Fri Oct 5 08:47:09 UTC 2018

Hello, 

> yes, i certainly did not mean my message to asad as suggesting we put our full
> muscle behind python 2.x! until proven wrong, i would like to stick to our bias
> for 3.x in NLPL—thus, hopefully help encourage folks to transition or start new
> projects in the modern era.

Ok, I see it the same way. 

> but i was tickled to confirm that i can make one TensorFlow installation (or,
> technically speaking, an overlay of two installations into the same target
> directory) work for both python variants. so, the exercise felt instructive
> :-).

I also agree here, it is not black and white issue. 

Martin 

> oe

> On Fri, 5 Oct 2018 at 09:50 Martin Matthiesen < [
> mailto:martin.matthiesen at csc.fi | martin.matthiesen at csc.fi ] > wrote:

>> Hello!

>> Asad is only one user needing 2.7 so far and the code does not seem to be
>> massively forked. I am a bit unsure whether we should officially support
>> Python2.7 with all tools or stick to 3 and support 2.7 on an as-needed basis.
>> This case was in any case a good reminder that the real world needs sometimes
>> differ from the planning :)

>> Regards,
>> Martin

>> --
>> Martin Matthiesen
>> CSC - Tieteen tietotekniikan keskus
>> CSC - IT Center for Science
>> PL 405, 02101 Espoo, Finland
>> +358 9 457 2376, [ mailto:martin.matthiesen at csc.fi | martin.matthiesen at csc.fi ]
>> Public key : [ https://pgp.mit.edu/pks/lookup?op=get&search=0x74B12876FD890704 |
>> https://pgp.mit.edu/pks/lookup?op=get&search=0x74B12876FD890704 ]
>> Fingerprint: AA25 6F56 5C9A 8B42 009F BA70 74B1 2876 FD89 0704

>>> From: "Asad Sayeed" < [ mailto:asayeed at mbl.ca | asayeed at mbl.ca ] >
>>> To: "Stephan Oepen" < [ mailto:oe at ifi.uio.no | oe at ifi.uio.no ] >
>>> Cc: "infrastructure" < [ mailto:infrastructure at nlpl.eu | infrastructure at nlpl.eu
>>> ] >
>>> Sent: Thursday, 4 October, 2018 23:40:33
>>> Subject: Re: [NLPL Task Force (A)] early feedback

>>> Hi,

>>> Yes, typing python2 or python2.7 works now. Thanks!

>>> Unfortunately I cannot install tensorflow_hub because it thinks that I am in a
>>> virtualenv and I'm pretty sure I am not. But I need that too. And pyhocon.

>>> The software I am trying to run is here, if you're wondering:

>>> [ https://github.com/luheng/lsgn | https://github.com/luheng/lsgn ]

>>> It needs the latest stuff. But under python2.7. But it's really very good...

>>> Yours,
>>> --Asad.

>>> On 2018-10-04 10:27 PM, Stephan Oepen wrote:

>>>> i think you are getting the wrong ’python‘ binary; try ’python2‘ or ’python2.7‘.
>>>> use

>>>> type -all python2

>>>> to confirm you end up with the binary from ’.../play/1.11/bin/‘.

>>>> oe

>>>> On Thu, 4 Oct 2018 at 22:23 Asad Sayeed < [ mailto:asayeed at mbl.ca |
>>>> asayeed at mbl.ca ] > wrote:

>>>>> Hi,

>>>>> I tried it, including the module purge, but then when I start Python 2.7
>>>>> and try to load tensorflow, I get:

>>>>> ***************

>>>>> -bash-4.1$ python
>>>>> Python 2.7.10 (default, Jul 1 2015, 11:02:23)
>>>>> [GCC Intel(R) C++ gcc 4.4 mode] on linux2
>>>>> Type "help", "copyright", "credits" or "license" for more information.
>>>>> >>> import tensorflow as tf
>>>>> Traceback (most recent call last):
>>>>> File "<stdin>", line 1, in <module>
>>>>> File
>>>>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/__init__.py",
>>>>> line 22, in <module>
>>>>> from tensorflow.python import pywrap_tensorflow # pylint:
>>>>> disable=unused-import
>>>>> File
>>>>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/__init__.py",
>>>>> line 49, in <module>
>>>>> from tensorflow.python import pywrap_tensorflow
>>>>> File
>>>>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py",
>>>>> line 74, in <module>
>>>>> raise ImportError(msg)
>>>>> ImportError: Traceback (most recent call last):
>>>>> File
>>>>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py",
>>>>> line 58, in <module>
>>>>> from tensorflow.python.pywrap_tensorflow_internal import *
>>>>> File
>>>>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py",
>>>>> line 28, in <module>
>>>>> _pywrap_tensorflow_internal = swig_import_helper()
>>>>> File
>>>>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py",
>>>>> line 24, in swig_import_helper
>>>>> _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname,
>>>>> description)
>>>>> ImportError:
>>>>> /cluster/software/VERSIONS/python_packages-2.7_6/lib64/libc.so.6:
>>>>> version `GLIBC_2.14' not found (required by
>>>>> /projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)

>>>>> Failed to load the native TensorFlow runtime.

>>>>> See
>>>>> [
>>>>> https://www.tensorflow.org/install/install_sources#common_installation_problems
>>>>> |
>>>>> https://www.tensorflow.org/install/install_sources#common_installation_problems
>>>>> ]

>>>>> for some common reasons and solutions. Include the entire stack trace
>>>>> above this error message when asking for help.

>>>>> ***************************

>>>>> So there is still a glibc discrepancy on the abel login nodes?

>>>>> Thanks!

>>>>> Yours,
>>>>> --Asad.

>>>>> On 2018-10-04 01:29 PM, Stephan Oepen wrote:
>>>>> > hi asad,

>>>>> > thanks for your feedback! i am glad you like the Abel cluster, even
>>>>> > though it cannot currently run what you would like to do :-).

>>>>> > it seems it did not take very long for an NLPL user to question our
>>>>> > assumption that folks should be able to make do with just Python 3.x
>>>>> > these days. challenge accepted!

>>>>> > i cannot yet promise this will play out in the long run, nor do i want
>>>>> > to promise we will commit to supporting it. but it appears i may have
>>>>> > managed to apply the ‘glibc gymnastics’ to a TensorFlow installation
>>>>> > that supports both Python 2.7 and 3.5.

>>>>> > i would encourage you to try the following:

>>>>> > module purge
>>>>> > module use -a /projects/nlpl/software/modulefiles
>>>>> > module load nlpl-play/1.11/2.7

>>>>> > does the above work for you? oe

>>>>>> On Thu, Oct 4, 2018 at 12:56 AM Asad Sayeed < [ mailto:asayeed at mbl.ca |
>>>>> > asayeed at mbl.ca ] > wrote:
>>>>> >> Hi,

>>>>> >> I've started trying to use the cluster. It's great but my current task
>>>>> >> seems to be impossible there, so I am running it for longer on local
>>>>> >> machines. The most recent issue is that I wanted to run a neural
>>>>> >> semantic role labeller that requires a recent tensorflow that runs on
>>>>> >> python2.7. I was only able to find a tensorflow that runs on python3,
>>>>> >> and my attempts to use a singularity container failed on abel even when
>>>>> >> I tried to package the container on my local system with the same
>>>>> >> singularity version that is on abel. Also virtualenv and user-level
>>>>> >> package install did not work for me because of an incompatible glibc
>>>>> >> library. Would there be any chance of getting a tensorflow for python2.7
>>>>> >> working on abel, along with tensorflow_hub?

>>>>> >> The purpose is to perform SRL on about 10 million sentences, which
>>>>> >> should take a day if I can run 400 processes with 10GB memory each (to
>>>>> >> hold glove vectors). I may in the future have even larger SRL-related
>>>>> >> tasks.

>>>>> >> Yours,
>>>>> >> --Asad.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nlpl.eu/archives/infrastructure/attachments/20181005/48b02b0e/attachment.htm>