[NLPL Task Force (A)] Python2?

Fri Oct 5 07:50:03 UTC 2018

Hello! 

Asad is only one user needing 2.7 so far and the code does not seem to be massively forked. I am a bit unsure whether we should officially support Python2.7 with all tools or stick to 3 and support 2.7 on an as-needed basis. 
This case was in any case a good reminder that the real world needs sometimes differ from the planning :) 

Regards, 
Martin 

-- 
Martin Matthiesen 
CSC - Tieteen tietotekniikan keskus 
CSC - IT Center for Science 
PL 405, 02101 Espoo, Finland 
+358 9 457 2376, martin.matthiesen at csc.fi 
Public key : https://pgp.mit.edu/pks/lookup?op=get&search=0x74B12876FD890704 
Fingerprint: AA25 6F56 5C9A 8B42 009F BA70 74B1 2876 FD89 0704 

> From: "Asad Sayeed" <asayeed at mbl.ca>
> To: "Stephan Oepen" <oe at ifi.uio.no>
> Cc: "infrastructure" <infrastructure at nlpl.eu>
> Sent: Thursday, 4 October, 2018 23:40:33
> Subject: Re: [NLPL Task Force (A)] early feedback

> Hi,

> Yes, typing python2 or python2.7 works now. Thanks!

> Unfortunately I cannot install tensorflow_hub because it thinks that I am in a
> virtualenv and I'm pretty sure I am not. But I need that too. And pyhocon.

> The software I am trying to run is here, if you're wondering:

> [ https://github.com/luheng/lsgn | https://github.com/luheng/lsgn ]

> It needs the latest stuff. But under python2.7. But it's really very good...

> Yours,
> --Asad.

> On 2018-10-04 10:27 PM, Stephan Oepen wrote:

>> i think you are getting the wrong ’python‘ binary; try ’python2‘ or ’python2.7‘.
>> use

>> type -all python2

>> to confirm you end up with the binary from ’.../play/1.11/bin/‘.

>> oe

>> On Thu, 4 Oct 2018 at 22:23 Asad Sayeed < [ mailto:asayeed at mbl.ca |
>> asayeed at mbl.ca ] > wrote:

>>> Hi,

>>> I tried it, including the module purge, but then when I start Python 2.7
>>> and try to load tensorflow, I get:

>>> ***************

>>> -bash-4.1$ python
>>> Python 2.7.10 (default, Jul 1 2015, 11:02:23)
>>> [GCC Intel(R) C++ gcc 4.4 mode] on linux2
>>> Type "help", "copyright", "credits" or "license" for more information.
>>> >>> import tensorflow as tf
>>> Traceback (most recent call last):
>>> File "<stdin>", line 1, in <module>
>>> File
>>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/__init__.py",
>>> line 22, in <module>
>>> from tensorflow.python import pywrap_tensorflow # pylint:
>>> disable=unused-import
>>> File
>>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/__init__.py",
>>> line 49, in <module>
>>> from tensorflow.python import pywrap_tensorflow
>>> File
>>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py",
>>> line 74, in <module>
>>> raise ImportError(msg)
>>> ImportError: Traceback (most recent call last):
>>> File
>>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py",
>>> line 58, in <module>
>>> from tensorflow.python.pywrap_tensorflow_internal import *
>>> File
>>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py",
>>> line 28, in <module>
>>> _pywrap_tensorflow_internal = swig_import_helper()
>>> File
>>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py",
>>> line 24, in swig_import_helper
>>> _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname,
>>> description)
>>> ImportError:
>>> /cluster/software/VERSIONS/python_packages-2.7_6/lib64/libc.so.6:
>>> version `GLIBC_2.14' not found (required by
>>> /projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)

>>> Failed to load the native TensorFlow runtime.

>>> See
>>> [
>>> https://www.tensorflow.org/install/install_sources#common_installation_problems
>>> |
>>> https://www.tensorflow.org/install/install_sources#common_installation_problems
>>> ]

>>> for some common reasons and solutions. Include the entire stack trace
>>> above this error message when asking for help.

>>> ***************************

>>> So there is still a glibc discrepancy on the abel login nodes?

>>> Thanks!

>>> Yours,
>>> --Asad.

>>> On 2018-10-04 01:29 PM, Stephan Oepen wrote:
>>> > hi asad,

>>> > thanks for your feedback! i am glad you like the Abel cluster, even
>>> > though it cannot currently run what you would like to do :-).

>>> > it seems it did not take very long for an NLPL user to question our
>>> > assumption that folks should be able to make do with just Python 3.x
>>> > these days. challenge accepted!

>>> > i cannot yet promise this will play out in the long run, nor do i want
>>> > to promise we will commit to supporting it. but it appears i may have
>>> > managed to apply the ‘glibc gymnastics’ to a TensorFlow installation
>>> > that supports both Python 2.7 and 3.5.

>>> > i would encourage you to try the following:

>>> > module purge
>>> > module use -a /projects/nlpl/software/modulefiles
>>> > module load nlpl-play/1.11/2.7

>>> > does the above work for you? oe

>>>> On Thu, Oct 4, 2018 at 12:56 AM Asad Sayeed < [ mailto:asayeed at mbl.ca |
>>> > asayeed at mbl.ca ] > wrote:
>>> >> Hi,

>>> >> I've started trying to use the cluster. It's great but my current task
>>> >> seems to be impossible there, so I am running it for longer on local
>>> >> machines. The most recent issue is that I wanted to run a neural
>>> >> semantic role labeller that requires a recent tensorflow that runs on
>>> >> python2.7. I was only able to find a tensorflow that runs on python3,
>>> >> and my attempts to use a singularity container failed on abel even when
>>> >> I tried to package the container on my local system with the same
>>> >> singularity version that is on abel. Also virtualenv and user-level
>>> >> package install did not work for me because of an incompatible glibc
>>> >> library. Would there be any chance of getting a tensorflow for python2.7
>>> >> working on abel, along with tensorflow_hub?

>>> >> The purpose is to perform SRL on about 10 million sentences, which
>>> >> should take a day if I can run 400 processes with 10GB memory each (to
>>> >> hold glove vectors). I may in the future have even larger SRL-related
>>> >> tasks.

>>> >> Yours,
>>> >> --Asad.

-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nlpl.eu/archives/infrastructure/attachments/20181005/1ff42fcc/attachment.htm>