[NLPL Task Force (A)] Python2?

Fri Oct 5 08:10:44 UTC 2018

yes, i certainly did not mean my message to asad as suggesting we put our
full muscle behind python 2.x!  until proven wrong, i would like to stick
to our bias for 3.x in NLPL—thus, hopefully help encourage folks to
transition or start new projects in the modern era.

but i was tickled to confirm that i can make one TensorFlow installation
(or, technically speaking, an overlay of two installations into the same
target directory) work for both python variants.  so, the exercise felt
instructive :-).

oe

On Fri, 5 Oct 2018 at 09:50 Martin Matthiesen <martin.matthiesen at csc.fi>
wrote:

> Hello!
>
> Asad is only one user needing 2.7 so far and the code does not seem to be
> massively forked. I am a bit unsure whether we should officially support
> Python2.7 with all tools or stick to 3 and support 2.7 on an as-needed
> basis.
> This case was in any case a good reminder that the real world needs
> sometimes differ from the planning :)
>
> Regards,
> Martin
>
> --
> Martin Matthiesen
> CSC - Tieteen tietotekniikan keskus
> CSC - IT Center for Science
> PL 405, 02101 Espoo, Finland
> +358 9 457 2376, martin.matthiesen at csc.fi
> Public key :
> https://pgp.mit.edu/pks/lookup?op=get&search=0x74B12876FD890704
> Fingerprint: AA25 6F56 5C9A 8B42 009F  BA70 74B1 2876 FD89 0704
>
> ------------------------------
>
> *From: *"Asad Sayeed" <asayeed at mbl.ca>
> *To: *"Stephan Oepen" <oe at ifi.uio.no>
> *Cc: *"infrastructure" <infrastructure at nlpl.eu>
> *Sent: *Thursday, 4 October, 2018 23:40:33
> *Subject: *Re: [NLPL Task Force (A)] early feedback
>
> Hi,
>
> Yes, typing python2 or python2.7 works now. Thanks!
>
> Unfortunately I cannot install tensorflow_hub because it thinks that I am
> in a virtualenv and I'm pretty sure I am not. But I need that too. And
> pyhocon.
>
> The software I am trying to run is here, if you're wondering:
>
> https://github.com/luheng/lsgn
>
> It needs the latest stuff.  But under python2.7.  But it's really very
> good...
>
> Yours,
> --Asad.
>
> On 2018-10-04 10:27 PM, Stephan Oepen wrote:
>
> i think you are getting the wrong ’python‘ binary; try ’python2‘ or
> ’python2.7‘.  use
>
> type -all python2
>
> to confirm you end up with the binary from ’.../play/1.11/bin/‘.
>
> oe
>
>
> On Thu, 4 Oct 2018 at 22:23 Asad Sayeed <asayeed at mbl.ca> wrote:
>
>> Hi,
>>
>> I tried it, including the module purge, but then when I start Python 2.7
>> and try to load tensorflow, I get:
>>
>> ***************
>>
>> -bash-4.1$ python
>> Python 2.7.10 (default, Jul  1 2015, 11:02:23)
>> [GCC Intel(R) C++ gcc 4.4 mode] on linux2
>> Type "help", "copyright", "credits" or "license" for more information.
>>  >>> import tensorflow as tf
>> Traceback (most recent call last):
>>    File "<stdin>", line 1, in <module>
>>    File
>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/__init__.py",
>>
>> line 22, in <module>
>>      from tensorflow.python import pywrap_tensorflow  # pylint:
>> disable=unused-import
>>    File
>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/__init__.py",
>>
>> line 49, in <module>
>>      from tensorflow.python import pywrap_tensorflow
>>    File
>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py",
>>
>> line 74, in <module>
>>      raise ImportError(msg)
>> ImportError: Traceback (most recent call last):
>>    File
>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow.py",
>>
>> line 58, in <module>
>>      from tensorflow.python.pywrap_tensorflow_internal import *
>>    File
>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py",
>>
>> line 28, in <module>
>>      _pywrap_tensorflow_internal = swig_import_helper()
>>    File
>> "/projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/pywrap_tensorflow_internal.py",
>>
>> line 24, in swig_import_helper
>>      _mod = imp.load_module('_pywrap_tensorflow_internal', fp, pathname,
>> description)
>> ImportError:
>> /cluster/software/VERSIONS/python_packages-2.7_6/lib64/libc.so.6:
>> version `GLIBC_2.14' not found (required by
>>
>> /projects/nlpl/software/play/1.11/lib/python2.7/site-packages/tensorflow/python/_pywrap_tensorflow_internal.so)
>>
>>
>> Failed to load the native TensorFlow runtime.
>>
>> See
>>
>> https://www.tensorflow.org/install/install_sources#common_installation_problems
>>
>> for some common reasons and solutions.  Include the entire stack trace
>> above this error message when asking for help.
>>  >>>
>>
>> ***************************
>>
>> So there is still a glibc discrepancy on the abel login nodes?
>>
>> Thanks!
>>
>> Yours,
>> --Asad.
>>
>> On 2018-10-04 01:29 PM, Stephan Oepen wrote:
>> > hi asad,
>> >
>> > thanks for your feedback!  i am glad you like the Abel cluster, even
>> > though it cannot currently run what you would like to do :-).
>> >
>> > it seems it did not take very long for an NLPL user to question our
>> > assumption that folks should be able to make do with just Python 3.x
>> > these days.  challenge accepted!
>> >
>> > i cannot yet promise this will play out in the long run, nor do i want
>> > to promise we will commit to supporting it.  but it appears i may have
>> > managed to apply the ‘glibc gymnastics’ to a TensorFlow installation
>> > that supports both Python 2.7 and 3.5.
>> >
>> > i would encourage you to try the following:
>> >
>> > module purge
>> > module use -a /projects/nlpl/software/modulefiles
>> > module load nlpl-play/1.11/2.7
>> >
>> > does the above work for you?  oe
>> >
>> > On Thu, Oct 4, 2018 at 12:56 AM Asad Sayeed <asayeed at mbl.ca> wrote:
>> >> Hi,
>> >>
>> >> I've started trying to use the cluster.  It's great but my current task
>> >> seems to be impossible there, so I am running it for longer on local
>> >> machines.  The most recent issue is that I wanted to run a neural
>> >> semantic role labeller that requires a recent tensorflow that runs on
>> >> python2.7.  I was only able to find a tensorflow that runs on python3,
>> >> and my attempts to use a singularity container failed on abel even when
>> >> I tried to package the container on my local system with the same
>> >> singularity version that is on abel.  Also virtualenv and user-level
>> >> package install did not work for me because of an incompatible glibc
>> >> library. Would there be any chance of getting a tensorflow for
>> python2.7
>> >> working on abel, along with tensorflow_hub?
>> >>
>> >> The purpose is to perform SRL on about 10 million sentences, which
>> >> should take a day if I can run 400 processes with 10GB memory each (to
>> >> hold glove vectors).  I may in the future have even larger SRL-related
>> >> tasks.
>> >>
>> >> Yours,
>> >> --Asad.
>> >>
>>
>>
>
>
-------------- next part --------------
An HTML attachment was scrubbed...
URL: <http://lists.nlpl.eu/archives/infrastructure/attachments/20181005/03f6a441/attachment.htm>