[NLPL Task Force (A)] [uninett.no #196965] Tensorflow issues, pt. 2
Vinit Ravishankar via RT
support at metacenter.no
Thu Oct 24 14:39:51 UTC 2019
Sure, I’m using my own Miniconda environment (Python 3.7), I’ve attached the output to `conda list’.
Apart from that, I’m running:
module load NCCL/2.4.8-gcccuda-2018b
module load OpenMPI/3.1.1-gcccuda-2018b
prior to submission (2019a appears to have issues with Horovod).
Incidentally, the Horovod github suggests that this might be a TensorFlow versioning issue, and that I should downgrade to 1.13.1. Unfortunately, when I downgrade (again, using conda), I can no longer import tensorflow, because it can’t find libcuda.so.1.
– Vinit
> On 24 Oct 2019, at 14:02, Henrik R. Nagel via RT <support at metacenter.no> wrote:
>
> Hi,
>
> Can you describe for us how we can reproduce the error message? What modules should we load, etc.?
>
> Best regards,
>
> Henrik
-------------- next part --------------
# packages in environment at /cluster/home/vinitr/.miniconda3/envs/nlp:
#
# Name Version Build Channel
_libgcc_mutex 0.1 main
_tflow_select 2.1.0 gpu anaconda
absl-py 0.7.1 pypi_0 pypi
alabaster 0.7.12 pypi_0 pypi
allennlp 0.8.4 pypi_0 pypi
astor 0.8.0 pypi_0 pypi
atomicwrites 1.3.0 pypi_0 pypi
attrs 19.1.0 pypi_0 pypi
awscli 1.16.196 pypi_0 pypi
babel 2.7.0 pypi_0 pypi
bintrees 2.0.7 pypi_0 pypi
binutils_impl_linux-64 2.31.1 h6176602_1
binutils_linux-64 2.31.1 h6176602_8
blas 1.0 mkl anaconda
blis 0.2.4 pypi_0 pypi
boto3 1.9.186 pypi_0 pypi
botocore 1.12.186 pypi_0 pypi
c-ares 1.15.0 h7b6447c_1001 anaconda
ca-certificates 2019.10.16 0
certifi 2019.9.11 py37_0
cffi 1.12.3 pypi_0 pypi
chardet 3.0.4 pypi_0 pypi
click 7.0 pypi_0 pypi
cloudpickle 1.2.2 pypi_0 pypi
colorama 0.3.9 pypi_0 pypi
conllu 0.11 pypi_0 pypi
cudatoolkit 10.0.130 0
cudnn 7.6.0 cuda10.0_0
cupti 10.0.130 0
cycler 0.10.0 pypi_0 pypi
cymem 2.0.2 pypi_0 pypi
docopt 0.6.2 pypi_0 pypi
docutils 0.14 pypi_0 pypi
editdistance 0.5.3 pypi_0 pypi
flaky 3.6.0 pypi_0 pypi
flask 1.1.1 pypi_0 pypi
flask-cors 3.0.8 pypi_0 pypi
ftfy 5.5.1 pypi_0 pypi
gast 0.3.2 py_0 anaconda
gcc_impl_linux-64 7.3.0 habb00fd_1
gcc_linux-64 7.3.0 h553295d_8
gevent 1.4.0 pypi_0 pypi
google-pasta 0.1.7 py_0 anaconda
greenlet 0.4.15 pypi_0 pypi
grpcio 1.22.0 pypi_0 pypi
gxx_impl_linux-64 7.3.0 hdf63c60_1
gxx_linux-64 7.3.0 h553295d_8
h5py 2.9.0 py37h7918eee_0 anaconda
hdf5 1.10.4 hb1b8bf9_0 anaconda
idna 2.8 pypi_0 pypi
imagesize 1.1.0 pypi_0 pypi
importlib-metadata 0.18 pypi_0 pypi
intel-openmp 2019.5 281 anaconda
itsdangerous 1.1.0 pypi_0 pypi
jinja2 2.10.1 pypi_0 pypi
jmespath 0.9.4 pypi_0 pypi
joblib 0.13.2 pypi_0 pypi
jsonnet 0.13.0 pypi_0 pypi
jsonpickle 1.2 pypi_0 pypi
jsonschema 3.0.1 pypi_0 pypi
keras-applications 1.0.8 py_0 anaconda
keras-preprocessing 1.1.0 py_1 anaconda
kiwisolver 1.1.0 pypi_0 pypi
langid 1.1.6 pypi_0 pypi
libedit 3.1.20181209 hc058e9b_0
libffi 3.2.1 hd88cf55_4
libgcc-ng 9.1.0 hdf63c60_0
libgfortran-ng 7.3.0 hdf63c60_0 anaconda
libprotobuf 3.9.2 hd408876_0 anaconda
libstdcxx-ng 9.1.0 hdf63c60_0
markdown 3.1.1 py37_0 anaconda
markupsafe 1.1.1 pypi_0 pypi
matplotlib 3.1.1 pypi_0 pypi
mkl 2019.5 281 anaconda
mkl-service 2.3.0 py37he904b0f_0 anaconda
mkl_fft 1.0.14 py37ha843d7b_0 anaconda
mkl_random 1.1.0 py37hd6b4f25_0 anaconda
mock 3.0.5 py37_0
more-itertools 7.1.0 pypi_0 pypi
mosestokenizer 1.0.0 pypi_0 pypi
murmurhash 1.0.2 pypi_0 pypi
ncurses 6.1 he6710b0_1
nltk 3.4.4 pypi_0 pypi
numpy 1.16.4 pypi_0 pypi
numpy-base 1.17.2 py37hde5b4d6_0 anaconda
numpydoc 0.9.1 pypi_0 pypi
openfile 0.0.7 pypi_0 pypi
openssl 1.1.1d h7b6447c_3
overrides 1.9 pypi_0 pypi
packaging 19.0 pypi_0 pypi
pandas 0.25.1 pypi_0 pypi
parsimonious 0.8.1 pypi_0 pypi
pip 19.1.1 py37_0
plac 0.9.6 pypi_0 pypi
pluggy 0.12.0 pypi_0 pypi
preshed 2.0.1 pypi_0 pypi
protobuf 3.8.0 pypi_0 pypi
psutil 5.6.3 pypi_0 pypi
py 1.8.0 pypi_0 pypi
pyasn1 0.4.5 pypi_0 pypi
pycparser 2.19 pypi_0 pypi
pygments 2.4.2 pypi_0 pypi
pyparsing 2.4.0 pypi_0 pypi
pyrsistent 0.15.3 pypi_0 pypi
pytest 5.0.1 pypi_0 pypi
python 3.7.3 h0371630_0
python-dateutil 2.8.0 pypi_0 pypi
pytorch-pretrained-bert 0.6.2 pypi_0 pypi
pytorch-transformers 1.0.0 pypi_0 pypi
pytz 2019.1 pypi_0 pypi
pyyaml 5.1 pypi_0 pypi
readline 7.0 h7b6447c_5
regex 2019.6.8 pypi_0 pypi
requests 2.22.0 pypi_0 pypi
responses 0.10.6 pypi_0 pypi
rsa 3.4.2 pypi_0 pypi
s3transfer 0.2.1 pypi_0 pypi
scikit-learn 0.21.2 pypi_0 pypi
scipy 1.3.0 pypi_0 pypi
seaborn 0.9.0 pypi_0 pypi
sentencepiece 0.1.82 pypi_0 pypi
setuptools 41.0.1 py37_0
six 1.12.0 pypi_0 pypi
snowballstemmer 1.9.0 pypi_0 pypi
sortedcontainers 2.1.0 pypi_0 pypi
spacy 2.1.4 pypi_0 pypi
sphinx 2.1.2 pypi_0 pypi
sphinxcontrib-applehelp 1.0.1 pypi_0 pypi
sphinxcontrib-devhelp 1.0.1 pypi_0 pypi
sphinxcontrib-htmlhelp 1.0.2 pypi_0 pypi
sphinxcontrib-jsmath 1.0.1 pypi_0 pypi
sphinxcontrib-qthelp 1.0.2 pypi_0 pypi
sphinxcontrib-serializinghtml 1.1.3 pypi_0 pypi
sqlite 3.28.0 h7b6447c_0
sqlparse 0.3.0 pypi_0 pypi
srsly 0.0.7 pypi_0 pypi
subword-nmt 0.3.6 pypi_0 pypi
tensorboard 1.14.0 py37hf484d3e_0
tensorboardx 1.8 pypi_0 pypi
tensorflow 1.14.0 gpu_py37h4491b45_0
tensorflow-base 1.14.0 gpu_py37h8d69cac_0
tensorflow-estimator 1.14.0 py_0
termcolor 1.1.0 pypi_0 pypi
tf-estimator-nightly 1.14.0.dev2019042301 pypi_0 pypi
thinc 7.0.5 pypi_0 pypi
tk 8.6.8 hbc83047_0
toolwrapper 1.0.0 pypi_0 pypi
torch 1.1.0 pypi_0 pypi
tqdm 4.32.2 pypi_0 pypi
unidecode 1.1.1 pypi_0 pypi
urllib3 1.25.3 pypi_0 pypi
wasabi 0.2.2 pypi_0 pypi
wcwidth 0.1.7 pypi_0 pypi
werkzeug 0.15.4 pypi_0 pypi
wheel 0.33.4 py37_0
word2number 1.1 pypi_0 pypi
wrapt 1.11.2 py37h7b6447c_0 anaconda
xz 5.2.4 h14c3975_4
zipp 0.5.2 pypi_0 pypi
zlib 1.2.11 h7b6447c_3
More information about the infrastructure
mailing list