[NLPL Task Force (A)] Storage alternatives

[Andrey Kutuzov]

The Python binary is usually located in the 'bin/' directory, not 'lib/'.

'lib/' contains installed Python packages, and for this I indeed can
easily believe in 5 GiB and more. But not the Python executable/binary
itself: it rarely exceeds a few MiB in size, whatever are the Cython
optimizations or anything.

But this is just to clear up the details. Of course the storage space
problem still persists, even if it is because of packages, not Python
binary.

18.11.2020 14:10, Vinit Ravishankar wrote:
> I am sure, I followed du -h and for eg. .miniconda3/envs/nlp/lib/python3.7 is 5 GiB. A quick Google search hints that this is because of Cython optimisation but I have no idea honestly.
> 
> The size of the binary is different for different environments, so it seems to depend on the packages installed.
> 
> – Vinit
> 
>> On 18 Nov 2020, at 14:08, Andrey Kutuzov <andreku at ifi.uio.no> wrote:
>>
>>
>> 18.11.2020 13:47, Vinit Ravishankar пишет:
>>> The space issues aren’t just the huggingface models (though those
>>> are obviously an issue too) - a single virtual environment is multiple
>>> gigabytes worth of libraries, where just the python3.7 binary is often
>>> ~5 gigabytes.
>> Are you sure about that? What can I do to make the Python binary 5 GB worth?
>>
>> Just interested, may be I don't understand something.
>>
>> -- 
>> Andrey
>> Language Technology Group (LTG)
>> University of Oslo
> 


-- 
Andrey
Language Technology Group (LTG)
University of Oslo