[NLPL Task Force (A)] Storage alternatives

[Vinit Ravishankar]

My bad, I assumed it was some sort of binary because it was called python3.7

– Vinit

> On 18 Nov 2020, at 14:20, Andrey Kutuzov <andreku at ifi.uio.no> wrote:
> 
> The Python binary is usually located in the 'bin/' directory, not 'lib/'.
> 
> 'lib/' contains installed Python packages, and for this I indeed can
> easily believe in 5 GiB and more. But not the Python executable/binary
> itself: it rarely exceeds a few MiB in size, whatever are the Cython
> optimizations or anything.
> 
> But this is just to clear up the details. Of course the storage space
> problem still persists, even if it is because of packages, not Python
> binary.
> 
> 18.11.2020 14:10, Vinit Ravishankar wrote:
>> I am sure, I followed du -h and for eg. .miniconda3/envs/nlp/lib/python3.7 is 5 GiB. A quick Google search hints that this is because of Cython optimisation but I have no idea honestly.
>> 
>> The size of the binary is different for different environments, so it seems to depend on the packages installed.
>> 
>> – Vinit
>> 
>>> On 18 Nov 2020, at 14:08, Andrey Kutuzov <andreku at ifi.uio.no> wrote:
>>> 
>>> 
>>> 18.11.2020 13:47, Vinit Ravishankar пишет:
>>>> The space issues aren’t just the huggingface models (though those
>>>> are obviously an issue too) - a single virtual environment is multiple
>>>> gigabytes worth of libraries, where just the python3.7 binary is often
>>>> ~5 gigabytes.
>>> Are you sure about that? What can I do to make the Python binary 5 GB worth?
>>> 
>>> Just interested, may be I don't understand something.
>>> 
>>> -- 
>>> Andrey
>>> Language Technology Group (LTG)
>>> University of Oslo
>> 
> 
> 
> -- 
> Andrey
> Language Technology Group (LTG)
> University of Oslo