Why are deep learning libraries so huge?

Question

I've recently downloaded all packages from PyPI. One interesting observation was that of the Top-15 of the biggest packages, all execept one are deep learning packages:

• mxnet: mxnet-cu90 (600 MB), mxnet-cu92, mxnet-cu101mkl, mxnet-cu101 (and 6 more mxnet versions)
• cntk: cntk-gpu (493MB)
• H2O4GPU (366MB)
• tensorflow: tensorflow-gpu (357MB), tensorflow

I looked at mxnet-cu90. It has exactly one huge file: libmxnet.so (936.7MB). What does this file contain? Is there any way to make it smaller?

I'm especially astonished that those libraries are so huge considering that one usually uses them on top of CUDA + cuDNN, which I thought would do the heavy lifting.

As a comparison, I looked at related libraries with which you can also build deep learning libraries:

• numpy: 6MB
• sympy: 6MB
• pycuda: 3.6MB
• tensorflow-cpu: 116MB (so the GPU version needs 241 MB more or around 3x the size!)

Answer 1

Deep learning frameworks are large because they package CuDNN from NVIDIA into their wheels. This is done for the convenience of downstream users.

CuDNN are the primitives that the frameworks call to execute highly optimised neural network ops (e.g. LSTM)

The unzipped version of CuDNN for windows 10 is 435MB.