Menu
Is there a straightforward way to find the GPU memory consumed by, say, an inception-resnet-v2 model that is initialized in tensorflow? This includes the inference and the backprop memories required.
896
TensorFlow (TF) GPU 1.6 and above requires cuda compute capability (ccc) of 3.5 or higher and requires AVX instruction support.
By default, TensorFlow maps nearly all of the GPU memory of all GPUs (subject to CUDA_VISIBLE_DEVICES ) visible to the process. This is done to more efficiently use the relatively precious GPU memory resources on the devices by reducing memory fragmentation. To limit TensorFlow to a specific set of GPUs we use the tf.
The average memory requirement is 16GB of RAM, but some applications require more memory. A massive GPU is typically understood to be a "must-have", but thinking through the machine learning memory requirements probably doesn't weigh into that purchase. However, it can make or break your application performance.
HOW TO CALCULATE GPU PERFORMANCE (RDNA 2 based) ROPS * Clock Speed = Fill Rate TMU * Clock Speed = Texture Fill Rate & RT perf (PEAK) CU * 64 * Clock Speed * 2 = TFLOPS (FP32 Perf) For XSX this gives 116Gpixel (Fill Rate), 380(texture / RT), & 12.1TFLOPS.
You can explicitly calculate the memory needed to store parameters, but I am afraid it would be difficult to compute the size of all buffers needed for training. Probably, a more clever way would be to make TF do it for you. Set the gpu_options.allow_growth config option to True and see how much does it consume. Another option is to try smaller values for gpu_options.per_process_gpu_memory_fraction until it fails with out of memory.
72
If you love us? You can donate to us via Paypal or buy me a coffee so we can maintain and grow! Thank you!
Donate Us With
