External GPU for machine learning [closed]

Question

I own MacBook Pro 15' Mid 2014 and I am thinking about buying Titan X GPU to speed up training of my neural networks. Titan will be connected via Thunderbolt 2 as an external GPU.

What sort of performance can I expect from this setup - will it be the same as if it was connected to the motherboard? Does the thunderbolt limit the speed of the GPU?

Answer 1

I recently connected a GTX 970 via Thunderbolt 2 to my Macbook Pro 13 Late 2013. GPU benchmarks yielded around 70% performance when compared to using a PCI-Express socket inside of my desktop as Thunderbolt 2 does limit the PCI-Express to X4 bandwidth speeds vs x16 in a desktop application.

The cheapest way of achieving this is to use an Akito 2 enclosure, remove the external case and rear to fit a large GPU, then you can wire a normal ATX power supply to the graphics card and to the Akito dock power source. You must provide 75 watts to the PCI-Express slot in order to use an external graphics card so using the power pack that comes with the Akito device is not possible.

There's a lot of information on the web about the Akito 2 set up I recommend you take a look.

Answer 2

Transferring data between the CPU and GPU is quite costly in machine learning, and can end up being a real bottleneck. Using an external graphics card will therefore have a significant impact on performance, and I definitely wouldn't recommend it.

I've done some ML on a 4 year old Macbook Pro and it worked fine for the dataset I was working on, however if you have some heavy number crunching to do you can't beat a desktop computer with a good dedicated graphics card.

If a desktop PC isn't possible you could also consider using an online service such as Amazon EC2 which offer servers with GPUs.

You'll also need to make sure that the framework you are using supports GPU acceleration (not all of them do) and that the performance increase is significant.