GPU for deep learning

28

7

I have a budget of ~10 kUSD to buy GPUs for neural network applications, mostly using Theano, Torch, TensorFlow and Caffe. Most programs I plan to run are for natural language processing. Computations are in float 32 and in CUDA. I live in the USA, and I don't pay for the electricity bills. The GPUs will be mounted to some computers running Ubuntu 14.04.3 LTS x64.

What are the graphic cards with the highest computational power / price ratio, given the above-mentioned conditions?

Franck Dernoncourt

Posted 2015-12-20T23:23:14.753

Reputation: 1 598

Haven't used them, but have you checked out Nvidia's GPU's?

Dawny33 2015-12-21T05:34:50.553

You'll definitely want to look into Nvidia's workstation cards and not desktop cards for this sort of thing since you're focusing on very specific and cumbersome tasks.Adam 2015-12-21T06:40:38.390

I don't have a full answer but Tesla M40's are the best for this type of work, although with your budget you will be restricted to 2 cards.Ctrl-alt-dlt 2016-02-05T11:59:57.213

Answers

17

I haven't personally used CUDA for this, though I am planning to. From my research I concluded, that you for sure want to have 'Computing Capability' at least equal to 3.5, as some libraries requre it already. The (tough to find) list of computing capability is under this link. From this list one can conclude, that having GTX 980 or Titan (both 5.2 score) is as good as you can get, but note, that if you are buing a graphics card only for this, Nvidia has an answer for professional and academic use named Tesla - it's just a computing box, far from graphics card (it has no display ports even!), costs from 3k $ for K20, to 5k $ for K80 model, and is a behemoth:

A quick comparision (CC stands for Compute Capability):

  • Tesla K20: for desktops, peek for float: 3.52 Tflops, 5 GB, 2496 CUDA cores, 2.9 k $, CC: 3.5
  • Tesla K40: for desktops, peek for float: 4.29 Tflops, 12 GB, 2880 CUDA cores, 3.1k $, CC: 3.5
  • Tesla K80: for servers, peek for float: 8 Tflops, 24 GB, 4992 CUDA Cores, 5k $, CC: 3.7

and customer grade, most popular and new graphics cards:

Also see comparision on wccftech.

To conclude: the commercial grade GPU seem to be more cost efficient, but only when we compare the specs. There can be other trade-offs, I am not awere of. Thus, I cannot confidently say "go with customer grade", but I can tell you what I would (will) do - I will buy GTX 960 or 970, because I plan on gaming and I'm quite cost limited, and this cards will do just fine for CUDA learning. If you buy for an institution, do not plan gaming, the calculations will go 24/7, consider the academic grade Teslas.

Also, if you'll be interested in boosting your 'conventional' integer based processing power on a high-end computation server, you may want to look up Xeon phi.

[EDIT] Please note, that switching from CPU based floating point arithmetic to GPU enchanced, is a change in quality, almost one order of magnitute, and will be very pronounced and noticible, but switching from ex. Tesla K20 to Tesla K40 will be just a change in quantity (K80 is just two K40 bundled together), so if you go for speed to price ratio, go with cheapest GPU acceleration, that'll work for you.

Piotr Falkowski

Posted 2015-12-20T23:23:14.753

Reputation: 740

4

As a note, Facebook uses the Nvidia Tesla M40 in their open sourced AI Hardware.

Andy 2015-12-22T00:24:01.487

8

Nvidia has just announced the Nvidia GTX 1080, which is ~25% faster than the Titan X and significantly cheaper ( 600 USD vs 1000 USD).

From http://www.anandtech.com/show/10304/nvidia-announces-the-geforce-gtx-1080-1070/2:

enter image description here

From http://wccftech.com/nvidia-geforce-gtx-1080-launch/:

enter image description here

It's difficult to find exhaustive specification comparison tables between the GTX 1080 and Titan X (I guess they should appear soon). A few more comparisons GTX 1080 vs Titan X:

  • 9 Tflops vs 6.1 Tflops

From some Nvidia official presentation:

enter image description here

enter image description here

The Nvidia GTX 1070 was also announced:

enter image description here

From cnn-benchmarks:

GTX 1080 > Titan X: Across all models, the GTX 1080 is 1.10x to 1.15x faster than the Titan X.

Franck Dernoncourt

Posted 2015-12-20T23:23:14.753

Reputation: 1 598

0

Nvidia announced on 2016-07-21 the new GTX Titan X:

  • Under 1200 USD
  • "Potentially 24% faster than GTX 1080; 60% faster than the old Titan X."
  • 11 teraflops of FP32 performance
  • 12GB of GDDR5X memory running at an effective 10GHz and attached to a wide 382-bit bus, resulting in a 480GB/s of memory bandwidth

A first benchmark (unfortunately they didn't keep the same CuDNN…):

enter image description here

Franck Dernoncourt

Posted 2015-12-20T23:23:14.753

Reputation: 1 598