What is the difference between CUDA core and CPU core?

Question

I worked a bit with CUDA, and a lot with the CPU, and i'm trying to understand what is the difference between the two. My I5 processor has 4 cores and cost $200 and my NVidia 660 has 960 cores and cost about the same.

I would be really happy if someone could explain what are the key differences between the two processing units architecture in terms of abilities pros and cons. For example, does a CUDA core have branch prediction?

Answer 1

It is a computer Architecture question which entails a long answer. I will try to keep it very simple on the risk of being inaccurate. You basically self-answered your question by asking do CUDA core handle branch prediction, the answer is NO. A CPU core has to handle each single operation a computer does, calculation, memory fetching, IO, interrupts, therefore it has a huge complex instruction set, and to optimize the speed of fetching instruction branch prediction is used.
Also it has a big cache and fast clock rate. To implement the instruction set you need more logic thus more transistors more cost per core compared to the GPU.

The GPU cores have less cache memory, simpler instruction and less clock rate per clock, however they are optimized to do more calculation as a group. The simple instructions set, the less cache memory makes them less expensive per core.

Answer 2

Cuda cores are more lanes of a vector unit, gathered into warps. In essence cuda cores are entries in a wider AVX or VSX or NEON vector.

The closest to a CPU core is an SMX. It can handle multiple contexts (warps, hyper threading, SMT), and has several parallel execution pipelines (6 FP32 for Kepler, 2 on Haswell, 2 on Power 8). And each SMX is independent, just as any core or a general purpose CPU.

This analogy is detailed further here: https://stackoverflow.com/a/36812922/6218300.

Answer 3

They are now in principle the same as CPU cores. It isn't long ago that this wasn't true for example they have been unable to process integer operands in 2005.

When comparing the CPU Core complexity of your 2-core i5 keep in mind that the original 80386 CPU had just about 275K transistors while a Core2Duo has about 230 Million. 1000 times more, so the numbers fit well.

The biggest difference is the memory handling which becomes even more complicated then the good old days when we need segmentation registers. There is no virtual memory etc. and it is the very thin bottleneck when you try to port your normal CPU programs but the real problem is that non local memory access is very expensive 400-800 cycles. They are using a technique that outside the GPU world only the SUN Niagara T1/T2 general purpose CPU had. While waiting for a memory access they schedule different set of threads with other instructions that are ready (called wraps). But if all the threads do is non-linear jumping around your data your performance just fails.

Answer 4

You need to understand the fundamental difference between CPU Vs GPU and the need for rise of GPGPU in recent tims. One of the informative course on this is available in Udacity

Also, this book might be helpful for beginner level programs.

Though this is not a programming question. Hope it might help someone.