Afaik, gpu cores a very stripped down cpu cores that do basic math, but what sort of math can cpus do that gpus can’t
I’m talking, cpu core vs Tensor/Cuda cores.
You must log in or register to comment.
Not so much GPUs themselves but GPU shader compilers really struggle on some code with a lot of branching and loops, operating on strings.
- It’s the GPU itself. GPUs work by grouping together multiple threads into a single thread group, which NVIDIA calls a warp and AMD/Intel call a wave. Every thread belonging to a given warp/wave has to take the exact same path through the code, so if you have heavily divergent branches where some threads in a warp/wave take one path and other threads take another path then that can kill performance as the code needs to be executed multiple times to cover all paths.
- GPUs typically don’t operate on strings. Strings aren’t even a supported data type in most GPU-oriented languages or frameworks, especially graphics. If you need to operate on strings on a GPU then you typically break the strings up into individual characters and treat each character as a simple integer, which is totally fine so long as you’re able to wrap your head around dealing with strings at such a low level.
I think it’s more about the efficiency of doing the math than the ability to perform the math.
CPUs and GPUs are both Turing complete, so from a computation perspective, they can technically do the same things. However each is optimized for a different set of metrics, so it may be wise to target a particular workload to one over the other.
The CPU is optimized for minimum latency, aka maximum single threaded performance. This is really important for a lot of workloads.
GPUs are optimized for maximum throughput and are willing to suffer higher latency to achieve it. There are many architectural differences as a result of this goal, for example: GPUs don’t have branch prediction, they also don’t have out of order processing, and also have much smaller caches per computational unit. All of these saves a ton of transistors, which can be used to add more cores.
So no point in making Linux or Windows that can run 100% within GPU and leaving a very minimal CPU to handle data transfer between drives, keyboard, etc and GPU.
I’d like to see someone with a 4090 GPU running Windows and a Motorola 68000 as the CPU to handle drives.
GPUs are not necessarily Turing complete, BTW.
I think it’s more about the instruction set. CPUs are designed to be general purpose and can handle any processing required. GPUs have a specialized instruction set but are highly parallel which is useful for those specialized workloads.
Get 1 answer in 10 microseconds: Use a CPU.
Get 1000 answers in 1000 microseconds: use a GPU.
GPUs are not good with branching, as in taking different paths based on previous calculations.
A CPU core has a variety of arithmetic and logic circuits making it good for general purpose compute, whereas a GPU core (CUDA or Stream processor) has a much more limited set of logic units, which allows each one to be much smaller physically. This is why a GPU can fit thousands of cores while standard desktop CPUs are rarely more than 12.
The eli5 is that one is like a very strong man, capable of carrying large objects or pulling a car.
The other is like a man with a thousand arms. They might not be very strong, but they can carry a thousand plates.
I like this one the best, only problem with this analogy is there is no common ground between pulling a car and carrying plates in terms of goal achieved.
How about a CPU can swing one 1000 pound hammer while a GPU can swing 1000 1 pound hammers?
CPUs are general purpose processor that do the general calculations and data. If a PC is handling AI algorithms, they are often done in the CPU. CPUs handle the more complex large file handling. CPUs handle the general interactions in a game. CPUs have fewer but massive complex processor cores optimized for large complex logic work.
GPUs are their own specialist computers optimized for complex graphics and physics vector calculations from hundreds of thousands of really tiny simple files. CPUs handle the general interactions in a game. GPUs handle the complex lighting, light rays, fragmentation physics and image 3D rendering. CUDA and Tensor Cores of a GPU are thousands of puny simple processors optimized for tiny random floating point calculations.
The real difference between a GPU and a CPU is the access to the rest of the hardware. You could do logical operations on both, but a GPU can’t address your hardware so it wouldn’t be overall very useful as the only processing unit.
Basic GPU handles algorithms for graphics and outputs to a display. Latest GPUs have specialized instructions to handle mathematical algorithms for parallelization so the Host OS hits certain routines it executes those on the GPU getting usually 8 - 32X speedup on execution over the CPU for mathematical routines.
I imagine it like a construction site for a new house. Do you want 10 master craftsman who can do everything or 300 guys off the street who kinda know what they’re doing and are really on there for grunt labor. For the tasks that require lots of manual labor you want the 300 guys, but for those tasks that aren’t easy or aren’t very parallel, you want fewer but more capable cores.
Think of it this way.
A CPU is a big digger, its bucket can dig a large chunk of earth with one scoop. Heavy lifting.
A GPU is a lot of small shovels, each digging a small amount of earth, but all at the same time.
So a CPU can do a large task at once, a GPU can do a lot of smaller tasks at once.
Which is great, the CPU is doing grunt work, the GPU is doing small calculations.
Both the GPU and the CPU being perfectly tasked for their specific purpose.
Security is an interesting reason that most people don’t think about.
When you run a program on your computer, you’re constantly swapping between user and privileged modes. For example, you don’t want a website reading the stuff from your harddrive. Any such attempts must go to the OS which will then say the website doesn’t have permission and refuse.
GPUs don’t have a privileged mode. This isn’t just because it wouldn’t be useful. To the contrary, webGL or WebGPU have massive security issues because you are executing third-party code using drivers which themselves generally have root access. GPUs don’t add such a mode because their hardware takes forever (in CPU terms) to get all the data loaded up and ready to execute. Moving to a different context every few microseconds would result in the GPU spending most of its time just waiting around for stuff to load up.
The solution to this problem would be decreasing latency, but all the stuff that does that for you is the same stuff that makes CPU cores hundreds of times larger than GPU cores. At that point, your GPU would just turn into an inferior CPU and an inferior GPU too.
my guess is they just have different instruction sets but both are able to do basic add, substract and multiply