If you have too many “slow” modes in a super computer you’ll hit a performance ceiling where everything is bottle necked by the speed of things that are not the CPU: memory, disk for swap, and network for sending partial results across nodes for further partial computing.

Source: I’ve hang up too much around people doing PhD thesis in these kinds of problems.