What is a GPU?

The GPU, or Graphics Processing Unit was initially intended to be secondary device that the CPU could offload the increasingly complex task of rendering (representing 3D meshes in one or more 2D images) onto. Because of this, GPUs had to be designed to perform many calculations very quickly in order to keep up with the stream of images needing to be displayed. This was accomplished by making the GPU highly parallel, allowing for multiple parts of an image to be rendered at once and thus reducing the total amount of time it took to render an image compared to the CPU.

To get an idea of the vast difference in computing that GPUs brought, check out this awesome visual demonstration by Mythbusters duo Adam Savage and Jamie Hyneman.

Early on, GPUs were highly specialized for rendering, meaning it was difficult if not impossible to harness

How GPUs Work

In essence, GPUs consist of dozens, sometimes hundreds, of individual execution units that can complete instructions in parallel with one another. Each unit is responsible for a single execution thread, which represents a single sequential process that can be run in parallel. To carry out a parallel task, threads are grouped together into thread blocks which all connect to the same shared memory pool and run the same process, called a kernel. Kernels are launched by the CPU, which then initializes a given set of threads to run as a block until the task is completed, at which point they are released from the block and are available for other tasks. Multiple kernels can also run parallel to one another on the same GPU provided there are enough threads available, allowing both data and task parallelism at the same time.