What is a GPU?

A graphics processing unit (GPU) is an electronic circuit that can perform mathematical calculations at high speed. Computing tasks like graphics rendering, machine learning (ML), and video editing require the application of similar mathematical operations on a large dataset. A GPU’s design allows it to perform the same operation on multiple data values in parallel. This increases its processing efficiency for many compute-intensive tasks.

A GPU is excellent at performing general-purpose parallel processing, but historically, this wasn’t always the case. As the name suggests, GPUs were initially designed for one specific task: controlling image display.

What is a GPU: Origin of the graphics processing unit

Before the GPU, we had dot matrix screens, which were released in the early 1900s. Vector and raster displays came next, and then later the first video game consoles and PCs were released. At the time, a non-programmable device called a graphics controller coordinated the display with the screen. Graphics controllers traditionally relied on the CPU (central processing unit) for processing, although some included on-chip processors.

Around the same time, there was a 3D imaging project concerned with generating a single pixel on a screen with a single processor. The goal was to produce an image that combines many pixels in a short amount of time. This project was the origin of the GPU as we know it.

It wasn’t until the late 1990s that the first GPUs were released. These were aimed at the gaming and computer-aided design (CAD) markets. The GPU integrated a previously software-based rendering engine and transformation and lighting engine with the graphics controller—all on a programmable chip.

Evolution of GPU technology

Nvidia was the first to market the single-chip GeForce 256 GPUs in 1999. The 2000s and 2010s marked a growth era where GPUs gained functions like ray tracing, mesh shading, and hardware tessellation. These led to increasingly advanced image generation and graphics performance.

It wasn’t until 2007 that Nvidia officially released CUDA, a software layer making parallel processing available on the GPU. Around this time, it became clear that GPUs were very effective at performing highly specific tasks. GPUs excel at tasks that require a large amount of processing power to achieve a particular outcome.

When Nvidia released CUDA, it opened up GPU programming to a wider audience. Developers could then program GPU technology for all sorts of different compute-intensive practical applications. GPU computing started to become far more mainstream.

GPUs are an in-demand chip for simulation and other emerging applications. They’re a key hardware technology in artificial intelligence and machine learning (AI/ML) applications.

GPUs can be used across a wide range of compute-intensive applications, including large-scale finance, defense applications, and research activities. Here are some of the most prevalent uses of GPUs today.

Gaming

The GPU’s first applications that extended beyond large business and government visualization applications were in personal gaming. They were used in the gaming consoles of the 1980s and still are in PCs and current gaming consoles. GPUs are essential for complex graphical rendering.

Professional visualization

GPUs are used in professional applications such as CAD drawing, video editing, product walkthroughs and interactivity, medical imagery, and seismic imaging. They are also applied to other complex image and video editing and visualization applications. Browser-based applications can even exploit the GPU through libraries such as WebGL.

Machine learning

Training a machine learning (ML) model requires a large amount of compute power. They can now run on GPUs for accelerated results. While it might take a long time to train a model on self-purchased hardware, you can achieve results quickly by using a cloud GPU.

Blockchain

Cryptocurrencies are built on blockchains. A particular type of blockchain, proof of work, typically heavily relies on GPUs for operation. Application-specific integrated circuits (ASIC), a similar but different chip, are now a common replacement for GPU processing for blockchain.

Proof of stake blockchain algorithmic proofs remove the need for massive amounts of compute power, but proof of work is still pervasive.

Simulation

Advanced simulation applications such as those used in molecular dynamics, weather forecasting, and astrophysics can all be accomplished through GPUs. GPUs are also behind a lot of applications in automotive and large vehicle design, including fluid dynamics.

Modern GPUs typically contain a number of multiprocessors. Each has a shared memory block, plus several processors and corresponding registers. The GPU itself has device memory on the board it is housed on, known as VRAM.

Each GPU works slightly differently depending on its purpose, the manufacturer, the specifics of the chip, and the software used for coordinating the GPU. For instance, Nvidia’s CUDA parallel processing software allows developers to specifically program the GPU with almost any general-purpose parallel processing application in mind.

GPUs can be standalone chips, known as discrete GPUs, or integrated with other computing hardware, known as integrated GPUs (iGPUs).

Discrete GPUs

Discrete GPUs exist as a chip that is fully dedicated to the task at hand. While that task has traditionally been graphics, discrete GPUs are now used as dedicated processing for tasks like ML or complex simulation.

When used in graphics, the GPU typically resides on a graphics card that slots into a motherboard. In other tasks, the GPU may reside on a different card or slot directly onto the motherboard itself.

Integrated GPUs

In the early 2010s, we started to see a move away from discrete GPUs. Manufacturers embraced the introduction of the combined CPU and GPU on a chip, known as the iGPU. The first of these iGPUs for PCs were Intel’s Celeron, Pentium, and Core lines. These remain popular across laptops and PCs.

Another type of iGPU is the system on a chip (SoC) that contains components like a CPU, GPU, memory, and networking. These are the types of chips typically found in smartphones.

Virtual

Like other types of computing hardware infrastructure, GPUs can also be virtualized. Virtualized GPUs are a software-based representation of a GPU that shares space alongside other virtual GPUs on cloud server instances. You can use them to run your workloads without having to worry about underlying hardware maintenance.

What is a cloud GPU?

A cloud GPU is a graphics processing unit hosted within the cloud and accessed via the internet. From a hardware standpoint, cloud GPUs are the same class of graphics processors used in on-premises systems, like those on a dedicated graphics card or video cards for enterprise use. However, these GPUs are inside servers and connected through high-speed interconnects, which serve a similar role to PCIe slots in traditional machines. You connect to the GPU through virtual machines or containers, not actually interacting with the physical hardware.

Cloud GPUs are often used for machine learning, deep learning, graphics rendering, or other workloads involving either compute graphics or large-scale data processing. GPUs excel at parallel operations, making them perfect for computationally intensive tasks like training AI models.

While local systems rely on integrated graphics processing units (integrated GPUs), cloud GPUs provide access to discrete GPUs. You can access these GPUs through the internet, offering consistent GPU performance for workloads that need maximum performance.

Being highly scalable, cloud GPUs are frequently used in workloads that are executed remotely and streamed to end users, like cloud gaming. Especially as they become more accessible, more companies are using cloud GPUs to power their parallel processing business use cases like ML applications.

The main difference between a CPU and a GPU is their purpose in a computer system. They have different roles depending on the system. For example, they serve different purposes in a handheld gaming device, a PC, and a supercomputer with several server cabinets.

In general, the CPU handles full system control plus management and general-purpose tasks. Conversely, the GPU handles compute-intensive tasks such as video editing or machine learning.

More specifically, CPUs are optimized for performing tasks such as:

• System management
• Multitasking across different applications
• Input and output operations
• Network functions
• Control of peripheral devices
• Memory and storage system multitasking

‘Graphics processing unit’ and ‘graphics card’ are often used interchangeably but these terms do not describe the same thing. Graphics cards are an add-in board (AIB) that slots into a place on a computer’s motherboard. Graphics cards are not built into the computer itself; they are interchangeable cards. A graphics card comes with a GPU on the board.

The GPU is the main component on graphics cards, alongside other components like Video RAM (VRAM) for video memory, ports (such as HDMI or DisplayPort), and a cooling component. However, a GPU can also be built directly into the motherboard or integrated as an all-in-one chip together with other components, such as an iGPU or SoC unit.

Amazon Web Services (AWS) offers Amazon Elastic Compute Cloud (Amazon EC2), the broadest and deepest compute platform.

Amazon EC2 has more than 500 instances and your choice of the latest processor, storage, networking, operating system, and purchase model. It can easily match the needs of your workload.

With Amazon EC2, it’s easy to rent and run GPUs in the cloud. Take advantage of GPUs for processing power for video editing, render graphics, artificial intelligence (AI), and other parallel processing capabilities.

Amazon EC2 instances can fit into almost any type of workload:

• P2 instances are intended for general-purpose GPU compute applications.
• Amazon EC2 P5 instances have eight built-in GPUs. They're the latest generation of GPU-based instances. They provide the highest performance in Amazon EC2 for deep learning and high performance computing (HPC).
• Amazon EC2 G5 instances are powered by AWS Graviton2 processors and feature NVIDIA T4G Tensor Core GPUs. They provide the best price performance in Amazon EC2 for graphics workloads such as Android game streaming.

Start using GPUs on AWS by creating an account today.