GPU Programming Model Introduction from a Big Data CEO

Solutions Review’s Expert Insights Series is a collection of contributed articles written by industry experts in enterprise software categories. In this feature, Row64 CEO Marc Stevens offers an introduction to the GPU programming model to help keep you on-trend.

Did you know that every day 2.5 quintillion bytes (aka 25 billion gigabytes) of new data is created? This is roughly double from just 3 years ago, and if trends continue likely just one fourth as much data as will exist by 2025!

For businesses, this growth in data can present an exciting (if not overwhelming) opportunity. Increased data points allow for more granular customer targeting and streamlined sales funnels, allowing businesses to pivot and adapt to trends in real time. However, analyzing these treasure troves of data often requires teams of data scientists— whose average salary per employee runs $150,000 a year.

Furthermore, developing and managing data processes often takes months, precious time which can cost business customers (especially during important sales opportunities of the year like the holiday season).

So why does parsing data cost so much time and money? It may be surprising to learn that much of the current bottleneck is actually hardware-based.

While companies are often upgrading their machines to the latest and greatest, the reality is that almost all companies are still using software that only runs on their CPUs, which are limited in how much data they can process—even on newer computers. To combat this, some finance and insurance companies are utilizing an emerging technology known as ‘GPU compute’. GPU compute shifts the computational burden of data processing from a computer’s generalized CPU to its highly specialized GPU—often achieving results that are 5-10x faster than a CPU alone, at vastly larger scales. For instance, while Microsoft Excel caps out at 1 million rows, GPU powered spreadsheets have been recorded manipulating dataset of up to 1 billion rows in real time.

So can your business take advantage of GPU compute? And if so, how do you do it? To answer these questions we dive deeper into the world of GPU programming, how it works and what you need to use it.

GPU Programming Model

The Secret Power Of “Low Code” Programming

When it comes to GPU programming, the first question many people ask is “Why can’t I just tell my programs to use my GPU”? After all, video games and editing software have been using powerful new graphics cards like the Nvidia RTX series for years.

The answer is that GPUs and CPUs actually speak different computing languages, Which means it’s not enough to just buy a new computer with a fast GPU, but you actually need software that can speak directly to the GPU. These languages (for example CUDA) are often referred to as “low level” code, and the reality is very few programs work with GPU languages at present.

The reason for this is because while GPUs are highly efficient at doing specialized tasks (primarily complex calculations in real time) they lack the versatility of a CPU—which can arrange and organize tasks to execute in order. Which means that outside of rendering sets of geometric pixel instructions into 3D worlds (aka graphics), the GPU hasn’t had a lot of computing tasks that suit its natural function.

However, recently, analysts at McKinsey as well as business schools like the Graziadio School at Pepperdine University have realized that mathematical functions— such as those required in spreadsheets or data analysis—greatly suit the hyper fast and focused nature of GPU processors.

The difficulty has been finding programmers who are skilled in GPU programming (which are far fewer than those programming for CPUs) and use their code to design programs and code snippets designed for data analysis.

The Importance of Implementing GPU Processing Into Your Data Workflow

The old adage goes: time is money—and when it comes to data analysis for business this is especially true.

The reality is that while data insights are tremendously useful (increasing new customer acquisition by an estimated 23x) 80 percent of time spent on data is simply prepping the data for use. That is to say a majority of time of data scientists is spent ingesting and cleaning data—not generating valuable insights.

This is where the importance of GPU compute comes into play. Unencumbered by hardware bottlenecks, the time to load and manipulate data is dramatically reduced. In studies, programs using GPU compute were able to sort and manipulate data up to 17x faster than legacy programs like Excel. Furthermore, the entire data process is often streamlined, as the processing power of GPUs allow all phases of the data process— including data ingest, manipulation, cleaning and visualization—all through one central program.

By unlocking the power of GPUs, companies can increase the efficiency of their existing teams by as much as 2-3x—with no budget increase to staff or hardware investments. It simply utilizes the power of your machine to its maximum efficiency, tapping into stores of processing power previously unused for your data process.

With this power, a small e-commerce business can easily process data from thousands of users to determine interest in new products, create hyper targeted sales and or track thousands of conversion metrics. High frequency traders can perform a myriad of pre-trade analytics, such as sentiment analysis and transaction cost analysis. Insurance companies are able to achieve increased granularity in their risk assessment and thus personalized pricing.

Ultimately, thus GPU programming is the means to scale data analysis without needing to add in more inputs. It simply increases computational efficiency. The application of scaled data has virtually infinite use, and as users increasingly migrate all aspects of their lives online, and companies who have the means to track and utilize data will be a decisive boost over those who don’t. Which is why when we say it comes to modernizing your data workflow for the future, turning to your GPU can provide serious benefits.