The Definitive Guide to Turning your Business and Data Center into an AI Factory

Hitachi Vantara’s Octavian Tanase offers the definitive guide to turning your data centers (and business) into an AI factory. This article originally appeared on Solutions Review’s Insight Jam, an enterprise IT community enabling the human conversation on AI.

AI factories and foundries moved into the popular lexicon when Nvidia CEO Jensen Huang used these terms in his recent keynote address. Traditionally, foundries transform raw materials into basic components, and factories assemble products. But AI factories and foundries are a new way to approach product creation and innovation – and they are poised to fundamentally change software development, the resources they consume and how they can be managed.

That is altering how businesses operate and increasing the value that they create and deliver.

Here’s how.

AI and GenAI Provide Opportunities

Companies are now beginning to leverage GenAI to improve their productivity. For example, Hitachi Vantara is using GenAI copilots and large language models (LLMs) to assist its customer service, marketing and sales teams. Joint research from MIT and Stanford indicates that GenAI can enable customer support agents to resolve 14% more issues per hour – and shows that GenAI drove productivity improvements of 34% for new and lower-skilled workers.

GenAI and AI are also changing the way software is built and how it works. Software engineers can become a lot more productive with a GenAI assist. McKinsey research indicates that with the help of GenAI, software developers can document code functionality for maintainability (which considers how easily code can be improved) in half the time, write new code in nearly half the time and optimize existing code (called code refactoring) in nearly two-thirds the time.

Companies are also starting to embed AI and machine learning (ML) into the software they build, enabling businesses to make better decisions based on a dynamic understanding of the customer and use case rather than static heuristics.

As organizations adopt and expand their use of GenAI and use more and more software with built-in AI and ML, they will want to consider how to prepare their IT infrastructure so they have the underlying power and flexibility they need. This presents an opportunity for organizations to upgrade and modernize their infrastructure in collaboration with trusted partners.

No Single Firm Will Own Everything Needed to Make GenAI efforts a Success

A lot of underlying hardware and software is needed to power AI and GenAI so companies can use these technologies to meet their goals. That creates the need to establish highly integrated processes throughout the product lifecycle and in the way that companies run their businesses.

Businesses and their key partners must ensure that they and every vendor in their supply chains comply with all relevant requirements and enforce best practices that fit the model of the company that is deploying AI. That includes following the right processes and having checks and balances on the materials they use, their manufacturing processes, the way they design software, and how they transport and deliver solutions. Alignment and tight integration are especially critical with GenAI, which requires significant compute and storage resources and, if left unchecked, can lead to runaway compute costs, power usage and carbon emissions.

A recent media report notes that one version of a single Nvidia Blackwell chip for data center use draws 1,200 watts of electricity, “an insane amount of power compared to just a few years ago.” GenAI, and the AI foundries and factories that enable GenAI applications, will need to rely on lots of compute, interconnect networks and storage to link the massive datasets to the compute. That means organizations also need optimization akin to FedEx’s approach of delivery, which involves choosing optimal routes and taking other steps to ensure timely delivery while reducing gas consumption and costs and limiting carbon emissions.

Consider This

Look at AI as a workload (or, to be specific, a suite of workloads), but understand that GenAI creates different workloads than companies have seen in the past. We are all learning about GenAI as we go, but one thing is clear: the way that organizations form their infrastructure around these new workloads must change, and putting the right infrastructure in place is not a trivial task. A big chunk of that infrastructure will be in the cloud, but on-premises infrastructure also has a role to play.

Deciding on and building the most appropriate systems in the cloud and on-prem to optimize efficiency will require a lot of analysis and expertise, so collaborate with partners that can position your GenAI efforts for success today and in the future. Partners that are innovative and have proven their expertise in deploying and running mission-critical infrastructure will empower you to get the greatest value from GenAI.

 Understand there are no quick answers; optimizing GenAI is an iterative process. Embrace solutions that simplify infrastructure and automation and partners that have the capability to help you with everything from data preparation (which includes removing unnecessary data and obfuscating sensitive data) to bringing data together on scalable, flexible and cost-effective data storage, to AI model training, to interference. Seek trusted partners with deep experience in your business vertical and with the kind of data-centric workflows you will be working with.

 Remember that AI and GenAI are all about data. So be sure you have the most relevant and complete data as well as the right data infrastructure to locate, secure and safeguard your data.

With the right approach and data infrastructure behind your GenAI and AI efforts, your business can become an AI factory of sorts, using software to optimize operations, differentiate offerings, and drive business growth.