Enterprises need to think beyond GPUs for agentic AI, analysts say

Because Agentic AI involves a different computing model than genAI training on GPUs, enterprises need to consider the hardware options and pricing models available through cloud providers. “It’s more about model management than about model building — and the CPU is critical in providing workflow management,” said Jack Gold, principal analyst at J. Gold Associates.

Pricing variations continue to be an issue. Straight CPU compute is not billed the same as heavy GPU use, making it difficult to nail down costs, Gold said. “GPUs in training use more electricity generically due to near 100% utilization in a training workload, whereas in general-purpose compute, servers and CPUs run more like 40% to 60% utilization,” he said. “But it’s highly variable depending on what the agent is doing.”

Gold predicts that 80% to 85% of AI workloads will move to inference in the next two to three years, especially as tools become more agentic. “CPUs take on a major significance in making everything work. It’s why all the hyperscalers are now loading up on CPUs, not just GPUs,” Gold said.

About Author

AndyC

Andy Curtis is an award-winning security consultant, researcher and public speaker. He has been working in the computer security industry since the early 1990s, having been employed by state and federal government, leading healthcare and banking providers across three continents. He has given talks about computer security for some of the world’s largest companies, worked with law enforcement agencies on investigations into hacking groups, and is a regular voice on TV and radio explaining IT security threats.

See author's posts