AI inference is getting cheaper, but your agents are getting more expensive

‘Swarms’ of agents eating up tokens

There is no doubt that AI delivers massive value, and is often better, and faster, than people at many routine tasks, the analysts pointed out. Agents can also more quickly identify patterns across siloed systems. For instance, Gartner has seen customer success agents reduce response times by 99%.

But as AI evolves, token usage increases, and token value is variable, Sommer and Zimmerhansl noted. Advanced AI agents that can reason already cost up to 150x more on a single task than basic AI chatbots.

A simple chatbot must read and interpret a request and quickly deliver a “probabilistically reasonable” answer, but agents, as they become more sophisticated, must think, question, and adapt when something goes wrong, and they increasingly run continuously, and often invisibly, in the background.

About Author

AndyC

Andy Curtis is an award-winning security consultant, researcher and public speaker. He has been working in the computer security industry since the early 1990s, having been employed by state and federal government, leading healthcare and banking providers across three continents. He has given talks about computer security for some of the world’s largest companies, worked with law enforcement agencies on investigations into hacking groups, and is a regular voice on TV and radio explaining IT security threats.

See author's posts