AI agents still need humans to teach them

The best performances came from agents with curated skills, which scored an average of 16.2 percentage points higher than agents with no skills, an indication that AI cannot yet do without human intervention. Even so, in 16 out of the 84 tasks human guidance had a negative impact on results.

Performance varied widely across industry sectors, with curated skills having the biggest impact in healthcare tasks, but only a small one for software engineering.

Agents asked to generate their own skills demonstrated no increase in performance, showing that AI still requires some human prompting to get the job done.

About Author

AndyC

Andy Curtis is an award-winning security consultant, researcher and public speaker. He has been working in the computer security industry since the early 1990s, having been employed by state and federal government, leading healthcare and banking providers across three continents. He has given talks about computer security for some of the world’s largest companies, worked with law enforcement agencies on investigations into hacking groups, and is a regular voice on TV and radio explaining IT security threats.

See author's posts