Artificial Intelligence is not that intelligent yet, Caution Apple researchers

Not-so-intelligent clever machines

The study reveals some capabilities in the existing models. For instance, ChatGPT-4o still managed to achieve a 94.9% precision rate in tests, even though this rate notably declined when researchers introduced a more intricate problem.

Thus far, that is satisfactory, but the success rate nearly fell apart — declining by up to 65.7% — when researchers altered the challenge by incorporating “seemingly pertinent but ultimately unimportant statements.”

These decreases in precision indicate the constraints present in the current LLM models, which fundamentally rely on pattern recognition to produce outcomes rather than employing any genuine logical reasoning. This signifies that these models “transform statements into operations without actually comprehending their significance,” as per the researchers’ statement.

About Author

AndyC

Andy Curtis is an award-winning security consultant, researcher and public speaker. He has been working in the computer security industry since the early 1990s, having been employed by state and federal government, leading healthcare and banking providers across three continents. He has given talks about computer security for some of the world’s largest companies, worked with law enforcement agencies on investigations into hacking groups, and is a regular voice on TV and radio explaining IT security threats.

See author's posts