LLM “Reasoning” Continues to be Deeply Flawed
As you may know, I have been harping on reasoning as a core challenge for deep learning for well over a decade, at least since a December 2012 New Yorker article:
“Realistically, deep learning is only part of the larger challenge of building intelligent machines. Such techniques lack ways of representing causal relationships (such as between diseases and their symptoms), and are likely to face challenges in acquiring abstract ideas … they have no obvious ways of performing logical inferences, and they are also still a long way from integrating abstract knowledge”
By now many, many others have pounded on the same point.
Silicon Valley’s response has always been to dismiss us.
- We got this covered, the Valley CEOs will tell you. . . .
- Give us money. Lots of money. Lots and lots of money.
- Scale Scale Scale.
- AGI is coming next year!
Well, AGI still hasn’t come (even though they keep issuing the same promises, year after year). LLMs still hallucinate and continue to make boneheaded errors.
And reasoning is still one of the core issues.
How serious an issue? A new review from Caltech and Stanford, called Large Language Model Reasoning Failure shows the latest deep learning systems — even the ones marketed as “reasoning” systems — (still) have major problems with reasoning.
Lots and lots of problems, everywhere they looked
Silicon Valley can do one of two things. It can continue to dismiss critics and criticism, praying for magical solutions, or it can face reality and start focusing on AI & Data Center Alternatives.