LLMs don’t do formal reasoning - and that is a HUGE problem

"We found no evidence of formal reasoning in language models …. Their behavior is better explained by sophisticated pattern matching -- so fragile, in fact, that changing names can alter results by ~10%!" ~ Mehrdad Farajtabar | Source

"There is just no way you can build reliable agents on this foundation, where changing a word or two in irrelevant ways or adding a few bit of irrelevant info can give you a different answer." ~ Gary Marcus | Source