AI Mimicry

 

 

On October 7, 2024, Apple published a research paper that discusses the limitations of mathematical Reasoning in large language models (LLMs). This is yet another proof that LLMs are about language and have very little in common with understanding the underlying meaning.

This is by the way quite similar to how human brains work, as we have (at least) two separate neural networks (and probably more) - one responsible for parsing and constructing language statements, and a completely separate one where the actual "thinking" happens. I posted a blog on that back in May.

Long story short, the Apple researchers point out that despite the overall heralded progress in LLMs, they are still bound by limits of pattern matching, when they "convert statements to operations without truly understanding their meaning". This "suggests deeper issues in reasoning processes" that can't be helped with fine-tuning or other refinements. In other words there is still this deep illusion of understanding as the models simply do not have an idea what they are taking about, while they talk (most of them very eloquently).

The paper also describes how the models can be easily distracted when prompted in a direction that doesn't precisely match any training data. Like when an AI video generating engine was asked for a cat drinking a can of beer it failed because there weren't videos of cats drinking beverages in the training data. Instead, the model pulled from what it had learned about videos of cats and videos of beer and combined them. The result was a cat with human hands.

Quoting Benj Edwards (Ars Technica):

One of the reasons OpenAI's GPT-4 turned heads in text synthesis is that the model finally reached a size where it was large enough to have absorbed enough information (in training data) to give the impression that it might be able to genuinely understand and model the world when, in reality, a key aspect of its success is that it "knows" far more than most humans and can impress us by combining those existing concepts in novel ways. With enough training data and computation, the AI industry will likely reach what you might call "the illusion of understanding".

In other words, the LLMs know large number of possible answers humans would have and select the most probable one, most of the time missing (or being distracted by) data which does not fit the training data. This, for example, fundamentally questions using the current AI technology in applications like self driving cars. The cars can repeat reactions which were in the training set, but would be unable to properly react to new situations (which happen on roads every day).