AI models master capabilities long before exhibiting them, research shows

Artificial intelligence (AI) models possess some capabilities long before they exhibit them during training, new research has shown. According to the research carried out by Havard and the University of Michigan, the models do not showcase these abilities until they need to in one way or another.

The research is one of the many that has been carried out to understand how AI models build their capabilities before showcasing them.

The study analyzed how AI models learn basic concepts like size and color, revealing they master the skills earlier than most tests suggest. The study also provided insight into the complexity of measuring an AI’s capabilities. “A model might appear incompetent when given standard prompts while actually possessing sophisticated abilities that only emerge under specific conditions,” the paper reads.

Research shows AI models internalize concepts

Havard and the University of Michigan are not the first to try to understand AI model capabilities, with researchers at Anthropic unveiling a paper titled ‘dictionary learning’. The paper discussed mapping out connections in their Claude language to specific concepts it understands. Although most of these researches took different angles, it is primarily to understand the AI models.

Anthropic revealed it found features that could be tied to different interpretable concepts. “We found millions of features which appear to correspond to interpretable concepts ranging from concrete objects like people, countries, and famous buildings to abstract ideas like emotions, writing styles, and reasoning steps,” the research revealed.

During its research, the researchers carried out several experiments using the diffusion model, one of the most popular architectures for AI. During the experiment, they realized the models had distinct ways to manipulate basic concepts. The patterns were consistent as the AI models showed new capabilities in different phases and a sharp transition point signaling when a new ability is acquired.

During the training, the models showed they had mastered concepts around 2,000 steps earlier than a standard test would detect. Strong concepts appeared around 6,000 steps and weaker ones were visible around 20,000 steps. After the concept signals were adjusted, they discovered a direct correlation with learning speed.

Researchers reveal methods to access hidden capabilities

The researchers used alternative prompting methods to reveal hidden capabilities before they were exhibited in standard tests. The rampant nature of hidden emergence has effects on AI evaluation and safety. For instance, traditional benchmarks may miss out on certain capabilities of the AI models, thereby missing both the beneficial and concerning ones.

During the research, the team figured out certain methods to access the hidden capabilities of the AI models. The research termed the methods linear latent intervention and over-prompting, as researchers made the models exhibit complex behaviors before they show in standard tests. Researchers also discovered that the AI models manipulated certain complex features before they could show them through standard prompts.

For instance, models could be prompted to generate ‘smiling women’ or ‘men wearing hats’ successfully before being asked to combine them. However, research showed they’ve learned to combine it earlier, but will not be able to showcase it through conventional prompts. The models showcasing capabilities can be said to be grokking, a situation where models exhibit perfect test performance after extended training. However, the researchers said there are key differences between both.

While grokking happens after several training sessions and involves refining several distributions of the same data sets, the research shows these capabilities emerge during active learning. The researchers noted that the models found new ways to manipulate concepts through change in phases rather than gradual representation improvements in grokking.

According to the research, it shows that AI models know these concepts, they are just unable to showcase them. It is similar to people watching and understanding a foreign movie but cannot speak the language. This shows that most models have more capabilities than they show, and it also shows the difficulty in understanding and controlling their capabilities.

From Zero to Web3 Pro: Your 90-Day Career Launch Plan