The Real Facts Behind AI’s Supposed Cognitive Decline

In today’s column, I examine the recent spate of claims that generative AI and large language models (LLMs) supposedly suffer from cognitive decline. This certainly sounds juicy. One related disconcerting factor is that comparing AI to the human condition associated with cognitive decline is an inapt analog and sadly another sign of excessive anthropomorphization of AI. In any case, the crux of the contention is that AI is presumably in some kind of state of decline regarding mustering computational and mathematical forms of exhibited intelligence.

Let’s talk about it.

This analysis of an innovative AI breakthrough is part of my ongoing Forbes column coverage on the latest in AI including identifying and explaining various impactful AI complexities (see the link here).

Human Cognitive Decline

Before we unpack the AI side of this matter, we ought to first establish the human facets of cognitive decline since that’s the barometer being used in these emerging claims.

According to the American Psychological Association (APA), the definition of cognitive decline consists of this aspect (excerpted from APA online dictionary):

• “Reduction in one or more cognitive abilities, such as memory, awareness, judgment, and mental acuity, across the adult lifespan.”
• “The presence and degree of decline varies with the cognitive ability being measured.”
• “Cognitive decline is a part of normal healthy aging, but a severe decline is not normative and could be symptomatic of disease.”
• “It is the primary symptom of disease-induced dementias, such as Alzheimer’s disease.”

I’m sure that we’ve all encountered someone who has been undertaken by dementia. They typically will be slow to process their thoughts and might at times wander from a given conversation. Forgetfulness is relatively common. The person gropes for words and has a tough time with a persistent train of thought.

As noted above, cognitive decline can range from minor to wholly significant. The aging process has a natural impact on cognition and leads to a semblance of cognitive decline. If you meet someone at the age of 40 and carry out various intelligence tests and revisit them when they are 80 years of age, the chances are that they will perform less capably on the intelligence test.

It would seem somewhat intuitive to suggest that people as they age are gradually going to reach a point in their later years that entails cognitive decline. Not everyone does so to the same degree. Not everyone does so at the same age. Nonetheless, it isn’t surprising that for example, a 40-year-old when they are 80 years old is likely to show signs of cognitive decline from that early base test to their latest intelligence test.

Generative AI And LLMs

Shift gears and focus on modern-day generative AI and LLMs. I will provide some quick background about the general nature of AI so that we are all on the same table as we get further into the unpacking of the premise of AI and cognitive decline.

First, the usual way of developing generative AI consists of doing a massive data training effort that involves scanning a wide swath of content on the Internet. The approach entails doing mathematical and computational pattern-matching of how humans write and make use of words. After performing this large-scale pattern matching, the AI system is tuned and then released for public usage. The AI appears to be fluent in natural language and can engage in human-like dialogue.

For more in-depth details on the building of AI, see my discussion at the link here.

Second, the AI makers are rapidly progressing their generative AI models. New techniques and ways to optimize the underlying pattern-matching are a fast-moving realm of research and practice, see for example breakthroughs that I’ve depicted at the link here and the link here. The aim is to make the AI bigger, faster, and more responsive.

Though an AI maker might decide to take an existing version and extend it, the likelier approach consists of starting over with the newest capabilities and performing the data training all over again. This makes abundant sense. Rather tha…

Revisiting The Decline Consideration

Now that we’ve gotten that matter addressed, I’d like to explore this topic with a fresh eye.

At the outset, let’s put aside the word “cognitive” since that is usually reserved to refer to humans. The same goes for the word “think” – which is popularly used when referring to AI but again is overloaded with meaning since we use that word when describing human thought and cognition.

I will simply revert to indicating that we can be interested in whether AI can be devised to improve over time or whether it might be devised such that it declines in capability over time. Notice that I didn’t have to use the word “cognitive” or “think” in expressing that consideration.

Is it possible that AI such as generative AI or LLMs might worsen or decline over time in their capabilities at responding to user prompts?

Yes, this is indeed possible.

Here’s how.

Suppose that we take an LLM and decide to give it some augmented data training. Our goal is that we hope the additional data will boost the AI. You can undertake such an effort readily using retrieval-augmented generation (RAG), as I describe at the link here.

But imagine that inadvertently the data used for this augmented effort was lousy data. It is replete with falsehoods and mistakes. The proper way to do things would be to have examined and screened the data beforehand. Assume for the sake of discussion that the data was just used as is. No sensibility checks and balances were performed.

What might happen to the LLM that is augmented in this manner?

Well, if we administer tests and those tests perchance tap into the pattern-match realm that involves that lousy data, the odds are that the AI will do less well on the test. The AI will be answering based on flawed data.

Voila, the AI has declined in performance.

Fine Tuning And Forgetting

You might be tempted to question whether having merely added lousy data to AI was the only way that the AI might decline in performance over time.

There are plenty of other avenues.

For example, suppose an AI maker opts to tweak their LLM.

One aspect that has come up frequently is that people want to have the so-called right-to-forget when it comes to generative AI, see my coverage at the link here. This entails having the AI maker delete out facets that are in the AI, that someone believes ought to be “forgotten” about them. There is a great deal of controversy over this matter and not everyone agrees that having AI be adjusted this way is the proper ethical or legal way to go (others insist that it is).

Anyway, envision that an AI maker is adjusting and refining their generative AI over time. They ought to do so with a careful hand. I mention this because they could lop out useful aspects that are crucial to their generative AI. If they aren’t mindful of the changes they are making, there is a solid chance that the AI will perform worse than it did before.

That would be a considered example of generative AI or LLM that has declined or performed worse over time.

Final Thoughts To Mull Over

A twist to that aspect is that the changes to the AI don’t necessarily have to be undertaken by the human hand. Some of the latest AI has been devised to be so-called self-healing, self-repairing, self-improvement, self-reflection, etc., see my coverage at the link here. It’s a dual-edged sword. The AI might improve itself. Yay. But there’s no guarantee that the upside is the only result. The AI might undercut itself. Sad face.

Again, you would contend that the generative AI has declined or worsened over time.

The most popular twist on this consists of using synthetic data for data training of generative AI and LLMs. It goes like this. Right now, the data on the Internet that is used for data training is substantially human-crafted writing. Generative AI meanwhile is being used to pump out responses, i.e. data, of which some or a lot is being posted to the Internet. This is referred to as synthetic data.

Some research studies have suggested that if you data train generative AI on principally synthetic data, this can lead to what is known as catastrophic model collapse. The idea is that you are using cloned data and the AI gets weaker and less viable accordingly. I’ve examined the truth of this claim, see the link here.

So, yet another pathway to having AI decline over time.

A final comment for now.

You might be familiar with Mark Twain and his famous satirical remarks about his supposed death in 1897 (he actually passed away in 1910), namely that such reports were greatly exaggerated. It’s a clever remark by the vaunted writer and humorist. There’s a lesson to be learned from that insight. Be cautious when you hear or read about AI and various eye-catching claims.

Dig in and find out the real story.