The Author, Unassisted · Ownership

Cognitive atrophy in you, and hallucinations in the machine, are symptoms of the same issue.

Human-written. Endgate Systems

This one may sting for people hard-set on disgracing LLM's for two issues that are the responsibility of the human to resolve.

AI hallucinations and cognitive atrophy. These are two of the biggest concerns with the LLM based systems that popularized "AI" as one of the most popular narratives on earth.

What most people do not recognize is that both of these issues: a machine that gets things wrong, and a human who erodes their thinking: are symptoms of the exact same issue.

Hallucinations

Let's look at hallucinations first: a machine fabricating information, routinely getting things wrong, and/or not properly delivering on tasks: often in a way that seems silly and hysterical within certain contexts.

Hallucinations are when you think the machine should absolutely be sufficient to do something correctly, yet it fails anyway.

Cognitive atrophy

Then we have cognitive atrophy: the reality in which a human mind degrades, and erodes its own thinking by ceding thought to a machine.

This is a valid concern for those utilizing AI systems, although the concern is ungrounded in where the issue actually comes from.

Hint: it's not the machine.

These issues emerge from the same place

What these two phenomenons have in common is a lack of literacy in the user regarding what an LLM is, what their strengths are, and what their weaknesses are. 

An LLM by nature is non-deterministic system. That is its greatest strength and weakness. Full stop.

It is the non-deterministic nature that made it groundbreaking for the machine learning space. It is what allowed engineers to take vast datasets and categorize information within them quickly, and with greater precision. It is what allowed a machine to work in unpredictable environment without gaps in logic and precision.

That same non-deterministic nature without proper guardrails, infrastructure, and logic, is what turns an LLM into a weakness. Without deterministic logic to guide non-determinism, you are left entirely with "best guess" randomness. Totally ungrounded.

If you expect an LLM to perform perfectly or with precision without structuring around it properly: you are probably the exact person who has a machine that routinely hallucinates, and you are probably the same person who is outsourcing thinking itself to the machine.

The ChatGPT culture and what it created

We have written extensively in our blog posts about the launch of ChatGPT, and investigating the maturity of releasing LLM's to the general public in the fashion that it did. The ChatGPT culture has only created inflation around how LLM's should behave.

The fact is that few people were taught about how an LLM actually works. Now we have millions of people treating it like an oracle, depending on it for everything, and getting upset when it does not work well.

Understand the stark contrast between a serious AI system: grounded in real datasets, hooked up to real file-systems with instruction sets, and routine workflows that it is calibrated to do well deterministically — and mindless dependence on an LLM alone, disconnected from real infrastructure, and ill-equipped to do its job well.

It is the ChatGPT culture of "just ask chat" instead of "use a machine to build real systems and workflows that work deterministically in service of your goals" — that creates over-hyped expectations around what these systems can do, and are meant to do well without assistance.

Two sides of the same coin

Cognitive atrophy is degraded mental performance on the human side. AI hallucination is degraded algorithmic performance on the machine side.

Isn't it interesting how a lack of literacy on the human side determines the entire outcome?  

Transformer based architecture (the engine behind the LLM) was designed in 2017 by Google because it works. However most people were never taught how it works, and what what conditions it works best in.

No one worries about cognitive atrophy when a machine is working semi-autonomously across a privately owned instruction-set and data set with dozens of different files and cleanly organized sets of rules: in order to produce something that a human could not have, or would not have willingly, on their own.

A machine is for serious work. Not casual chatting a 3AM.