Today’s large language models have evolved into statistical parrots that sound very reliable, but are they?
“The human and tangible approach with which Large Language Models (LLMs) formulate answers to our questions today causes our trust in classic models to decline,” begins Matthijs van den Langenberg, AI Adoption Lead Benelux at Lenovo, during a roundtable discussion organized by ITdaily.
“AI models are actually advanced data analyses that can convert structured data into natural language,” says Véronique Van Vlasselaer, Head of AI & Data Science at SAS EMEA, who compares them to statistical parrots.
Three other experts are also at the table: Wilco Zwerus, Client Solutions Sales Director at Dell Technologies, Joris Dresselaers, Business Area Director at Visma, and Gianni Cooreman, Senior Director Solution Engineering for Salesforce Benelux. Where does the (sometimes misplaced) trust in AI models come from, and how do companies deal with this?
Statistical parrots
“Today’s LLMs are actually very advanced systems for data analysis,” says Van Vlasselaer. “Something like this has existed for much longer than today. In recent decades, solutions mainly concerned analysis based on structured data; today, technology has made it possible to analyze unstructured data such as text.”
LLMs are statistical parrots.
Véronique Van Vlasselaer, Head of AI & Data Science bij SAS EMEA
For that reason, LLMs are sometimes called statistical parrots, she notes. According to van den Langenberg, this makes the models accessible to everyone, but it also has a downside.
Deterministic versus probabilistic
According to Van Vlasselaer, there is a striking paradox in how people deal with AI. “Classic, deterministic models work on the basis of clear logic and always give the same reproducible answer, but often encounter distrust: a system that indicates something is potentially fraudulent must be able to justify its reasoning immediately.”
“LLMs, on the other hand, or probabilistic models, inspire much more confidence, precisely because they formulate answers in natural language and therefore feel like a trusted conversation partner.”
She finds this discrepancy striking: “People trust the probabilistic, and therefore inherently uncertain, models more than the deterministic systems that are specifically designed to be reliable and verifiable.”
Humanity wins
Everyone around the table agrees with this paradox. “AI models talk with feeling,” Cooreman notes. “They can proclaim lies with full confidence, but if you don’t double-check the outcome, you’ll fall for it completely.” Moreover, the trust seems mainly applicable to today’s ChatGPTs. Van Vlasselaer: “There is 200 percent more trust in generative AI than traditional AI.”
Van den Langenberg also acknowledges that making technical tools non-technical makes them more accessible to the public. “Most people don’t understand the very technical tools, which makes them trust them less.”
Dresselaers adds: “There is a danger in blindly trusting AI. He illustrates this with an example from the accounting world. If an accountant files a tax return based on answers from AI, which may contain incorrect information, it can have far-reaching consequences.” In short, without a critical eye, you often end up deceived, with all the resulting consequences.
Data as a foundation
In addition to trust in the AI model itself, there is another important step that companies (sometimes unconsciously) skip: trust in the data itself. “Often there is a lot of old data in models that is no longer relevant at all,” says Zwerus. Van den Langenberg also confirms: “Many companies struggle with data that is simply not properly organized. Then they are no longer data lakes, but data swamps.”
Companies are not dealing with data lakes, but data swamps.
Matthijs van den Langenberg, AI Adoption Lead Benelux at Lenovo
Guardrails
Furthermore, the participants notice a lack of clear AI frameworks. “Governance is one of the most difficult factors in the implementation of AI,” says van den Langenberg. According to him, many companies do not yet have an internal framework in place. He compares it to the lanes on a highway. You can change lanes, but if there are no guardrails, you’ll fly off the road.
Not only is the framework for the internal use of AI by employees important, but also the framework in which the so-called AI agents must operate. “AI agents take over the steering completely to reach the destination,” Dresselaers adds.
An AI agent only becomes truly good because of the rules you give it.
Matthijs van den Langenberg, AI Adoption Lead Benelux at Lenovo
“An AI agent only becomes truly good because of the rules you give it,” van den Langenberg notes. “For example, we have an internal legal agent that only looks at certain documents or guidelines, and is therefore well-defined. Precisely because there are things it cannot and may not do, it is better at what it can do.”
AI checking AI
Finally, Van Vlasselaer points to a phenomenon that is appearing more and more frequently: LLM as a judge, or the use of one AI model to evaluate the output of another AI model. “But who judges the judge then?”, Dresselaers asks rhetorically.
The experts agree: the use of AI requires a critical, human eye. Moreover, it is important to start at the beginning: is my data in order? Then establish a clear framework, so that everyone stays neatly within the guardrails of the same highway. Finally, always check the output. A statistical parrot may sound convincing, but that doesn’t make it reliable.