Something I’ve known is true for a long time, but have realized even more as time goes on, is that most people aren’t really that smart. That’s really not an insult as much as it’s a liberating revelation.
As someone who has struggled with self-confidence, it’s been very important for my success to understand that most people’s “intelligence” is actually BS masked with confidence.
I’ve met so many people – ones really good at something – that don’t start their own business because they mistakenly think entrepreneurs have it all together. They think they could never do it because, at first glance, anyone running their own business looks really impressive.
It’s not always true. In fact, it’s usually not true. And, honestly, it’s quite terrifying because you suddenly realize your future is in greater jeopardy with a “stable job” than working for yourself.
So, why am I going on about this?
Because the AI hype is being pushed by people who sound really, really smart but are also just ordinary people who are better at sounding smart than being smart.
After reading several industry people discussing OpenAI’s September 5th post, “Why language models hallucinate,” I noticed they all commented on OpenAI’s remarks that the solution is surprisingly simple.
The biggest, most core problem with LLMs… the one that started with telling people to eat rocks and drink bleach, the one that is still being sued for allegedly being complicit in suicides… has a “surprisingly simple” solution?
And nobody thought of this? More on that, but first:
The article summarizes the paper like this:
- Models are trained to reward ANY answer to a query more than no answer at all. So, if I ask the model to tell me how many people visit Helsinki from Canada every year, and it doesn’t yet “know,” it is incentivized to guess because getting lucky and guessing the right answer is worth more than saying it doesn’t know.
- LLMs are simply “next word predictors.” So if you take trillions of words, you’ll pretty quickly train a model to understand correct grammar and spelling. But, given it’s not an actual human with true reasoning ability, when you apply the same method to less common data, it hallucinates.
- “the straightforward solution is to penalize confidently generated errors more than you penalize uncertainty. There is a straightforward fix. Penalize confident errors more than you penalize uncertainty, and give partial credit for appropriate expressions of uncertainty.”
This is where I’m unsure if the OpenAI team was too incompetent to see this problem from the beginning, too far into training to change their methods before launching ChatGPT, or more likely: a deliberate decision because humans actually do crave any answer over no answer.
I’m, of course, fully speculating here, but one thing Sam Altman is very good at is persuasion. He understands how humans of all kinds think, and that is a gift.
Launching ChatGPT in its original stage, almost certainly knowing it was going to deliver billions, if not trillions, of wrong answers to people is laughable, if not despicable and absurdly negligent.
You will probably never convince me that the OpenAI team wasn’t fully aware of this all along.
I understand “move fast, break things” is how we operate in this era. It’s how I always operated as well.
But I also didn’t have tens of millions (now hundreds of millions) of people, clueless to how the technology works, using my product.
Then again, perhaps it is a human problem, and OpenAI simply brought it to light.
After all, “ChatGPT can make mistakes.” has been underneath that text field this whole time, but, like all fine-print, we choose, or are conditioned, to ignore it.
We all want what we want, and we want it right now. The instant gratification society we live in is not going away.
If Sam Altman, or someone else at OpenAI, chose to reward their original models for guessing over accuracy because they knew it would lead to a more viral, widely talked about experience, then I’m not sure if they disgust me – or inspire me.
It was smart in that it kicked off a revolution prematurely, one that clearly wasn’t ready. One that Google itself, the actual discoverers of the Transformer architecture, wasn’t even moving forward with until they were forced.
This paper is a joke because it’s an admission of either massive incompetence or being the greatest illusionists of all time.
And that makes total sense because, like my initial point, people are either a lot dumber than you think – or they confidently tell you what you want to hear.
Sounds familiar.
