“The Illusory Superpowers of Oversized Linguistic Models: A Tongue-in-Cheek Exploration”
“Large Language Models’ Emergent Abilities Are a Mirage”
“As it turns out, machines built for the sole purpose of ‘reading’ vast amounts of text from the internet – nearly a trillion words might be able to learn something. Just like how a toddler might learn to swear if you always mumble expletives while they put together their toy building blocks.”
Oh, the shock, the horror, an AI that learns from absorbing vast quantities of data. Truly a groundbreaking revelation. But seriously folks, this isn’t a plot from a science fiction flick, so let’s put down the popcorn and get back to reality. In a fascinating article that recently surfaced on WIRED, Katherine Rieder steers us through the exciting world of ‘large language models’. These are not your everyday chatbots that call you incessantly during your lunch break. No, these are machines, like GPT-3, that can comprehend and generate human-like text, and they achieve it through a training method called ‘machine learning’.
Ever shared a secret with a friend and found that the next day the entire town knows about it? Well, that’s effectively what these sophisticated language models are designed to do, albeit for the greater good. They gobble up mountains of internet text – we’re talking about nearly a trillion words folks – encode it, learn from it, and then tu-ta-loo, they’re able to answer your questions, draft an email or even help you pen your next novel.
Wait, doesn’t that sound eerily similar to how our human brains learn too? We consume data (read, experience, listen), store it away in our little gray cells and then leverage it when needed – during a trivia night, for instance. Only difference is, we need every bit of those countless sleepless nights, heartbreaks, laughter, integrating, forgetting, and relearning to make sense of everything, while these models just need a steady Wi-Fi.
Now, let’s take a step back and look at the bigger picture. The rise of large language models begs a critical question: how quickly can these systems adapt and learn? More importantly, do these models end up learning things we didn’t initially anticipate them to learn? Kind of like when your pet dog learns to open the front door – a neat trick until you find your living room covered with confetti of paper towel.
In essence, that’s what the researchers at OpenAI set out to investigate. They found that GPT-3 did indeed learn many unexpected revelations. Shockingly (not), the model picked up biases present in the data it was fed. Guess Garbage In, Garbage Out (GIGO) holds true even for AI models, huh?
So, the next time your AI autocomplete suggests something downright bonkers, remind yourself that it’s merely parroting the vast sea of information it’s been exposed to. Also, be sure to cut it some slack. After all, we’re all a product of what we consume, aren’t we? Human or AI.