The AI Industry’s Scaling Frenzy: A Comedy of Errors with a Cliffhanger Ending!
“The AI Industry’s Scaling Obsession Is Headed for a Cliff”
“The AI field is well aware of the scaling problem. CPU consumption by AI has been doubling every 3.4 months, leading to a 300,000 times increase between 2012 and 2018 (Stanford University’s AI Index 2019 report).”
The fetish for ‘scaling’ is something within the artificial intelligence (AI) industry that you should be aware of. Let’s face it, this obsession is teetering on the edge of a cliff and there’s a good chance we’ll all come to regret it. As it’s known, AI’s CPU consumption has been doubling every 3.4 months. Simply amazing or should we say, alarming? This escalation has led to a spectacular surge of 300,000 times between 2012 and 2018. Stanford University’s AI Index 2019 report isn’t just making things up, y’know.
Even with the current situation, there’s a hearty dose of belief with a side of aversion to the idea that this ‘scaling’ approach will continue to be the golden egg of AI. It seems like we’ve forgotten that obsessively manufacturing larger models without much thought for their efficiency, or lack thereof, is akin to going on a road trip without a plan. Sure, it could be ‘fun’, but eventually, you’re going to run out of fuel miles away from a gas station. And nobody enjoys getting stuck in the middle of nowhere.
Miniaturization. This term may not be a swanky buzzword in AI circles, but it attempts to create a balance between the size of the models and their ability to function effectively. But, guess what? It too seems to be hanging on by a thread. Stay tuned folks, the best is yet to come.
Size isn’t everything, despite what they say. True greatness lies within efficiency—think Antman. Ditching the trend of bigger is better, a gallant few are battling the status quo. They’re fostering AI systems that are lean, mean, and phenomenally smart. Imagine how transformative it would be if this became the norm?
Let’s be realistic for a moment, the obsession around scaling is not going to disappear overnight. After all, who doesn’t like big toys? But imagine living in a world where AI isn’t just about how much CPU you can guzzle. Oh, what a world that would be!
In the end, it’s important to remember that there’s a cliff coming up. We have a choice – continue sprinting towards the edge like Thelma and Louise, or we put on the brakes and reconsider. Let’s hope wisdom prevails, or we’re all in for a pretty free fall.