AI Actually in Your Car? Qualcomm's Next-Gen Chips Are Poised to Bring It There
The new horsepower wars are happening on the semiconductor level, with AI at the center of it.
Artificial intelligence isn't just taking over the news cycle. It's taking over our lives. Or at least that's what a lot of experts in the tech space are saying it will do. We've seen AI applied in all sorts of interesting ways, and all of the advanced AI experiences we as consumers can sample today have one thing in common: They run on a computer that's far away from you.
0:00 / 0:00
Most of today's AI agents require so much computing horsepower that they must live on a server farm or supercomputer somewhere else. That means occasionally high latency as your request must go out over the internet and back, a minor annoyance that comes along with some disconcerting implications regarding the privacy of your data.
Qualcomm, a technology supplier for many of the world’s top automakers, including BMW, Mercedes-Benz, and Rivian, wants to bring that AI tech a little closer. The company has introduced a series of new chips capable of making the next generation of software-defined vehicles so powerful that they can run AI right there in the dashboard, creating a new era of in-car assistants that will make today's most advanced voice recognition features look positively remedial.
The new horsepower wars are digital, and with chips like these, plus a major partnership with Google, we're potentially just a few years away from taking supercomputer-like AI power on the road.
Bringing AI to the Edge
Artificially intelligent agents like OpenAI's GPT or Meta's Llama are a little tricky to compare because they were all developed and trained in different ways to satisfy different needs. But one way of stacking them up against each other is to look at their relative sizes.
To do this, you can compare the number of so-called parameters contained within a model. This is, basically, the number of interconnections between points of logic within the model's digital brain. More parameters have previously meant a bigger and more advanced AI agent that behaves more realistically and comprehensively.
Early models, like OpenAI's GPT-3, clocked in at a whopping 175 billion connections. When it launched, it was so big it was impossible—or at least impractical—to run locally. But that's changing. In just the few years since, software developers have dramatically pared those models back without reducing their functionality.
"If you take a look at Llama as an example, this year's Llama 3, 8 billion parameter model outperforms last year's Llama 2, 70 billion parameter model," Durga Malladi said. He's SVP and GM of technology planning and edge solutions at Qualcomm. "We went from 175 billion parameters to 8 billion parameters in two years, and the quality of the models has only increased."
That's like going from a 6.0-liter V-8 to a 1.5-liter three-cylinder and getting more power, more torque, and better fuel economy out of the deal, only the reduction in size is a few orders of magnitude bigger.
Eight billion parameters is still quite large, but it's reaching a point where these more compact AI agents could be run not on the cloud but on a local device. Running this kind of technology locally is broadly called edge computing, and that's the major push for Qualcomm's new tech.
Rise of the NPU
To give cars a major computing power boost, Qualcomm is introducing two upgraded system-on-chip (SoC) semiconductor processors, the Snapdragon Cockpit Elite and the Snapdragon Ride Elite. These are an evolution of the platforms introduced at CES in 2022.
Snapdragon Cockpit Elite is designed to power the next-generation in-car experiences, offering three times the computing and graphics processing power of the previous generation. That's enough to simultaneously drive up to 16 individual displays, each with a 4K resolution and featuring the latest 3D rendering technologies like ray tracing. In other words, it's plenty of pixels to dazzle even the most jaded of gamers in your family while ensuring you never see the tiniest hint of lag when swiping through infotainment menus.
Meanwhile, the Snapdragon Ride Elite SoC is more focused on the nuts and bolts of keeping the car operating safely, with support for more than 80 simultaneous sensors, including high-resolution radar, imaging, and lidar. That's more sensors than Waymo uses for its sixth-generation self-driving car.
Indeed, Ride Elite, Qualcomm says, will have enough digital horsepower to provide up to SAE Level 4 autonomy. This means cars that can fully drive themselves in limited situations, even if you're not paying attention.
Beyond more power, a big part of what these chips offer is an upgraded component called a neural processing unit, or NPU. If you're familiar with the idea of a graphics processing unit, or GPU for accelerating computer graphics, it's the same concept. Here, though, that chip is optimized for the sorts of complex logic that virtual brains demand. Qualcomm's NPU, called Hexagon, is said to deliver 12 times the AI performance of Qualcomm's current in-car chips.
With the power to run AI agents locally, there's the potential to raise the in-car experience to new levels.
Google Onboard
Google's been dabbling with in-car technology for years now, first keeping it at arm's length with phone projection in Android Auto, then getting deeper with the Android Automotive Operating System, or AAOS, found in cars like the Polestar 3. Now Google is getting deeper still, announcing a partnership with Qualcomm to create a "standardized reference framework for the development of generative AI-enabled digital cockpits and software-defined vehicles."
That's a lot of jargon. It basically boils down to creating a series of templates and standards, all making it easier for automakers to implement more advanced user experiences in their cars. Clearly, the manufacturers need some help. We've seen numerous cars suffer delays at the hand of software implementation woes, like Porsche's Macan Electric and Volvo's EX90.
With the rise of AI, things will only get more complicated. Qualcomm and Google promise this new framework will not only handle a lot of the heavy lifting for AI integration but can also simplify more basic tasks like handling internet connectivity and software upgradability.
AI in the Car
It's possible to interact with voice-activated AI agents in some cars today, and there's more coming, with manufacturers like Volkswagen and Mercedes-Benz bringing AI right into the dashboard.
But again, with those systems, the AI you're chatting with while you drive is running elsewhere, somewhere far off in the cloud. Running these AI agents right in the dashboard, Qualcomm says, will open the door to some new experiences.
For one thing, running these things locally provides immediate access to all the sensors in the car. Mark Granger, Qualcomm's director of automotive product management, described the following hypothetical: "For instance, you're on a road trip, you see a red thing off in the distance, and the car knows from its interior cameras that you're pointing in a certain direction."
That AI agent would be listening, too, using the in-cabin microphones. With full awareness of what's going on in the cabin, the car's AI could automatically identify that distant object for you and tell you all about it, without you even having to ask. It could potentially even make you reservations for dinner if that identified place happens to be a restaurant. This is the kind of advanced in-car scenario we've heard about for years, now made potentially more easily by the ability of an in-car AI agent to have all the contextual information it needs directly from the cameras and microphones. To do that in the cloud would require passing a huge volume of data back and forth.
However, not all the examples presented were quite so compelling. Gretchen Effgen, Google's director of automotive partnerships, said tech like this will create "a truly personalized driving experience," with the car automatically setting the navigation to take you to your Tuesday morning workout class or suggesting a coffee break if you've been driving for too long. These features have been commonplace on phones and in cars for years.
When asked for more concrete examples of things that would drive better in-car experiences, Qualcomm's Nakul Duggal, group GM of automotive, industrial, and cloud, theorized that an in-car AI agent could monitor the behavior of a young driver in your household, hopefully with more precision than Chevrolet's Teen Driver system, which is almost a decade old.
So the promise of in-car, local AI still remains to be seen from a user experience standpoint, but there is one area where AI in the car has a clear advantage: safety.
AI at the Wheel
When it comes to next-generation levels of active safety, more and better sensors are required, and that means ever-increasing amounts of data. "The car is a source of immense data, data that can be used for improving the driving behavior of the AV system, or data that can be used for multiple personalized services for the user by the ecosystem," Anshuman Saxena said. He's vice president of product management and head of advanced driver assistance systems (ADAS) and automated driving products at Qualcomm.
Dealing with those reams of data is an area where AI can help. The idea is to provide next-level sensor fusion, combining all those sensors into a single vision of the world outside the car—and inside it, too.
Local AI running in the car could help filter out a lot of the noise, dramatically reducing the volume of data from those sensors. This could enable ever more advanced ADAS and, eventually, full-autonomy systems to see the world more accurately by paying more attention to what matters.
This sort of filtering could have other benefits, too. Anyone who's left their Tesla parked at the airport for more than a few days knows the impact Sentry Mode can have on your battery. With better AI running locally and with its more efficient chips, Qualcomm says you could run a system like that on the car for weeks with hardly any impact on battery life.
AI All the Way Down
According to Qualcomm CEO Cristiano Amon, we're about to enter into a boom of AI, a technology revolution the likes of which we haven't seen since the smartphone revolution. Within a few years we went from being blissfully unaware of what an app was to suddenly having dozens of the things ruling our lives.
Amon says we'll soon be in the same boat but with AI agents, a shift that will "break the paradigm of the app construct."
The hope, though, is that everything will integrate seamlessly. Your car's AI will be able to talk to your bank's AI so it can pay at the pump for you, and to your kid's school's AI so that it'll know if classes were let out early due to weather—all without you having to fumble between apps.
Facilitating this will be something called the AI Hub, which Qualcomm's Malladi called "AI agent orchestration." In this, one AI acts like an orchestra conductor, talking to many other AI instances in the background.
The concept is a bit worrying, an artificial agent with tendrils working its way into all the digital aspects of your life, but by bringing these agents into the car, you'd have access to the entirety of your digital data by voice, gesture, or however you care to interact.
"The bottom line is, when you put it all together, the AI agent becomes that one single entity, which is the starting point that does everything else for you," Malladi said. And, in theory, by having all that running within the car, your personal data will be less exposed.
So are you ready to hand over the keys of your life and your car to AI? The good news is you have a little time to ponder that question. Qualcomm's next-gen chips won't be available to manufacturers until sometime next year, and it'll likely be another few years after that before they start showing up in cars on dealership lots.
