Nvidia vs FSD explanation by Huang

Worth noting that Nvidia has attempted to enter the self-driving business several times before. With this new attempt, they seem to be embracing the wrong approach (same as FSD).

Do they believe in lidars?

Yes, in a press Q&A they’ve stated lidar and sensor redundancy are needed for L4. https://youtu.be/EzAVW1VgzcI?si=OgMKYJCS6_UIShKq 12:02

BB/QNX for the win.

Meaning they don't work with lidar either? I'm confused what you mean by wrong approach

This adds competition to Tesla's personal vehicle based city ADAS system and autonomous taxi service, and can be applied to any other vehicle brand. Their OEM competitors chose to not sign a contract to use Tesla's ADAS system... but now have an open source solution they can all use and benefit from, suddenly creating competition across all OEMs... not just one or two. It's another domino falling for Tesla: * They've lost an enormous amount of engineering and administrative talent. * They've lost their head start in battery tech and battery energy density and cost advantage, almost completely reliant on 3rd party suppliers, as they have been for most of their history. We haven't even heard of Tesla working on tech that's currently being developed and implemented like Sodium and SSBs. * Failed their pivot to produce their own cells which they claimed would be among the best in the market. * Lost their EV supply chain lead with few if any other companies sharing their 'vertically integrated' parts supply. * Competition is quickly rising for home and grid battery storage, including from the big cell suppliers who have no extra cell supplier middle man to increase their costs. * Lost their driving efficiency and range superiority crown (or close enough) * Lost charging network superiority in the US (never had it in other nations) on account of losing their anti-competitive advantage of locking other vehicles out of their network and their vehicles out of other networks. * Lost their huge lead on infotainment and OTA updates * Falling way behind on new models and trims, and reducing overall costs for EVs. * Seeing both production and sales growth decline, versus the 50% 2020-2030 CAGR on vehicle sales growth they claimed for 2.5 years between early 2021 and late 2023. * Loads of humanoid robotics companies showing demos that are far superior to everything Tesla has shown. Tesla has still yet to show any AI driven capabilities of their robots; they've often falsely portrayed their robots as acting autonomously when they were actually being remote piloted by a person. * Their CEO has significantly declined in mental stability, and significantly inclined in alienating would be buyers with far right pro-nazi rhetoric. Tesla stock now completely relies on Elon Musk's lies and the manipulated market's incessant need to push Tesla stock high enough where simple market index weighting can take hold and create an environment that recursively pushes the stock up. Part of that can be justified with the risk that Musk's claims about their technology eventually come true, but that's always been partially dependent on these technologies being implemented AND Tesla having a monopoly on them. The longer time goes on, and the more competition arises, the less this becomes true. Investors now have a plethora of competitors they can distribute their money between, rather than throwing it all on Tesla. \_\_\_\_\_ My guess is still that Tesla's next move is to allow their stock price to correct down, forcing out the investors who don't support Tesla's purchase of xAI, or putting the fear of God into those investors that failed to vote for the purchase last time, enabling the proposal to pass in the next vote. With Tesla stock lower, and the valuation of the private xAI company likely holding steady or even going up, the purchase through an all share's sale will net Elon Musk, his friends, and his family a much larger share of Tesla. Once accomplished, Musk (likely with the help of Trump) will find a way to re-pump the stock value, thus driving his paper wealth through the roof, easily over $1 trillion. Musk, like most billionaires, is a narcissistic man with severe inferiority and God complexes, surrounded by people who only see personal achievement and life meaning through the accumulation of wealth and power. Musk is an incredibly unhappy man, who thinks being the richest and thus most politically powerful person in the world will gain him more worship, and make him feel important and good about himself. It won't... because that's not how it works.... but it's the only thing Musk really has. He lied himself to massive levels of wealth. He's gotten away with it. To Musk, this is all just a game that he wants to dominate. It really is as simple as that.

It’s not clear that self-driving can be solved with a neural network only. It’s still new technology with problems such as repeatability. I wrote safety-critical avionics software for 10 years. Regulators depend on deterministic behavior in order to certify a system. But an AI might respond differently each time it’s trained. This is why we see so much “two steps forward but one step back” in NN solutions. Can this method get us from 99% of the way there to the 99.99999% that’s needed? Only time will tell.

If it is wrong, you need to use testing and data to prove it is wrong or prove the other tech is more superior. You cannot just say wrong with your feeling. As a tech, I can only say something is wrong based on the calculation not feeling.

I don't know what the internal status of NVidia's autonomous driving software stack is like, but you can see what they've had released for several months [here](https://github.com/NVlabs/alpamayo). The trained model weights and data sets (video/LIDAR/radar captures from real world driving, presumably what was used in training) are not provided directly in the repository but there are links where you can request access. I don't have a super deep understanding of how these models work and I haven't spent a ton of time on the source code but it looks like the inference model takes as input supplied camera images and produces vehicle trajectories as outputs. I don't think there's any scaffolding in place to interface this into a proper closed loop system, be it simulation or actual real world vehicles. The limited readme indicates: >Important notes: >\- Alpamayo-R1 is provided solely for research, experimentation, and evaluation purposes. >\- Alpamayo-R1 is not a fully fledged driving stack. Among other limitations, it lacks access to critical real-world sensor inputs, does not incorporate required diverse and redundant safety mechanisms, and has not undergone automotive-grade validation for deployment. So I think while the model was trained using LIDAR and radar the inference stack only includes camera, but acknowledges that this is a requirement gap and that additional sensors are critical inputs. What we see being demonstrated appears to be not just the standard NVidia provided software/hardware stack but also a lot of additional integration work (sensors and software expansion) that I think has been done by Mercedes or its third party contracts.

To be fair, Tesla's ADAS also "lacks access to critical real-world sensor inputs, does not incorporate diverse and redundant safety mechanisms, and has not undergone automotive-grade validation"... and yet is deployed 😂 /s <-- just in case Reddit's in a mood today

Of course, Jensen would be positive about a customer.............................Crapshoot.

no self-respecting intelligent person would want to be within 1000' or ten managers of Elon

yeah that's my take too. The ["Bitter Lesson"](https://en.wikipedia.org/wiki/Bitter_lesson) is what they're trying to leverage, but I think expert systems / General AI is necessary ... e.g. the car should be able to navigate through any arbitrary collection of traffic control devices that the average human could.

Hyped by Huang?

The Mercedes that Nvidia is collaborating on is absolutely decked out with sensors - beyond cameras, near and far range radar, lidar - even ultrasonic for immediate proximity navigation.

Ultrasonic, as in, regular parking sensors?

There are many different types of NNs. I agree Transformers only NN is not enough for safety critical applications. There are many research directions for hybrid networks. AlphaFold for example is a transformers plus a 3D logic NN. XPeng uses a transformer VLA plus an actor-critic adversarial network for control, then maybe another NN for precise controls.

Yeah I worded that badly (I think I was making edits and messed it up). Ultrasonic is pretty common so I didn't mean to emphasize that like I did. Yes you are correct that they are common -- my main point was intended to be the camera/radar/lidar.

Fair enough. It's not like I'm some maestro myself. Dual radars and lidar sounds like a pretty impressive sensor suite though. And expensive.

If this article is to be believed, then the planned L4 Mercedes/Nvidia reference car will have: * 14 cameras * 9 radars * 1 lidar * 12 ultrasonic Pretty stacked, indeed

NVIDIA is very successfully in this business with the leading players. Baidu, WeRide, Pony all run NVIDIA in the robotaxi market. NIO BYD Geely, Merc and others on the consumer side.

BB/QNX