How Is End-to-End Model Changing Autonomous Driving?

What comes to mind when you think of a car driving itself? Probably dozens of sensors placed along the road, massive maps, and engineers who pre-wrote rules for every situation, right? Well, with FSD version 12, Tesla turned that picture upside down: there are no longer thousands of hand-written rules in the mix; raw camera footage goes in, and steering-brake-throttle commands come out. So what does telling a neural network "learn to drive" actually mean? How is this transition changing the world of autonomous driving - and where is it getting stuck?
The Classic Modular Architecture: A Factory Line Inside Your Driver Assistant
Before talking about end-to-end, it is essential to understand the old setup, because most systems on the roads today still work on this logic. Modular architecture sets up autonomous driving software like a factory line: each stage does its own job and passes its output to the next. From Cruise, one of the first winning vehicles of the DARPA Grand Challenge, to Mobileye and the open-source Autoware, this approach has a long history.
- Perception: LiDAR, camera and radar data are processed and objects are detected; algorithms like YOLO and VoxelNet work in this layer.
- Prediction: Pedestrians and other vehicles are modeled with their short- and long-term trajectories.
- Planning: The route is calculated with classical algorithms like A* or Dijkstra, while local maneuvers are computed with methods like RRT.
- Control: Usually a Model Predictive Controller (MPC) kicks in and physically steers the vehicle.
The biggest advantage of this structure is transparency: you can test each module separately and find out which stage got what wrong. But its biggest weakness is hidden right there - error propagation. If the perception module misclassifies a pedestrian, that error is carried all the way to the end of the chain, and the planning module makes its decision with already corrupted data. Moreover, training each module separately requires serious domain expertise and computing power; staged processing is an inefficiency in its own right.
Key Takeaways
- Tesla FSD, with version 12 (released to customers in early 2024), switched entirely to an end-to-end neural network architecture; today you are on version 14.
- In modular architecture, perception, prediction, planning and control are separate modules; in end-to-end, they all merge into a single neural network.
- FSD's official name is still FSD (Supervised): not a fully autonomous car, but an assistance system that requires a continuously attentive driver.
- The biggest problem with end-to-end is explainability: formal verification of neural networks is still an unsolved engineering problem.
The End-to-End Approach: Millions of Examples Instead of Thousands of Rules

In end-to-end driving, the idea is strikingly simple: you feed raw sensor data - camera, LiDAR, radar, GPS - into a single giant neural network and get driving commands or a driving plan directly from the network. Perception, prediction and planning cease to be separate software and melt into a single program. Tesla entered this road with FSD v12, and in early 2024 the first end-to-end version that drivers experienced behind the wheel emerged. Today the system has evolved up to v14.
Let me explain the difference with an analogy: a modular system is a student who has memorized traffic rules; it has a ready-made rule for every situation. An end-to-end system, on the other hand, is an experienced driver who sat beside you and was watched for thousands of hours; it may not be able to explain one by one what it does and why, but it senses the right move. The neural network learns not only what objects are, but also how human drivers react to them.
So How Is the Network Trained?
There are two main paths. The first is imitation learning (behavior cloning): the network learns to copy the behavior of human drivers across millions of kilometers. A nice method, but it has two weaknesses - covariance shift and causal confusion; in other words, the network can learn incorrectly what is actually important. The second is reinforcement learning: through trial and error in simulation, the network is trained with rewards. NVIDIA has a history in this area stretching from DAVE-2 in 2016 to today's DRIVE Thor, which offers chip power of up to 1,000 TOPS; Wayve and Comma.ai are other names following the same philosophy.
Modular or End-to-End? A Concrete Comparison
When we place the two approaches side by side, the picture becomes clear. The advantages of end-to-end architecture as listed by researchers are these: reduced error propagation, computational efficiency, relative ease of training and, interestingly, resilience to attacks - since it cannot be known from the outside which specific input triggers the system's driving behavior, malicious manipulation also becomes harder.
| Feature | Modular Architecture | End-to-End Architecture |
|---|---|---|
| Structure | Perception → prediction → planning → control as separate modules | A single neural network from raw sensor data to control |
| Development method | Hand-written rules + separately trained modules | Learning from data (imitation/reinforcement learning) |
| Transparency | High; each module can be tested separately | Low; the decision mechanism is a black box |
| Error propagation | Present; perception errors are carried along the chain | Reduced |
| Data requirement | Domain expertise per module | A very large dataset is mandatory |
The other side's heavy artillery, however, is transparency. You can audit a modular system piece by piece; whereas you can only partially see what is going on inside the neural network's head. Although the research world develops partial answers to this criticism with attention maps and interpretable outputs, the matter deepens when safety certification comes into play - we will come to that shortly.
Tesla vs. Waymo: Same Goal, Opposite Philosophies
The split on the sensor side is the most visible face of the architecture debate. Tesla's approach is camera + neural network, without LiDAR - they argue that this path is necessary for a system that can navigate on a global scale. Waymo, on the other hand, is the exact opposite: a multi-sensor architecture that stacks sensors on top of each other, with 13 cameras and 4 LiDAR units in its latest vehicle. Waymo co-CEO Dmitri Dolgov is the person who openly said that in 2026 the camera-only approach has plateaued, meaning it has stalled before reaching full autonomy. The Tesla camp, on the other hand, argues that billions of miles of evidence have been produced with FSD.
In fact, the picture looks a bit grayer when viewed from behind the scenes: It is stated that many players, including Tesla and Waymo, effectively use hybrid architectures, meaning a combination of deterministic route planning and neural-network-based perception; Momenta and Aurora are also counted on this list. So the answer to the question "is everything in one network, or is everything in separate modules" is, in most real-world practice, "a mix of both." When it comes to sensor comparison, LiDAR is superior in low-light and nighttime conditions; camera-only systems, meanwhile, are energy efficient and do not require pre-mapping.
The Black Box Problem: Explainability and Verifiability
This is the technically most crucial part of the matter. The automotive world's safety standards - ASIL, ASPICE, AUTOSAR, MISRA - demand that what the system does be defined and proven at the requirement level. In an end-to-end neural network, it is already difficult to write a requirement definition such as "the car will make this decision in this situation"; and the formal verification of neural networks is still an unsolved engineering problem. In other words, no matter how well the network drives, when engineers have to prove every safety requirement, they may not have sufficient tools at hand.
We see its concrete consequence on the liability side. Mercedes-Benz announced that it assumes legal responsibility for DRIVE PILOT, the world's first certified Level 3 system, and received approval from the German authority for speeds of up to 95 km/h on highways - according to SAE J3016 levels, at Level 3 responsibility shifts from the driver to the manufacturer. The reason is partly architectural: a modular system with clearly defined boundaries is something one can declare "the system drives under these conditions" and put a signature on. Putting the same signature on a black-box network is far more difficult. The new research trend is large language models and vision-language models that add a scene-understanding and reasoning layer to neural networks - an attempted answer to the explainability problem.
Promise vs. Reality: What Is the Current State Today?
Everything up to this point was exciting; now for the cold shower. Let's look at the official name of Tesla FSD: FSD (Supervised). In other words, the system bearing the name "full self-driving" is, by its official definition, an advanced driver assistance system designed to be used entirely with an attentive driver, and it does not turn Tesla into a fully autonomous vehicle. The driver must be ready to intervene at any moment - in the SAE classification, this is still Level 2 territory. With FSD v13, the system can be engaged in a parked vehicle and reverse maneuvering is included; the capabilities are growing, but the classification is not changing.
On the other side, Waymo is effectively running Level 4 operations with its robotaxi service scaled to 14 US cities. SAE's latest revision also renames Levels 0-2 as "driver support systems" and Levels 3-5 as "automated systems," emphasizing that full automation, i.e. Level 5, is still a long way off. Industry-wide, this reflects the transition period from a wave of hype to real-world deployment: Level 2+ systems have become widespread, robotaxis are limited to certain cities, and Level 5 is still a goal for everyone.
In summary: How Much Did the Shift from Writing Rules to Learning Change Things?
The direct answer is this: The end-to-end architecture moved the decision-making software in autonomous driving from hand-written rules to a single neural network that learns from data; this brought more natural driving behavior and reduced error propagation, but created new and still-unsolved challenges on the explainability, verifiability, and certification side. Tesla is the most aggressive player in this bet, and the system after FSD v12 really became something different - but unsupervised full autonomy is still not delivered. The modular approach hasn't left the stage either; Level 3-4 systems that take on responsibility using Waymo's sensor stack are being certified with this philosophy. The winning architecture of the future will likely be hybrid structures that combine the best of both worlds - and that race is being run right now.
Sources
- Recent Advancements in End-to-End Autonomous Driving using Deep Learning: A Survey - arXiv
- Autonomous driving: Modular pipeline Vs. End-to-end and LLMs
- Navigating Autonomous Driving: Comparing Modular, End-to-End, and Hybrid
- End-to-end learning vs modular trade-offs
- Integrating Modular Pipelines with End-to-End Learning
- End-to-end system architectures in autonomous driving