How Do Autonomous Vehicles Communicate with Pedestrians?

You look the driver in the eye as you are about to cross the road; the driver subtly signals slowing down, and in a single second two strangers reach an understanding. But what if there is no one in the driver's seat? How exactly do autonomous vehicles communicate with pedestrians - and is that communication really what we think it is? For someone passionately devoted to cars, this question turns into an incredibly fun journey, because the answer is far more curious than you might expect.
Pedestrian Detection: How Does the Vehicle See You?

First, let's be clear about one thing: the autonomous vehicle's eyes are not like your eyes. Lidar, radar, and cameras work together to turn everything around them into a mixture of point clouds and images; then artificial intelligence sifts through that mass to pick out the cyclist, the scooter rider, and the pedestrian. So saying the vehicle "sees" you actually means it is continuously recalculating you. Sounds like science fiction, doesn't it?
The truly crucial part begins after detection: prediction. In pedestrian intention prediction research, there is a three-stage arrangement - input, feature extraction, and prediction. The vehicle collects all signals directed at the pedestrian, encodes them into meaningful patterns, and finally seeks an answer to this question: will this person step into the road, or stay on the sidewalk? What you do with a single glance is a mathematical race repeated thousands of times per second in the background.
Key Takeaways
- Many pedestrians believe they make eye contact with an approaching vehicle - yet most of them never even see the driver.
- Without eye contact, about 55% of drivers do not stop at pedestrian crossings.
- In intention prediction, the most critical signal is not the pedestrian but the situation itself; when context is removed, performance drops sharply.
- Robotic eyes on vehicles increase the sense of safety, but participants consider them unnecessary.
- An expected collision count of 4 to 5.5 per million vehicle-km still makes pedestrian-autonomous vehicle interaction risky.
The Eye Contact Myth: What Are We Trusting When No One Is Watching You?
This is where things get truly interesting. Driver-pedestrian eye contact experiments revealed something striking: many people are convinced they make eye contact with the driver of an approaching vehicle. However, the experiment recordings show that no one actually saw that driver. In other words, the human brain can generate a sense of trust not from a real gaze, but from assuming one. Striking, in every sense of the word.
Meanwhile, there are situations where eye contact really does work: pedestrians who lock eyes with the driver feel safer while crossing, and stop rates at pedestrian crossings change - without eye contact, about 55% of drivers do not even stop. But pay attention: this cue only becomes decisive in situations where vehicle motion cues are ambiguous. And in an autonomous vehicle, this cue is already absent. So the traditional "I looked at the driver, and they saw me" trust becomes unfounded in the age of driverless vehicles. So what has taken its place?
Stop or Go? The Anatomy of the Stop-Go Decision
The moment autonomous systems struggle most is that moment of hesitation at intersections and crossings. Researchers here rigorously distinguish between two different concepts: crossing intention, meaning the pedestrian's internal decision, and crossing prediction, meaning the vehicle's forecast that "this person will be on the road." To get a feel for the difference, think of it this way: a friend deciding to high-five you versus that high-five actually landing. The vehicle has to predict both correctly.
An incorrect prediction hurts at both ends: either there is needless hesitation and traffic grinds to a halt, or an unsafe maneuver is attempted. That is why intent prediction models are constantly evolving. Recently, a lightweight Transformer model combining four behavioral streams - attention and gaze, position, context and interaction - predicted intersection decisions with high accuracy. Even more interesting are the ablation, or component-removal, experiments: when context information is removed from the model, performance collapses. In short, the vehicle's strongest signal is not the pedestrian themselves, but the context the pedestrian is in.
In this field there are also labeled datasets relying not only on imagery but on human drivers' natural language explanations, and LSTM-based studies on the right-of-way problem at unsignalized intersections. And at the end of 2025 came a major leap: a system called OmniPredict, using a multimodal large language model, emerged as the first model to predict human pedestrian behavior in real time. The vehicle is slowly learning that quick intuition inside your head.
Putting a Face on the Vehicle: Do eHMI Experiments Really Work?
Driver and pedestrian used to communicate through informal cues like hand gestures and facial expressions. Autonomous vehicle developers tried to carry these cues over to the machine, and external communication interfaces - that is, eHMI experiments - began. In one of the first important experiments, a van representing the autonomous vehicle presented information telling pedestrians when they could cross; afterwards, screens, light strips and even robotic eyes mounted on the vehicle were tried.
Field study results for the robotic eyes are quite mixed - and in my view that is exactly why they are valuable. Among people who notice the eyes, the feeling of safety increases, and a separate 360° video experiment showed that these eyes could reduce potential traffic accidents. But the same participants think the eyes are not "necessary." So a cute vehicle face appeals to us, but it is not essential. The industry side says something similar: for pedestrians to feel at ease, the vehicle must express its behavior in a way humans naturally understand, and an analysis published in Science emphasizes that pedestrians at intersections expect confirmation from the vehicle that it has "seen" them.
Only there is a strange paradox here that must not be skipped: no matter how much you improve communication design, you may not be able to prevent risky user behavior. A child pedestrian study revealed that even when autonomous vehicles light up the green signal and slow down away from intersections, children still exhibit riskier crossing behavior. In other words, writing "you may cross" on a screen cannot stop a child from saying "then I'll run." No matter how good the technology is, the human side remains the most variable variable.
Why Do Accidents Happen at Exactly This Moment of Interaction?
Now let's move on to the brutal part. Pedestrian–autonomous vehicle interaction safety models calculate the expected number of collisions in interactions as between 4 and 5.5 per million vehicle-km - and this value is clearly qualified as a worrying level of risk. Moreover, the origin of classic pedestrian accidents lies in the same region: about half of pedestrian accidents occur at points far from crossing facilities, mostly alongside parked vehicles. So the problem erupts not mainly at moments when "the vehicle doesn't see me," but at moments when the vehicle has to negotiate with me.
There are pedestrian types that even algorithms struggle with: pedestrians carrying large items, walking dogs, or wearing unusual clothing can delay or confuse detection systems. Partial concealment, crowded scenes, and ambiguous cues are also among the pain points. The first pedestrian death caused by an autonomous vehicle in Arizona in 2018 happened in exactly this kind of scenario: the pedestrian was on the road at a distance from which the vehicle could not stop. An early detection unfairness study in this field is also still remembered; the claim that pedestrians with darker skin were harder to detect sparked debate at the time, and this issue is still one that must be taken seriously today.
The transparency gap: who shares data, and who doesn't?
Another tension in the industry is the sharing of safety data. Waymo publishes research showing its vehicles are five times safer than human drivers and 12 times safer in pedestrian interactions, and voluntarily discloses its safety data. On the other hand, after industry criticism, Tesla stopped publishing detailed safety data. One side of the market is spilling data while the other is drawing the curtain - leaving the public without a fair means of comparison.
There is also an interesting skew on the perception side: people blame autonomous vehicles more than human drivers, even when they are not at fault. In other words, the machine's mistake is unforgivable while the human's mistake becomes ordinary. This gap between public perception and data is wide enough to overshadow the technology's real achievements.
So How Do Autonomous Vehicles Ultimately Come to Terms with Pedestrians?
Here is the clear answer I promised: autonomous vehicles come to terms with pedestrians not through eye contact, but by reading behavior. Sensors continuously track the pedestrian, and artificial intelligence combines speed, position, gaze, and context to predict intent; when necessary, eHMI displays or vehicle behavior - a clear slowing down, a clear stop - carries the message outward. The language of the new agreement is not the old glance, but movement and context. You tell it your intention with your walk, and it tells you its decision with its brakes.
My summary is this: the machine side makes better predictions every year, but the interaction never stops being a two-way game. Without transparency in data sharing, humility in eHMI design, and a bit more patience from us users, this deal remains incomplete. If it were you, would you trust the eyes, or would you trust the sound of the brakes? I'd say the latter - though the charm of robotic eyes is also worth mentioning.
Sources
- Pedestrian Crossing Intent Prediction via Psychological Features and Transformer Fusion
- Pedestrian Intention Prediction for Autonomous Vehicles
- Pedestrian Intention Prediction Using LSTM for Right-of-Way
- Eye Contact Between Pedestrians and Drivers
- Safe Crossings: The Power of Eye Contact
- External Human-Machine Interfaces for Autonomous Vehicles
- Evaluation of Vehicle-to-Pedestrian Communication Displays
- A Field Study on Pedestrians' Thoughts toward a Car with Eyes
- Can AI read humans' minds? A pedestrian behavior model
- Waymo Safety Impact