Inside Mercedes’ plan to deliver hands-free driving to the masses

I’m sitting in the passenger seat of a special Mercedes Benz S-Class on a blindingly bright California spring day, casually watching an autonomous delivery robot roll through a crosswalk on its way to deliver someone’s takeout meal in Santa Monica. The test driver next to me chuckles as we’re about to merge onto the highway for a demonstration of Mercedes’ Drive Pilot system, a conditional Level 3 automated driving system that consumers may be able to order by the end of this year.

Mercedes is aiming to be the first automaker to bring legal Level 3 automated driving off the test track to the masses in its full-size, luxury S-Class vehicles. The question is whether it should, especially considering the Everest-sized challenges that lay ahead — even if the economic opportunities include cornering a piece of the estimated $220.4 billion autonomous driving market.

The stakes are high, too. The Mercedes Level 3 system has to handle multiple tasks all at once, including recording and exchanging vast amounts of data and giving ample time and warnings for the human driver to take back control when something goes sideways. There are the legal risks that Mercedes has pledged it will take on when the system is engaged, and there are even geopolitical ones: Mercedes uses the Russian GLONASS system for its global positioning information in Germany, for instance.

And yet, Mercedes is plowing ahead despite the risks, because the opportunity is just too vast to ignore. While other manufacturers like Tesla claim to have fully autonomous driving systems, Mercedes is the first to pass the required legal hurdles in the U.S. and in Germany to offer the conditional system to consumers. While the timeline is a bit fuzzy because Mercedes is still working through those legal requirements, the system could be in consumers’ hands and their driveways as soon as mid-2023.

The technology

In the trunk of one of the four development vehicles parked in the garage of the Proper Hotel in Santa Monica, sits a huge chassis of computer components. When we arrive, the trunk is open to let the components breathe, according to the test driver. There’s no room for your coveted golf bags or luggage here.

These components register, record, manage and upload as much as 2.87 GB of data per minute when the car is in regular operation. If an incident occurs while the vehicle is underway, say, for example, someone cuts off the development vehicle in traffic and forces a panic stop, the system takes in as much as 33.73 gigabytes of data so that engineers can take a closer look at what happened and improve the system.

Customers who own the S-Class vehicles equipped with the Drive Pilot system will not have to contend with computer components hogging up the trunk space. Instead, the vehicle will still b they will still be present in the cars so that the Level 3 system will work and be able to process and store large amounts of data. Some of that data will be kept on board, while much of it will be uploaded to a secure cloud system.

That data all comes from a variety of sensors around the vehicle, a few of which will be new to future S-Class vehicles that have been ordered with the new Drive Pilot system. While the company wouldn’t disclose specific costs of the system, representatives did say that it will cost as much as their top-of-the-line Burmester audio system. That audio system on the S-Class is a $6,700 option alone, yet requires the addition of a separate $3,800 package, bringing the rough total to around $10,500. That’s getting close to the cost of Tesla’s “Full-Self Driving” system, which currently is a $12,000 option.

The conditional Level 3 Drive Pilot system builds on the hardware and software used by Mercedes’ Level 2 ADAS system known as Distronic. It adds a handful of additional advanced sensors as well as software to support the features. Key hardware systems that will be added to future S-Class vehicles configured with the Drive Pilot upgrade include an advanced LiDAR system developed by Valeo SA, a wetness sensor in the wheel well to determine moisture on the road, rear-facing cameras and microphones to detect emergency vehicles, and a special antenna array located at the rear of the sunroof to help with precise GPS location.

The Valeo LiDAR system is more advanced than what is on the current generation of S-Class, in that it scans at a rate of 25 times per second at a range of 200 meters (approximately 650-plus feet). This is the second generation of the system, according to the Valeo spokesperson at the event. The system sends out lasers which then create points in space to help the AI classify the type of object in and around the path of the vehicle, whether its human, animal, vehicle, tree, or building. From there, the AI uses data from the other sensors around the car, to determine more than 400 different projected paths for both itself and the potential paths for the vehicles, pedestrians, and motorcyclists around it, and choose the safest route through.

The wetness sensor is a small round audio sensor positioned at the rear of the front driver wheel well and it determines how damp the road surface is. When the road is wet, droplets are thrown up against it creating an audible patter. When the system “hears” that patter, Drive Pilot will be disabled and the human in the driver seat will need to take over.

The antenna array on the roof of the S-Class uses a variety of different satellites to pinpoint the exact location of the vehicle within a few centimeters. It is precise enough to recognize which lane the vehicle is in on the highway. Mercedes says it relies on Galileo, and GPS in the U.S., and the Russian GLONASS system for this positioning information in Germany. These precise GPS points are integrated into an HD map which then helps the system navigate the real world.

These sensors are added to those already present in the Distronic system, which includes interior cameras to ensure that the driver is paying attention, as well as radar, ultrasonic, and 3D cameras outside. The added hardware is there to ensure that each system has redundancy and provides a more accurate view of both the interior and exterior of the vehicle as the system navigates the environment and, unlike the Tesla system, ensures that the driver is actually paying attention and not sleeping or watching a movie while operating the system.

There ‘s a reason for all of this precise and specialized equipment. Mercedes-Benz has taken on the responsibility, including the liability, for the safe operation of the system. The legal ramifications could be immense should something go wrong and a crash occur while the system is in use by a consumer.

New rules for Level 3 operation

Mercedes has used vehicles just like this one to test its Drive Pilot on more than 50,000 miles of roads in California and Nevada, where the company currently has conditional licenses to run the system.

Once the legal hurdles are passed, which Mercedes says it expects to happen by the end of the year, the systems will be available on properly equipped S-Class vehicles, when driven in specific conditions. However, it will still be limited.

The system will only be available in states where it’s legal (California, Nevada, and Florida currently). Cross the border into say, Arizona or Utah with an S-Class equipped with Drive Pilot, and the system will not be available. It’s geo-fenced.

In addition to the state location, the system won’t engage unless the vehicle is on clearly marked, divided highways, freeways, or interstates driving in a traveling lane, not in an exit lane. While out on our drive, the test driver moved over to take an exit, and the system turned off and requested that he take over as soon as he indicated that he was changing lanes.

And even when all of these requirements are met, the system is only available up to speeds of 40 mph (60 kmh).

The ride

Inside, the vehicle looks almost identical to an S-Class with one key difference: On the steering wheel sits a pair of buttons that fall directly under the driver’s thumbs. These buttons, engraved with the image of the front of a car with the letter ‘A’ over the top, are used to initiate the Level 3 system when the external conditions are met. Lighting around the buttons and on the steering column turns white when the system is available, and green-blue when it is engaged.

Our short ride took us down the 10 freeway in Los Angeles towards downtown LA and back to Santa Monica. Traffic was heavy stop-and-go, and there were plenty of opportunities for the system to fail. Within the first few minutes on the freeway, we encountered various road obstacles like plastic bags, cardboard boxes and more than one oblivious Angeleno making panic stops and randomly cutting into our lane of travel.

In the short periods where the system was available when all conditions were met, its operating appeared to be seamless. The handover was smooth and almost unnoticeable. The driver engaged the system, took his hands and feet off the controls, and let the car drive itself, all while keeping his attention on the road ahead.

The system uses maximum following distance when it’s engaged, so the gap was quite large between the S-class and the car ahead. Surprisingly, and sadly, no one decided to jump into that gap while the system was engaged, so we didn’t get to experience what might happen if a human made a sudden lane change in front of the car while operating the conditional Level 3 system. When the system lost the needed information, say when the lane markers (known sometimes as oreos), became faint, an audible tone would sound, and a message would appear for the driver to take over. At that point, the test driver would take control of the vehicle.

All in, the system was only engaged for maybe 10 minutes in total over our 30-minute ride. Each engagement was relatively short as traffic sped up to over 40 mph, or the system lost the required information to manage the driving. The very short ride along didn’t give us enough time to evaluate the system, but it did offer a glimpse of just how Level 3 autonomy may work in the very near future. The real question, however, is how the system will behave in customers’ hands, and whether or not, even the very well-off, will purchase the technology.