Recently, the CARJAM TV published a new autonomous driving video of the famous smart car from Mercedes Benz – S500 Intelligent Drive, a normal S500 type equipped with intelligent sensors and systems. This video clip lasts for 11 minutes, demonstrating the S500’s capabilities in handling complex environments and challenge situations (Read More Here). In August of 2013, the S500 has already completed its first open drive, named “Bertha Benz Memorial Route”.
Daimler is in charge of the design and development of intelligent system inside the S500. Today let’s have a deeper understanding into this vehicle. All the contents are based on open public data.
A regular S500 is used as the base for adoption, equipped with all emergency braking systems enabled as underlying protection. Another reason is the enough space inside S500, which allows the installments of all the sensors and PCs.
Only two types of sensors are used for the environment perception tasks: the cameras and the radars. One stereo pair is installed on the roof of driving seat, which is aimed to detect the obstacles ahead. Two monocular cameras are looking forward and backward respectively, providing a wide view angle up to 90 degree. The short range radar (SRR) systems are designed for blind spot monitoring, with 150 degree coverage; while the long range radar (LRR) systems promise a perception distance up to 200 meters.
A 3-layer system structure serves as the core of S500.
GPS-may also combined with IMU-localization system is not shown in this figure, but from other confidential sources, this system is equipped and provides a coarse global localization. In addition, two digital maps are used for refining local localization up to centimeter level, of which the Feature Map is mainly based on matching feature correspondences, namely visual slam or visual odometry, and Lane Map uses the structured lane elements, like lane markings or curbs, for a refined localization.
Besides, the normal modules on S500, like distronic radars and ESP offer information directly to emulation, without any fusion or processing by the planning module.
Stereo Vision & Object Recognition：
The front stereo pairs provide a depth description of the outside world. Then some hypotheses will be generated by analyzing the clustered disparities. Though not shown in the figure, the “Stixel” is very possible to be the method for clustering and decreasing the amount of data. Classification and validation on these hypotheses could be based on the mono image. The last stage is the tracking of confirmed moving objects, extending kalman filter (EKF) is the method adopted for tracking, from confidential sources.
Traffic Light Recognition：
One front mono camera provides the detection of traffic light. The wide view angle and high resolution of this camera ensure that the traffic light can be seen both from large distance and when the vehicle is right stopped in front of the stopping line. In order to decrease the false alarms and uncertainties, a traffic light map is also used to provide the prior knowledge about the position and height of the traffic light in next intersection. One benefit of this map is when the vehicle is driving on a multi-lane road, and aiming to go straight, as shown below, only the traffic light for the straight direction is needed to be detected. The centimeter level localization of the vehicle can tell which traffic light should be observed.
Map & Localization：
This is a core of autonomous driving, especially when we still have the uncertainties lying with the perception. The map provides information like stop line, traffic light, and planned trajectory.
The four-layer map architecture serves different purposes. A Navigation map is stored as GPS Definition File (GDF), providing the basic topology information of lane splitting\lane conjunction\intersection. For planning layer, a 3D map, called 3D corridors, and traffic rules are the prior for conducting the planning function. Above all is the localization layer. In this layer, two methods are used for a refined localization: one using edge descriptors, while another using lane marking and curbs.
Different elements contained in the maps have different accuracy, shown in the figure. This shows why the localization layer is so important, as the lane marking map and curb map have accuracy up to 5cm. This satisfies the desired 10 cm accuracy in map. The map is generated and cooperated with Nokia HERE department, one of the world’s leading map suppliers.
For online localization function, S500 uses visual based localization by matching visually detected land marks with those stored in the map. Then the actual pose of the vehicle is computed. To mention, this ‘pose’ is the position and heading (let’s define a 2D surface) of the vehicle respected to the map, or rather, where is the vehicle in the map. Once we know the global position of the map, we can compute the global position of the vehicle. This work is done with KIT.
S500 has a 3-stage loop: object classification and prediction, behavior generation and trajectory planning.
Once one object is classified as a dynamic object, some predicted paths will be generated based on the attributes of objects and driving lanes. Probabilities of these predicted paths show which is the most likely path for one object to follow.
A hierarchical concurrent state machine is used to determine vehicle behavior in individual situations. This work is also collaborated with KIT, and this technology is from the former AnnieWay who has attended the Urban Challenge.
Then the trajectory planner is triggered by different behaviors, and generated by the localization and object information.
S500’s performance under different situations:
Daimler also showed several cases of S500 during the “Bertha Benz Memorial Route” test, including: overland, inner city, parked cars, cyclists, pedestrians, intersections, turning, and roundabouts.