It is evident that self-driving vehicles are going to reshape the traditional transportation systems in the near future through enhancement in safe and smart mobility. However, the first ones will likely still need to periodically have the driver take manual control of the vehicle. Now, researchers at Germany’s Fraunhofer Institute for Optronics, System Technologies, and Image Exploitation developed a new AI system that could check if the driver is currently able to do so by monitoring what they’re doing.
The core of the development lies in machine learning algorithms and processes, such as Artificial Intelligence (AI). The algorithms analyze the camera data in real-time and find out whether the driver is on the phone, playing with the children, or looking at the passenger’s cell phone.
The technology is designed to work with cameras located inside the vehicle’s cabin. It supports both traditional video cameras and infrared cameras that can see in the dark, as well as 3D cameras that measure the distance between objects and the camera.
The onboard computer analyzes video from cameras in the passenger compartment in real-time: it uses AI-based algorithms to determine the position of the driver and other passengers and then matches these positions with those already known, for example, face-to-face communication, sleep, or road monitoring. The system can also recognize that the driver is on the phone.
In a situation where the driver needs to take control, the system warns the person behind the wheel and also checks if he is ready to steer. In case the driver is not ready, the system will calculate and how long it will take them to focus back on the road based on their current activity. Therefore, the AI waits for full readiness and only then turns off the autopilot. The technology respects privacy from the outset and thus complies with the strict regulations and high level of data protection awareness in the EU.
The AI system abstracts images of the driver or passengers to form a digital skeleton and consequently deduces the activity using the skeletal movement and supplementary object recognition.
“The algorithms can thereby tell whether someone is sleeping or looking at the street, how distracted the person is, and how long it will take them to focus back on the road,” explains Michael Voit, Group Manager at Fraunhofer IOSB. “We can not only detect the activities of the driver but those of all passengers, too – both in the front and back of the vehicle,” explains Michael Voit, Group Manager at Fraunhofer IOSB. “The technology is ready for pilot production. We are already in contact with companies who want to use our technology.“
Driver monitoring is to become mandatory in automated cars, no matter their level of automation. Using Fraunhofer IOSB technology, vehicle manufacturers can not only meet this requirement but also make numerous visions in terms of autonomous driving a reality. When used in conjunction with body pose identification and activity detection, the system can determine which parking space the user is pointing to at that very moment. The system can also help with the safety aspects of driverless vehicles. Additionally, it could ensure that the driver and all passengers comply with safety regulations and fasten their seatbelts.
