DIME – multimodal utvärdering av förarnedsättning
Kort beskrivning
DIME-projektet (Driver Impairment Multimodal Evaluation) utvecklar framtidens teknik för att upptäcka om en förare är påverkad – även när personen försöker att dölja det. Med hjälp av ett avancerat system baserat på djupinlärning som är framtaget av Smart Eye analyseras förarbeteende, röst och biobeteendemått som hjärtfrekvens och ögonrörelser. Målet är att skapa en algoritm som säkert kan identifiera kognitiv nedsättning, särskilt orsakad av alkohol, och integrera den i ett intelligent övervakningssystem. Tekniken väntas lanseras kommersiellt år 2028.
Beskrivning av projektet (på engelska):
Background
Legislation is being introduced that requires vehicle manufacturers to equip new cars with systems capable of monitoring various forms of cognitive impairment. According to the EU GSR (New Vehicle General Safety Regulation), these systems include Driver Drowsiness and Attention Warning (DDAW) – mandatory for all new and existing vehicle types from July 2024 – and Advanced Driver Distraction Warning (ADDW), mandatory from July 2026.
The upcoming Euro NCAP (European New Car Assessment Program) requirement for new vehicles to include Driver Monitoring Systems that can detect behavioral signs of intoxication, starting in 2026, highlights the urgent need for effective strategies to combat impaired driving.
Intoxication and cognitive impairment are major contributors to traffic accidents and reduced road safety. So far, this issue has mainly been addressed through law enforcement and alcohol ignition locks. This is expected to change as more responsibility is placed on in-vehicle technology, and car manufacturers are required to include driver monitoring systems. Understanding how to develop algorithms that can detect deviations from normal driving behavior at an early stage is key to preventing accidents.
Purpose
The Driver Impairment Multimodal Evaluation (DIME) project, proposed by Smart Eye and the University of Gothenburg (UGOT), aims to:
- develop and validate an advanced multimodal algorithm for detecting cognitive impairment in drivers, particularly alcohol intoxication; and
- integrate this algorithm into a driver monitoring system, demonstrated in simulated environments and designed for computational efficiency.
The long-term goal of DIME is to generalize the algorithm to detect various forms of cognitive impairment among drivers. To achieve this, multiple modalities will be fused and analyzed:
i) Driver state monitoring: bodily and facial indicators of impairment.
ii) Speech and language monitoring: interactive speech analysis.
iii) Biobehavioral monitoring: including eye tracking, heart rate, and driving behavior such as steering and braking patterns.
Research questions
DIME will address innovation in the following areas:
i) Fusion modeling of in-cabin driver behavior: combining Smart Eye’s Driver Monitoring System (frontal analysis) and Cabin Monitoring System (postural analysis).
ii) Speech recognition and interaction: identifying intoxicated speech patterns and using interactive LLM-based approaches to generate sufficient speech for reliable detection.
iii) Biobehavioral evaluation and modeling: integrating biophysiological variables (heart rate, heart rate variability, eye behavior) with driving activity (steering, acceleration, braking).
iv) Multimodal fusion: combining in-cabin behavior, speech patterns, and biobehavior to enable holistic and complementary intoxication detection capabilities.
Methods
Modeling: The project will model the following modalities:
i) in-cabin driver monitoring using computer vision;
ii) speech monitoring and LLM-based dyadic interactions (prompt-engineered);
iii) biobehavioral modeling using eye-tracking data, heart rate, and driving behavior (e.g., steering, braking).
Deep learning approaches will be used, with pre-trained models for (ii) and existing Smart Eye data for (iii). Fine-tuning will be performed for each modality using data collected in driving simulator studies.
Testing:
Two simulation studies will be conducted with intoxicated participants (in accordance with GDPR) using driver simulators. These studies will evaluate biobehavioral markers of intoxication and the multimodal model’s ability to detect such states.