Consumer Reports (CR), the independent, nonprofit, and nonpartisan member organization, welcomes the opportunity to provide comments on the National Highway Traffic Safety Administration’s (NHTSA) proposed information collection on modern voice command interfaces. CR supports this effort and appreciates the agency’s continued work to better understand the effects of in-vehicle technologies on driver distraction.
Because this notice requests comments on the utility of the proposed study and on ways to enhance the information to be collected, below we offer several areas where we believe the design of the study could be strengthened.
As described in the notice, the study will compare three common tasks performed through the voice command interface with the same three tasks performed manually through the touchscreen interface, along with one additional voice-only task, while collecting various measures of driver performance and workload. The notice also describes a simulator familiarization drive, but it does not explicitly identify a baseline condition with no secondary task or benchmarking conditions for the study drivers themselves. That means the study appears better positioned to show whether voice-command interactions are better or worse than visual-manual interactions, as opposed to showing the level of impact each mode has relative to ordinary driving in relation to established NHTSA reference measures. NHTSA’s own research on the driver workload and relative risk associated with in-vehicle voice control systems has used driving-only baseline comparisons, measuring performance while participants were just driving and not engaging in any secondary task, and also has discussed results in relation to no-task baseline driving. That research also incorporated a standardized benchmark task (N-back) to help interpret cognitive workload levels and support comparisons across studies. It found that some measures, such as tactile detection response task (TDRT) performance, were sensitive to the difference between voice-task conditions and driving without a secondary task, while other measures were not. A comparable baseline condition here could therefore make the study more useful by allowing NHTSA to assess not only the relative differences between the two modes, but also the magnitude of distraction associated with each mode relative to driving without a secondary task.
The proposed collection should also include a way to measure how demanding participants found each task. The proposed information collection identifies several objective indicators of distraction and cognitive workload, including eye-tracking metrics, pupil diameter, heart-rate variability, and TDRT performance. A more subjective post-task workload assessment could provide useful additional context for interpreting those measures. It would also allow NHTSA to evaluate whether participants’ own perceptions align with the observed effects. Including a subjective workload measure could then improve the utility of the study by helping to identify cases in which a task, for instance, appears manageable to the driver even though the objective indicators suggest meaningful distraction or workload.
The study could also be designed to better capture the demands of voice-command use. As proposed, participants will use a voice command interface with which they are already familiar in a vehicle they drive regularly, and the study will document errors made during training before the task drives begin. That approach is understandable, but it appears likely to produce data reflecting relatively favorable conditions. NHTSA’s prior research has found that drivers often had trouble using voice control systems, that interactions frequently included speech-recognition and other errors, and that evaluation based solely on error-free trials would not be representative of many interactions commonly experienced by drivers using these systems. The extra effort required to repeat, attempt to correct, or abandon a failed voice interaction may itself increase distraction and should be reflected in the protocol.
Consumer Reports appreciates NHTSA’s continued work to better understand driver distraction, particularly as it may be linked to the use of built-in vehicle technologies. The proposed collection addresses important issues and we look forward to the insights this research may provide.