Neural correlates of human-machine trust in autonomous vehicles context. (English) Zbl 07916512
Bountis, Tassos (ed.) et al., Chaos, fractals and complexity. Selected papers based on the presentations at the 8th summer school-conference on dynamical systems and complexity, Crete, Greece, July 18–26, 2022. Cham: Springer. Springer Proc. Complex., 245-262 (2023).
Summary: Poor mental states-such as fatigue, low vigilance and low trust-in-automation-have been known to interfere with the appropriate use and interaction with vehicular automation. This has spurred strong interest in driver state monitoring systems (DSMS) that support adaptive interfacing between human drivers and automated driving system to enhance road safety and driver experience. While there have been thriving developments in fatigue and vigilance monitoring, research on trust monitoring is still in its infancy. Trust-in-automation has predominantly been measured subjectively via self-report measures, with fewer studies attempting to measure trust objectively owing to the difficulties in capturing this relatively abstract mental state. Nevertheless, recent progress has unveiled promising potential for objective trust monitoring that can be implemented in future intelligent vehicles. This review presents a framework for understanding the cognitive, affective and behavioural components of driver trust, and surveys current approaches and developments in objective trust measurement in autonomous vehicle contexts using behavioural and brain-based techniques. Approaches are evaluated for strengths and limitations in both their conceptual validity in capturing trust-relevant information, measure reliability, and their practical value in real-world driving settings. Future directions for improving trust monitoring towards practical implementation are also discussed.
For the entire collection see [Zbl 1533.37005].
For the entire collection see [Zbl 1533.37005].
MSC:
| 37-XX | Dynamical systems and ergodic theory |
Keywords:
human-machine interaction; trust; brain; neuroimaging; electroencephalography; autonomous vehicles; drivingReferences:
| [1] | Teoh, E.R., Kidd, D.G.: Rage against the machine? Google’s self-driving cars versus human drivers. J. Saf. Res. 63, 57-60 (2017) |
| [2] | Seet, M., Bezerianos, A., Panou, M., Bekiaris, E., Thakor, N.V., Dragomir, A.: Individual susceptibility to vigilance decrement in prolonged assisted driving revealed by alert-state wearable EEG Assessmen. IEEE Trans. Cognit. Dev. Syst. (2022). In Press |
| [3] | Wang, H., Dragomir, A., Abbasi, N.I., Li, J., Thakor, N.V., Bezerianos, A.: A novel real-time driving fatigue detection system based on wireless dry EEG. Cognit. Neurodyn. 12(4), 365-376 (2018) |
| [4] | Chung, W.Y., Chong, T.W., Lee, B.G.: Methods to detect and reduce driver stress: a review. Int. J. Automot. Technol. 20(5), 1051-1063 (2019) |
| [5] | Sciaraffa, N., Di Flumeri, G., Germano, D., Giorgi, A., Di Florio, A., Borghini, G., Vozzi, A., Ronca, V., Varga, R., van Gasteren, M., Babiloni, F.: Validation of a light EEG-based measure for real-time stress monitoring during realistic driving. Brain Sci. 12(3), 304 (2022) |
| [6] | Hergeth, S., Lorenz, L., Krems, J.F.: Prior familiarization with takeover requests affects drivers’ takeover performance and automation trust. Hum. Factors 59(3), 457-470 (2017) |
| [7] | Gold, C., Körber, M., Lechner, D., Bengler, K.: Taking over control from highly automated vehicles in complex traffic situations: the role of traffic density. Hum. Factors 58(4), 642-652 (2016) |
| [8] | Miller, D., Johns, M., Mok, B., Gowda, N., Sirkin, D., Lee, K., Ju, W.: Behavioral measurement of trust in automation: the trust fall. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, pp. 1849-1853. SAGE (2016) |
| [9] | Collet, C., Musicant, O.: Associating vehicles automation with drivers functional state assessment systems: a challenge for road safety in the future. Front. Hum. Neurosci. 13, 131 (2019) |
| [10] | Larue, G.S., Rakotonirainy, A., Pettitt, A.N.: Driving performance impairments due to hypovigilance on monotonous roads. Accid. Anal. Prev. 46(3), 2037-2046 (2011) |
| [11] | Schmidt, E.A., Schrauf, M., Simon, M., Fritzsche, M., Buchner, A., Kincses, W.E.: Drivers’ misjudgement of vigilance state during prolonged monotonous daytime driving. Accid. Anal. Prev. 41(5), 1087-1093 (2009) |
| [12] | Cummings, M.L., Bauchwitz, B.: Safety implications of variability in autonomous driving assist alerting. IEEE Trans. Intell. Transp. Syst. 23(8), 12039-12049 (2022) |
| [13] | Kong, W., Lin, W., Babiloni, F., Hu, S., Borghini, G.: Investigating driver fatigue versus alertness using the granger causality network. Sensors 15(8), 19181-19198 (2015) |
| [14] | Di Flumeri, G., Ronca, V., Giorgi, A., Vozzi, A., Aricò, P., Sciaraffa, N., Zeng, H., Dai, G., Kong, W., Babiloni, F.: EEG-based index for timely detecting user’s Drowsiness occurrence in automotive applications. Front. Hum. Neurosci. 16 (2022) |
| [15] | Bose, R., Wang, H., Dragomir, A., Thakor, N.V., Bezerianos, A., Li, J.: Safety implications of variability in autonomous driving assist alerting. IEEE Trans. Cogn. Dev. Syst. 12(2), 323-331 (2019) |
| [16] | Lei, S., Roetting, M.: Influence of task combination on EEG spectrum modulation for driver workload estimation. Hum. Factors 53(2), 68-179 (2011) |
| [17] | Mabry, J.E., Glenn, T.L., Hickman, J.S.: Commercial motor vehicle operator fatigue detection technology catalog and review. Technical Report (2019) |
| [18] | Metcalfe, J.S., Marathe, A.R., Haynes, B., Paul, V.J., Gremillion, G.M., Drnec, K., Atwater, C., Estepp, J.R., Lukos, J.R., Carter, E.C., Nothwang, W.D.: Building a framework to manage trust in automation. In: Micro-and Nanotechnology Sensors, Systems, and Applications IX, pp. 351-361. SPIE (2017) |
| [19] | Molnar, L.J., Ryan, L.H., Pradhan, A.K., Eby, D.W., Louis, R.M.S., Zakrajsek, J.S.: Understanding trust and acceptance of automated vehicles: an exploratory simulator study of transfer of control between automated and manual driving. Transp. Res. F: Traffic Psychol. Behav. 58, 319-328 (2018) |
| [20] | Kyriakidis, M., de Winter, J.C., Stanton, N., Bellet, T., van Arem, B., Brookhuis, K., Martens, M.H., Bengler, K., Andersson, J., Merat, N., Reed, N.: A human factors perspective on automated driving. Theor. Issues Ergon. Sci. 20(3), 223-249 (2019) |
| [21] | Hoff, K.A., Bashir, M.: Trust in automation: integrating empirical evidence on factors that influence trust. Hum. Factors 57(3), 407-434 (2015) |
| [22] | Schaefer, K.E., Chen, J.Y., Szalma, J.L., Hancock, P.A.: A meta-analysis of factors influencing the development of trust in automation: implications for understanding autonomy in future systems. Hum. Factors 58(3), 377-400 (2016) |
| [23] | Drnec, K., Marathe, A.R., Lukos, J.R., Metcalfe, J.S.: From trust in automation to decision neuroscience: applying cognitive neuroscience methods to understand and improve interaction decisions involved in human automation interaction. Front. Hum. Neurosci. 10, 290 (2016) |
| [24] | Hopko, S.K., Mehta, R.K.: Trust in shared-space collaborative robots: shedding light on the human brain. Hum. Factors 00187208221109039 (2022) |
| [25] | Kohn, S.C., De Visser, E.J., Wiese, E., Lee, Y.C., Shaw, T.H.: Measurement of trust in automation: a narrative review and reference guide. Front. Psychol. 12 (2021) |
| [26] | He, D., DeGuzman, C.A., Donmez, B.: Anticipatory driving in automated vehicles: the effects of driving experience and distraction. Hum. Factors 00187208211026133 (2021) |
| [27] | Victor, T.W., Tivesten, E., Gustavsson, P., Johansson, J., Sangberg, F., Ljung Aust, M.: Automation expectation mismatch: incorrect prediction despite eyes on threat and hands on wheel. Hum. Factors 60(8), 1095-1116 (2018) |
| [28] | Schoettle, B., Sivak, M.: A survey of public opinion about autonomous and selfdriving vehicles in the U.S., the U.K., and Australia (2014). http://deepblue.lib.umich.edu/handle/2027.42/108384 |
| [29] | Piao, J., McDonald, M., Hounsell, N., Graindorge, M., Graindorge, T., Malhene, N.: Public views towards implementation of automated Vehicles in urban areas. Transp. Res. Proc. 14, 2168-2177 (2016) |
| [30] | Lee, J.G., Kim, K.J., Lee, S., Shin, D.H.: Can autonomous vehicles be safe and trustworthy? Effects of appearance and autonomy of unmanned driving systems. Int. J. Hum. Comput. Interact. 31(10), 682-691 (2015) |
| [31] | Jurecki, R., Poliak, M., Jaśkiewicz, M.: Young adult drivers: simulated behaviour in a car-following situation. Promet-Traffic Transp. 29(4), 381-390 (2017) |
| [32] | Hoff, K.A., Bashir, M.: Trust in automation: integrating empirical evidence on factors that influence trust. Hum. Factors 57(3), 407-434 (2015) |
| [33] | Young, M.S., Stanton, N.A.: Back to the future: brake reaction times for manual and automated vehicles. Ergonomics 50, 46-58 (2007) |
| [34] | Manser, M.P., Creaser, J., Boyle, L.N.,: Behavioural adaptation: methodological and behavrioal issues. In Behavioural Adaptation and Road Safety: Theory, Evidence and Action, pp. 339-359 (2013) |
| [35] | Körber, M., Cingel, A., Zimmermann, M., Bengler, K.: Vigilance decrement and passive fatigue caused by monotony in automated driving. Proc. Manuf. 3, 2403-2409 (2015) |
| [36] | Walker, G.H., Stanton, N.A., Salmon, P.: Trust in vehicle technology. Int. J. Veh. Des. 70, 157 (2016) |
| [37] | König, M., Neumayr, L.: Users’ resistance towards radical innovations: the case of the self-driving car. Transp. Res. F: Traffic Psychol. Behav. 44(3), 42-52 (2017) |
| [38] | Liu, P., Yang, R., Xu, Z.: How safe is safe enough for self-driving vehicles? Risk Anal. 39(2), 315-325 (2019) |
| [39] | Price, M.A., Venkatraman, V., Gibson, M.C., Lee, J.D., Mutlu, B.: Psychophysics of trust in vehicle control algorithms. (SAE Technical Paper) (2016). https://doi.org/10.4271/2016-01-0144 |
| [40] | Bellem, H., Schonenberg, T., Krems, J.F., Schrauf, M.: Objective metrics of comfort: Developing a driving style for highly automated vehicles. Transp. Res. F: Traffic Psychol. Behav. 41, 45-54 (2016) |
| [41] | Hergeth, S., Lorenz, L., Vilimek, R., Krems, J.F.: Keep your scanners peeled: gaze behavior as a measure of automation trust during highly automated driving. Hum. Factors 58, 509-519 (2016) |
| [42] | Shinohara, Y., Currano, R., Ju, W., Nishizaki, Y.: Visual attention during simulated autonomous driving in the US and Japan. In: Proceedings of the 9th International Conference on Automotive User Interfaces and Interactive Vehicular Applications, Oldenburg, pp. 144-153 (2017) |
| [43] | Howard, D., Dai, D.: Public perceptions of self-driving cars: the case of Berkeley, California. In: Proceedings of the 93rd Transportation Research Board Annual Meeting, Washington, D.C., pp. 1-21 (2014) |
| [44] | Bansal, P., Kockelman, K.M.: Are we ready to embrace connected and self-driving vehicles? A case study of Texans. Transportation 45(2), 641-675 (2018) |
| [45] | Haghzare, S., Campos, J.L., Bak, K., Mihailidis, A.: Older adults’ acceptance of fully automated vehicles: effect of exposure, driving style, age, and driving conditions. Accid. Anal. Prev. 150, 105919 (2021) |
| [46] | Jing, P., Du, L., Chen, Y., Shi, Y., Zhan, F., Xie, J.: Factors that influence parents’ intentions of using autonomous vehicles to transport children to and from school. Accid. Anal. Prev. 152, 105991 (2021) |
| [47] | Kaye, S., Lewis, I., Forward, S., Delhomme, P.: A priori acceptance of highly automated cars in Australia, France, and Sweden: a theoretically-informed investigation guided by the TPB and UTUAT. Accid. Anal. Prev. 137, 105441 (2020) |
| [48] | Faber, K., van Lierop, D.: How will older adults use automated vehicles? Assessing the role of AVs in overcoming perceived mobility barriers. Transp. Res. Part A Policy Pract. 133, 353-363 (2020) |
| [49] | Phipps-Nelson, J.O., Redman, J.R., Rajaratnam, S.M.: Temporal profile of prolonged, nighttime driving performance: breaks from driving temporarily reduce time on task fatigue but not sleepiness. J. Sleep Res. 20, 404-415 (2011) |
| [50] | Finkleman, J.M.: A large database study of the factors associated with workinduced fatigue. Hum. Factors 36, 232-243 (1994) |
| [51] | Xianglong, S., Hu, Z., Shumin, F., Zhenning, L.: Bus drivers’ mood states and reaction abilities at high temperatures. Transp. Res. Part F Traffic Psychol. Behav. 59, 436-444 (2018) |
| [52] | Strayer, D.L., Drews, F.A.: Cell-phone-induced driver distraction. Curr. Dir. Psychol. Sci. 16, 128-131 (2007) |
| [53] | Arexis, M., Maquestiaux, F., Gaspelin, N., Ruthruff, E., Didierjean, A.: Attentional capture in driving displays. Br. J. Psychol. 108, 259-275 (2017) |
| [54] | Chi, Y.M., Wang, Y.T., Wang, Y., Maier, C., Jung, T.P., Cauwenberghs, G.: Dry and noncontact EEG sensors for mobile brain-computer interfaces. IEEE Trans. Neural Syst. Rehabil. Eng. 20(2), 228-235 (2011) |
| [55] | Seet, M., Harvy, J., Bose, R., Dragomir, A., Bezerianos, A., Thakor, N.: Differential impact of autonomous vehicle malfunctions on human trust. IEEE Trans. Intell. Transp. Syst. 23(1), 548-557 (2022) |
| [56] | Park, C., Shahrdar, S., Nojoumian, M.: EEG-based classification of emotional state using an autonomous vehicle simulator. In: 2018 IEEE 10th Sensor Array and Multichannel Signal Processing Workshop (SAM), pp. 297-300. IEEE (2018) |
| [57] | Belcher, M.A., Huang, I., Battacharya, S., Hairston, D.W., Metcalfe, J.S.: EEG-based prediction of driving events from passenger cognitive state using Morlet Wavelet and Evoked Responses. Transp. Eng. 8, 100107 (2022) |
| [58] | DSouza, K., Dang, T., Metcalfe, J.S., Battacharya, S.: Brain-based indicators of passenger trust during open-road driving. In: 2021 IEEE 94th Vehicular Technology Conference (VTC2021-Fall), pp. 1-6. IEEE (2021) |
| [59] | Akash, K., Hu, W.L., Jain, N., Reid, T.: A classification model for sensing human trust in machines using EEG and GSR. ACM Trans. Interact. Intell. Syst. (TiiS) 8(4), 1-20 (2018) |
| [60] | Harmon-Jones, E., Gable, P.A.: On the role of asymmetric frontal cortical activity in approach and withdrawal motivation: an updated review of the evidence. Psychophysiology 55(1), e12879 (2018) |
| [61] | Oh, S., Seong, Y., Yi, S., Park, S.: Neurological measurement of human trust in automation using electroencephalogram. Int. J. Fuzzy Logic Intell. Syst. 20(4), 261-271 (2020) |
| [62] | Dragomir, A., Omurtag, A. : Brain’s networks and their functional significance in cognition. In: Handbook of Neuroengineering, pp. 1-30 (2021) |
| [63] | Vinck, M., Oostenveld, R., Van Wingerden, M., Battaglia, F., Pennartz, C.M.: An improved index of phase-synchronization for electrophysiological data in the presence of volume-conduction, noise and sample-size bias. Neuroimage 55(4), 1548-1565 (2011) |
| [64] | Xu, T., Dragomir, A., Liu, X., Yin, H., Wan, F., Wang, H.: An EEG study of human trust in autonomous vehicle basing on graphic theoretical analysis. Front. Neuroinform. 16, 907942 (2022) |
| [65] | Pinti, P., Tachtsidis, I., Hamilton, A., Hirsch, J., Aichelburg, C., Gilbert, S., Burgess, P.W.: The present and future use of functional near-infrared spectroscopy (fNIRS) for cognitive neuroscience. Ann. N. Y. Acad. Sci. 1464(1), 5-29 (2020) |
| [66] | Sibi, S., Baiters, S., Mok, B., Steiner, M., Ju, W.: Assessing driver cortical activity under varying levels of automation with functional near infrared spectroscopy. In: 2017 IEEE Intelligent Vehicles Symposium (IV), pp. 1509-1516. IEEE (2017) |
| [67] | Balters, S., Baker, J.M., Geeseman, J.W., Reiss, A.L.: A methodological review of fNIRS in driving research: relevance to the future of autonomous vehicles. Front. Hum. Neurosci. 15, 637589 (2021) |
| [68] | Perello-March, J.R., Burns, C.G., Woodman, R., Elliott, M.T., Birrell, S.A.: Using fNIRS to verify trust in highly automated driving. IEEE Trans. Intell. Transp. Syst. (2022) |
| [69] | Unni, A., Trende, A., Pauley, C., Weber, L., Biebl, B., Kacianka, S., Lüdtke, A., Bengler, K., Pretschner, A., Fränzle, M., Rieger, J.W.: Investigating differences in behavior and brain in human-human and human-autonomous vehicle interactions in time-critical situations. Front. Neuroergonomics 3 (2022) |
| [70] | Parasuraman, R., de Visser, E., Wiese, E. and Madhavan, P.: Human trust in other humans, automation, robots, and cognitive agents: neural correlates and design implications. In: Proceedings of the Human Factors and Ergonomics Society Annual Meeting, vol. 58, no. 1, pp. 340-344. ISAGE (2014) |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
