Determining the effect of emotional images brightness on EEG signals by classification algorithms. (English) Zbl 1507.92051
Summary: This study demonstrated the effects of image brightness on brain activity in neuroscience research, in which the brightness of emotional images had not been taken into account. Electroencephalography recordings from 14 electrode sites of 31 healthy participants were examined during the presentation of original and bright versions of neutral, pleasant and unpleasant images. Power spectra of the recordings were obtained using the short time Fourier transform. The features were extracted from the power spectra for specific time-frequency windows and data obtained from features were classified using support vector machine (SVM), partial least squares regression (PLSR) and k-nearest neighbor (k-NN) algorithms between the original and bright groups for three emotional contents. New features were created with feature combinations providing high classification accuracy. The data obtained from new features were reclassified using SVM, PLSR, k-NN and voting methods between the original and bright groups for three emotional contents. The classification results revealed that the datasets obtained for the original and bright versions of neutral, pleasant and unpleasant images could be separated with 71–81% accuracy. The brightness effect occurred predominantly in the frontal and central regions. This effect was observed in the early time window of visual processing for pleasant and unpleasant images, and in the late time window for neutral images. The findings emphasize that image brightness of affects the power of brain activity and therefore, is an important parameter to be considered in neuroscience research.
MSC:
| 92C55 | Biomedical imaging and signal processing |
References:
| [1] | Aftanas, LI; Varlamov, AA; Pavlov, SV; Makhnev, VP; Reva, NV, Time-dependent cortical asymmetries induced by emotional arousal: EEG analysis of event-related synchronization and desynchronization in individually defined frequency bands, International Journal of Psychophysiology, 44, 67-82 (2002) · doi:10.1016/S0167-8760(01)00194-5 |
| [2] | Aydin, S., Demirtaş, S., Ateş, K., Tunga M.A. (2016) Emotion recognition with eigen features of frequency band activities embedded in induced brain oscillations mediated by affective pictures. Int. J Neural Syst doi:10.1142/S0129065716500131 |
| [3] | Balconi, M.; Brambilla, E.; Falbo, L., Appetitive vs defensive responses to emotional cues autonomic measures and brain oscillation modulation, Brain Research (2009) · doi:10.1016/j.brainres.2009.08.056 |
| [4] | Balconi, M.; Falbo, L.; Brambilla, E., BIS/BAS responses to emotional cues: Self report, autonomic measure and alpha band modulation, Personality and Individual Differences, 47, 858-863 (2009) · doi:10.1016/j.paid.2009.07.004 |
| [5] | Bamidis, PD; Klados, MA; Frantzidis, C.; Vivas, AB; Papadelis, C.; Lithari, C.; Pappas, C., A framework combining delta event-related oscillations (EROs) and synchronisation effects (ERD/ERS) to study emotional processing, Comput. Math. Methods Med. Intell. Neurosci. (2009) · doi:10.1155/2009/549419 |
| [6] | Bekhtereva, V.; Craddock, M.; Müller, MM, Attentional bias to affective faces and complex IAPS images in early visual cortex follows emotional cue extraction, NeuroImage, 112, 254-266 (2015) · doi:10.1016/j.neuroimage.2015.03.052 |
| [7] | Boulesteix, AL; Strimmer, K., Partial least squares: A versatile tool for the analysis of high-dimensional genomic data, Briefings in Bioinformatics, 8, 32-44 (2007) · doi:10.1093/bib/bbl016 |
| [8] | Clayson, PE; Larson, MJ, The impact of recent and concurrent affective context on cognitive control: An ERP study of performance monitoring, International Journal of Psychophysiology, 143, 44-56 (2019) · doi:10.1016/j.ijpsycho.2019.06.007 |
| [9] | Costa, T.; Cauda, F.; Crini, M.; Tatu, MK; Celeghin, A.; De Gelder, B.; Tamietto, M., Temporal and spatial neural dynamics in the perception of basic emotions from complex scenes, Soc. Cogn. Affect. Neurosci., 9, 1690-1703 (2014) · doi:10.1093/scan/nst164 |
| [10] | Cover, T. M., & Hart, P. E. (2012). Nearest Neighbor Pattern Classfication, I 1-28 |
| [11] | Cuthbert, BN; Schupp, HT; Bradley, MM; Birbaumer, N.; Lang, PJ, Brain potentials in affective picture processing: Covariation with autonomic arousal and affective report, Biological Psychology, 52, 95-111 (2000) · doi:10.1016/S0301-0511(99)00044-7 |
| [12] | Delplanque, S.; N’diaye, K.; Scherer, K.; Grandjean, D., Spatial frequencies or emotional effects?. A systematic measure of spatial frequencies for IAPS pictures by a discrete wavelet analysis, Journal of Neuroscience Methods, 165, 144-150 (2007) · doi:10.1016/j.jneumeth.2007.05.030 |
| [13] | Delplanque, S.; Silvert, L.; Hot, P.; Rigoulot, S.; Sequeira, H., Arousal and valence effects on event-related P3a and P3b during emotional categorization, International Journal of Psychophysiology (2006) · doi:10.1016/j.ijpsycho.2005.06.006 |
| [14] | Donchin, E.; Coles, MGH, Is the P300 component a manifestation of context updating?, The Behavioral and Brain Sciences, 11, 357 (1988) · doi:10.1017/S0140525X00058027 |
| [15] | Eroğlu, K.; Kayıkçıoğlu, T.; Osman, O., Eff ect of brightness of visual stimuli on EEG signals, Behavioral Brain Science (2020) · doi:10.1016/j.bbr.2020.112486 |
| [16] | Feng, C.; Wang, L.; Liu, C.; Zhu, X.; Dai, R.; Mai, X.; Luo, YJ, The time course of the influence of valence and arousal on the implicit processing of affective pictures, PLoS ONE, 7, 1-9 (2012) · doi:10.1371/journal.pone.0029668 |
| [17] | Frantzidis, CA; Bratsas, C.; Klados, MA; Konstantinidis, E.; Lithari, CD; Vivas, AB; Papadelis, CL; Kaldoudi, E.; Pappas, C.; Bamidis, PD, On the classification of emotional biosignals evoked while viewing affective pictures: An integrated data-mining-based approach for healthcare applications, IEEE Transactions on Information Technology in Biomedicine, 14, 309-318 (2010) · doi:10.1109/TITB.2009.2038481 |
| [18] | Ganin, IP; Kosichenko, EA; Kaplan, AY, Properties of EEG Responses to Emotionally Significant Stimuli Using a P300 Wave-Based Brain-Computer Interface, Neuroscience and Behavioral Physiology, 48, 1093-1099 (2018) · doi:10.1007/s11055-018-0672-7 |
| [19] | Ghodrati, M.; Ghodousi, M.; Yoonessi, A., Low-level contrast statistics of natural images can modulate the frequency of event-related potentials (ERP) in humans, Frontiers in Human Neuroscience, 10, 1-12 (2016) · doi:10.3389/fnhum.2016.00630 |
| [20] | Giannakopoulos, T.; Pikrakis, A., Signal transforms and filtering essentials, Introd. to Audio Anal. (2014) · doi:10.1016/b978-0-08-099388-1.00003-0 |
| [21] | Gianotti, LRR; Faber, PL; Schuler, M.; Pascual-Marqui, RD; Kochi, K.; Lehmann, D., First valence, then arousal: The temporal dynamics of brain electric activity evoked by emotional stimuli, Brain Topography (2008) · doi:10.1007/s10548-007-0041-2 |
| [22] | Goto, N.; Lim, XL; Shee, D.; Hatano, A.; Khong, KW; Buratto, LG; Watabe, M.; Schaefer, A., Can brain waves really tell If a product will be purchased? Inferring consumer preferences from single-item brain potentials, Frontiers in Integrative Neuroscience, 13, 1-13 (2019) · doi:10.3389/fnint.2019.00019 |
| [23] | Groen, IIA; Silson, EH; Baker, CI, Contributions of low- and high-level properties to neural processing of visual scenes in the human brain, Transaction of the Royal Society B Biological Sciences Philos (2017) · doi:10.1098/rstb.2016.0102 |
| [24] | Güntekin, B.; Başar, E., A new interpretation of P300 responses upon analysis of coherences, Cognitive Neurodynamics (2010) · doi:10.1007/s11571-010-9106-0 |
| [25] | Güntekin, B.; Başar, E., Event-related beta oscillations are affected by emotional eliciting stimuli, Neuroscience Letters (2010) · doi:10.1016/j.neulet.2010.08.002 |
| [26] | Güntekin, B.; Başar, E., Event-related beta oscillations are affected by emotional eliciting stimuli, Neuroscience Letters, 483, 173-178 (2010) · doi:10.1016/j.neulet.2010.08.002 |
| [27] | Güntekin, B.; Başar, E., A review of brain oscillations in perception of faces and emotional pictures, Neuropsychologia (2014) · doi:10.1016/j.neuropsychologia.2014.03.014 |
| [28] | Güntekin, B.; Femir, B.; Gölbaşı, BT; Tülay, E.; Başar, E., Affective pictures processing is reflected by an increased long-distance EEG connectivity, Cognitive Neurodynamics, 11, 355-367 (2017) · doi:10.1007/s11571-017-9439-z |
| [29] | Hansen, BC; Jacques, T.; Johnson, AP; Ellemberg, D., From spatial frequency contrast to edge preponderance: The differential modulation of early visual evoked potentials by natural scene stimuli, Visual Neuroscience, 28, 221-237 (2011) · doi:10.1017/S095252381100006X |
| [30] | Hansen, BC; Johnson, AP; Ellemberg, D., Different spatial frequency bands selectively signal for natural image statistics in the early visual system, Journal of Neurophysiology, 108, 2160-2172 (2012) · doi:10.1152/jn.00288.2012 |
| [31] | Harris, JM; Ciorciari, J.; Gountas, J., Consumer neuroscience and digital/social media health/social cause advertisement effectiveness, Behavioral Science (2019) · doi:10.3390/bs9040042 |
| [32] | Hess, RF; Plant, GT, The effect of temporal frequency variation on threshold contrast sensitivity deficits in optic neuritis, Journal of Neurology, Neurosurgery and Psychiatry, 46, 322-330 (1983) · doi:10.1136/jnnp.46.4.322 |
| [33] | Huerta, M.; Leiva, V.; Lillo, C.; Rodríguez, M., A beta partial least squares regression model: Diagnostics and application to mining industry data, Applied Stochastic Models in Business and Industry, 34, 305-321 (2018) · Zbl 1414.62509 · doi:10.1002/asmb.2278 |
| [34] | Jing, K.; Mei, Y.; Song, Z.; Wang, H.; Shi, R., How Do Price and Quantity Promotions Affect Hedonic Purchases? An ERPs Study, Frontiers in Neuroscience, 13, 1-9 (2019) · doi:10.3389/fnins.2019.00526 |
| [35] | Johannes, S.; Münte, TF; Heinze, HJ; Mangun, GR, Luminance and spatial attention effects on early visual processing, Cognitive Brain Research (1995) · doi:10.1016/0926-6410(95)90008-X |
| [36] | Keil, A.; Stolarova, M.; Moratti, S.; Ray, WJ, Adaptation in human visual cortex as a mechanism for rapid discrimination of aversive stimuli, NeuroImage, 36, 472-479 (2007) · doi:10.1016/j.neuroimage.2007.02.048 |
| [37] | M.G. Kounelakis, M. Zervakis, X. Kotsiakis, (2007) The Impact of Microarray Technology in Brain Cancer, Elsevier B.V. doi:10.1016/B978-044452855-1/50015-5. |
| [38] | Kurt, P.; Eroğlu, K.; Bayram KGüntekin, TB, The modulation of delta responses in the interaction of brightness and emotion, International Journal of Psychophysiology (2017) · doi:10.1016/j.ijpsycho.2016.11.013 |
| [39] | Kutas, M., Mccarthy, G., & Donchin, E. (1977). Augmenting mental chronometry: the P300 as a measure of stimulus evaluation time, Science (80-). 197: 792-795. |
| [40] | Luck, S. J. (2005). An introduction to the event-related potential technique, MA: MIT Press, Cambridge. |
| [41] | Lakens, D.; Fockenberg, DA; Lemmens, KPH; Ham, J.; Midden, CJH, Brightness differences influence the evaluation of affective pictures, Cognition and Emotion, 27, 1225-1246 (2013) · doi:10.1080/02699931.2013.781501 |
| [42] | Lang, P. J., Bradley, M. M., & Cuthbert, B. N. (1997). International affective picture system (IAPS): Technical manual and affective ratings, NIMH Central Study Emotional Attention pp. 39-58. |
| [43] | Leyh, R.; Heinisch, C.; Kungl, MT; Spangler, G., Attachment Representation Moderates the Influence of Emotional Context on Information Processing, Frontiers in Human Neuroscience, 10, 1-12 (2016) · doi:10.3389/fnhum.2016.00278 |
| [44] | Li, W.; Liu, Z., A method of SVM with normalization in intrusion detection, Procedia, Environmental Sciences, 11, 256-262 (2011) · doi:10.1016/j.proenv.2011.12.040 |
| [45] | Liu, N.; Wang, K.; Jin, X.; Gao, B.; Dellandréa, E.; Chen, L., Visual affective classification by combining visual and text features, PLoS ONE, 12, 6-9 (2017) · doi:10.1371/journal.pone.0183018 |
| [46] | Ma, H.; Mo, Z.; Zhang, H.; Wang, C.; Fu, H., The temptation of zero price: Event-related potentials evidence of how price framing influences the purchase of bundles, Frontiers in Neuroscience, 12, 1-8 (2018) · doi:10.3389/fnins.2018.00251 |
| [47] | Ma, Q.; Wang, X.; Shu, L.; Dai, S., P300 and categorization in brand extension, Neuroscience Letters (2008) · doi:10.1016/j.neulet.2007.11.022 |
| [48] | Magliero, A.; Bashore, TR; Coles, MGH; Donchin, E., On the Dependence of P300 Latency on Stimulus Evaluation Processes.pdf, Psychophysiology, 21, 171-186 (1984) · doi:10.1111/j.1469-8986.1984.tb00201.x |
| [49] | Martini, N.; Menicucci, D.; Sebastiani, L.; Bedini, R.; Pingitore, A.; Vanello, N.; Milanesi, M.; Landini, L.; Gemignani, A., The dynamics of EEG gamma responses to unpleasant visual stimuli: From local activity to functional connectivity, NeuroImage, 60, 922-932 (2012) · doi:10.1016/j.neuroimage.2012.01.060 |
| [50] | Mavratzakis, A.; Herbert, C.; Walla, P., Emotional facial expressions evoke faster orienting responses, but weaker emotional responses at neural and behavioural levels compared to scenes: A simultaneous EEG and facial EMG study, NeuroImage (2016) · doi:10.1016/j.neuroimage.2015.09.065 |
| [51] | McFarland, DJ; Parvaz, MA; Sarnacki, WA; Goldstein, RZ; Wolpaw, JR, Prediction of subjective ratings of emotional pictures by EEG features, Journal of Neural Engineering, 14, 1-9 (2017) · doi:10.1088/1741-2552/14/1/016009 |
| [52] | Migliore, S.; Curcio, G.; Porcaro, C.; Cottone, C.; Simonelli, I.; D’aurizio, G.; Landi, D.; Palmieri, MG; Ghazaryan, A.; Squitieri, F.; Filippi, MM; Vernieri, F., Emotional processing in RRMS patients: Dissociation between behavioural and neurophysiological response, Multiple Sclerosis and Related Disorder, 27, 344-349 (2019) · doi:10.1016/j.msard.2018.11.019 |
| [53] | Miskovic, V.; Schmidt, LA, Cross-regional cortical synchronization during affective image viewing, Brain Research, 1362, 102-111 (2010) · doi:10.1016/j.brainres.2010.09.102 |
| [54] | Montagu, JD; Coles, EM, Mechanism and Measurement of the Galvanic Skin Response: An Addendum, Psychological Bulletin, 69, 74-76 (1968) · doi:10.1037/h0025305 |
| [55] | Müller, MM; Gundlach, C., Competition for attentional resources between low spatial frequency content of emotional images and a foreground task in early visual cortex, Psychophysiology, 54, 429-443 (2017) · doi:10.1111/psyp.12792 |
| [56] | Olofsson, JK; Nordin, S.; Sequeira, H.; Polich, J., Affective picture processing: An integrative review of ERP findings, Biological Psychology (2008) · doi:10.1016/j.biopsycho.2007.11.006 |
| [57] | Olofsson, JK; Polich, J., Affective visual event-related potentials: Arousal, repetition, and time-on-task, Biological Psychology (2007) · doi:10.1016/j.biopsycho.2006.12.006 |
| [58] | Peli, E., Contrast in complex images, Journal of the Optical Society of America, 7, 2032-2040 (1990) · doi:10.1364/JOSAA.7.002032 |
| [59] | Sánchez-Reolid, R.; García, A.; Vicente-Querol, M.; Fernández-Aguilar, L.; López, M.; González, A., Artificial Neural Networks to Assess Emotional States from Brain-Computer Interface, Electronics, 7, 384 (2018) · doi:10.3390/electronics7120384 |
| [60] | Schettino, A.; Keil, A.; Porcu, E.; Müller, MM, Shedding light on emotional perception: Interaction of brightness and semantic content in extrastriate visual cortex, NeuroImage, 133, 341-353 (2016) · doi:10.1016/j.neuroimage.2016.03.020 |
| [61] | Singh, MI; Singh, M., Development of a real time emotion classifier based on evoked EEG, Biocybern, BioMedical Engineering, 37, 498-509 (2017) · doi:10.1016/j.bbe.2017.05.004 |
| [62] | Smith, N. K., Cacioppo, J. T., Larsen, J. T., & Chartrand, T. L. (2003). May I have your attention, please: Electrocortical responses to positive and negative stimuli. |
| [63] | Tian, Y.; Zhang, H.; Pang, Y.; Lin, J., Classification for Single-trial N170 during responding to facial picture with emotion, Frontiers in Computational Neuroscience (2018) · doi:10.3389/fncom.2018.00068 |
| [64] | Torralba, A.; Oliva, A., Statistics of natural image categories, Network: Computation in Neural Systems, 14, 391-412 (2003) · doi:10.1088/0954-898X_14_3_302 |
| [65] | Valberg, A. (2005). Light Vision Color. Wiley. |
| [66] | Valdez, P.; Mehrabian, A., Effects of color on emotions, Journal of Experimental Psychology: General, 123, 394-409 (1994) · doi:10.1037/0096-3445.123.4.394 |
| [67] | Vasilios, V.; Gasteratos, A., The Human Visual System (2006) · doi:10.1016/B978-0-12-405906-1.00002-7 |
| [68] | L. S. Vogel EK,, The visual N1 component as an index of a discrimination process, Psychophysiology, 37, 190-203 (2000) · doi:10.1111/1469-8986.3720190 |
| [69] | Yusoff, N.; Anuar, NN; Reza, MF, The effect of sex on the electropsychological process of emotional arousal intensity, Malaysian Journal of Medical Science, 25, 103-110 (2018) · doi:10.21315/mjms2018.25.3.10 |
| [70] | Zamuner, E., The Role of the Visual System in Emotion Perception, Acta Anal., 28, 179-187 (2013) · doi:10.1007/s12136-012-0151-7 |
| [71] | Zhang, Q.; Lee, M., Emotion development system by interacting with human EEG and natural scene understanding, Cognitive Systems Research (2012) · doi:10.1016/j.cogsys.2010.12.012 |
| [72] | Zhang, W.; Lu, J.; Liu, X.; Fang, H.; Li, H.; Wang, D.; Shen, J., Event-related synchronization of delta and beta oscillations reflects developmental changes in the processing of affective pictures during adolescence, International Journal of Psychophysiology (2013) · doi:10.1016/j.ijpsycho.2013.10.005 |
| [73] | Z.H. Zhou, (2011) Cost-sensitive learning, Lect. Notes Comput. Sci. (Including Subser. Lect. Notes Artif. Intell. Lect. Notes Bioinformatics). 6820 LNAI doi:10.1007/978-3-642-22589-5_2. |
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.
