×

Identifying dominant industrial sectors in market states of the S&P 500 financial data. (English) Zbl 1539.91143

Summary: Understanding and forecasting changing market conditions in complex economic systems like the financial market is of great importance to various stakeholders such as financial institutions and regulatory agencies. Based on the finding that the dynamics of sector correlation matrices of the S&P 500 stock market can be described by a sequence of distinct states via a clustering algorithm, we try to identify the industrial sectors dominating the correlation structure of each state. For this purpose, we use a method from explainable artificial intelligence (XAI) on daily S&P 500 stock market data from 1992 to 2012 to assign relevance scores to every feature of each data point. To compare the significance of the features for the entire data set we develop an aggregation procedure and apply a Bayesian change point analysis to identify the most significant sector correlations. We show that the correlation matrix of each state is dominated only by a few sector correlations. Especially the energy and IT sector are identified as key factors in determining the state of the economy. Additionally we show that a reduced surrogate model, using only the eight sector correlations with the highest XAI-relevance, can replicate 90% of the cluster assignments. In general our findings imply an additional dimension reduction of the dynamics of the financial market.
{© 2023 The Author(s). Published on behalf of SISSA Medialab srl by IOP Publishing Ltd}

MSC:

91G80 Financial applications of other theories
62P05 Applications of statistics to actuarial sciences and financial mathematics
91B80 Applications of statistical and quantum mechanics to economics (econophysics)

Software:

TensorFlow

References:

[1] Abadi, Met al.2015TensorFlow: large-scale machine learning on heterogeneous systemswww.tensorflow.org
[2] Aroussi, R2022yfinance 0.1.70https://pypi.org/project/yfinance/
[3] Ashwin, P.; Timme, M., When instability makes sense, Nature, 436, 36-37 (2005) · doi:10.1038/436036b
[4] Böhm, W.; Hornik, K., Generating random correlation matrices by the simple rejection method: why it does not work, Stat. Probab. Lett., 87, 27-30 (2014) · Zbl 1298.65077 · doi:10.1016/j.spl.2013.12.012
[5] BaFin—Federal Agency for Financial Services Supervision2018Big data meets artificial intelligencewww.bafin.de/SharedDocs/Downloads/EN/dl_bdai_studie_en.html
[6] Brown, S. J., The number of factors in security returns, J. Finance, 44, 1247-62 (1989) · doi:10.1111/j.1540-6261.1989.tb02652.x
[7] Chollet, Fet al.2015Kerashttps://github.com/fchollet/keras
[8] Dose, V.; Menzel, A., Bayesian analysis of climate change impacts in phenology, Glob. Change Biol., 10, 259-72 (2004) · doi:10.1111/j.1529-8817.2003.00731.x
[9] Fernex, D.; Noack, B. R.; Semaan, R., Cluster-based network modeling—from snapshots to complex dynamical systems, Sci. Adv., 7, eabf5006 (2021) · doi:10.1126/sciadv.abf5006
[10] Geron, A., Hands-On Machine Learning With Scikit-Learn, Keras and Tensorflow (2019), Sebastopol, CA: O’Reilly Media, Sebastopol, CA
[11] Giada, L.; Marsili, M., Algorithms of maximum likelihood data clustering with applications, Physica A, 315, 650-64 (2002) · Zbl 1001.65013 · doi:10.1016/S0378-4371(02)00974-3
[12] Gozgor, G.; Lau, C. K M.; Lu, Z., Energy consumption and economic growth: new evidence from the OECD countries, Energy, 153, 27-34 (2018) · doi:10.1016/j.energy.2018.03.158
[13] Graben, P. B.; Hutt, A., Detecting recurrence domains of dynamical systems by symbolic dynamics, Phys. Rev. Lett., 110 (2013) · doi:10.1103/PhysRevLett.110.154101
[14] Grojean, C.; Paul, A.; Qian, Z.; Strümke, I., Lessons on interpretable machine learning from particle physics, Nat. Rev. Phys., 4, 284-6 (2022) · doi:10.1038/s42254-022-00456-0
[15] Heßler, M.; Kamps, O., Bayesian on-line anticipation of critical transitions, New J. Phys., 24 (2022) · doi:10.1088/1367-2630/ac46d4
[16] Heckens, A. J.; Guhr, T., A new attempt to identify long-term precursors for endogenous financial crises in the market correlation structures, J. Stat. Mech. (2022) · Zbl 1539.91140 · doi:10.1088/1742-5468/ac59ab
[17] Heckens, A. J.; Guhr, T., New collectivity measures for financial covariances and correlations, Physica A, 604 (2022) · Zbl 07582070 · doi:10.1016/j.physa.2022.127704
[18] Heckens, A. J.; Krause, S. M.; Guhr, T., Uncovering the dynamics of correlation structures relative to the collective market motion, J. Stat. Mech. (2020) · Zbl 1459.91223 · doi:10.1088/1742-5468/abb6e2
[19] Hutt, A.; Svensen, M.; Kruggel, F.; Friedrich, R., Detection of fixed points in spatiotemporal signals by clustering method, Phys. Rev. E, 61, R4691-3 (2000) · doi:10.1103/PhysRevE.61.R4691
[20] Kasperowicz, R., Electricity consumption and economic growth: evidence from Poland, J. Int. Stud., 1, 46-57 (2014) · doi:10.14254/2071-8330.2014/7-1/4
[21] Kauffmann, J.; Esders, M.; Ruff, L.; Montavon, G.; Samek, W.; Müller, K-R, From clustering to cluster explanations via neural networks, IEEE Trans. Neural Netw. Learn. Syst., 1, 1-15 (2022) · doi:10.1109/TNNLS.2022.3185901
[22] Koeppe, A.; Bamer, F.; Selzer, M.; Nestler, B.; Markert, B., Explainable artificial intelligence for mechanics: physics-explaining neural networks for constitutive models, Front. Mater., 8, 636 (2022) · doi:10.3389/fmats.2021.824958
[23] Laloux, L.; Cizeau, P.; Bouchaud, J-P; Potters, M., Noise dressing of financial correlation matrices, Phys. Rev. Lett., 83, 1467-70 (1999) · doi:10.1103/PhysRevLett.83.1467
[24] Landecker, W.; Thomure, M. D.; Bettencourt, L. M A.; Mitchell, M.; Kenyon, G. T.; Brumby, S. P., Interpreting individual classifications of hierarchical networks, pp 32-38 (2013)
[25] Lu, W-C, Electricity consumption and economic growth: evidence from 17 Taiwanese industries, Sustainability, 9, 50 (2017) · doi:10.3390/su9010050
[26] Münnix, M. C.; Shimada, T.; Schäfer, R.; Leyvraz, F.; Seligman, T. H.; Guhr, T.; Stanley, H. E., Identifying states of a financial market, Sci. Rep., 2, 644 (2012) · doi:10.1038/srep00644
[27] MacQueen, J., Some methods for classification and analysis of multivariate observations, vol 5.1, pp 281-97 (1967) · Zbl 0214.46201
[28] Mantegna, R. N., Degree of correlation inside a financial market, AIP Conf. Proc., 411, 197-202 (1997) · doi:10.1063/1.54189
[29] Mantegna, R.; Stanley, H., An Introduction to Econophysics. Correlations and Complexity in Finance (2000), Cambridge: Cambridge University Press, Cambridge · Zbl 1138.91300
[30] Markowitz, H., Portfolio selection, J. Finance, 7, 77 (1952) · doi:10.2307/2975974
[31] Marsili, M., Dissecting financial markets: sectors and states, Quant. Finance, 2, 297-302 (2002) · Zbl 1405.91755 · doi:10.1088/1469-7688/2/4/305
[32] Marti, G.; Nielsen, F.; Bińkowski, M.; Donnat, P., A review of two decades of correlations, hierarchies, networks and clustering in financial markets, Progress in Information Geometry: Theory and Applications, pp 245-74 (2021), Cham: Springer, Cham · Zbl 1471.91547
[33] McKinney, W.; van der Walt, S.; Millman, J., Data structures for statistical computing in Python, pp 56-61 (2010)
[34] Molnar, C., Interpretable Machine Learning. A Guide for Making Black Box Models Explainable (2022), Munich: Leanpub, Munich
[35] Montavon, G.; Binder, A.; Lapuschkin, S.; Samek, W.; Müller, K-R; Samek, W.; Montavon, G.; Vedaldi, A.; Hansen, L. K.; Müller, K-R, Layer-wise relevance propagation: an overview, Explainable AI. Interpreting, Explaining and Visualizing Deep Learning, pp 193-209 (2019), Cham: Springer, Cham
[36] Montavon, G.; Samek, W.; Müller, K-R, Methods for interpreting and understanding deep neural networks, Digit. Signal Process., 73, 1-15 (2018) · doi:10.1016/j.dsp.2017.10.011
[37] Neubauer, M. S.; Roy, A., Explainable AI for high energy physics (2022)
[38] Plerou, V.; Gopikrishnan, P.; Rosenow, B.; Nunes Amaral, L. A.; Stanley, H. E., Universal and nonuniversal properties of cross correlations in financial time series, Phys. Rev. Lett., 83, 1471-4 (1999) · doi:10.1103/PhysRevLett.83.1471
[39] Reback, Jet al.2022pandas-dev/pandas: Pandas (v1.4.3)https://zenodo.org/record/3509134
[40] Rinn, P.; Stepanov, Y.; Peinke, J.; Guhr, T.; Schäfer, R., Dynamics of quasi-stationary systems: finance as an example, Europhys. Lett., 110 (2015) · doi:10.1209/0295-5075/110/68003
[41] Rudin, C., Stop explaining black box machine learning models for high stakes decisions and use interpretable models instead, Nat. Mach. Intell., 1, 206-15 (2019) · doi:10.1038/s42256-019-0048-x
[42] S&P Dow Jones Indices2018Sector classificationwww.msci.com/documents/10199/4547797/Effective+Until+February+28,+2014.xls(Accessed: 3 March 2022)
[43] Samek, W.; Montavon, G.; Vedaldi, A.; Hansen, L. K.; Müller, K-R, Explainable AI. Interpreting, Explaining and Visualizing Deep Learning (2019), Cham: Springer, Cham
[44] Sandoval, L., A map of the brazilian stock market, Adv. Complex Syst., 15 (2012) · doi:10.1142/S0219525912500427
[45] Sandoval, L.; Franca, I. D P., Correlation of financial markets in times of crisis, Physica A, 391, 187-208 (2012) · doi:10.1016/j.physa.2011.07.023
[46] Schäfer, R.; Guhr, T., Local normalization. Uncovering correlations in non-stationary financial time series, Physica A, 389, 3856-65 (2010) · doi:10.1016/j.physa.2010.05.030
[47] Shrikumar, A.; Greenside, P.; Kundaje, A.; Precup, D.; Teh, Y. W., Learning important features through propagating activation differences, vol 70, pp 3145-53 (2017), PMLR
[48] Stepanov, Y.; Rinn, P.; Guhr, T.; Peinke, J.; Schäfer, R., Stability and hierarchy of quasi-stationary states: financial markets as an example, J. Stat. Mech. (2015) · Zbl 1456.62115 · doi:10.1088/1742-5468/2015/08/P08011
[49] van den Berg, MKuiper, O2020Xai in the financial sector. A conceptual framework for explainable AI (XAI)www.internationalhu.com/research/projects/explainable-ai-in-the-financial-sector
[50] von der Linden, W.; Dose, V.; von Toussaint, U., Bayesian Probability Theory. Applications in the Physical Sciences (2014), Cambridge: Cambridge University Press, Cambridge · Zbl 1358.62008
[51] Zhang, J.; Bargal, S. A.; Lin, Z.; Brandt, J.; Shen, X.; Sclaroff, S., Top-down neural attention by excitation backprop, Int. J. Comput. Vision, 126, 1084-102 (2018) · doi:10.1007/s11263-017-1059-x
This reference list is based on information provided by the publisher or from digital mathematics libraries. Its items are heuristically matched to zbMATH identifiers and may contain data conversion errors. In some cases that data have been complemented/enhanced by data from zbMATH Open. This attempts to reflect the references listed in the original paper as accurately as possible without claiming completeness or a perfect matching.