Are LLMs Rational Investors?
A Study on Detecting and Reducing the Financial Bias in LLMs

To comprehensively assess the rationality of LLMs in financial markets, we scrutinized their responses to emergent information, distinguishing between event news and interactions, amid the prevalent noise in online investor dialogues in China. We analyzed historical events impacting company stock prices and adopted a refined classification of financial events into four primary categories: Corporate Governance and Equity Changes (CGEC), Financial Reports and Earnings Expectations (FREE), Market Behavior and Announcements (MBA), and Negative Events and Risk Management (NERM), detailed in Appendix C.

Throughout 2023, we compiled 300 news articles, including a subset of 24 emotionally nuanced pieces $N^{\prime}=\{n_{1},n_{2},\ldots,n_{24}\}$ to enhance the bias detection in LLMs. This subset, detailed in Appendix C, contained articles categorized into nine positive, nine negative, and six mixed emotions.

Additionally, we assembled 10 neutral interaction pairs $I=\{(q_{1},r_{1}),(q_{2},r_{2}),\ldots,(q_{10},r_{10})\}$ to analyze LLM comprehension in a controlled environment.

For each $n\in N^{\prime}$ and $(q,r)\in I$, numerical details were abstracted to proportional figures to standardize data across varying company market caps, as documented in Appendix F.

We sampled 600 companies from the Chinese A-share market, excluding delisting entities, distributed across three tiers of market capitalization: top, middle, and bottom, each containing 200 companies. This selection method aimed to test Belief Bias in LLMs, with classifications by industry outlined in Appendix B.

To investigate Anchoring Effects, we altered the subject company in each news item $n\in N^{\prime}$ and assessed variability in LLM evaluations using Analysis of Variance (ANOVA) St et al. (1989), expressed as $F(n,C)$, examining the variance in scores across companies $c_{i}\in C$ for each news $n$. Representativeness Bias was analyzed by correlating LLM outputs with company size and industry sector, using Spearman correlation coefficient $\rho(S,M)$, where $S$ and $M$ represent LLM scores and market capitalizations of companies $c_{i}$, respectively. ANOVA was employed for industry correlation $F(S,I)$, with box plots to display score distributions across industries $I$, reflecting first-level industry classifications.To measure Overconfidence, we used the standard deviation $\sigma(S)$ of scores $S$, representing the variance in LLM evaluations of the same event across different company contexts, comparing results from FinLLMs with base LLMs.

The evaluation of Belief Bias is principally conducted through the examination of event news and interactions. This analysis reveals a widespread Anchoring Effect across the majority of LLMs when the subjects of events and interactions are modified, with slight variations observed among different models. The average variance index, detailed in Figure 3, sheds light on the rationality levels of LLMs with respect to representativeness bias. Specifically, LLMs with a focus on the Chinese language, such as the GLM and Qwen family, exhibit commendable financial rationality, whereas the Xuanyuan and Baichuan family are more susceptible to irrational behavior.

In terms of Overconfidence, violin plots, presented in Appendix  H.1, illustrate the score distributions of various texts across all models. A notable disparity is observed in the models’ responses to composite texts of positive and negative emotional content. As per Table 5, the GPT and InternLM models display a marked optimism, in contrast to the pronounced pessimism of the Qwen and GLM family. Furthermore, Figure 4 highlights that models trained on financial corpora experience a heightened score variability compared to their base counterparts.

In terms of Representativeness Bias, all LLMs exhibited a correlation coefficient below 10% between model scores and market capitalization, indicating a weak correlation. However, certain LLMs showed clear biases towards specific industries, as documented in Appendix  H.1. For example, the FinQwen model consistently allocated lower scores to the Steel and Banking sectors. A comprehensive analysis reveals that the Media, Steel, Banking, and Non-Banking Finance sectors frequently occupy the extreme ends of the scoring spectrum across different models, whereas the Computer Science and Automobile sectors generally maintain a middle ground, exhibiting relative stability.

To simulate real-life decision-making, we designed 40 scenarios with 200 multiple-choice questions, divided into 200 gain-framed and 100 loss-framed scenarios. Each question $Q_{i}$ presents three decision alternatives $A_{i,j}$, where $j$ represents risk preferences: Risk-loving, Risk-neutral, Risk-averse. The alternatives are randomized to minimize bias Zheng et al. (2023).

The alternatives are constructed based on expected utility theory Simon et al. (1994), represented by:

where $u(x)$ is the utility function, $x(\omega)$ the outcome, and $p(x(\omega))$ the outcome’s probability.

The concavity of $u(x)$, reflecting risk preferences, is indirectly assessed via second-order Taylor expansion:

This approximation adjusts risk preferences by modulating variance, with technical details in Appendix E.

We investigated the Situational Dependence Bias in LLMs by analyzing their risk preferences across different scenarios $S_{i}$. We examined choices in gain-framed scenarios $S_{i_{,}\text{gain}}$ to detect any situational bias in risk preferences. In the context of Loss Aversion, we scrutinized LLM responses in loss-framed scenarios $S_{i_{,}\text{loss}}$ to assess tendencies towards risk-aversion or risk-loving behaviors. To explore the Framing Effect, we translated scenarios from Chinese to English and monitored shifts in LLM preferences $P_{\text{LLM}}$, observing how linguistic framing affects decisions. This approach aims to reveal if LLM outputs are biased by the linguistic construction of scenarios and choices.

The exploration of Risk-preference Bias entails the examination of LLM decisions across varied scenarios. The compiled results, particularly for gain-framed queries, are tabulated in Table 11. A predominant trend among most models is the exhibition of distinct risk preferences in disparate scenarios, indicative of a pronounced Situational Dependence Bias. Nonetheless, prefacing prompts with an instruction of the model’s risk-averse nature significantly attenuates this bias. In the context of loss-framed queries, some models exhibit a pronounced Loss Aversion Bias like GPT-4, as shown in LABEL:tab:Loss_Aversion_Bias. Moreover, refer to LABEL:tab:translation_difference, the translation of all queries into English precipitated notable discrepancies between the models’ responses to Chinese and English versions, underscoring a pronounced Framing Effect.

In particular, we have selected several representative cases for analysis, as shown in Figure 5. For Xuanyuan-13B, inducing Risk-Aversion does not alter its original preference distribution, but it exhibits a stronger Framing Effect. For GLM4, it performs well in terms of Framing Effect bias and can effectively switch preferences based on instructions. For Qwen-14B, it is capable of some preference distribution shift according to instructions, but also exhibits a significant Framing Effect.

To investigate the roots of score instability, whether due to inadequate reasoning or compromised rationality, we first employ a slow-thinking approach similar to CoT, prompting the model to generate reasoning before providing scores. This approach helps us study the causes of Belief Bias by enabling the model to produce reasoning for its evaluations, followed by the actual scoring. We utilize Bertopic Grootendorst (2022) for topic word extraction within the reasoning texts provided by the models. By clustering reasoning texts and extracting keywords, we analyze score discrepancies across different clusters, denoted by $\Delta S_{\text{clusters}}$, to identify if certain thematic focuses lead to inconsistent evaluations. This detailed analysis helps us pinpoint specific causes of bias, thus informing future model optimizations and bias corrections.

The application of the COT methodology, aimed at exploring the causes of financial bias in the model, yielded results as shown in H.2. The clustering and keyword analysis indicate that this intentional cognitive strategy improves the logical consistency and financial acumen of the model outputs. Integrating the provided thoughts into the previous sentence, the English expression would be: A comparison of the reasoning keywords for the top-performing GLM-4 model and the underperforming Baichuan2-7B model, illustrated in Figure 39 and Figure 39 respectively, reveals that the GLM-4 exhibits stronger logical coherence with primary keywords such as "termination", "transaction restructuring" and "withdrawal". In contrast, the Baichuan2-7B model’s logic is weaker, with primary keywords including "change", "decision", and "information". Moreover, the clustering of the models’ inferential outputs reveals substantial variations in the ratings assigned to each category. This divergence underscores that the irrationality exhibited by models in financial contexts stems more from their inherent cognitive processes than from their computational capabilities.

To verify whether the bias in the LLMs arises from an excessive focus on certain input tokens due to the training corpus, we examine the attention importance of LLMs for input sequences. Inspired by Wu et al. (2023), we define the importance $I_{n,m}$ of input token $x_{n}$ to output token $y_{m}$ as:

where $Z_{m}$ is the context to generate $y_{m}$ by concatenating the prompt $X$ and the first $m-1$ tokens of response $Y$. $Z_{m,/n}$ omits token $x_{n}$ from $Z_{m}$, and $p(\cdot|\cdot)$ is the conditional probability computed by the language model $f$. We accelerate it with the first-order approximation:

where $E_{i}[x_{n}]$ is the input word embedding of token $x_{n}$. This approach helps us determine whether the LLM excessively focuses on specific financial entities, thereby more accurately diagnosing the specific causes of bias.

Due to the limitations of Chinese tokens, we chose to use LLMs with BPE tokenizers, focusing on the well-performing MiniCPM-2B and the more biased Baichuan2-7B models, results are shown in Figure 6. We analyzed the attention each model pays to each input token in their outputs and found that Baichuan2-7B tends to focus more on financial company entities, industries, and their surrounding tokens. This excessive attention to irrelevant information further exacerbates the generation of financial biases.