-
Optimizing Performance: How Compact Models Match or Exceed GPT's Classification Capabilities through Fine-Tuning
Authors:
Baptiste Lefort,
Eric Benhamou,
Jean-Jacques Ohana,
David Saltiel,
Beatrice Guez
Abstract:
In this paper, we demonstrate that non-generative, small-sized models such as FinBERT and FinDRoBERTa, when fine-tuned, can outperform GPT-3.5 and GPT-4 models in zero-shot learning settings in sentiment analysis for financial news. These fine-tuned models show comparable results to GPT-3.5 when it is fine-tuned on the task of determining market sentiment from daily financial news summaries source…
▽ More
In this paper, we demonstrate that non-generative, small-sized models such as FinBERT and FinDRoBERTa, when fine-tuned, can outperform GPT-3.5 and GPT-4 models in zero-shot learning settings in sentiment analysis for financial news. These fine-tuned models show comparable results to GPT-3.5 when it is fine-tuned on the task of determining market sentiment from daily financial news summaries sourced from Bloomberg. To fine-tune and compare these models, we created a novel database, which assigns a market score to each piece of news without human interpretation bias, systematically identifying the mentioned companies and analyzing whether their stocks have gone up, down, or remained neutral. Furthermore, the paper shows that the assumptions of Condorcet's Jury Theorem do not hold suggesting that fine-tuned small models are not independent of the fine-tuned GPT models, indicating behavioural similarities. Lastly, the resulted fine-tuned models are made publicly available on HuggingFace, providing a resource for further research in financial sentiment analysis and text classification.
△ Less
Submitted 22 August, 2024;
originally announced September 2024.
-
Examining Independence in Ensemble Sentiment Analysis: A Study on the Limits of Large Language Models Using the Condorcet Jury Theorem
Authors:
Baptiste Lefort,
Eric Benhamou,
Jean-Jacques Ohana,
Beatrice Guez,
David Saltiel,
Thomas Jacquot
Abstract:
This paper explores the application of the Condorcet Jury theorem to the domain of sentiment analysis, specifically examining the performance of various large language models (LLMs) compared to simpler natural language processing (NLP) models. The theorem posits that a majority vote classifier should enhance predictive accuracy, provided that individual classifiers' decisions are independent. Our…
▽ More
This paper explores the application of the Condorcet Jury theorem to the domain of sentiment analysis, specifically examining the performance of various large language models (LLMs) compared to simpler natural language processing (NLP) models. The theorem posits that a majority vote classifier should enhance predictive accuracy, provided that individual classifiers' decisions are independent. Our empirical study tests this theoretical framework by implementing a majority vote mechanism across different models, including advanced LLMs such as ChatGPT 4. Contrary to expectations, the results reveal only marginal improvements in performance when incorporating larger models, suggesting a lack of independence among them. This finding aligns with the hypothesis that despite their complexity, LLMs do not significantly outperform simpler models in reasoning tasks within sentiment analysis, showing the practical limits of model independence in the context of advanced NLP tasks.
△ Less
Submitted 26 August, 2024;
originally announced September 2024.
-
Stress index strategy enhanced with financial news sentiment analysis for the equity markets
Authors:
Baptiste Lefort,
Eric Benhamou,
Jean-Jacques Ohana,
David Saltiel,
Beatrice Guez,
Thomas Jacquot
Abstract:
This paper introduces a new risk-on risk-off strategy for the stock market, which combines a financial stress indicator with a sentiment analysis done by ChatGPT reading and interpreting Bloomberg daily market summaries. Forecasts of market stress derived from volatility and credit spreads are enhanced when combined with the financial news sentiment derived from GPT-4. As a result, the strategy sh…
▽ More
This paper introduces a new risk-on risk-off strategy for the stock market, which combines a financial stress indicator with a sentiment analysis done by ChatGPT reading and interpreting Bloomberg daily market summaries. Forecasts of market stress derived from volatility and credit spreads are enhanced when combined with the financial news sentiment derived from GPT-4. As a result, the strategy shows improved performance, evidenced by higher Sharpe ratio and reduced maximum drawdowns. The improved performance is consistent across the NASDAQ, the S&P 500 and the six major equity markets, indicating that the method generalises across equities markets.
△ Less
Submitted 12 March, 2024;
originally announced April 2024.
-
Can ChatGPT Compute Trustworthy Sentiment Scores from Bloomberg Market Wraps?
Authors:
Baptiste Lefort,
Eric Benhamou,
Jean-Jacques Ohana,
David Saltiel,
Beatrice Guez,
Damien Challet
Abstract:
We used a dataset of daily Bloomberg Financial Market Summaries from 2010 to 2023, reposted on large financial media, to determine how global news headlines may affect stock market movements using ChatGPT and a two-stage prompt approach. We document a statistically significant positive correlation between the sentiment score and future equity market returns over short to medium term, which reverts…
▽ More
We used a dataset of daily Bloomberg Financial Market Summaries from 2010 to 2023, reposted on large financial media, to determine how global news headlines may affect stock market movements using ChatGPT and a two-stage prompt approach. We document a statistically significant positive correlation between the sentiment score and future equity market returns over short to medium term, which reverts to a negative correlation over longer horizons. Validation of this correlation pattern across multiple equity markets indicates its robustness across equity regions and resilience to non-linearity, evidenced by comparison of Pearson and Spearman correlations. Finally, we provide an estimate of the optimal horizon that strikes a balance between reactivity to new information and correlation.
△ Less
Submitted 9 January, 2024;
originally announced January 2024.
-
Detecting and adapting to crisis pattern with context based Deep Reinforcement Learning
Authors:
Eric Benhamou,
David Saltiel,
Jean-Jacques Ohana,
Jamal Atif
Abstract:
Deep reinforcement learning (DRL) has reached super human levels in complex tasks like game solving (Go and autonomous driving). However, it remains an open question whether DRL can reach human level in applications to financial problems and in particular in detecting pattern crisis and consequently dis-investing. In this paper, we present an innovative DRL framework consisting in two sub-networks…
▽ More
Deep reinforcement learning (DRL) has reached super human levels in complex tasks like game solving (Go and autonomous driving). However, it remains an open question whether DRL can reach human level in applications to financial problems and in particular in detecting pattern crisis and consequently dis-investing. In this paper, we present an innovative DRL framework consisting in two sub-networks fed respectively with portfolio strategies past performances and standard deviations as well as additional contextual features. The second sub network plays an important role as it captures dependencies with common financial indicators features like risk aversion, economic surprise index and correlations between assets that allows taking into account context based information. We compare different network architectures either using layers of convolutions to reduce network's complexity or LSTM block to capture time dependency and whether previous allocations is important in the modeling. We also use adversarial training to make the final model more robust. Results on test set show this approach substantially over-performs traditional portfolio optimization methods like Markowitz and is able to detect and anticipate crisis like the current Covid one.
△ Less
Submitted 9 November, 2020; v1 submitted 7 September, 2020;
originally announced September 2020.
-
Raman-assisted crystallography reveals end-on peroxide intermediates in a nonheme iron enzyme
Authors:
Gergely Katona,
Philippe Carpentier,
Vincent Nivière,
Patricia Amara,
Virgile Adam,
Jérémy Ohana,
Nikolay Tsanov,
Dominique Bourgeois
Abstract:
Iron-peroxide intermediates are central in the reaction cycle of many iron-containing biomolecules. We trapped iron(III)-(hydro)peroxo species in crystals of superoxide reductase (SOR), a nonheme mononuclear iron enzyme that scavenges superoxide radicals. X-ray diffraction data at 1.95 angstrom resolution and Raman spectra recorded in crystallo revealed iron-(hydro)peroxo intermediates with the (h…
▽ More
Iron-peroxide intermediates are central in the reaction cycle of many iron-containing biomolecules. We trapped iron(III)-(hydro)peroxo species in crystals of superoxide reductase (SOR), a nonheme mononuclear iron enzyme that scavenges superoxide radicals. X-ray diffraction data at 1.95 angstrom resolution and Raman spectra recorded in crystallo revealed iron-(hydro)peroxo intermediates with the (hydro)peroxo group bound end-on. The dynamic SOR active site promotes the formation of transient hydrogen bond networks, which presumably assist the cleavage of the iron-oxygen bond in order to release the reaction product, hydrogen peroxide.
△ Less
Submitted 16 December, 2014;
originally announced December 2014.