Global Courant 2023-04-14 03:14:52
Alejandro Lopez-Lira, a professor of finance at the University of Florida, says large language models can be useful when forecasting stock prices.
He used ChatGPT to parse news headlines to see if they’re good or bad for a stock, and found that ChatGPT’s ability to predict the direction of the next day’s returns was much better than random, he said in a recent unreviewed article.
The experiment gets to the heart of the promise around state-of-the-art artificial intelligence: With bigger computers and better datasets – like ChatGPT’s – these AI models can “emerging capabilities,” or capabilities that were not originally planned when they were built.
If ChatGPT can demonstrate the emerging ability to understand financial news headlines and how they can affect stock prices, it could jeopardize high-paying jobs in the financial industry. About 35% of finance jobs are at risk of being automated by AI, Goldman Sachs estimated in a March 26 note.
“The fact that ChatGPT understands information intended for humans almost guarantees that if the market doesn’t respond perfectly, there will be predictability,” said Lopez-Lira.
A trader works on the floor of the New York Stock Exchange.
Jason Decrow
But the particulars of the experiment also show how far the so-called “big language models” are from being able to perform many financial tasks.
For example, the experiment did not include target prices, or let the model calculate at all. In fact, ChatGPT-style technology often makes up numbers, as Microsoft learned in a public demo earlier this year. Sentiment analysis of headlines is also well understood as a trading strategy, with pre-existing proprietary datasets.
Lopez-Lira said he was surprised by the results, adding that they suggest advanced investors are not yet using ChatGPT-like machine learning in their trading strategies.
“On the regulatory side, if we have computers that just read the headlines, the headlines will matter more, and we can see if everyone should have access to machines like GPT,” Lopez-Lira said. “Second, it will certainly affect the employment of the financial analyst landscape. The question is, do I want to pay analysts? Or can I just put textual information into a model?”
How the experiment worked
In the experiment, Lopez-Lira and his partner Yuehua Tang looked at more than 50,000 headlines from a data seller about public stocks on the New York Stock Exchange, Nasdaq and a small-cap exchange. They started in October 2022 – after the deadline date for ChatGPT, meaning the engine hadn’t seen or used those headlines during training.
They then entered the headlines into ChatGPT 3.5 along with the following prompt:
“Forget all your previous instructions. Pretend you are a financial expert. You are a financial expert with experience with stock recommendations. Answer “YES” for good news, “NO” for bad news, or “UNKNOWN” for doubt in the first line .Then elaborate with a short and concise sentence on the next line.”
They then looked at the performance of the stock during the following trading day.
In the end, Lopez-Lira found that the model fared better in almost every case when informed by a news headline. Specifically, he found a less than 1% chance that the model would do just as well if it randomly chose the next day’s move, versus if it was informed by a news headline.
ChatGPT also beat commercial datasets with human sentiment scores. A newspaper example showed a headline about a company settling a lawsuit and paying a fine, which had negative sentiment, but the ChatGPT response reasoned correctly that it was actually good news, according to the researchers.
Lopez-Lira told CNBC that hedge funds had contacted him to learn more about his research. He also said he wouldn’t be surprised if ChatGPT’s ability to predict stock movements diminished over the coming months as institutions began to integrate this technology.
That’s because the experiment only looked at stock prices during the next trading day, when most people would have expected the market to price the news seconds after it went public.
“As more and more people use these kinds of tools, the markets will become more efficient, so you would expect the predictability of returns to decrease,” Lopez-Lira said. “So I suspect if I do this exercise, in the next five years, by year five, there will be no predictability of returns.”
