Investing in the Web Study Finds ChatGPT Struggles with Financial Accuracy Across Key Topics
New research has raised red flags about relying on AI for financial advice. A study by broker comparison platform Investing in the Web revealed that more than 33% of answers generated by ChatGPT to finance and investment-related questions were incorrect. The researchers asked 100 questions covering a broad range of financial topics—from saving for a child’s education to comparing average pensions with salaries, and evaluating the risks and rewards of investing in gold. The findings suggest a need for caution when using generative AI for important financial decisions.
The study also asked both evergreen questions, where the right answer will always remain relatively consistent, and current questions, where the answers are more dependent on recent information.
The responses from ChatGPT were considered ‘correct’ by the industry experts in 65% of cases, which is a reasonably high result. However, in 29% of cases, the response was incomplete or misleading, and in 6% of cases, the answer was entirely wrong.
This means that over one third of the questions posed to the AI bot were answered incorrectly or unsuitably.
These findings have sparked concerns from investment experts that a high proportion of people who use ChatGPT may ask for financial advice and unknowingly receive a poor response.
Pedro Braz, CEO of Investing in the Web, reflected on the findings of the research, offering the following conclusion:
“As with any other tool, ChatGPT serves its purpose if used in moderation. A good rule of thumb is to use it early in the research process, to help clarify terms or shed light on a general topic. Using it to ask more personalised questions is where the risk of miscommunication comes into play.
“It is always advisable to ask for and check the listed sources from ChatGPT’s answers, and to cross-reference its main points with other trusted sites, especially when dealing with financial inquiries that may have a significant impact on our lives.
“For information and data that is subject to change such as interest rates, the current performance of specific stocks, or the average pension, it is usually best to go to the very source of the information, rather than asking AI chatbots for data, as ChatGPT has well-recognised issues with up-to-date information.
“Overall, ChatGPT search is a welcome application that can save us a lot of time, but it can also misdirect us if we’re not careful while using it. Where the subject matter is as serious as personal finances, overuse of AI can have a significant negative impact.”
