In a new paper, Oxford researchers shed light on the potential risks posed by large language models (LLMs), such as ChatGPT and Bard, arguing that their utilization in scientific research carries inherent dangers due to the possibility of generating false responses. The researchers emphasize the need for restrictions on the use of LLMs to safeguard the integrity of scientific inquiry.
Key Points:
- Trust issues with LLMs:
- LLMs, which power artificial intelligence (AI) chatbots, are capable of producing human-like text responses.
- Oxford researchers caution that users tend to place excessive trust in these models, viewing them as reliable human-like resources.
- Brent Mittelstadt, director of research at the Oxford Internet Institute, highlights the design of LLMs as contributors to the issue, presenting them as helpful agents capable of confidently answering various questions.
- Risk of False Information:
- The concern stems from the fact that LLMs do not guarantee accurate responses.
- False outputs can result from the datasets used to train these models, which often contain content from the internet, including “false statements, opinions, jokes, and creative writing,” according to the researchers.
- LLMs can generate false information (hallucination) that appears convincing to users.
- Opaque Datasets and Biases:
- The lack of transparency regarding the datasets used in training LLMs is a significant issue.
- An investigation into Bard’s dataset revealed content from various internet forums, personal blogs, and entertainment websites, raising concerns about potential biases in the information provided.
- The researchers highlight that subtle inaccuracies or biases in LLM outputs may require specific expertise to identify.
- Concerns in Scientific References:
- One specific concern raised is the potential fabrication of references to scientific articles by LLMs.
- Mittelstadt notes that these models might invent references, and without fact-checking, users may unknowingly rely on fabricated information.
- Recommended Use as a “Zero-Shot Translator”:
- The researchers suggest redefining the role of LLMs in scientific research, recommending their use as “zero-shot translators.”
- This involves providing the model with inputs containing reliable information or data and a request to perform a specific task with that data.
- Responsible use of LLMs is crucial, especially in generating and disseminating scientific articles, to avoid potential harm.
- Nature’s Precautionary Measures:
- Nature, a leading scientific publication, has implemented measures to address concerns related to LLMs.
- LLM tools, including ChatGPT, are not accepted as credited authors on research papers, and authors are required to disclose the use of large language models in their papers.
The cautionary findings from Oxford researchers highlight the need for responsible and transparent use of LLMs in scientific research to mitigate the risks associated with potential misinformation and biases.