Clinicians have voiced concerns that ChatGPT is not yet suitable for clinical applications, highlighting its inconsistent responses to identical questions posed at different times. For instance, a physician noted that the same prompt might yield a comprehensive response in the morning and a useless one in the afternoon. While these observations are valid, expecting unwavering consistency and uniformly excellent responses from Large Language Models (LLMs), including ChatGPT, is not a feasible adoption standard.
A common cause of user confusion is their anthropomorphizing of ChatGPT, where they attribute human-like cognitive abilities to the model. This phenomenon isn't exclusive to ChatGPT but extends to other advanced technologies, such as autonomous vehicles. In this instance, ChatGPT's sophisticated language comprehension and communication capabilities can misleadingly suggest it possesses human-like understanding, memory, and problem-solving capabilities that it does not.
However, ChatGPT does share a fundamental trait with humans: the capacity to learn and update its knowledge base over time. As its knowledge increases, its responses to similar or identical prompts will also change. As a result, while strategies exist to enhance response consistency, updated knowledge, short memory, and lack of human-like understanding make predictable consistency unattainable.
In summary, expecting ChatGPT to deliver identical answers to the same questions across different sessions is an impractical measure of its utility. Instead, healthcare professionals should treat each interaction with ChatGPT as a unique consultation, akin to seeking a second opinion from a peer. It's also crucial to critically assess the relevance and accuracy of ChatGPT's responses. Adopting this perspective and approach should lead to more realistic expectations of ChatGPT, particularly in professional environments like healthcare, where quality and consistency are highly prized.