ChatGPT in Medical School and Residency - Joseph Varon and Paul E. Marik

As we navigate the rapidly evolving landscape of medical education, the integration of artificial intelligence (AI) tools like ChatGPT has become increasingly prevalent. While we strongly advocate for leveraging technology to enhance medical practice, we are compelled to express our concerns regarding medical students' and residents' over-reliance on ChatGPT. This manuscript explores the risks associated with using ChatGPT in medical education, particularly the growing trend of students uploading images to the AI model without first developing basic interpretation skills.

Introduction to ChatGPT in Medical Education 

ChatGPT, a cutting-edge natural language processing model developed by OpenAI, has been recognized for its potential to revolutionize medical education by offering personalized learning experiences and enhancing clinical reasoning skills. However, its adoption raises significant ethical and educational concerns. The model's ability to generate text and answer questions based on vast datasets can lead to dependency on AI for answers, potentially undermining the development of critical thinking . and clinical skills among medical learners. 

Studies have highlighted AI's potential to enhance medical education and emphasize the need for careful integration to avoid diminishing human expertise. (1) 

The Risk of Automation Bias 

One of the primary dangers of relying on ChatGPT in medical education is the development of automation bias. This phenomenon occurs when users overtrust AI outputs, leading to a diminished ability to evaluate information critically. In the clinical setting, such bias can result in misdiagnosis or inappropriate treatment if AI recommendations are not appropriately scrutinized. For instance, if a student relies solely on ChatGPT to interpret radiological images without understanding the underlying principles of radiology, they may fail to recognize errors or inaccuracies in the AI's output. This concern is further exacerbated by the potential for AI to produce false or misleading results, or "hallucinations," which can vary across different patient populations. The Brookings Institution has noted that these risks necessitate a cautious approach to AI integration in healthcare. (2)

Lack of Basic Radiological Skills 

A concerning trend among medical students is the practice of uploading images to ChatGPT to obtain diagnoses without first learning how to read X-rays themselves. This approach hinders the development of essential radiological skills and fosters a dangerous reliance on AI for critical decision-making. Radiology is a complex field that requires a deep understanding of anatomy, pathology, and the nuances of image interpretation. While AI can assist in identifying patterns and abnormalities, it cannot replace the human judgment and expertise that are crucial in clinical practice. Research has shown that AI tools can diagnose medical issues with high accuracy, but they should be used with human expertise. (3)

Ethical and Legal Concerns 

The use of ChatGPT in medical education also raises ethical and legal concerns. The model's training data may contain biases, which can be perpetuated in its outputs, potentially leading to discriminatory practices in healthcare. Furthermore, using AIgenerated content in academic and clinical settings poses challenges related to accountability, copyright, and the integrity of medical research. Ethical challenges include algorithmic discrimination, privacy concerns, and the allocation of medical responsibility. These issues have been highlighted in discussions on the moral aspects of AI implementation in medical education. (4)

Impact on Clinical Judgment and Patient Care 

The over-reliance on ChatGPT can erode the clinical judgment of future physicians, as they may not develop the critical thinking skills necessary for nuanced decision-making in complex clinical scenarios. Critical reasoning based on a patient's history and physical examination is the foundation of clinical medicine. Clinical educators should engage students in structured diagnostic reasoning to develop problem-solving abilities. The use of AI in this process will likely short-circuit this development, compromising the students' critical thinking skills. Patient care is not just about diagnosing conditions but also about understanding the human aspect of medicine, including empathy and communication skills. While AI can assist in diagnosis and treatment planning, it cannot replace the empathetic and personalized care patients expect from their healthcare providers. The role of AI in healthcare is multifaceted, offering opportunities and challenges that must be carefully managed. (5) 

Moreover, the "lazy doctor" effect—where physicians rely exclusively on AI for diagnosis and treatment options—can lead to a progressive loss of practical skills and intellectual creativity in solving medical problems. This trend is concerning, as it may result in a generation of less adept doctors who struggle to manage complex cases without AI support.

Balancing Technology and Traditional Skills 

To mitigate these risks, it is essential to strike a balance between leveraging AI tools and maintaining traditional clinical skills. Medical education should focus on integrating AI as a complementary tool rather than replacing human expertise. This includes ensuring that students learn basic radiological skills, such as interpreting X-rays, while also using AI for enhanced analysis. Balancing technology with traditional skills is crucial for maintaining high-quality patient care. (6)

Susceptibility to Bias and Reinforcement of the Status Quo 

AI is vulnerable to bias, which can arise at various stages of its development and deployment. These biases often reflect and amplify human prejudices or systemic inequalities embedded in the data and processes used to create AI systems. Bias frequently originates from training data—if datasets are not diverse or representative, AI models may produce discriminatory results. Additionally, human annotators may introduce subjective biases during data labeling, influenced by cultural or personal perspectives. AI bias can perpetuate misinformation that disproportionately affects marginalized groups, including racial minorities. Moreover, AI has the potential to reinforce the medical status quo and downplay alternative healthcare interventions, such as naturopathy and homeopathy, if not carefully managed. AI systems are only as good as the data they are trained on. If the data predominantly reflect conventional medical practices, AI may reinforce these norms, potentially overlooking alternative approaches. Much of AI research in healthcare focuses on conventional medicine, which might overshadow the development of AI tools for alternative health practices. Furthermore, the pharmaceutical industry has a significant influence on the development and application of medical AI, particularly in areas such as drug discovery, clinical trials, and personalized medicine.

Recommendations for Safe Integration 

1. Curriculum Integration: AI education should be incorporated into the medical school curriculum to teach students about the benefits and limitations of AI tools like ChatGPT. 

2. Critical Thinking Development: Educational programs should emphasize the development of critical thinking and clinical judgment skills to prevent overreliance on AI. 

3. Ethical Considerations: Ethical training should be provided to address potential biases in AI outputs and ensure transparency in AI use. 

4. Regulatory Frameworks: Establishing regulatory frameworks to govern the use of AI in medical education and practice is crucial to safe- guarding patient care and privacy. 

5. Independent Research: Users should independently verify medical literature and the citations provided by AI to ensure accuracy and minimize bias. 

Conclusion 

While AI has the potential to revolutionize healthcare by improving diagnostic accuracy and efficiency, its integration into medical education must be approached with caution. The dangers of relying solely on ChatGPT for answers without cultivating fundamental clinical skills pose significant risks to the quality of future healthcare. As we continue to progress, we must adapt technology to complement rather than replace human expertise.

References

1. Nguyen T. ChatGPT in medical education: a precursor for automation bias? JMIR Med Educ. 2024;10(1):e50174. doi:10.2196/50174 

2. Brookings Institution. Risks and remedies for artificial intelligence in health care. Brookings Institution Press. 2022. 

3. Montana G, Patel R, Davis T. AI trained on Xrays can diagnose medical issues as accurately as doctors. J Med Imaging. 2023;10(2):1-4. 

4. Zheng Y, Chen L, Wang Z et al. Biomedical ethical aspects towards the implementation of artificial intelligence in medical education. J Med Ethics. 2023;49(3):1-10. doi:10.1007/ s40670-023-01815-x 

5. Kang K, Lee J, Kim H et al. The role of AI in healthcare: challenges and opportunities. J Med Syst. 2023;47(3):532-540. 

6. Patel R, Davis T, Lee S et al. Balancing technology and traditional skills in medical education. Med Educ. 2023;57(1):1-10.