2025 Southeastern Residency Conference

Background: 
Effective communication skills are essential for pharmacy students, as they directly impact patient care and counseling quality. Traditional methods such as role-playing, simulated patients, and virtual patient interactions are commonly used in pharmacy education to enhance these skills. However, these approaches have notable limitations, including increased faculty workload, the need for trained standardized patients, and the high cost and potential technical challenges of virtual simulation software. The integration of artificial intelligence (AI) presents an innovative solution to address these challenges. The purpose of this study was to assess first-year pharmacy student confidence gains in communicating with patients using an AI chatbot in a communications course. 


Methods: 
First-year pharmacy students at the University of Georgia College of Pharmacy participated in this study. ChatGPT prompts were developed to simulate patient interactions, guiding the chatbot to engage with students as if they were patients picking up a new prescription. Embedded rubrics assessed students' soft skills, accuracy of medication counseling, and use of PRIME questions. The chatbot provided real-time feedback on strengths and areas for improvement. 
Electronic surveys measured students' confidence before and after a graded standardized patient activity. Confidence was assessed across seven domains: patient introduction, patient interviewing, use of probing questions, active listening, nonverbal communication, patient-centered education, and provision of accurate medication information. Students initially registered for ChatGPT and completed a pre-survey capturing demographics and baseline counseling confidence. They then engaged in a chatbot-guided counseling session before a week-long independent practice period. A post-survey followed the first graded standardized patient session to evaluate changes in confidence. Students had the option to use the chatbot again before a second graded encounter, after which they completed the same post-survey. Descriptive statistics analyzed demographic data, and a Wilcoxon Signed Rank Test assessed changes in confidence based on pre- and post-survey results following chatbot utilization. 


Results
A total of 145 first-year pharmacy students enrolled in the Pharmacy Intercommunications course at the University of Georgia. Of these, 117 completed all components of pre- and both post-surveys. Among pre-survey respondents (n=126), 70.3% reported pharmacy work experience, with 60.7% (n=88) having experience in a community setting. Additionally, 57% (n=72) had at least one year of work experience. Prior to the study, 62.8% had used ChatGPT, while 35.9% reported feeling comfortable using it. A Wilcoxon Sign-Rank Test indicated that median post-survey ranks for confidence across all 7 confidence domains were statistically significantly higher than the pre-survey ranks. The largest changes were seen in confidence introducing oneself to the patient and stating the purpose of the interview (Z=6.82, p<0.001), interviewing and assessing patient knowledge by using PRIME questions (Z= 6.954, p<0.001), using probing questions to clarify information (Z=6.941, p<0.001), tailoring counseling to meet patient specific needs (Z= 5.457, p<0.001), and providing complete and accurate information during the counseling session (Z= 5.19, p<0.001). A Wilcoxon Sign-Rank Test comparing confidence between the first post-survey and the second post-survey indicated that median post-survey ranks for confidence were statistically significant across all domains with the exception of confidence in the ability to demonstrate active listening (Z=0.885, p=0.376) and confidence in communicating interest and confidence through body language (Z=1.825, p=0.068).  


Conclusions
Incorporating an AI chatbot as a counseling tool in the classroom resulted in a statistically significant increase in first-year pharmacy students’ confidence when counseling a graded simulated patient. These findings highlight the potential of AI as an effective educational tool to enhance student learning.