Researchers conducted an online experiment with 371 participants who viewed a COVID-19 screening session between a hotline agent - chatbot or human - and a user with mild or severe symptoms.
They studied whether chatbots were seen as being persuasive, providing satisfying information that likely would be followed. Their results showed a slight negative bias against chatbots' ability, perhaps due to recent press reports. When the perceived ability is the same, however, participants reported that they viewed chatbots more positively than human agents, which is good news for health care organizations struggling to meet user demand for screening services.
"The primary factor driving user response to screening hotlines - human or chatbot - is perceptions of the agent's ability," said Alan Dennis, the John T. Chambers Chair of Internet Systems at Kelley and corresponding author of the paper, "User reactions to COVID-19 screening chatbots from reputable providers." "When ability is the same, users view chatbots no differently or more positively than human agents."
Other authors on the paper, forthcoming in the Journal of the American Medical Informatics Association, are Antino Kim, assistant professor of operations and decision technologies at Kelley; and Sezgin Ayabakan, assistant professor of management information systems, and doctoral candidate Mohammad Rahimi, both at Temple University's Fox School of Business.
Even before the pandemic, chatbots were identified as a technology that could speed up how people interact with researchers and find medical information online.
"Chatbots are scalable, so they can meet an unexpected surge in demand when there is a shortage of qualified human agents," Dennis, Kim and their co-authors wrote, adding that chatbots "can provide round-the-clock service at a low operational cost.
"This positive response may be because users feel more comfortable disclosing information to a chatbot, especially socially undesirable information, because a chatbot makes no judgment," researchers wrote. "The CDC, the World Health Organization, UNICEF and other health organizations caution that the COVID-19 outbreak has provoked social stigma and discriminatory behaviors against people of certain ethnic backgrounds, as well as those perceived to have been in contact with the virus. This is truly an unfortunate situation, and perhaps chatbots can assist those who are hesitant to seek help because of the stigma."
The primary factor driving perceptions of ability was the user's trust in the provider of the screening hotline.
"Proactively informing users of the chatbot's ability is important," the authors wrote. "Users need to understand that chatbots use the same up-to-date knowledge base and follow the same set of screening protocols as human agents... Because trust in the provider strongly influences perceptions of ability, building on the organization's reputation may also prove useful."
Alan R Dennis, Antino Kim, Mohammad Rahimi, Sezgin Ayabakan.
User reactions to COVID-19 screening chatbots from reputable providers.
Journal of the American Medical Informatics Association, 2020. doi: 10.1093/jamia/ocaa167