How accurate are online symptom checkers?

From paper by Adam Ceney, Stephanie Tolond, Andrzej Glowinski, Ben Marks, Simon Swift, and Tom Palser.

In 2013, the World Health Organisation reported that the world was short of 7.2 million healthcare professionals of all disciplines. This is predicted to grow to 12.9 million by 2035. Primary care is particularly affected, with severe shortages being reported across the world. For example, the United States alone is predicted to have a shortfall of between 7,300 and 43,100 primary care physicians by 2030. In the United Kingdom, a joint report by the Health Foundation, King’s Fund and Nuffield Trust in March 2019 suggested that there were 6% fewer general practitioners (GP) practising in September 2018 than in September 2015. This report further predicted there would be a shortfall of 11,500 general practitioners by 2028/29 which could become a threat to the very sustainability of primary care.

This shortage has several consequences, including reduced access to health care, rising healthcare inequity, longer waiting times and increased use of emergency services. As a result, there has been increasing interest in harnessing novel technologies, principally Artificial Intelligence (AI) techniques such as machine learning and deep learning, to reduce clinician workload, manage finite resources and help patients access the most appropriate care pathway more rapidly.

One manifestation of these new technologies is Computerised Diagnostic Decision Support (CDDS) programmes. Two broad types exist. The first are professional decision aids, most of which help stratify a patient’s risk of a particular condition or outcome and thereby help a medical professional decide what management plan they should institute. The second are “symptom checkers” which are targeted primarily at patients. Over the last decade, web and app-based symptom checkers and symptom assessment tools have proliferated – both in terms of geographical reach and capability, with multiple tools now in existence which claim to use AI algorithms and chatbot functionality. Their first aim is to provide information to a patient that will help them to determine the likely cause of their symptoms. The second is to provide triage, based on the symptoms presented, and to advise an individual whether they should seek medical advice, and if so at what level i.e., hospital, general practice, self-care at home, and the level of urgency that is required.

The currently available symptom checkers require the user to respond to a series of questions and provide a diagnosis and advice on the appropriate next step. There is variability between the symptom checkers in the way individuals are asked to interact and there are different approaches to displaying endpoints for diagnostic and triage advice.

One potential benefit of such systems is the ability to identify life-threatening issues, for example, stroke or heart failure, and advise the patient to seek emergency care. They also have the potential to identify less severe conditions that could be self-managed at home, thus providing reassurance to an individual by supplying them with accurate information and reducing the impact on the demand for healthcare services by decreasing the likelihood individuals will access care when they can self-manage.

However, these benefits can also have the opposite impact, if a symptom checking system calculates a diagnosis that is not appropriate and suggests that the individual does not seek care when they should. This could lead to false reassurance and potential harm. Individuals who could self-manage but are advised to seek care could suffer harm from unnecessary anxiety as well as creating extra demand and costs on already scarce healthcare resources, particularly in primary care.