Claude, ChatGPT confidently missed skin cancers in darker skin
SAN DIEGO — Two popular AI chatbots demonstrated limited accuracy in identifying skin cancers in darker skin tones, yet expressed relatively high confidence in their diagnoses, researchers reported.
For the study, presented at the American Society for Dermatologic Surgery Annual Meeting, researchers asked Claude and ChatGPT AI models to rate their diagnostic confidence on a scale of 1% to 100%. For lesions in Fitzpatrick skin types V to VI — the darkest skin colors — the models reported confidence levels of 65% to 70% when their diagnoses were correct and 60% to 65% when they were incorrect.