In 2018, the researcher Pranav Rajpurkar was working on an algorithm that could find blood clots in patients’ legs from ultrasound images. It spotted them very well, but when he went looking for what the algorithm had picked up on in the images to make its predictions, he saw it had been cheating: it was looking at the metadata in the top right corner of every ultrasound.
This got him thinking about how to evaluate whether AI models are actually pointing at the right spots on medical images. He designed what he calls a “pointing game” between radiologists and AI algorithms. “If you ask a person and an algorithm to point at a spot, are they near each other?” said Rajpurkar, an assistant professor of biomedical informatics at Harvard Medical School.