When a patient asked Laura Ferris, an assistant professor of dermatology at the University of Pittsburgh School of Medicine, her opinion on smartphone apps that purport to distinguish between benign and malignant skin lesions, Ferris realized she'd never used one and decided to run images of melanomas through a few of the apps herself.
"When I saw the first few results come back of them being missed, I really started to get concerned," Ferris says in a school video. So she decided to investigate further, and reports this week in JAMA Dermatology that three out of four smartphone apps her team tested incorrectly described at least 30 percent of melanomas as "unconcerning."
The news comes the same week a Pew Research Center study found that 35 percent of Americans seek out medical diagnoses (not just remedies or information) online.
For the melanoma study, researchers uploaded 188 images of skin lesions to each of the four applications (they decided "not to make a direct statement about a particular app" and are thus not naming the apps they studied). The apps analyzed the images in different ways, including automated algorithms and images reviewed by an anonymous board-certified dermatologist.
It turns out that of the four melanoma apps studied, the most accurate one was not only the most expensive (costing $5 per image) but also was the only one to rely on the dermatologist to review the images. (In this case, just one of the 53 melanomas uploaded was incorrectly identified as benign.)
These results could be particularly problematic for those without the resources to go to an actual dermatologist or afford the most expensive app, especially if they take an "unconcerning" result as confirmation that they need not spend any more money investigating the area in question.
"If [users] see a concerning lesion but the smartphone app incorrectly judges it to be benign, they may not follow up with a physician," says Ferris, whose study was funded in part by the National Institutes of Health. "Technologies that decrease the mortality rate by improving self- and early-detection of melanomas would be a welcome addition to dermatology, but we have to make sure patients aren't being harmed by tools that deliver inaccurate results."
The researchers further warn that even though the free or low-cost apps are specified as educational only, they are not subject to regulatory oversight and should be used with caution.