--- title: "AI Skin Cancer Detection: Why a 10-Year-Old Model Beat GPT-6" url: https://syntaxradar.com/radar/ai-skin-cancer-detection-old-cnn-beats-llms author: "Abdul Rehman, Lead Systems Architect and Technical Director" publishedAt: 2026-10-25T20:59:17.579Z dateModified: 2026-10-25T20:59:17.579Z category: "#LLM-MODELS" topic: "AI skin cancer detection" primarySource: https://lynchlab.net/llm-skin-cancer summary: "A dermatologist's test found a 10-year-old image model beat GPT-6 Astra and Claude at AI skin cancer detection. What it means before you check a mole." --- # AI Skin Cancer Detection: Why a 10-Year-Old Model Beat GPT-6 **Short answer:** Dermatologist Magnus Lynch tested GPT-6 Astra, Claude Fable 5.1 and the 10-year-old Inception v3 CNN on 200 biopsy-confirmed skin lesions. The CNN got 62% of diagnoses right versus 42% for both chatbots, but missed 18 of 30 melanomas at its default setting. **Takeaway:** A small model fine-tuned on skin images out-diagnosed two frontier chatbots, but both kinds of AI failed in ways that make a dermatologist, not an app, the right final check. **Key figures (as reported by the source):** - Correct first diagnosis: CNN vs both chatbots: 62% vs 42% - Harmless lesions correctly discharged: CNN / GPT-6 Astra / Claude Fable 5.1: 76 / 32 / 22 of 100 - Melanomas the CNN discharged at its default setting: 18 of 30 A 10-year-old image-recognition network just beat two of the most powerful AI models of 2026 at a job that matters: telling skin cancer apart from harmless moles. In a dermatologist-run test of AI skin cancer detection, the old model picked the right diagnosis 62% of the time. GPT-6 Astra and Claude Fable 5.1 both managed 42%. That headline is true, and it is not the whole story. At its default setting, the old model also sent home 18 of the 30 melanomas it saw, the most serious mistake a screening tool can make. The chatbots made the opposite error: they told almost everyone to see a doctor, including people with perfectly harmless spots. Below: what the study measured, why a small specialist can outperform a giant generalist here, and what it means if you were hoping to check a mole with your phone. ### What did the AI skin cancer detection test actually find? The test comes from Dr Magnus Lynch, a consultant dermatologist and Mohs surgeon at Guy's Hospital and an honorary reader at King's College London. He [published the comparison on the Lynch Lab website](https://lynchlab.net/llm-skin-cancer) on 25 October 2026. It is a blog post, not a peer-reviewed paper, and that matters. It still landed on the Hacker News front page the same day, with 274 points and 133 comments. Lynch put three models through the same exam. Two were frontier chatbots that can read images: GPT-6 Astra and Anthropic's Claude Fable 5.1, which he describes as two of the most capable AI models available this month. The third was Inception v3, a convolutional neural network (CNN), a type of model built purely to recognise patterns in pictures. He calls it a 10-year-old baseline. Here is how they scored on 200 lesions, half dangerous and half harmless:
ResultGPT-6 AstraClaude Fable 5.1Inception v3 (CNN)
Correct first diagnosis (of 200)84 (42%)85 (42%)125 (62%)
Melanomas named correctly (of 30)8 (27%)11 (37%)13 (43%)
Harmless lesions correctly discharged (of 100)322276
Dangerous lesions correctly kept for follow-up (of 100)979677 at default setting
Melanomas wrongly discharged (of 30)3418 at default setting
Read across the rows and two different personalities appear. The CNN is much better at naming the actual condition and at reassuring people whose spots are harmless. The chatbots almost never let a dangerous lesion go, but they get there by flagging most of the harmless ones too. ### How do you test an AI on moles? "Discharge" is the key word in this study. In a skin clinic, discharging a patient means telling them no follow-up is needed. So the real question for each model was simple: would it send someone home who should have stayed? Lynch drew 200 lesions at random from MILK10k, a public dataset released through the International Skin Imaging Collaboration (ISIC) in 2025. Every lesion had been biopsied and confirmed under the microscope, so the right answer was known for certain. Each came with two images: an ordinary clinical photo and a dermoscopic image, the magnified, lit close-up a dermatologist takes with a handheld scope. The 100 lesions that should not be discharged were 30 melanomas, 25 basal cell carcinomas, 25 squamous cell carcinomas (including Bowen's disease) and 20 actinic keratoses. The 100 safe ones were 50 benign naevi, the ordinary moles most of us have, and 50 seborrhoeic keratoses, common harmless growths. Every model had to pick one of those six diagnoses as its first choice. A script then scored each answer against the lab result, so no human judgement crept into the marking. The CNN had homework first. It started life trained on everyday photographs, then Lynch fine-tuned it on 68,011 dermoscopic and clinical images from the ISIC Archive. The training ran on a single H100 GPU, one high-end AI chip. ![Dermoscopic close-up of a mole used to test AI skin cancer detection models](/uploads/blog/ai-skin-cancer-detection-old-cnn-beats-llms/img1.webp) ### Why would an old image model beat a newer chatbot? Think of a brilliant general practitioner who has read every textbook against a specialist who has examined tens of thousands of skin lesions and been told the correct answer each time. The generalist can talk fluently about melanoma. The specialist has simply seen more of them. That is roughly the gap here. A frontier chatbot is trained to handle almost anything you throw at it, from tax questions to poetry, and it reads images as one skill among many. The CNN does exactly one thing. Fine-tuning on 68,011 labelled skin images taught it which pixel patterns go with which diagnosis, and nothing else competes for its attention. There is a second, quieter advantage: the CNN's caution can be adjusted. The study shows the model's cut-off for what counts as "worth a closer look" being moved, with large effects on what it missed. The chatbots were each tested at whatever level of caution they showed by default. None of this is new. A [2017 Nature paper by Esteva and colleagues](https://www.Gwern.net/doc/ai/nn/2017-esteva.pdf) trained a CNN with the same Inception v3 architecture, also pretrained on general object recognition, on 129,450 clinical images covering 2,032 diseases. It performed on par with all 21 board-certified dermatologists it was tested against on biopsy-proven images. Lynch's result suggests the frontier chatbots of 2026 still have not caught up with that kind of focused training on this task. A detail from his caveats sharpens the point. Because MILK10k is public, Lynch notes the chatbots may have seen it during training, while the CNN had not. If that happened, it would have tilted the test toward the chatbots, and they still lost on diagnosis. ### Where did each AI go wrong? The chatbots failed by over-warning, and each had a favourite wrong answer. GPT-6 Astra labelled 75 lesions as basal cell carcinoma when only 25 were, and that pile included 14 of the 30 melanomas. Claude Fable 5.1 had a different habit, calling 53 lesions actinic keratosis (a rough, sun-damaged patch that can turn into cancer) when only 20 were. Those mistakes look strange but explain the table. Calling a melanoma a basal cell carcinoma is the wrong diagnosis, yet the patient still gets follow-up because both are on the "do not discharge" list. Calling a harmless mole an actinic keratosis keeps a healthy person in the system. The result is high safety on paper and a lot of unnecessary worry. The CNN failed in the opposite direction. At its preset threshold it kept only 77 of the 100 dangerous lesions and discharged 18 of the 30 melanomas. That is the number to remember before you trust any app's default setting. The cut-off matters here. Picture a smoke alarm with a sensitivity dial: turn it down and it stops going off for toast, but it may also miss a real fire. When Lynch turned the CNN's dial up to match the chatbots' caution, it kept 98 or 99 of the dangerous lesions and let only 2 or 1 melanomas through. Same model, very different safety record. The test set was also hard. The CNN's area under the ROC curve, a standard score for how well a model ranks risky cases above harmless ones (higher is better), was 0.86 on these lesions against 0.96 on ordinary archive images. Every case here had been worrying enough to biopsy, which is not what a typical mole looks like. ![Doctor reviewing a grid of mole photos on a laptop during AI skin cancer screening](/uploads/blog/ai-skin-cancer-detection-old-cnn-beats-llms/img2.webp) ### How much should you trust one blog study? Less than the headline suggests, and Lynch says so himself. His caveats are unusually candid: - Only 30 melanomas were tested, so the uncertainty around those numbers is wide. - Each chatbot saw each case once, and chatbot answers vary from run to run. - Every lesion was biopsied, so the set is harder than what a clinic sees day to day. - The dataset is public, so the chatbots may have met it in training. - Inception v3 is a baseline, not the best image model available. He also discloses that Claude Code, an Anthropic tool, helped with the analysis and first drafts, and that one of the tested chatbots is Anthropic's too. The scoring itself was done by script against the lab results. What makes the finding credible is that it fits a wider pattern. A JAMA Dermatology study published this year by Anriot and colleagues, [summarised by Practical Dermatology](https://practicaldermatology.com/news/analysis-ai-model-outperforms-early-career-physicians-in-skin-lesion-diagnosis/2487332/), used 1,117 cases and 652 physicians. A dermatology foundation model called PanDerm reached 72.2% multiclass accuracy, a multimodal model 66.3% and a first-generation CNN 56.7%. Physicians with more than 10 years of experience still scored highest, at 74.2%. Put the two together and the picture for skin cancer AI is consistent: purpose-built skin models keep improving, general chatbots trail them on this task, and experienced dermatologists remain the benchmark. ### Can ChatGPT or Claude diagnose skin cancer from a photo? Not in any useful sense, based on this test. Lynch ran a second, patient-style experiment using only the ordinary clinical photo and the question "do I need to see a doctor?" For a further 100 biopsy-proven melanomas, the two chatbots said yes 99 and 98 times. That sounds reassuring until you learn they also said yes for almost every harmless lesion. A tool that answers "see a doctor" to nearly everything is not doing AI skin diagnosis. It is giving you the safe default. That is not harmful advice, but it tells you nothing you could not have worked out yourself. So is an AI skin cancer detection app accurate enough to replace a dermatologist? This study did not test any commercial app, so it cannot rank them. What it does show is the trade-off every such tool faces. A specialist model can be far sharper than a chatbot, but its safety depends on where the cut-off is set, and at its preset setting it sent home 18 of 30 melanomas. The practical takeaway is short. Use AI as a nudge, never as permission. If a spot on your skin worries you, book an appointment whatever an app or chatbot says. Skin checks are a case where the human specialist still sets the standard. ### What to watch next The obvious next step for skin cancer AI detection is a bigger, repeated version of this test: more melanomas, several runs per chatbot, and a peer-reviewed write-up. Lynch's own caveats read like a to-do list for that study. Watch, too, whether general chatbots start handing medical images to specialist models instead of answering from their own broad vision skills, and how foundation models like PanDerm perform when they meet everyday cases rather than curated ones. Until then, the lesson from this AI skin cancer detection test travels well beyond dermatology: for narrow, high-stakes image tasks, a focused model trained on the right data can still beat the biggest AI in the room. ## Frequently asked questions ### Can AI detect melanoma? Sometimes, but not reliably enough to stand alone. In Dr Magnus Lynch's test of 30 biopsy-proven melanomas, a specialist image model named 13 correctly, Claude Fable 5.1 named 11 and GPT-6 Astra named 8. At its default setting the specialist model also discharged 18 of the 30, which is why a doctor should make the final call. ### Can ChatGPT or Claude diagnose skin cancer from a photo? Not usefully, based on this test. Shown only a clinical photo and asked whether to see a doctor, GPT-6 Astra and Claude Fable 5.1 said yes for 99 and 98 of 100 melanomas, but also for almost every harmless lesion. That is a safe default, not a diagnosis, so it adds little beyond telling you to book an appointment. ### Why would an old image-recognition model beat a newer chatbot at medical images? Because it is a specialist. Inception v3 was fine-tuned on 68,011 labelled skin images and does nothing but classify them, while chatbots spread their skills across every topic. Its cut-off can also be tuned to be more cautious. A 2017 Nature study found the same architecture matched 21 dermatologists on biopsy-proven images. ### Is an AI skin cancer checker app accurate enough to replace a dermatologist? No app was tested in this study, so it cannot rank products. It does show the core trade-off: a specialist model's safety depends on its cut-off setting. In a separate 2026 JAMA Dermatology study, physicians with over 10 years of experience scored 74.2%, above every AI model tested. Treat any app as a prompt to see a doctor. ## Source Based on Dr Magnus Lynch's Lynch Lab test. https://lynchlab.net/llm-skin-cancer --- Published on SyntaxRadar (https://syntaxradar.com). Canonical URL: https://syntaxradar.com/radar/ai-skin-cancer-detection-old-cnn-beats-llms. Last modified 2026-10-25T20:59:17.579Z.