Development of AI for dermatoscopic image analysis

Analysis of dermatoscopic images: where accuracy is lost

Analysis of dermatoscopic images rests on the homogeneity of input data, and it is almost never present. Images come from different equipment: contact and non-contact modes, different optics, polarization, lighting.

A model trained on one stream sees an unfamiliar distribution of brightness and color on another — recognition accuracy drops before a complex lesion even appears in the frame. This is not a rare case but a typical consequence of mixing sources in a single dataset.

The second source is the image itself.

Glare from immersion medium, hair, shadows along the lens edge, and compression artifacts are often learned by the model as a feature: if glare appeared more often in images of benign lesions, the network starts relying on the glare rather than on the pigment network structure. Outside the training set, such a feature disappears, and model errors become unpredictable.

Annotation quality adds a third layer. Different specialists draw boundaries differently and interpret borderline cases differently: disagreement in labels on the same image is sometimes comparable to disagreement between classes.

If this is not measured and resolved, the model averages contradictory answers and loses confidence on everything that resembles a disputed case.

The fourth cause is class imbalance. Rare but clinically significant categories are represented by a few images, and the model shifts toward frequent ones. Formally, the metric looks decent, but in practice exactly what the system is deployed for is missed.

Add drift: equipment, settings, and operator change — and the input data moves away from what the model was trained on. We break down how this is addressed.

What changes in clinical work after deployment

Deployment changes not so much the accuracy of an individual report as the workflow of the office: images are queued by priority rather than reviewed in order of arrival.

Therefore, we measure the effect at the department level — by time to report, the share of disagreements between specialists, and the number of lesions sent for follow-up monitoring.

• Early detection of neoplasms. Suspicious structures are marked on the image, and it rises to the front of the queue: time to review is reduced from several days to one shift.
• Reporting speed. Contours are annotated, a draft description is ready — the specialist reviews and edits instead of describing from scratch, and 2–3 times less time is spent per case in the workflow.
• Consistency of reports. A common feature scale and a unified description template remove discrepancies in wording between doctors.
• Reduced miss rate. Each image is read independently by a specialist and the model; a discrepancy goes to a second read rather than remaining unnoticed.
• Predictable appointment workload. The volume and complexity of the queue are visible — scheduling is based on actual data, not on an average.

The final report remains with the doctor: the system prepares the annotation and draft but does not replace the decision. Tuning is performed on the open ISIC Archive and on your annotated cases, so validation reflects the real practice of the department, not an averaged picture.

Model deployment modes: cloud, local environment, hybrid

A dermatoscopic image is an image of a patient's skin linked to a follow-up history, so medical data protection is the main constraint here, not deployment convenience.

The choice of environment determines both the response latency at the appointment and who is responsible for the compute node and model updates.

We break the task down into three deployment options and select according to the clinic's regulations, not the other way around.

The difference between them is whether the image leaves the perimeter, whether a permanent communication channel is needed, and how new weights are delivered after fine-tuning.

In practice, a hybrid mode is most often assembled: closed cases are processed locally, while image quality checks and weight updates run in an external environment using de-identified data. This way, the clinic does not pay for idle equipment and retains control over what goes where.

Stages of solution development and deployment

Work proceeds in iterations: each stage ends with a verifiable artifact — a data map, a split scheme, an error matrix on an independent sample. You can stop at any step without losing what has already been achieved.

1. Image dataset audit. We review the archive volume, source devices, availability of histological confirmation, annotation history, duplicates, and low-quality frames. The output is a data map and a list of gaps to close before training.
2. Validation scheme design. We split by patients, not by images: otherwise frames of the same lesion would fall into both training and validation, and the metrics would be inflated. We fix the target metric and threshold before work begins.
3. Data preparation. We bring images to a unified form, remove acquisition defects, and re-annotate part of the sample blindly — this is how we assess inter-specialist agreement and the upper quality bar.
4. Model training and validation. First a baseline, then the main model. We separately review probability calibration and robustness to different devices, and analyze errors by groups of similar lesions.
5. Solution acceptance. A run on a holdout sample according to the agreed protocol: metrics with confidence intervals, review of false misses. The result is accepted together with the doctor, not based on a single number.
6. Handover to operation. Integration into the working environment, logging of responses, data drift monitoring, and a model fine-tuning procedure. After that — support with quality control on new images.

What we hand over: project artifacts

Model handover is not only weights and metrics on a test set. An environment without an integration scheme, access rules, and procedures lasts only until the first scanner update or image format change.

Therefore, we fix tangible artifacts — each can be opened, checked, and handed over to another engineer.

• Configured processing environment. The image path from upload to report recording: frame quality check, normalization, analysis, result storage. Each step leaves a log entry, and a failure does not lead to silent image loss.
• Integration scheme with the medical information system. Which fields are sent and received, where the original image is stored, how the report is linked to the patient record. Your integrator performs configuration according to this scheme — there is no hard dependency on us.
• Access rules and roles. Who sees images, who sees only reports, who exports the array for quality review. Data access is logged, and personal information is de-identified before entering the environment.
• Solution documentation. Architecture diagram, field formats and purposes, dependency list, and instructions for deployment from scratch on a clean environment.
• Staff workflow procedures. What the doctor does, what the model does, where a second-specialist review is mandatory, and where a recorded discrepancy goes.
• Quality control protocol. A set of annotated cases with a known answer, sensitivity and specificity by neoplasm classes, and the threshold below which the system marks an image as uncertain.
• Update and rollback procedure. How a new model version is released, on what data it is validated, and how to return to the previous one if quality drops. Along with the weights and environment, we hand over rights — the team continues work without us.

Case study: metrics on a stream of dermatoscopic images

Before automation, the stream of dermatoscopic images was reviewed manually: images were distributed across shifts, and during peak weeks the queue reached several days.

The main risk was not speed — two specialists gave different assessments of the same image, and some lesions requiring further examination were classified as “monitor.”

We worked with a dermatology center that had accumulated archives of images with histology-confirmed outcomes.

From them, we assembled a control sample and immediately separated it from the training one: annotation followed dermatoscopic patterns, and disputed cases were described by a panel, not a single annotator.

The trigger threshold was selected so that an increase in sensitivity would not turn into a stream of unnecessary biopsy referrals.

On the control sample, sensitivity reached 0.94 versus 0.86 for routine reading, and the share of missed lesions decreased by about three times. Specificity was held at 0.88 — otherwise the load on doctors would increase due to false alarms, and the time gain would be eroded.

In the stream, the model does not make a diagnosis but sorts images: low-confidence images go to the doctor first, and the time to review a priority case decreased from days to hours.

The team recalculates the metrics on the control sample with each model update — degradation on new devices and different optics becomes visible before rollout to all offices.

How detection differs from image classification

Classification answers the question “what is in the image overall”: the model receives an image and returns a label with the probability of belonging to the target class.

Detection answers the question “where exactly”: the output is bounding box coordinates or a mask of the region of interest, and there may be several suspicious lesions in the frame.

For dermatoscopy, the difference is fundamental — the lesion often occupies a small part of the frame, and the decision depends on whether the doctor needs visual support.

The approach is determined by the problem statement, not by implementation convenience. If the scenario is sorting a stream of images into “requires attention” and “routine monitoring,” classification of the whole frame is sufficient.

If the solution must show the specialist a specific region and its boundaries, detection is indispensable: a classifier over the whole frame will give an answer but will not explain what it relies on.

Annotation follows from the choice of approach and determines the project budget. For classification, a label per image is enough, but background and acquisition artifacts begin to influence the model: it learns from incidental features rather than the lesion.

For detection, bounding boxes or masks from a domain specialist are needed, along with resolution of disputed cases between annotators and discrepancy control — the volume of manual work and the qualification requirements for the annotator grow manifold.

The metrics are also separated. For classification, we look at sensitivity and specificity at the selected threshold and separately at the balance of misses and false alarms.

For detection, localization quality is assessed first — what share of regions were found and how accurately the boundaries matched — and only then the diagnosis quality within the found region.

These numbers must not be mixed: high classification accuracy masks localization misses, and vice versa.

In practice, a hybrid is more common: a detector isolates the region of interest, and a second-stage classifier produces a report for it.

This increases robustness to background but doubles annotation requirements, so at the start we fix the use case and target metrics — both the annotation volume and what exactly goes into acceptance depend on them.

Common technical questions about the project

We collected the questions most often heard at technical meetings: data, equipment, closed environment, and the life of the solution after launch. If a topic was not covered enough, we will go into more detail with an engineer.

Is different equipment an obstacle?

We bring images to a unified form at the input: scale, color, lighting. We calculate metrics separately by device groups — if performance drops, we fine-tune the model on the problematic group.

Operation without external traffic

Yes, the solution is deployed in the clinic's closed environment: images do not leave its network. The result arrives directly in the medical information system — the doctor does not need to open a separate window.

What happens after launch?

Once a quarter, we recalculate metrics on fresh annotations: the equipment fleet and the nature of cases change. We validate the update on a holdout sample and roll it out with the ability to roll back to the previous version.

Let's discuss the task: what data is needed to start

A ready annotated corpus of images is not needed for a substantive conversation.

It is enough to describe how the image path is currently structured — from the dermatoscope to the doctor's report: this is enough to discuss the task substantively and return with a project estimate and work plan.

The starting data can be “raw”: frames — without selection or cleaning, descriptions — in free form. Below is what actually removes uncertainty and sets the boundaries of the future solution.

• Study flow: how many examinations per day and week, who captures images — a doctor or an assistant, one device or several different ones.
• Image examples: a few dozen frames from different dermatoscopes and under different lighting, including disputed and unsuccessful ones.
• Current review workflow: which types of neoplasms the doctor distinguishes, where they more often have doubts, and whom they refer for further evaluation.
• Reference for validation: biopsy reports or the agreed opinion of several doctors — these are used to measure review quality.
• Storage requirements: where data must be stored, who has access, and whether a separate environment is needed for de-identified images.
• Integration points: the doctor's workstation or the record system — in what form the result must be returned and with what latency.
• Constraints: what must not leave the perimeter, and which de-identification rules are already in place at the clinic.

If some items are not yet defined — no problem, we will go through them on a call and fix them in the stage plan. Describe the task in your own words — we will return with an estimate and a timeline proposal.