Strabismus
Two eyes under a penlight. One eye drifts in, out, up and down in turn, showing the four forms of strabismus.
Normal — both reflections sit centredEsotropia — one eye turns in, toward the noseExotropia — one eye turns out, toward the earHypertropia — one eye turns upHypotropia — one eye turns down

AIMLA 2025 · Paper 1140

Which way is that eye turning?

Strabismus is a misalignment of the eyes. We trained four neural networks to recognise its four main forms, and a normal pair, from a single cropped photograph.

Walk through the pipelineSee it classify test photos

The problem

A penlight and a trained eye

Clinicians find strabismus with hands-on tests. In the Hirschberg test they shine a penlight at both eyes: when the eyes are aligned, its reflection sits in the same spot on each. When one eye turns, that eye's reflection sits off-centre, which is what the animation above shows.

Tests like this need a specialist in the room, and reading them is partly a judgement call. We wanted to know whether a neural network could read the same misalignment from a photograph, and tell its kinds apart.

  • Hypertropia · turns up

    One eye sits higher than the other, so its line of sight points upward.

  • Hypotropia · turns down

    One eye sits lower than the other, so its line of sight points downward.

  • Normal · aligned

    Both eyes point the same way, and a penlight's reflection lands in the same spot on each.

  • Esotropia · turns in

    One eye turns inward, toward the nose, while the other looks straight ahead.

  • Exotropia · turns out

    One eye turns outward, toward the ear, while the other looks straight ahead.

The pipeline

Six steps from a photo to a class

This is the flowchart from our paper. Every box opens its step: what we did there, why, and the real images and code behind it.

Along the way you can follow one photograph — an eye turning inward — from the moment it was collected to the moment it joined the training set.

The result

84%

accuracy from EfficientNet-B7 on photos it had never seen

It beat the three other networks on every metric we measured. The test set was held back from training entirely, so these are photos the models met for the first time.

See every metric and confusion matrix →
  • EfficientNet-B784.00%EfficientNet-B7Accuracy 84.00%Precision 85.00%Recall 83.00%F1 score 84.40%
  • ResNet5074.00%ResNet50Accuracy 74.00%Precision 79.00%Recall 73.00%F1 score 74.00%
  • VGG1966.67%VGG19Accuracy 66.67%Precision 67.45%Recall 66.67%F1 score 67.00%
  • AlexNet59.26%AlexNetAccuracy 59.26%Precision 58.88%Recall 59.26%F1 score 58.71%
Accuracy on the held-out test set, as published in the paper (Table I). Hover or focus a bar for all four metrics.