Can AI Rate Photo Quality? I Tested 3 Models Against People

Have you ever taken multiple shots of the same moment and then had trouble picking the best one? Now imagine doing that across your collection of thousands of photos.

Sounds impossible. So I decided to try building a local AI workflow to see if it could help. Could AI tell me which photos were bad, okay, good, or great? And could I trust its ratings?

Short answer: the best model I tested, MUSIQ, put photos in nearly the same order people did. Its labels were the weak part: on phone photos they often ran a step or more too harsh, so a photo people called “great” could come out “good” or even “okay.”

Here’s how I tested it, what I learned, and what you can try too.

Can it recognize better-looking photos?

My constraint: everything had to run locally on my Mac. So I could only pick from models that were free, ran locally, and actually ran on my machine. I compared three. To judge them, I needed photos people had already rated. KonIQ-10k¹ contains about 10,000 everyday photos with crowd ratings. I used separate groups from its official test split: 179 photos to set up the ratings and 52 others to evaluate them. None were my personal photos.

The models produce numbers. My code turns those numbers into four labels: bad, okay, good, or great. The boundaries between labels are called cutoffs. I adjusted them separately for each model using the first group, then left them unchanged for the evaluation.

52 evaluation photos, 13 per rating. Ranking agreement of 1.0 means the same order; 0.947 does not mean 94.7% of ratings were correct. These packaged versions ran on my MacBook Pro’s graphics chip (M5 Max, 128 GB). One photo changes rating accuracy by about two percentage points.

MUSIQ,² from Google researchers, led on all three measures in this small test. It got all 13 “great” photos right, and every missed rating was one category away. The other models were LIQE³ and Q‑ReAlign Mini.⁴

There was a catch. This MUSIQ version had learned from KonIQ’s training photos, so the collection was familiar territory. I wanted to know whether it would work on the kind of photos I actually take.

Does it work on phone photos?

I tested the KonIQ-trained MUSIQ on 100 public smartphone photos from SPAQ,⁵ with 25 in each human-rating category. These photos were 4160 by 3120 pixels, and this model had not trained on SPAQ.

At full size, it matched the human rating on only 43 photos. Shrinking each photo so its longer side was 1,024 pixels raised that to 58.

The 25 photos people rated “great” show what changed:

Recorded results for the same 25 public phone photos. Only the image preparation changed; the model and rating cutoffs stayed the same.

Before resizing, it called almost every one “okay.” Afterward, it agreed with people on 14 of them. Resizing helped, but it didn’t make the labels dependable.

Across all 100 photos, ranking agreement improved from 0.78 to 0.89. Processing took about 0.23 seconds per photo either way. This test used Google’s original TensorFlow model on my Mac’s main processor, rather than the faster packaged implementation in the first table.

Why does it call good photos “okay”?

A number doesn’t come with a universal meaning. Even after converting scores to a common 0–1 scale, different models put their scores in different places. Choosing the boundaries between ratings is called calibration.

When I first split scores evenly at 0.4, 0.6, and 0.8, MUSIQ did badly. Setting the boundaries on the separate calibration group fixed most of that on KonIQ:

Those KonIQ boundaries still ran too harsh on phone photos, even resized. My next experiment is to set them using about 150 separate, human-rated phone photos, then evaluate on the original 100. I haven’t done that yet, so I can’t say how much it will help.

What do I actually trust it to do?

A quality score isn’t the same as knowing which picture matters to you. A blurry photo can still be a keeper.

Today MUSIQ rates photos in my local AI photo library alongside measures of sharpness, brightness, and contrast. When I find near-duplicate photos, it breaks ties after sharpness and faces. A score helps suggest a keeper; I still review the choice.

I also changed how I ran it. The comparison used IQA-PyTorch,⁶ whose noncommercial license didn’t fit what I wanted for this tool. I switched to Google’s original Apache 2.0 model files. On the same 52 photos, the two implementations produced nearly the same ordering, with agreement of 0.99, though the original model ran more slowly on my main processor.

For technical readers, here are more details

The phone-photo comparison

I used Google’s TensorFlow2 MUSIQ koniq-10k model, version 1, from Google’s model downloads.⁷ Each image was scored once at full size and once after resizing. The model’s output was divided by 100. The rating boundaries were 0.470, 0.677, and 0.723, fitted on the separate KonIQ calibration group and held fixed for both runs.

The resize preserved aspect ratio, rounded each output dimension to the nearest pixel, and used TensorFlow’s area resize. Pixels were clipped to 0–255, converted to unsigned 8-bit values, and encoded as JPEG at quality 95. Thus this comparison tests the resize-and-re-encode preparation together. Images already at or below 1,024 pixels on their longer side were left unchanged.

Timing included resizing and JPEG encoding when used, plus inference. It excluded file reading, model loading, and the first image’s warm-up timing. The recorded averages were 0.238 seconds at full size and 0.232 seconds after resizing. Repeat both conditions on the same images, without adjusting the cutoffs between runs.

The model comparison and calibration

The comparison used IQA-PyTorch model identifiers musiq (KonIQ-trained), liqe_mix (the original authors’ LIQE), and qrealign-mini (0.8B). Scores were normalized using each model’s declared minimum and maximum, then clipped to 0–1.

The crowd scores in my metadata were on a 0–100 scale, derived from the dataset’s human ratings. My categories were: bad below 40; okay from 40 to under 60; good from 60 to under 80; great at 80 or above. These are my categories, not words the crowd directly assigned.

For each of the three boundaries, the script tried the observed calibration scores and chose the threshold that best separated the categories below it from those above it. Boundaries were kept in ascending order. The recorded boundaries, rounded to three decimals, were:

A score equal to a boundary enters the higher category. Google’s TensorFlow MUSIQ had its own calibrated boundaries, listed in the phone-photo comparison above; I did not assume the two implementations had identical scores.

Ranking agreement is the correlation between the two ordered rank lists. The original script assigned consecutive ranks even to tied values, rather than averaging tied ranks. These are the recorded results, not a recomputation with tie-adjusted Spearman correlation. For a new evaluation, use a standard tie-aware implementation and report that choice.

The graphics-chip timing excluded model loading and a warm-up prediction, but included the package’s per-image call and image loading. It is not a controlled processor-speed comparison with the TensorFlow run. These details let you repeat the approach; reproducing the exact figures also requires the original image selections and pinned software versions.