How I Made My Phone Videos Searchable with Local AI

Ever tried to find one moment in years of phone videos? They’re much harder to search than photos. Photos are easy once an AI has written a short description of each one. But a video is a fast flip-book of still pictures called frames, and at 24 frames a second, one minute holds 1,440 of them. The model that describes my photos takes about 3.5 seconds a picture. Describing every frame would turn each minute of video into about 84 minutes of work.

I wanted my videos searchable anyway, without sending them to an AI company, as part of my local AI photo library.

Short answer: Don’t show the AI the whole video. Show it one picture a second with the time printed on each, ten pictures at a time, then have it join its notes into one story. Here are the specifics: what the model can and can’t see, why ten pictures is the limit, how the pieces are joined, and how I tested it.

What the model actually sees

My constraint: everything had to run locally on my Mac. So I could only pick from models that were free, ran locally, and actually ran on my machine. I use Google’s Gemma 4¹ (the 26B-A4B version), a model that can look at pictures and answer questions about them. It came out ahead in my photo tests.

One frame a second, stamped with its time

I take one frame for each second of video. Why not fewer? With a frame every 2 or 3 seconds, the model caught 9 of 10 events in a test video and missed the last one both times.

Each frame gets its time printed in the corner, like “12s”. When I gave the time as text beside the picture instead, the model shifted what it saw by a second.

Ten frames at a time

My first try sent a whole minute, 60 frames, in one request, or call. It came back as garbage.

It wasn’t out of room: thirty frames were well within what the model can read. It just couldn’t follow a sequence that long. (Another model may stop sooner or later.) So each call gets at most 10 frames.

Separate chunks beat a running summary

I tried two ways to join ten-second chunks into one account, on a three-minute test video with 10 events I knew about. In a running summary, each chunk gets the story so far and adds to it. With separate chunks, each is described on its own, and one last call joins the notes.

Three-minute test video, one frame a second, 18 chunks. Events found counts the expected events a description mentions; it doesn’t take off points for anything made up.

That scoring is why the running summary looked fine. It wasn’t. It got slower every round and believed its own story. Once it had written “a GOODBYE sign,” it kept the sign, even though 9 frames showed three red balls. Separate chunks have no story to defend.

Videos longer than three minutes get one more layer: each three-minute stretch becomes its own story, then those get joined.

Test on a video where you know the answer

How do you grade a description of a phone video? You’d have to watch the whole thing next to the text. So I had code draw test videos instead: simple scenes where I chose every event.

The slips that remained came from single chunks, like a ball rolling the wrong way. I haven’t graded real phone videos the same way. So this shows the method holds together, not how accurate it is on real footage.

One idea that looked like a time-saver was skipping frames that barely change. It helped only on drawn video. In five of six real phone clips, nearly every second changed a lot, so I didn’t build it.

At the test’s pace, about 18 seconds per minute of video, a four-minute clip would take about 75 seconds and a ten-minute one about three minutes. I haven’t timed anything longer than six minutes, though. My real videos now go through the same method. If yours sit unsearched too, draw a test video first. Then you know every answer before you trust the descriptions.

For technical readers, here are more details

All tests ran on October 13, 2026, on a MacBook Pro with an M5 Max chip and 128 GB of memory, using an 8-bit copy of Gemma 4 26B-A4B. The test videos were drawn with the Pillow image library, so every event and its time was known. Six of my own iPhone clips were used only for counting how much each frame changed, never described.

Thirty frames came to only about 8,500 tokens (the small pieces of text or picture a model reads), roughly 280 a picture even at 512 pixels, so the 10-frame limit isn’t a memory limit. Each chunk is told its seconds and asked to compare every frame with the one before, and a video’s chunk calls run 8 at a time. Every joined story is capped at 3,000 characters, checked by code; one that runs long gets one more try, told by how much.

Frames are read once a second with ffmpeg, the same reader the photo quality and face steps use. In five of the six phone clips, nearly every second changed by 40 to 175 out of 255 in the most-changed patch of a small gray copy, which is why skipping repeat frames saved almost nothing.

This is custom code for my own library, not a package you can install. The method carries over; the limits are for this model, so test yours.