Evals
Can AI set the focal point of a photo?
Every crop of a photo should keep the part that matters. On new images an editor can click that spot. On older content nobody ever did, so I tested whether a model can do it instead.
One photo, many crops
A site rarely shows a photo the way it was uploaded. The same image becomes a square on a card, a wide band at the top of an article and a tall picture on a phone. Each of those is a crop, and a crop has to decide what to cut away.
The focal point is the spot that every crop must keep: the face, the eyes, the thing the photo is about. In Drupal, the Focal Point module lets an editor click that spot when they add an image, and the image styles crop around it. Without a focal point, the crop keeps the centre and hopes for the best.
The problem with older content
Setting a focal point takes an editor a second, on the day the image is uploaded. The trouble is everything that was uploaded before. A site that has been running for a few years has thousands of images, and none of them have a focal point. When you add a new layout with square cards or tall phone images, every one of those old images gets cropped around its centre. Heads get cut off, the person the story is about ends up outside the frame, and the card shows a wall.
Migrations make it worse. Content moved over from another system often has no focal point at all, or has one stored in a way that is hard to extract: in the old system's own crop data, in a custom field, or as a crop that was baked into the image file itself.
Fixing that by hand means opening every image, finding the spot and saving it again. Nobody budgets for that, so it doesn't happen. That makes it a job worth handing to a model: look at the photo, say where the focal point is, and do it for the whole media library in one go.
How I tested it
I picked 19 photos from Pexels: people, groups, a dog, a car, a camera dial, a stadium and a couple of photos where the subject is tiny. All the photos in this post come from Pexels. I marked the focal point on each one myself. That is the answer every method is measured against.
Each method got one of three prompts that explain what a focal point is: a short one (18 tokens), a medium one (228) and a long one (1,529). None of them mentions what is in these photos. Every combination ran three times, so you can see how much the answers move between runs.
Short (18 tokens)
Medium (228 tokens)
Long (1,529 tokens)
Each method adds its own short instruction on how to answer (pixels, a grid cell, a number on a box). Tokens counted with OpenAI's o200k_base tokenizer.
| Method | How it answers |
|---|---|
| Centre (baseline) | Always the middle of the photo. What you get today without a focal point. |
| Ask for x,y | The model sees the photo and its size and answers x and y in pixels. |
| Rulers | Rulers from 0 to 100 are drawn on all four sides; the model reads the spot off them. |
| Grid, 2 or 3 steps | The model picks a cell in a numbered 4x4 grid, then sees that area zoomed in from the full-size photo and points in it (the 3-step version grids it once more). |
| Cursor loop | The model answers, sees a marker drawn at its answer and moves it, up to three times, until it is at least 80% sure. |
| Jev box | TypeSafe's Jev decision model draws a box around the focal point; the answer is its centre. |
| Describe, Jev boxes | The language model describes the focal point in words and Jev boxes that description. |
| Set-of-Mark | Jev boxes eight general candidates, they are numbered on the photo and the model picks one. |
| Qwen point and box | Qwen2.5-VL's own pointing and box output. |
| Object detector, word rule | YOLO11x, Grounding DINO or Florence-2 find objects; take the best person, face or animal, else the best object. |
| Object detector, LLM picks | The detector's boxes are numbered on the photo and a model picks one. |
The models: GPT-5.5 with reasoning and GPT-5.4 mini without, both from OpenAI; Claude Haiku 5.5 from Anthropic, only asked for x and y; Jev from TypeSafe; and two small models I ran on my own graphics card, Gemma 4 E2B and Qwen2.5-VL 3B, through Ollama. The object detectors ran on the same card, one model at a time.
The score is the distance between a method's answer and my point, in percent of the photo's diagonal. 5% is close enough that you would hardly notice; at 20% the crop is probably about something else. I also cut four crops around every answer (square, 16:9, 9:16 and a close square thumbnail) and counted how often my point was still inside.
The multi-step methods, step by step
One real run each, on photo 1 with GPT-5.5 and the short prompt, picked because the methods disagree. The images are exactly what the model was sent. Rulers and the grid went for the man walking away; the cursor loop caught its own mistake.
See where they put it
Here are all 19 photos with my point (the white ring) and the answers. Pick the methods, models and prompt on the left. Small dots are the three runs; the square is the middle of the three, which is what you would use if you ran a method three times and wanted one answer. Switch on crop frames to see what a site would cut, or open the Crops view to compare one method's crops with mine.
Where each method put the focal point
Answers from every run, as measured. All photos from Pexels. Distances are in % of the photo's diagonal; the middle of three runs is the median of their x and of their y.
The results
In % of the photo's diagonal. The centre baseline is 14.4%.
| Method | Mean distance |
|---|---|
| Cursor loop (GPT-5.5) | 5.8% |
| Ask for x,y (GPT-5.5) | 6.2% |
| Rulers (GPT-5.5) | 7.5% |
| Grid, 2 steps (GPT-5.5) | 8.4% |
| Grid, 3 steps (GPT-5.5) | 8.6% |
| Jev box (Jev) | 8.7% |
| Ask for x,y (Claude Haiku 5.5) | 8.9% |
| Rulers (GPT-5.4 mini) | 9.7% |
| Cursor loop (GPT-5.4 mini) | 9.8% |
| Grid, 2 steps (GPT-5.4 mini) | 9.9% |
| Ask for x,y (GPT-5.4 mini) | 9.9% |
| Grid, 3 steps (GPT-5.4 mini) | 10% |
| Describe, Jev boxes (GPT-5.4 mini) | 10.3% |
| Qwen point (Qwen2.5-VL 3B) | 10.6% |
| Grounding DINO boxes, LLM picks (GPT-5.4 mini) | 10.6% |
| Set-of-Mark (GPT-5.4 mini) | 11.6% |
| Florence-2 boxes, LLM picks (GPT-5.4 mini) | 12.1% |
| Describe, Jev boxes (Gemma 4 E2B) | 13% |
| YOLO11x boxes, LLM picks (GPT-5.4 mini) | 13.4% |
| Grounding DINO, word rule (No LLM) | 13.5% |
| Qwen box (Qwen2.5-VL 3B) | 13.6% |
| Rulers (Qwen2.5-VL 3B) | 13.7% |
| Grounding DINO boxes, LLM picks (Gemma 4 E2B) | 13.7% |
| Set-of-Mark (Gemma 4 E2B) | 14.3% |
| Centre (baseline) (No LLM) | 14.4% |
| Rulers (Gemma 4 E2B) | 14.4% |
| YOLO11x, word rule (No LLM) | 16.2% |
| Florence-2, word rule (No LLM) | 16.8% |
| Florence-2 boxes, LLM picks (Gemma 4 E2B) | 16.9% |
| YOLO11x boxes, LLM picks (Gemma 4 E2B) | 17.2% |
| Grid, 2 steps (Gemma 4 E2B) | 24.5% |
| Grid, 3 steps (Gemma 4 E2B) | 25.3% |
| Ask for x,y (Gemma 4 E2B) | 25.8% |
| Cursor loop (Gemma 4 E2B) | 26% |
| Ask for x,y (Qwen2.5-VL 3B) | 27.1% |
| Method | Model | Mean distance | Median | Within 5% | Crops kept | $ per 1,000 photos | Seconds per photo |
|---|---|---|---|---|---|---|---|
| Cursor loop | GPT-5.5 | 5.8% (best) | 4.0% | 61% | 96% | $53.51 | 24.4 |
| Ask for x,y | GPT-5.5 | 6.2% | 4.1% | 60% | 96% | $15.81 | 6.8 |
| Rulers | GPT-5.5 | 7.5% | 3.9% | 61% | 93% | $23.31 | 9.5 |
| Grid, 2 steps | GPT-5.5 | 8.4% | 4.6% | 61% | 91% | $35.64 | 13.7 |
| Grid, 3 steps | GPT-5.5 | 8.6% | 4.5% | 59% | 92% | $58.07 | 22.2 |
| Jev box | Jev | 8.7% | 5.0% | 52% | 89% | $0.06 | 0.5 |
| Ask for x,y | Claude Haiku 5.5 | 8.9% | 3.8% | 63% | 89% | $0.19 | 1.3 |
| Rulers | GPT-5.4 mini | 9.7% | 5.2% | 48% | 87% | $0.87 | 1.2 |
| Cursor loop | GPT-5.4 mini | 9.8% | 7.1% | 35% | 88% | $1.85 | 2.8 |
| Grid, 2 steps | GPT-5.4 mini | 9.9% | 5.5% | 46% | 89% | $1.86 | 2.9 |
| Ask for x,y | GPT-5.4 mini | 9.9% | 6.4% | 36% | 88% | $0.75 | 1.3 |
| Grid, 3 steps | GPT-5.4 mini | 10.0% | 5.8% | 44% | 89% | $2.97 | 4.0 |
| Describe, Jev boxes | GPT-5.4 mini | 10.3% | 5.9% | 43% | 87% | $0.88 | 1.7 |
| Qwen point | Qwen2.5-VL 3B | 10.6% | 4.8% | 51% | 86% | $0.00 | 0.3 |
| Grounding DINO boxes, LLM picks | GPT-5.4 mini | 10.6% | 5.5% | 42% | 85% | $0.84 | 1.3 |
| Set-of-Mark | GPT-5.4 mini | 11.6% | 8.4% | 37% | 83% | $0.92 | 1.5 |
| Florence-2 boxes, LLM picks | GPT-5.4 mini | 12.1% | 7.9% | 33% | 83% | $0.82 | 1.7 |
| Describe, Jev boxes | Gemma 4 E2B | 13.0% | 7.6% | 37% | 81% | $0.03 | 0.8 |
| YOLO11x boxes, LLM picks | GPT-5.4 mini | 13.4% | 13.8% | 18% | 81% | $0.74 | 1.1 |
| Grounding DINO, word rule | No LLM | 13.5% | 11.6% | 26% | 78% | $0.00 | 0.2 |
| Qwen box | Qwen2.5-VL 3B | 13.6% | 11.5% | 17% | 79% | $0.00 | 0.3 |
| Rulers | Qwen2.5-VL 3B | 13.7% | 12.5% | 16% | 78% | $0.00 | 0.3 |
| Grounding DINO boxes, LLM picks | Gemma 4 E2B | 13.7% | 11.6% | 28% | 78% | $0.00 | 0.5 |
| Set-of-Mark | Gemma 4 E2B | 14.3% | 11.4% | 22% | 76% | $0.04 | 0.8 |
| Centre (baseline) | No LLM | 14.4% | 12.9% | 5% | 79% | $0.00 | 0.0 |
| Rulers | Gemma 4 E2B | 14.4% | 12.7% | 4% | 79% | $0.00 | 0.2 |
| YOLO11x, word rule | No LLM | 16.2% | 11.7% | 16% | 78% | $0.00 | 0.0 |
| Florence-2, word rule | No LLM | 16.8% | 13.9% | 11% | 79% | $0.00 | 0.5 |
| Florence-2 boxes, LLM picks | Gemma 4 E2B | 16.9% | 17.3% | 19% | 76% | $0.00 | 0.8 |
| YOLO11x boxes, LLM picks | Gemma 4 E2B | 17.2% | 17.2% | 12% | 74% | $0.00 | 0.3 |
| Grid, 2 steps | Gemma 4 E2B | 24.5% | 24.6% | 2% | 65% | $0.00 | 0.8 |
| Grid, 3 steps | Gemma 4 E2B | 25.3% | 25.7% | 4% | 65% | $0.00 | 1.2 |
| Ask for x,y | Gemma 4 E2B | 25.8% | 28.2% | 6% | 68% | $0.00 | 0.2 |
| Cursor loop | Gemma 4 E2B | 26.0% | 27.6% | 5% | 68% | $0.00 | 0.6 |
| Ask for x,y | Qwen2.5-VL 3B | 27.1% | 24.7% | 4% | 65% | $0.00 | 0.3 |
GPT-5.5 came closest. Simply asked for x and y, it landed 6.2% from my point on average, with a median of 4.1%, and 96% of the crops kept my point, for about $16 per 1,000 photos and almost seven seconds per photo. Letting it check and move its own answer (the cursor loop) got it to 5.8%, but that costs more than three times as much and takes 24 seconds per photo. Rulers and grids didn't beat the plain question.
Claude Haiku 5.5 was the cheap surprise. Asked for x and y, it had the best median of all, 3.8%, and the most answers within 5% of my point, for 19 cents per 1,000 photos and just over a second per photo. When it missed, though, it missed by a lot, so its average is 8.9%. GPT-5.4 mini, at four times the price and just as fast, was less accurate on every measure.
Jev was the other surprise. A box around the focal point, take its centre: 8.7% on average, 89% of the crops kept, for six cents per 1,000 photos and half a second per photo. That is about 250 times cheaper than GPT-5.5 for an answer that is a little further off.
The small local models split. Qwen2.5-VL 3B, asked to point in its own format, got a median of 4.8% and was very steady between runs, but when it missed, it missed by a lot. Gemma 4 E2B, asked for x and y, did worse than always picking the centre.
The object detectors were the most interesting failure. Grounding DINO nearly always found a box around the right thing: if you could pick the best of its top ten boxes, you would be 3.5% from my point. But neither a simple rule nor Gemma could tell which box mattered, so they ended up at 13.5% to 17%, about as good as the centre. GPT-5.4 mini picked better: 10.6% with Grounding DINO's boxes. That is still behind asking the same model for x and y, partly because a person's box has its middle on the chest, not the eyes.
The long prompt helped the small models the most. GPT-5.5 didn't need it: asked for x and y, it did best with the short prompt (5.9% against 6.4% with the long one).
Models evolve
We came up with the rulers, the grid and the cursor loop because they gave much better results with older models like GPT-4.1, and we weren't the only ones: the Set-of-Mark paper showed in 2023 that numbered marks on an image made GPT-4V far better at pointing at things, and developers on OpenAI's forum reported that a numbered grid helped GPT-4o and GPT-4.1 give usable coordinates. Like a lot of things with AI, the magic workarounds aren't needed anymore: GPT-5.5 does best when you simply ask it for x and y. The same goes for the prompt: with GPT-4.1 we had to spell out what a focal point is in a long prompt, while GPT-5.5 did best with the 18-token one. The cursor loop is still marginally better (5.8% against 6.2%), but at more than three times the cost and 24 seconds per photo, it's probably not worth it.
Jev and GPT-5.5, side by side
Averages hide how different these two feel in use. Here both crop the same photos at the speed and price I measured, on their way to 1,000 photos. Speed it up to see where the bill ends.
- Jev 1,000 photos in 9 min of model time, for $0.10.
- GPT-5.5 1,000 photos in 1 h 29 min of model time, for $11.25.
Each at its measured average time and cost per photo, with the prompt that worked best for it: GPT-5.5 asked for x,y with the short prompt, Jev's box with the long one. The 19 photos repeat with the next run's answers. The blue square is the answer, the white ring my point, the clear window the crop.
What does this mean for AI Image Crop?
For AI Image Crop, Jev support and image classification support go without saying. We can probably turn off the loop as well, if you use a frontier model. The module already has a bulk action for existing content, and that is where Jev shines: a whole media library for a few cents, at half a second per image. An object detection model like Grounding DINO would let a site do the same on its own server, without sending its images anywhere. And the loop didn't help the smaller models either (GPT-5.4 mini went from 9.9% to 9.8%), so one call with a good model is the way to go.
Conclusion
- If you want exactness, a frontier model that you simply ask for x and y does the trick: GPT-5.5 landed 6.2% from my point on average and kept it in 96% of the crops.
- Jev's box is very good and very cheap: 8.7% for six cents per 1,000 photos, at half a second each. In most cases, that is good enough.
- Qwen2.5-VL 3B is supposed to be built for exactly this kind of pointing, so take its result with a grain of salt. We might have done something wrong.
- Grounding DINO runs on a server's CPU (25 to 40 seconds per photo on my laptop), so if you want something that runs on your own machine and at least picks a real subject, this is the way to go. On its own it is only a little better than the centre (13.5% against 14.4%), but with a small model choosing among its boxes, it got to 10.6%.
- The prompt size mattered most for the small models. The long prompt (1,529 tokens) took Gemma 4 E2B from 20.8% to 16.8%, Qwen2.5-VL from 17.1% to 13.5% and Jev from 8.9% to 7.4%. GPT-5.5, GPT-5.4 mini and Claude Haiku hardly changed, and GPT-5.5 asked for x and y did best with the 18-token prompt, so with a frontier model a short prompt is enough and costs half as much.
Next: manual cropping
A focal point tells a crop what to keep, but sometimes an editor wants to choose the crop itself: exactly this square, exactly this band. We will do something similar for manual cropping later, and see whether a model can pick the crops an editor would.
Chapters
16 min read
Other ways to consume this
Listen to this article
13:36Podcast: Can AI set the focal point of a photo?
Podcast episode
5:15Workflows of AI Podcast
Listen in your podcast app
Transcript
- AI disclosure
- This podcast episode is created and spoken by AI based on articles written on workflows-of-ai.com.
- Kwame Kwilson
- Six cents. That's what it cost Jev to set the focal point on a thousand photos. GPT-5.5 did the same job for about sixteen dollars. Today we're talking about Marcus's test: can AI decide where a photo should be cropped?
- Jane Olson
- And it's a real problem. Every site that's been around for a few years has thousands of images with no focal point. Add a square card layout and suddenly every old photo gets cropped around its centre.
- Erin Milleu
- Which is how you end up with a lovely card showing someone's shirt and no head. And migrations make it worse, as the post says. The focal point is either missing, or buried in the old system's crop data where nobody can get it out.
- Kwame Kwilson
- So how bad is the centre, really? Marcus marked the focal point on nineteen Pexels photos himself, and measured every answer as a distance in percent of the photo's diagonal. Just cropping around the middle lands fourteen point four percent away.
- Jane Olson
- And on a tall phone crop, that's often the whole subject gone.
- Erin Milleu
- Quite. And to be fair, nobody is going to open ten thousand images and click on each one.
- Jane Olson
- Erin would. She'd read the fifteen hundred token prompt first, though. Twice. For pleasure.
- Erin Milleu
- I did read it. It's rather good, actually.
- Kwame Kwilson
- Let's talk about that prompt and the tricks. There were rulers drawn around the photo, a numbered grid that zooms in, and a cursor loop where the model checks its own crosshair.
- Erin Milleu
- Those came from older models like GPT-4.1, which were genuinely poor at saying where something is. The Set-of-Mark paper showed the same thing with GPT-4V back in 2023. Numbers on the image helped enormously.
- Jane Olson
- And now they mostly don't. With GPT-5.5, just asking for x and y beat the rulers and the grid.
- Erin Milleu
- The cursor loop is the exception, but only just. Five point eight percent against six point two. For more than three times the cost and twenty-four seconds a photo.
- Jane Olson
- Which nobody's going to pay for a whole media library. There's a lovely example in the post, though, where the loop catches itself.
- Erin Milleu
- Photo one. It put the crosshair on a man walking away, said it was nine percent confident, and moved it to the two small figures Marcus had marked.
- Kwame Kwilson
- So, the practical question. Jev or a frontier model?
- Jane Olson
- Jev, for most sites. A box around the focal point, take the centre. Eight point seven percent on average, half a second per photo, and it costs next to nothing. For backfilling old content, that's the answer.
- Erin Milleu
- I'd push back on that. GPT-5.5 kept Marcus's point in ninety-six percent of the crops. Jev kept it in eighty-nine. That's seven more bad crops in every hundred, and on a news site those are faces.
- Jane Olson
- Seven in a hundred, against no focal point at all. I'll take that trade.
- Erin Milleu
- Perhaps. I'd still want the better model on anything with people in it.
- Kwame Kwilson
- What about running it yourself? Marcus also tested object detectors.
- Jane Olson
- That's the bit I like. Grounding DINO runs on your own server, even on a CPU, so your images never leave the building. And the right box was nearly always there. If you'd picked the best of its top ten boxes, you'd be three point five percent from Marcus's point.
- Erin Milleu
- If. That's the whole problem. With the simple rule it landed at thirteen point five percent. The centre is fourteen point four. It's barely better than doing nothing.
- Jane Olson
- With GPT-5.4 mini choosing the box it got to ten point six.
- Erin Milleu
- Which is worse than just asking GPT-5.4 mini for x and y. The middle of a person's box is their chest, not their eyes.
- Jane Olson
- Fine, but it at least picks a real subject. That's not nothing.
- Kwame Kwilson
- Last one. Did the prompt size matter?
- Erin Milleu
- For the small models, a lot. The long prompt took Gemma from about twenty-one percent to under seventeen.
- Kwame Kwilson
- And Jev went from eight point nine to seven point four with the long prompt.
- Jane Olson
- But GPT-5.5 did best with the eighteen token one. So with a frontier model, keep it short and pay half as much.
- Erin Milleu
- And take the Qwen result with a grain of salt. It's meant to be built for pointing, and Marcus says himself they might have done something wrong.
- Jane Olson
- For AI Image Crop it's pretty clear, then. Add Jev, add object detection, and switch the loop off when you're using a frontier model.
- Kwame Kwilson
- For me the number that sticks is that crosshair on photo one. Nine percent confident, then two hundred and thirty-four pixels to the left, right onto the people that mattered. Read the full post, with all nineteen photos and the side-by-side race, on workflows-of-ai.com.
Narrated slides: Can AI set the focal point of a photo?
The post in three sentences
Sites with years of content have thousands of images without a focal point, and nobody will go back and set them by hand. I asked models to set them on 19 Pexels photos and measured how far they landed from my own points. GPT-5.5 came closest, Jev was nearly as good at a fraction of the cost and time, and most small local models and object detectors barely beat a crop around the centre.