The task
Marketplaces require the main product photo to have a white background, sharpness, normal exposure and at least 1,000 px on the short side. The seller shoots on a phone, uploads, and finds out about the problem when the listing is already up and not selling, or when moderation has rejected it. Checking every photo by hand takes too long.
The open brief from OpenCV and AWS set two conditions: OpenCV 5 must do substantial image analysis, and a noticeable part of the solution must live on AWS. Novelty was not required. Separate categories existed for agentic vision (Agentic Vision: the OpenCV result must change the agent’s next step, with a decision trace, an evaluation of successes and failures, and a human in the loop) and for COOL, the OpenCV build for Graviton processors. We entered the general track and both categories.
Why, and what the idea was
An ordinary auto-filter, “found a defect, applied a fix”, often makes things worse: brightens the photo and burns out a white product, cuts out the background and eats half the object. On our held-out set, 255 of 1,032 photos “fixed” by such an open-loop pipeline fail the re-check.
The idea: the agent does not trust its own fix. After every correction it measures the photo again and rolls back whatever did not help or introduced a new defect. For every photo one of four verdicts: the photo is fine as is; fixed and verified, the corrected version can be uploaded; needs a reshoot, with a specific reason (for example, motion blur during the shot); hand to a human. Below the result you see the trace: what the agent found, what it tried, what it kept and what it rolled back.
We deliberately stuck to one role: a quality filter with verifiable autonomy. No new features were added, the Bedrock planner was not switched on, and the value of verification was proven with numbers, not words.
How it went
| When | What happened |
|---|---|
| Day 1, morning | Brief rules read from the primary source, the idea approved: an agent for checking product photos. A plan with verifiable criteria and a cost estimate. OpenCV 5.0.0 already installed locally |
| Day 1 | The open Amazon Berkeley Objects dataset (CC BY 4.0): 358 photos, 184 product types. An evaluation set of 3,276 photos assembled (252 base x 13 defect variants), dev 1,508 / test 1,768. The agent core, on dev 74.7% -> 91.4%, pytest 8/8 |
| Day 1, ~13:30 | AWS account created. It turned out the plan was paid and credits were $0 instead of the promised $100, every expense comes off the card. A $1 budget alert set right away |
| Day 1, ~14:10 | The “free only” condition: dropped the always-on EC2 and deployed the service on Lambda arm64 within the always-free quotas. Metrics on Lambda matched the Mac to 4 decimal places |
| Day 1, ~14:20-15:05 | COOL measured against regular OpenCV on Graviton4, the agent’s decisions identical. Servers deleted, the COOL subscription cancelled. “Scene protection” added after the agent “fixed” 10 of 40 interior photos by cutting out the scene (after the fix, 1 of 40) |
| Day 1, ~17:10 | A single run on the held-out test: 89.7%. Draft video of 3:09 and Devpost copy |
| Day 1, ~17:40 | Council of 5 advisors: stick to one role, do not switch on Nova, no new features, prove the value of verification with numbers |
| Day 1, ~18:10 | Ablation (the agent versus the same pipeline without re-measuring), manual labelling of 106 real photos, README and report rewritten, video re-cut to 3:37, a repeat COOL run with a per-stage profile |
| Day 1, ~21:25 | Submitted on Devpost, both categories ticked |
| Day 6 | A second council with other participants’ data. Done: a kill switch with self-recovery and a threshold of 1,000 requests an hour, statistics with confidence intervals and a McNemar test, a limits map (40 photos x 5 defects x 7 severity levels) |
| Day 7 | A new header on Devpost: a 2-minute judge’s path, the numbers in plain words, the limits map in the gallery |
| Day 12 | A test on 24 phone photos under a protocol fixed in advance. The result was a failure, published as is. A block “how to shoot a photo that will pass” added on Devpost, with conditions taken from the thresholds in the code |
| Day 13 | A scan of the whole repository history for secrets and personal data, the repository made public, tag v1.1-judging. Lambda updated and frozen. The whole Devpost page proofread, mismatches in the numbers fixed |
| Day 15 | A line about the price of strictness (of 73 good photos 30 pass on their own, 31 go to a human, 12 to a reshoot), a response time measurement from Kyiv, MIT licence, a GitHub Release with the report and archive |
| Day 17 | Edits on Devpost closed |
| Day 21 | A “judges” council, scoring by the brief’s rubric. Nothing in the submission was changed. The judge’s path added to the YouTube video description |
Tools and method
A four-step agent loop. Perceive: OpenCV 5 measures sharpness, exposure, noise, background via a LAB mask, framing and pHash for duplicates. Decide: rules pick the permitted actions. Act: 6 tools, crop, white balance, exposure, GrabCut background replacement, NLM denoising, sharpening. Verify: re-measure and decide whether to keep the fix or roll it back. A limit of 6 steps, product color protection. Verifying a fix is 5 gates, not one rule: the policy itself, GrabCut mask agreement (IoU), whether the fix introduced a new defect, whether it fixed what it was meant to, and a dE color change budget.
The default planner runs on rules. There is an optional one on Amazon Bedrock (Nova) with a permitted-action check, covered by tests but switched off in the live loop: Bedrock costs money.
Runs on AWS Lambda arm64 (Graviton), Python 3.12, 2,048 MB, a Function URL over HTTPS. Dependencies pinned: opencv-python-headless 5.0.0.93, numpy 2.2.6. Photos are processed in memory and not stored. Money protection: a $1 a month budget alert by email, a CloudWatch alarm when the hourly call threshold is exceeded, a separate kill-switch Lambda sets concurrency to 0, and the site answers 429. The switch re-enables itself on the all-clear, but after 3 triggers in a day it stays off until manually re-enabled. No long-lived keys, build and deploy from AWS CloudShell with deploy/*.sh scripts. About 965 lines of Python in the agent core and Lambda, 8 unit tests, 29 commits.
COOL was measured separately on an EC2 c8g.large (Graviton4) from the Marketplace image “Cloud Optimized OpenCV 3.1”, next to regular pip OpenCV on the same server. The servers deleted themselves after the run.
Video: the intro shot on a phone, English subtitles, scene voice-over by Gemini TTS, language check with Whisper, assembled by a Python script.
Result
One web page on AWS Lambda, no login. Six ready samples can be checked in one click, or you can upload your own photo, including from a phone camera. The judge’s path we wrote out on Devpost and in the video description: the “color cast” sample is fixed on its own, the trace shows the white balance kept after re-measuring; the “motion blur” sample goes to a reshoot with a hint; the “lifestyle” sample (about 5 s) shows the whole loop, the agent tries to clean the background, measures, rolls back its own fix and hands the photo to a human.
The held-out set of 1,768 photos, no thresholds tuned on it, a single run. The agent: 1,584 correct decisions, 89.6% (95% interval 88.1-90.9%). The same rules and tools without re-measuring: 1,507, 85.2% (83.5-86.8%). Pairwise, the agent is right and the open-loop pipeline is wrong on 97 photos, the reverse on 20 (McNemar p = 2.6e-13). On the 371 photos where the agent rolled back its own fix, it is right 94.6% of the time versus 73.9%. By defect type: heavy blur, low resolution, darkness and a busy background 100%, clean photos accepted 97%, motion blur 85%, overexposure only 33%, the weakest spot.
On 106 real catalogue photos labelled before the run (33 with a problem, 73 good), the agent stopped 27 of 33 problem photos versus 11 of 33 for the open-loop pipeline. The price of that strictness: 31 of 73 good photos go to a human.
COOL on Graviton4, two full runs, 12 launches, identical decisions: box blur x2.87, Otsu x1.94, CLAHE x1.40, but adaptiveThreshold x0.77 and Canny x0.90. The agent as a whole only x1.03 (344.5 versus 356.1 ms per photo): about 80% of our perception is written in NumPy, where COOL does not reach.
Response time of the live site from Kyiv: lifestyle 5.42-5.47 s, the other samples 0.90-2.70 s. AWS cost about $0.13 in total, that is EC2 for roughly 50 minutes of COOL measurements. Lambda, CloudWatch and SNS live within the always-free quotas. By our estimate the service costs up to $0.05 per 1,000 photos, and about 160 thousand photos a month fit within the free quota.
What we do not know yet: the results will be announced on 10 November. Our own “judges” council (a simulation, not the real jury) gave 6.6 out of 10: documentation strongest (8.4), real-world usefulness weakest (5.7). We will add it when the results come.
Mistakes and what we took from them
