AI-Generated Product Descriptions at Scale: What Actually Breaks at 3,800 SKUs

The four-week rollout that shouldn't have taken four weeks

Somebody in a meeting said "let's move the whole site to AI-generated product descriptions and see what happens." It shouldn't have been a big project. It was a catalogue of about 3,800 SKUs across one distributor's site and I had already built the description generator for a different client. Six weeks of nights and weekends later, we had a system that actually shipped, and a good portion of what I learned was about what breaks when you try to generate content at volume.

What I'm not going to pretend is that it was all success. About a third of the catalogue came out fine on the first pass. Another third needed a human to touch it. The last third is where the interesting work was, and where most people doing this at scale get stuck.

First, the volume problem nobody warns you about

At 3,800 SKUs, doing it one at a time with a chat window is impossible. You need a pipeline. I built mine with a Python script that reads the product feed, builds a prompt per SKU from a template, calls an API, and writes the response back to a staging table. Total runtime for the full catalogue: about four hours on a laptop with a free API tier that rate-limited to 60 calls a minute.

That's where the trouble began. At 60 calls a minute across four hours, you're making about 14,000 API calls. Roughly one in two hundred failed — timeouts, rate limits, the occasional content that came back truncated mid-sentence. So the pipeline needed retry logic and, once I added it, needed to be idempotent so retrying didn't produce duplicate descriptions. I ended up with a two-column table: SKU and content hash. If the hash of a successful response matched the stored one, skip. That alone cut the time on re-runs from hours to minutes.

The other thing I underestimated was cost. At $0.002 per thousand input tokens, 3,800 SKUs averaging 800 input tokens each is about $6. That sounds great until you add retries, prompt engineering iterations and a second model for the hard cases, and the bill reaches about $40. Still cheap. But if you're evaluating tooling that charges $0.05 a product, you're paying roughly a dollar a hundred products, or $38 for a catalogue this size, and that's before retries. The math changes fast when your catalogue is 40,000 SKUs.

The part that actually fails: factual accuracy

Nobody hiring a content tool cares about throughput after a week. They care that the description says 1,200 lumens when the spec sheet says 900. I hit that on maybe 4% of the first-pass outputs. The model was inventing plausible numbers when the input was ambiguous or missing.

The fix I settled on was boring — a JSON input schema with every field optional and a hard rule that the model must not generate any number it wasn't given. Then I ran a validation pass that extracted all digits from the output and diffed them against the input. Any number that appeared in the text but not in the input was flagged for review. That caught about 80% of the factual problems. The remaining ones were things like unit conversions and range expansions ("holds up to 30kg" when the input said "15kg per hook") which the digit diff missed because 30 wasn't in the text before.

A secondary validator that checks multipliers and sums is worth writing once and reusing forever. Mine is about 200 lines and has saved me from more embarrassing content than I want to admit.

What quality looks like when you measure it

I ran a blind test with the client's product manager. Fifty SKUs, ten AI-generated descriptions mixed with forty written by their existing copywriter, shuffled without labels. She rated each on fact accuracy, tone fit and whether she'd publish it as-is. Each rating on a simple good/needs-edit/reject scale.

The results: 62% of AI outputs would have been published as-is. 26% needed an edit. 12% were rejected. For the human-written set, the numbers were 71%, 24%, 5%. That's closer than I expected, honestly, but the 5% reject rate for human copy versus 12% for AI is also where the trust issue lives. One bad description published under your brand name is the kind of thing customers screenshot.

The interesting part: when I looked at which SKUs the AI missed, they were almost all the ones with sparse input data. SKUs with full spec sheets, multiple images and a good category assignment came out at 78% publish-as-is. SKUs with a name and a price, nothing else, came out at 31%. If you're going to run this at scale, your data quality is a bigger lever than your prompt.

The client's actual decision after this was to run the full process on the 2,100 SKUs with adequate data and hand-write the rest. That took about seven hours of copywriter time against a projected twenty, and their total time saved was around a hundred and thirty hours. Net, they were happy. But they weren't happy enough to run it on the sparse 1,700 SKUs, and I don't think they were wrong.

The problem the demos don't mention: language and tone

Chinese-built ecommerce copy has a distinctive flavour in English — long, adjective-heavy, and prone to "high quality" and "exquisite craftsmanship" and "intelligent design" used as filler. If your prompt says "professional English" the output is usually better than the original human translation, but it still reads as translated. That's fine for B2B catalogue pages nobody reads, and fatal for DTC product descriptions.

The fix that worked for me was a two-stage process. First, a "de-marketing" pass that strips every adjective that isn't tied to a claim in the spec sheet. Second, a "rewrite in the voice of a technical writer" pass with three real short examples from the brand's own writing. That cost twice as much in API calls and was worth it every single time. The client's PM stopped being able to tell which of hers had been rewritten, which was the empirical confirmation I wanted.

If you're buying this as a service from a supplier, ask what their evaluation is. A specific number beats a brochure, and any partner who can't give you one is either measuring the wrong thing or not measuring.