Keeping a Large E-commerce Catalog Fully Indexed (Shopify, WooCommerce, Magento)
Why product and category pages fall out of the index faster than any other page type, and the specific fixes that keep a large, fast-changing catalogue visible.
Why catalogues are the hardest pages to keep indexed
A blog post is written once and mostly stays put. A product page isn't: prices change, variants go in and out of stock, seasonal items get added and archived every quarter, and whole collections get restructured as the catalogue grows or a site gets redesigned. Every one of those changes is an event a search engine has to notice and re-crawl to stay accurate — and on a store with thousands of SKUs, that's thousands of small changes happening every single week, the overwhelming majority of which never get flagged for a re-crawl on their own because nothing outside the store's own database knows they happened. The result, on almost every catalogue past a certain size, is the same uneven picture: some pages are indexed and accurate, some are indexed and quietly stale, and some — often the newest, most commercially important ones — never made it into the index at all.
Where the URL count actually comes from
A typical store's indexable surface isn't just product pages — it's the homepage, every category and subcategory, every individual product, and often every variant of that product (size, colour, bundle, material) as its own separate URL depending on the platform's configuration. Shopify and WooCommerce both generate variant and filtered-collection URLs by default unless a store owner has specifically configured canonical handling to prevent it, which is exactly why a store with only a few hundred distinct products can easily have several thousand technically-indexable URLs in total, the large majority of which change independently of each other on completely different schedules — a size variant going out of stock doesn't touch the parent product's own description or price at all.
The three most common causes of catalogue indexing gaps
New product pages that outrun the crawler — a product goes live and starts being actively sold and advertised before it's ever indexed, meaning paid traffic works but organic search traffic simply can't find it yet. Discontinued or out-of-stock pages left live indefinitely with no clear signal about their status, so crawl budget keeps getting spent re-checking pages that no longer matter to the business at all. And duplicate or near-duplicate URLs generated by filtering and sorting parameters (?sort=price, ?color=red, ?page=2) competing directly with the canonical product or category page for the same limited crawl attention. Each of the three has a genuinely different fix, but all three trace back to the same root cause: catalogue changes outpacing whatever the crawler was ever going to notice purely on its own initiative.
Fixing new-product lag
The direct fix is submitting new and updated product URLs the moment they go live, rather than waiting for the site's next scheduled crawl cycle to discover them organically, which on a lower-authority or less-frequently-crawled domain can genuinely take weeks. That's the specific gap IndexNow closes: instead of a new product sitting unindexed while shoppers who search for it can't find your listing, submission happens the same moment the product page is published, and the engines that support IndexNow — Bing, Yandex, Seznam.cz, and Naver — queue it for a faster crawl, typically indexed within minutes rather than however long the site's normal crawl cadence would otherwise have taken for that particular page.
Fixing discontinued and out-of-stock pages
A page for a product that's genuinely gone should either redirect to its closest current replacement with a real 301, or return an honest 404 or 410 if there's truly nothing sensible to send visitors to instead — leaving it live indefinitely with only an "out of stock" banner and no other signal just keeps consuming crawl budget on a page that no longer serves the business or the visitor. If the product is expected back in stock on a known timeline, keeping the page live and indexed is the right call; if it's permanently discontinued, letting it actually leave the index is better for both crawl efficiency and for shoppers who'd otherwise land on a dead end from a stale search result.
Fixing filter and sort-parameter duplication
Every ?sort=, ?filter=, and ?page= variant of a category page is technically a distinct URL as far as a crawler is concerned, and nothing automatically tells it that all of those variants are really the same underlying listing shown in a different order or with a different subset visible. A canonical tag on every filtered variant, pointing back at the clean, unfiltered category URL, tells search engines unambiguously which version is the one worth indexing and ranking — so crawl budget goes toward the actual page that matters instead of being thinly split across dozens of near-identical parameter combinations of what is, in substance, the same listing shown a slightly different way.
Bulk-submitting a full catalogue migration
A full platform migration, a bulk price update, or a site-wide URL structure change can touch every single URL in a catalogue at once — that's exactly the case bulk API submission was built for, rather than a one-by-one dashboard workflow that would take a dedicated person days to work through manually. A 5,000-URL Enterprise-tier credit balance comfortably covers a mid-sized catalogue's full resubmission in a single batch after a migration; a 10,000-credit Agency package covers running that same resubmission process across several client stores from one account without buying separate packages for each. Either way, the goal right after a migration is the same regardless of scale: get the new, correct URL set back in front of search engines as fast as possible, rather than waiting for slow organic re-discovery to gradually catch up on its own timeline while rankings and traffic quietly erode.
Ongoing maintenance beats one-time fixes
A catalogue isn't a project you finish once and move on from, it's something you keep current indefinitely — the same way a redirect map for discontinued products or a canonical strategy for filtered URLs only actually helps if it keeps getting applied to every new product going forward, not just retrofitted onto the ones that happen to be live today. Pairing an automatic submission step with every product create and update event turns catalogue indexing into routine, invisible maintenance instead of an occasional cleanup project someone has to schedule time for, and any submission that does get stuck refunds its credit automatically after 7 days rather than silently wasting it on a search engine that never actually queued the page.