How to Fix WooCommerce Crawl Issues: Faceted Nav & Budget

WooCommerce can generate hundreds of thousands of crawlable URLs from a few hundred products — variations, filters, attribute combinations, pagination. Without intervention, Google wastes crawl budget on these and misses your real product pages. This guide covers WooCommerce-specific crawl management. Pair with ecommerce crawl guide and WooCommerce SEO.

Step 1: Audit current crawl status

Search Console → Coverage report. Note 'Excluded' URLs, particularly 'Crawled — currently not indexed' and 'Discovered — not indexed' which indicate budget waste.

Step 2: Identify faceted navigation URLs

Look at Search Console for URLs with ?filter_, ?attribute_, ?orderby parameters. Each represents a potentially-crawlable but low-value page. Count them — if over 10x your real product count, you have a faceted-nav problem.

Step 3: Implement parameter handling

Choose one approach: A) AJAX-only filters that don't change URLs (best for crawl); B) Canonical tags pointing back to the base category from all faceted variants; C) noindex on faceted URLs via Yoast/Rank Math conditional rules; D) Block faceted parameters in robots.txt.

Step 4: Configure robots.txt to block low-value paths

Block: /cart/, /checkout/, /my-account/, /?add-to-cart=, /?orderby=, /*?filter_*. Allow: /product/, /shop/, /product-category/. Add via Yoast/Rank Math robots.txt editor.

Step 5: Prune sitemap aggressively

Include only canonical product URLs and category landing pages. Exclude variation child products, out-of-stock if discontinued, low-stock-low-traffic items. Yoast WooCommerce SEO includes per-product sitemap inclusion toggle.

Step 6: Configure crawl rate in Search Console

For large WooCommerce stores (5000+ products), set crawl rate to 'Higher' in Search Console legacy settings if available, or via Bing Webmaster Tools.

Step 7: Monitor crawl stats over time

Search Console → Settings → Crawl stats. Watch the trend. After interventions, you should see total crawl requests stable or down, but indexed product count up.

Frequently Asked Questions

How many faceted-nav URLs is too many?

Rule of thumb: total crawlable URLs should be no more than 5x your product count. A 1000-product store with 20,000 indexable URLs has a faceted-nav problem. Google will struggle to discover and prioritise your real product pages.

Should I noindex or canonical faceted URLs?

Canonical is safer if the faceted URL has any inbound links (preserves equity). Noindex is more decisive but Google still has to crawl the URL to read the noindex tag. Best: both — set canonical to parent, add noindex via robots meta. For Google this is consistent; for Bing/Yandex it's clear too.

Why does Search Console show thousands of 'Excluded' URLs?

Usually a combination of: faceted navigation crawl waste, internal-search URLs leaked into discovery, pagination beyond reasonable depth, and outdated URLs from old plugins. Audit and prune. The 'Excluded' count isn't a bug — it's a crawl-budget signal.

Do I need a paid SEO plugin to handle WooCommerce crawl?

Free Yoast/Rank Math handle the basics: robots.txt, basic noindex rules, sitemap exclusions. For per-product/per-category sitemap toggles and advanced faceted handling, the paid versions (Yoast WooCommerce SEO, Rank Math Pro) are worth the cost for larger stores (500+ SKUs).

Will fixing crawl issues improve organic traffic?

If you have a real crawl-budget problem (Google not discovering new products or not re-crawling existing ones often), yes. Fixed crawl issues typically improve discovery and freshness, which translates to higher impressions. Won't help if your products simply aren't ranking for queries.

🕷️ Audit WooCommerce crawl

Identify crawl traps, parameter URLs and indexation issues.

Run Crawl Audit →