An indexation audit helps you separate normal search-engine processing from problems that prevent important URLs from appearing in search. This workflow shows how to compare submitted and indexed pages, inspect URL-level signals, prioritize fixes, and record results in a repeatable audit sheet. It is designed for publishers, ecommerce sites, SaaS websites, and niche sites that need a practical way to investigate pages not indexed without treating every exclusion as an error.
Overview
Indexation is the stage at which a search engine stores and considers a crawled page for its search results. A page can be technically reachable yet still be unindexed because it has a noindex directive, points to another canonical URL, redirects, appears substantially duplicated, offers little unique value, or has not been processed yet.
An indexation audit should answer four questions:
- Which URLs matter to the business or reader?
- Which of those URLs are submitted, crawled, indexed, excluded, or unknown?
- What signal is most likely preventing indexation?
- What fix should be tested first, and how will the result be verified?
Do not use the raw percentage of indexed URLs as the only success measure. A large sitemap may contain tag pages, filtered URLs, expired content, redirects, or intentionally excluded pages. A better measure is the percentage of indexable, valuable URLs that are indexed and receiving appropriate visibility.
For related technical checks, use this technical SEO checklist for sites that publish at scale. If important pages are indexed but losing visibility, a content decay audit is a separate and useful follow-up.
How to estimate
Estimate the scale and urgency of an indexation problem before changing templates or publishing more content. Start with a defined URL set, then calculate three simple measures.
Indexation rate: indexed priority URLs divided by total priority URLs, multiplied by 100.
Exclusion rate: excluded priority URLs divided by total priority URLs, multiplied by 100.
Fix coverage: priority URLs with an assigned, testable action divided by total priority URLs, multiplied by 100.
For example, if a site has 240 priority URLs and 198 are indexed, its indexation rate is 82.5%. If 27 are excluded and 15 have not yet been classified, the immediate task is not necessarily to force all 42 into the index. First classify the 27 exclusions and investigate the 15 unknowns. Some excluded URLs may be correctly excluded, while a small group of commercially important pages may justify urgent work.
Use a prioritization score to make decisions consistently:
Priority score = business value + organic demand + internal-link importance + technical confidence
Score each factor from 0 to 3. Business value reflects conversions, revenue, leads, or strategic importance. Organic demand reflects the relevance and opportunity of the page's target queries. Internal-link importance reflects whether the URL is a hub, category, or key destination. Technical confidence reflects how clearly the cause and fix are understood. Investigate high-value pages with clear technical causes first. This prevents the team from spending time on low-value URLs simply because they are easy to count.
Inputs and assumptions
Collect inputs from several sources rather than relying on one report. Interface names and report availability can change, so treat the workflow as a set of checks rather than a fixed sequence.
- URL inventory: export URLs from XML sitemaps, a crawl, the CMS, analytics, and important landing-page lists. Deduplicate URLs after normalizing protocol, host, trailing slash, and obvious tracking parameters.
- Search Console data: review indexing reports for patterns, then use URL Inspection for representative URLs and the highest-priority individual pages. Inspection results are useful for URL-level checks but should not be treated as a complete site inventory.
- Sitemap comparison: compare URLs declared in sitemaps with crawlable, canonical, and indexed URL sets. Sitemaps should generally contain the URLs you want search engines to consider, not every URL the site can generate.
- Crawl data: record status codes, blocked resources where relevant, redirect chains, canonical targets, robots directives, meta robots tags, link depth, and whether the URL is discoverable through internal links.
- Content signals: flag near-duplicates, empty templates, thin pages, outdated pages, faceted variants, and pages whose main content is unavailable to a crawler. These are investigation signals, not automatic proof of a problem.
Use this spreadsheet-ready audit template:
| Field | What to record |
|---|---|
| URL | Normalized page address |
| Page type | Article, category, product, landing page, archive, or other |
| Priority | High, medium, or low, with the score if used |
| Sitemap status | Included, absent, or uncertain |
| HTTP status | 200, redirect, client error, server error, or unknown |
| Canonical target | Self, another URL, missing, or conflicting |
| Robots signal | Blocked, noindex, indexable, or needs review |
| Inspection result | Indexed, excluded, not tested, or pending |
| Likely cause | Specific diagnosis, not just “not indexed” |
| Action and owner | Fix, responsible person, and target date |
| Verification date | Date to retest and record the outcome |
Worked examples
Example 1: An intentional noindex. A set of internal search-result URLs appears in a crawl but is not intended to rank. The pages contain a noindex directive and are absent from the XML sitemap. Mark them as correctly excluded. Do not remove the directive merely to improve the site's indexation percentage.
Example 2: A canonical conflict. A product guide returns a successful response, contains useful original content, and is linked from the site, but its canonical points to a broader category page. If the guide is the preferred search result, correct the canonical relationship and make sure internal links and sitemap entries consistently reference the guide. Recheck the canonical after deployment.
Example 3: A sitemap and internal-link mismatch. A group of new articles is listed in the sitemap but can only be reached through an archive several clicks deep. The crawl finds them inconsistently, and inspection has not confirmed processing. Improve contextual internal links, confirm that the pages return a successful response, and keep only preferred canonical URLs in the sitemap. An internal linking strategy can help map hubs, supporting articles, and important destinations.
Example 4: Duplicate or low-value variants. Filtered URLs create many versions of the same category. Instead of requesting indexing for every variant, define which combinations deserve standalone pages. Canonicalize, redirect, block crawling where appropriate, or use another controlled exclusion method according to the site's architecture. Keep the decision documented so future templates do not recreate the problem.
For each fix, record the baseline, the change made, the URLs affected, and the expected signal. A useful status sequence is identified → diagnosed → fixed → recrawled or requested → verified. A request for recrawling is not proof of indexation; verification requires checking the page again and reviewing whether search visibility or indexed status changes over time.
When to recalculate
Run a light indexation review monthly for active sites. Compare submitted versus indexed priority URLs, inspect new exclusion patterns, and sample recently published pages. Run a deeper audit after a migration, redesign, CMS change, URL restructuring, large-scale publishing project, or template update. Also revisit the audit when organic impressions fall unexpectedly or when crawl reports show a new concentration of redirects, errors, blocked URLs, or duplicate signals.
Refresh the worksheet and reporting definitions at least every six months. Search Console interfaces, documentation, and available crawl data can change, so confirm that the fields still mean what your team assumes they mean. Revisit your priority scores when business goals, product lines, content categories, or target queries change.
To finish an audit, take these actions:
- Filter the sheet to high-priority URLs with an unresolved technical cause.
- Fix one cause pattern at a time rather than changing several sitewide directives simultaneously.
- Validate status codes, canonicals, robots directives, sitemap inclusion, and internal links after deployment.
- Record a verification date and compare the result with the baseline.
- Close correctly excluded URLs as intentional, not as failures.
This process turns “pages not indexed” from a vague complaint into a managed technical SEO queue. Repeating it monthly and after major releases makes indexation changes easier to explain, prioritize, and correct.
