For pages not indexed, start with the exclusion reason and last crawl date in Search Console. Then inspect the HTTP response, crawler access, noindex instructions and canonical target. If there is no technical barrier, review internal discovery and content value. A successful live test or indexing request does not establish that indexing has completed.
Decide which pages should be discoverable in search
Separate products, services, categories, filters and archives. Not every URL needs to be an independent search result. Prioritise current, useful pages serving your intended customers rather than treating every exclusion in a technical report as an error.
Record creation or revision dates, the template used and the routes by which visitors can reach each page. Comparing an affected group with a working equivalent can reveal a shared difference. Prioritise by business and user impact as well as the number of URLs.
Distinguish Google's recorded version from the current page
Inspect one exact URL and record its last crawl, fetch result, crawling permission, indexing permission and canonical information. Run the live test separately when Google’s stored view predates a fix. Keep the dates with both results: indexed information and the server’s current response are different observations.
Do not treat “Discovered – currently not indexed” and “Crawled – currently not indexed” as the same fault. For the former, investigate discovery and fetching; for the latter, examine received content, similar pages and any reported Google-selected canonical. Canonical information may not be available for every state. Sample other URLs from the template before generalising across the site.
Review access and publishing instructions systematically
Open the URL directly without an administrator session, then inspect redirects and the final HTTP status. A 200 response containing an empty template or error message does not establish usable content. Check the page source and response headers for noindex rather than relying only on an SEO plugin’s settings screen.
A robots.txt block can prevent Google from reading a page’s noindex instruction. For JavaScript sites, verify that the main text exists in rendered HTML too. If every new service template retains a staging noindex, correct the shared source; adding paragraphs to individual service pages will not remove that barrier.
| Observation | Evidence | First check |
|---|---|---|
| Fetch failure | Final HTTP status and response body | Server, CDN or redirect source |
| Noindex exclusion | Meta tag and X-Robots-Tag | Unintended publishing instruction |
| Another canonical selected | Declared and Google-selected URLs | Equivalence and conflicting links |
| Discovered, not indexed | Discovery and last crawl information | Internal discovery and fetching conditions |
| Crawled, not indexed | Received HTML and similar pages | Main content and distinct page value |
Review the value of a technically available page
If a service page contains only broad promotional phrases, identify which information would help a customer decide. Is its scope genuinely different from neighbouring pages? Improve missing answers, concrete delivery details or verified information rather than adding words merely to increase length.
If a separate page is unnecessary, plan consolidation through both editorial and technical review. Do not remove an address without considering current links and user tasks. An indexing investigation can also reveal an unclear role within the website.
Follow the correction through the same examples
Record the change, owner and verification URLs. Review the same addresses and group afterwards, noting the dates represented in the new report. A submitted request is not an indexing result.
When a shared template issue is found, add it to the publication checks for future pages. The aim is to prevent recurrence as well as resolve the existing list.