Technical SEO audit: diagnose crawl and content problems

1. Separate discovery, crawling and indexing

A sitemap or internal link can help a search engine discover a URL. A successful fetch provides content to process. Indexing is a separate decision, and appearing for a useful query is another. These stages need different evidence.

When a page is missing from Google, inspect its Search Console URL status as well as the audit. An unknown URL, a crawled-but-not-indexed page and a page excluded by noindex are different problems. The Google SEO Starter Guide explains the role of crawlable content and useful links. Avoid treating every absence from results as a penalty or blaming a recent deployment without a timeline.

2. Read the crawl sample before the score

Review requested URLs, final destinations, successful HTML responses and deferred pages. Several requests can end at one page. Conversely, two URLs with different query parameters may contain genuinely different public content. Count assessed content rather than equating request count with unique pages.

Sitelemetry distinguishes HTML content from resources such as text/plain files. A text file should not fail checks intended for an HTML title or H1. Secondary account and intentionally excluded pages also need different interpretation from an explicitly requested commercial page. On a small or single-page site, a sparse internal-link graph is not by itself evidence that you should manufacture extra pages.

3. Common findings and contextual priority

These examples show repair priority in context; engine severity and business urgency are not identical. Keep the affected URL and observed value in the ticket.

A length warning is an editing cue, not a universal ranking formula. A concise, accurate title is more useful than padding text to satisfy a counter.

4. Repair discovery and canonical signals first

1. Identify the preferred public URL for each important page.
2. Make internal links point directly to that URL instead of chains of redirects.
3. Ensure the chosen URL returns the intended content and an appropriate status.
4. Align its canonical, sitemap entry and navigation links.
5. Review indexing directives against the purpose of the page.
6. Inspect necessary scripts and data requests if important content is generated in JavaScript.

Do not canonicalize every page to the home page. The Google canonical guidance concerns consolidating duplicate or closely related versions, not erasing distinct product pages. Equally, do not make private routes crawlable to solve a public rendering dependency; identify the specific resource the public page needs.

5. Improve the content and verify the rendered page

Use a specific title, a clear main heading and introductory text that states what the page offers. Write a description that helps a searcher decide whether this page answers their need. Add descriptive links from relevant pages. Keep structured data consistent with what a person can actually read.

For a hypothetical product page with content loaded by an API, inspect both the initial response and the browser-rendered result. A static audit can correctly report missing text that appears only after JavaScript. Use Search Console’s live rendering tools to establish what Google receives; the JavaScript SEO guidance describes the distinction. Prefer resilient delivery of essential public content.

6. Do not mistake a heuristic for a search rule

There is no universal word count that makes a page valuable. A short refund policy and a detailed engineering guide have different purposes. Multiple headings, missing optional schema or a small link graph do not automatically mean a page is unrankable.

Robots restrictions are also contextual: a login page is not expected to behave like a public guide. Blocking crawling does not provide access control, and it is not a reliable substitute for a page-level indexing directive. Read the dedicated robots and indexing guide before changing broad rules. Preserve the public content you want indexed and keep authenticated data protected by actual authentication.

7. Close the loop with comparable evidence

• Retest the same URLs after cache and deployment changes settle.
• Confirm the final response, canonical and indexing directives.
• Check the important text in initial and rendered HTML.
• Verify sitemap and internal links use the intended URL.
• Record unresolved pages that were outside the sample.
• Track Search Console impressions and clicks over comparable complete periods.

A successful live test means Google can access the current page; it does not guarantee inclusion or a position. Search traffic also does not measure every direct or MCP visit. Use technical checks to validate the repair and search data to observe the later outcome. Start with the page that has clear demand or commercial importance, then examine shared templates using the full audit workflow.