Free SEO audit with 10 action points - $2K worth of value
Rasesh Koirala

Indexing

Search engine pipeline: crawl, index, rank, AI Overview

Crawling gets Google to your page. Indexing is what happens next. It is the process by which Google analyses a page's content and decides whether to add it to its database - the index - from which all search results are drawn. A page that is not in the index cannot appear in search results, regardless of how good it is.

What happens during indexing

According to Google's own Search Central documentation, indexing involves Google analysing a page's text content, title elements, alt attributes, images, videos, and other key tags. It then determines what the page is about, how useful it is, and whether it duplicates content that already exists in the index.

One step that surprises many people: Google renders the page. It processes the HTML, CSS, and JavaScript the way a browser would, to see what a visitor would actually see. This matters because content loaded via JavaScript that Google cannot render will not be indexed. Sites built as single-page applications, or pages that rely heavily on client-side rendering to display their main content, can run into serious indexing problems for this reason.

Mobile-first indexing also applies here. Google uses the mobile version of your page as the primary version it indexes. If your mobile and desktop versions differ significantly - for example, if your mobile version has less content - what gets indexed is the mobile version. This is now the standard for all websites Google indexes.

Why pages get excluded from the index

Being crawled does not guarantee being indexed. Google actively filters out pages it considers low-value. Common reasons a crawled page does not get indexed include:

A noindex directive. A noindex meta tag in the page's HTML tells Google not to index it. This is useful for pages you legitimately want to keep out of search results - admin pages, thank-you pages, duplicate content - but a misconfigured noindex on an important page is one of the most common technical SEO disasters.

Duplicate or near-duplicate content. When multiple pages contain very similar content, Google picks one to index as the canonical version and may ignore the rest. This is particularly relevant for ecommerce sites with product variants, filtered pages, or paginated content. The canonicalisation page covers how to manage this.

Thin or low-quality content. Google's documentation explicitly states that quality is a primary factor in indexing decisions. Pages with very little content, pages that do not add anything beyond what already exists on other pages, or pages that do not meet Google's helpful content standards may be crawled repeatedly but never indexed.

Soft 404s and error states. Pages that return a 200 status code but display "no results found" or similar empty states may be treated as low-value and excluded. Google looks at what a page actually contains, not just whether it technically loaded.

How long indexing takes

Research by IndexCheckr analysing data from 16 million pages found that 93.2% of pages that Google indexes are indexed within the first six months of publication. Pages on high-authority sites with strong internal linking can be indexed within hours or days. New sites, pages with no internal links, or pages on sites with known quality issues can take weeks or never be indexed at all.

Submitting a page via the URL Inspection tool in Google Search Console signals Google to prioritise it for crawling and indexing. It does not guarantee immediate inclusion but is consistently the fastest route for new or updated content.

How to check your indexing status

The URL Inspection tool in Search Console tells you whether a specific page is indexed, when it was last crawled, how Google rendered it, and any reasons it may have been excluded. It is the most reliable way to diagnose indexing problems on individual pages.

The Index Coverage report (now called the Indexing report in newer versions of Search Console) shows your site's indexing status across all pages. It breaks down pages into categories including indexed, not indexed, and excluded - with reasons for each. Checking this report regularly is one of the most valuable habits in technical SEO.

Indexed is not the same as ranking

An indexed page is eligible to appear in search results. Whether it actually appears - and where - is determined by ranking. An Ahrefs study of over one billion web pages found that 96.55% of indexed pages receive zero organic traffic from Google. Being indexed is a necessary condition for ranking, not a sufficient one.

What determines ranking is covered in the next page.

Next: Ranking: how results are ordered