What is indexing in SEO?
You have written the article. It is well-structured, thoroughly researched, published on a clean URL. Nothing happens. Weeks pass. Still nothing. The most common explanation: the page is not indexed.
Indexing is the process by which Google adds a crawled page to its search database, the Google Search Index. Until a page is in that database, it is invisible to search. It does not matter how good the content is, how many backlinks point to it, or how well the on-page elements are optimized. Not indexed means not findable.
Shwetank Ojha's audits of stalled content pipelines find the same diagnosis more often than any other: teams troubleshooting a "ranking problem" when the real issue is that the page never entered the index in the first place. You cannot rank a page Google has never stored.
Go deeper: the index is bigger and more selective than most people picture
A sense of scale helps. Google's index spans hundreds of billions of web pages, and roughly 15% of searches it handles on any given day have never been searched before, a constant stream of genuinely new queries against an index that never stops growing. Most of the web doesn't get the benefit of that scale, though: the majority of pages online receive effectively zero organic search traffic, often because they were never indexed at all, not because they ranked poorly once they were.
During indexing, Google also runs a clustering step worth understanding: it groups together pages it finds with substantially similar content, then selects one canonical version from each group to actually show in results. The other pages in that cluster still exist and may even still be indexed, but they're treated as alternate versions, not separate ranking opportunities. This is why duplicate or near-duplicate pages so often cannibalize each other instead of both ranking.
The full pipeline
- Crawling: Googlebot visits the page and reads the HTML content.
- Processing: Google analyzes the content, topic, quality, language, links, structured data, E-E-A-T signals.
- Indexing: If the page passes quality thresholds, Google stores it in its index with metadata about what it contains.
- Ranking: When a user searches, Google retrieves relevant pages from the index and ranks them by relevance, authority, and quality.
What prevents indexing
- Noindex tag: <meta name="robots" content="noindex"> explicitly instructs Google not to add the page to its index. Correct for admin pages, checkout flows, thank-you pages. Catastrophic when applied accidentally to content you want ranking.
- Canonical tag conflicts: A rel="canonical" tag pointing to a different URL signals that the canonical URL is the official version. Google may not index the non-canonical page.
- Thin or duplicate content: Google applies quality filters. Pages with very little unique content or pages that closely mirror others on your site may be excluded entirely.
- robots.txt blocking: If Googlebot cannot crawl a page, it cannot make an indexing decision.
- Domain-level quality filters: Sites with large volumes of thin, low-quality, or manipulative pages may see indexing suppressed across the entire domain.
Go deeper: "Discovered" and "Crawled" are not the same exclusion
Search Console's Coverage report uses two specific, easily confused statuses worth knowing apart. "Discovered, currently not indexed" means Google knows the URL exists, usually from a sitemap or a link, but hasn't crawled it yet, often because of crawl budget prioritization on a large site. "Crawled, currently not indexed" means Google visited the page and decided, after reading it, not to add it to the index, typically a quality or duplication judgment. The first is a discovery problem with a queue behind it. The second is a quality problem Google has already evaluated and rejected. Treating them as the same issue wastes the diagnostic value Search Console is handing you for free.
How to check index status
- Google Search Console → Coverage report: The definitive diagnostic view. Categorizes all known URLs into indexed, not indexed (with reasons), and excluded.
- URL Inspection Tool: Shows the last crawl date, Google's rendered version, and any detected indexing issues for that specific URL.
- site: operator: Searching site:oshwetank.com in Google returns an estimate of indexed pages from your domain.
Speeding up indexing: IndexNow
One real lever most indexing guides skip entirely. IndexNow is a protocol, backed jointly by Microsoft Bing and Yandex (and supported in varying capacity by other engines), that lets you actively push a URL notification the moment you publish or update a page, rather than waiting for a crawler to discover it on its own schedule. Submitting through the URL Inspection Tool covers Google specifically; IndexNow covers the rest of the index ecosystem in one ping rather than waiting on each engine's own crawl cadence. For sites publishing frequently or correcting content on a deadline, this is the difference between hours and weeks.
Indexing and AI search
AI platforms like ChatGPT Search, Perplexity, and Google's AI Overviews primarily source their answers from content that is indexed and ranked in traditional search. A page not in Google's index is almost certainly not being considered as a source by AI engines. Indexing is the baseline. AI engines then apply additional filters for author credibility, structural quality, and citation richness before selecting a source.
Frequently asked questions
How long does it take Google to index a new page?
Highly variable. A new article on a high-authority domain can appear in search results within hours. A page on a new domain might wait several weeks. Two things accelerate it: submitting the URL via Google Search Console's URL Inspection Tool, and adding internal links from already-indexed pages.
What is the difference between crawled and indexed?
Crawled means Googlebot found and read the page. Indexed means Google evaluated it as worth adding to its database. A page can be crawled but not indexed, if Google's quality filters determine the content does not meet the threshold for inclusion.
My page is indexed but not ranking, why?
Indexing is the entry requirement, not the destination. Being indexed means you are eligible to rank. Actually ranking on page one requires relevance, authority (credible external sources vouching for your domain), and quality that satisfies E-E-A-T standards at a level competitive with what is already ranking.
What's the difference between "Discovered" and "Crawled" but not indexed in Search Console?
"Discovered, currently not indexed" means Google knows the URL exists but hasn't crawled it yet, usually a queue or crawl-budget issue. "Crawled, currently not indexed" means Google read the page and chose not to index it, almost always a content quality or duplication judgment. The fix for each is different: the first needs better discovery signals, the second needs a better page.
Does submitting a sitemap guarantee indexing?
No. A sitemap tells Google a page exists; it doesn't obligate Google to index it. Google still applies the same quality and duplication filters regardless of how it learned about the URL. Sitemaps speed up discovery, not the indexing decision itself.
Is there a free indexing checker?
Yes. Google Search Console's URL Inspection Tool is the authoritative source, since it's Google reporting its own decision, not a third-party guess. The site: search operator gives a fast, rough estimate without logging in. Third-party "index checker" tools exist too, but they're inferring from a Google search result, not querying the index directly, so Search Console wins whenever the two disagree.
