Indexing

What's the difference between crawling and indexing?

Crawling is when search robots read your pages; indexing is when they analyze what they read and add it to the list used for search. A page can be crawled but not indexed, and being indexed doesn't guarantee it will appear near the top for any particular query.

At the indexing stage, Google analyzes body text, title tags, image descriptions (alt text), images, videos and more. When there are several similar pages, it treats them as one group and only shows the representative page (the canonical page) in search results.

Naver goes through a similar process. When you tell it a URL through a crawl request or a sitemap, its robots visit in order of priority, and only the crawled documents that pass its indexing criteria appear in search.

How do I check whether my page has been indexed?

The most accurate way is to enter the URL into the URL Inspection tool in Google Search Console and the URL check in Naver Search Advisor. A "site:" search is only a rough check and isn't enough to pinpoint which pages are missing.

Naver's URL check shows separately whether the page was crawled, whether it can be indexed and whether it is indexed, and it also tells you whether the title and description are read correctly. Google's URL Inspection even shows the date Google last read the page and the URL it chose as the canonical page.

The result count for a "site:yourbusiness.kr" search is a rough estimate and can change from day to day. Rather than panicking when the number drops, it's more accurate to watch the trend in indexed pages in Search Console's Page indexing report.

1. 01Go to Google Search Console and paste the full URL of the page you want to check into the search bar at the very top.
2. 02On the results screen, check the indexing status and, if the page isn't indexed, the reason given.
3. 03If it isn't indexed and there's nothing to fix, click "Request indexing" once.
4. 04In Naver Search Advisor's webmaster tools, select your site and enter the same URL into the URL check.
5. 05Check the crawl and indexing status and how the title and description were extracted, and if needed, request a web page crawl from the Request menu.

What does "Crawled - currently not indexed" mean?

It means Google has read the page but hasn't added it to its search list yet. You'll see it in Search Console's Page indexing report, along with a similar status, "Discovered - currently not indexed," which means Google knows the URL but hasn't read the page yet.

Google lists common reasons for pages not being indexed: low-quality content, a setting that blocks indexing, or a site structure that makes indexing hard. Pages with just one photo and a one-line description, or pages almost identical to other pages, often get caught here.

Seeing this status doesn't automatically mean there's a problem. Right after launch it often clears up on its own over time, so start with the pages whose content you can strengthen, and check again a few weeks later.

• Add substance: include information found only on that page, like prices, hours and how it works.
• Clean up duplicates: merge near-identical pages into one or set a canonical URL.
• Connect links: link to the page from the menu or related posts.
• Check again: a few weeks later, use URL Inspection to see whether the status has changed.

When and how often should I request indexing?

Requesting indexing once per page is enough when you've launched a new page or made major changes to important content. Google says requesting the same URL multiple times won't get it crawled faster, and there's a quota on individual URL requests.

Naver also states that crawl requests aren't processed in real time and can take from at least a day to several weeks depending on priority. There's no need to request the same URL every day, and even a successful crawl doesn't guarantee search visibility.

If you have many pages or they change often, submitting a sitemap is more efficient than requesting them one by one. If you post blog articles regularly, just set your sitemap to update automatically.

How do I keep a page out of search results?

Mark the page with noindex. You can add <meta name='robots' content='noindex'> to the page code, or send it from the server as an X-Robots-Tag response header. Google says it drops a page from search results when it sees this.

Don't also block that page in robots.txt. If search robots can't get into the page, they can't read the noindex either, so the URL may stay in search if it's linked from elsewhere.

If you've deleted a page that already appears in Naver search, first delete the original page and confirm it returns a 404 (page not found) response, then apply for web page search removal from the Request menu in Search Advisor. It has to be re-indexed, so it won't disappear immediately.

• Confirmation pages like "Booking complete" or "Payment complete"
• Design drafts and test pages still in progress
• Members-only pages (blocking these with a login comes first)

What are the common reasons a page isn't indexed?

The most common reason is a noindex tag or robots.txt block added during the build that was never removed after launch. Be sure to ask your web agency whether this item is on their launch checklist.

Next comes content that's too thin or overlaps with other pages. If the preview URL and the live URL are both open, the same content exists in two places, and the wrong one may be chosen as the canonical page.

On pages that draw their content only with JavaScript, search robots may see the content late or miss it. It's safer to make sure important text is there as text from the moment the page first loads.

• A noindex tag or a site-wide robots.txt block still in place after launch
• Duplicates caused by preview or test URLs being open to search
• Orphan pages with no links at all from the menu or other pages
• Pages with only photos and almost no text
• A server that's often slow or returns errors