Login - Jit4All

Tell search engines whether a page should appear in search results.

Robots meta tags tell search engines whether to index a page and whether to follow links from that page.

After page identity, mobile setup, and canonical URL, the next question is whether search engines should show the page at all.

Most public learning pages should be indexable. But some pages should stay out of search results, such as login pages, thank-you pages, test pages, and duplicate utility pages.

The robots meta tag is placed in the head because it gives instructions to search engines before they decide what to do with the page.

Use this carefully. A wrong noindex can remove an important page from search.

<head>
  <title>Jit HEAD - Learn The HTML Head</title>
  <meta name="description" content="Learn practical HTML head tags for search, sharing, and browser setup.">

  <meta name="robots" content="index,follow">
  <meta name="googlebot" content="index,follow">
</head>

name="robots" targets search engine crawlers generally.

content="index,follow" means the page may appear in search and links may be followed.

noindex asks search engines not to show the page in search results.

nofollow asks search engines not to follow links on the page.

googlebot is a Google-specific version. Use it only when you need Google-specific behavior.

You can tell search engines which pages belong in results.

Wrong robots tags can hide pages that should be public.

Ask ChatGPT: "Should this page be indexed or noindexed based on its purpose?" Then make the final decision yourself.

Add <meta name="robots" content="index,follow">.

Explain why noindex would be appropriate or not appropriate.

Check that no important live page accidentally has noindex.

Copying noindex from a test page to a live page.

Thinking robots tags protect private data. They do not replace login security.

Use robots meta tags to explain whether search engines should index a page and follow its links.