seonetlifyrobots.txtastrojsx-robots-tag

Why is my astrojs website blocked from indexing?

I'm hosting an astrojs website on netlify. I keep getting a low lighthouse score for SEO, with the reason being that the page is blocked from indexing. The image is attached below. SEO section of lighthouse report

I'm following the steps mentionned in the docs, but it doesn't seem to be effective. For context, here's what my head tag of the layout looks like:

  <head>
    <meta charset="UTF-8" />
    <link rel="icon" type="image/svg+xml" href="/swift-logo.svg" />
    <meta name="viewport" content="width=device-width, initial-scale=1.0" />
    <link rel="preconnect" href="https://fonts.googleapis.com" />
    <link rel="preconnect" href="https://fonts.gstatic.com" crossorigin />
    <meta
      name="description"
      content="My page"
      data-rh="true"
    />
    <link rel="sitemap" href="/sitemap-index.xml" />
    <meta name="robots" content="index,follow" />
    <title>{title}</title>
  </head>

This is the robots.txt file that I have in my public folder

User-agent: *
Allow: /
Sitemap: https://mysite.netlify.app/sitemap-index.xml

I also tried using this package, but it didn't work.

Keep in mind that no where in my code is noindex included. Not even in the head element of the hosted website, when checking using the devtools.

If you need any more context, please let me know. I'd appreciate any help!

Solution

  • As @mb21 mentionned in his comment, turns out I was checking my netlify preview page instead of the production one... This i kind of odd since I'm using CI/CD, and I assumed the changes would automatically be pushed to the main url "https://mysite.netlify.app", but it appeared that I can check out the production deploy using a button (see the image below), which would take me to "https://main--mysite.netlify.app", where the production deploy is hosted.

    deploy page

    So I'm now getting 100 on SEO on the main site :D

    seo