Are you redirecting everything on www to non-www? If so, you don't really need a robots.txt to be served for the www subdomain. Google will ignore the original robots.txt file if it is given a 301 anyway.
Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Best posts made by Xiano
-
RE: I have two robots.txt pages for www and non-www version. Will that be a problem?posted in Technical SEO
-
RE: Blocking Standard pages with Robots.txt (t&c's, shipping policy, pricing & privacy policies etc)posted in Reporting & Analytics
I wouldn't block them. While it's unlikely to affect the rank of your other pages, it may result in a poorer user experience, e.g. if someone were to be searching for one of your policies in Google, it would not be returned.
I'm afraid I'm not an expert on Shopify at all, so I can't answer why they wouldn't have been blocked.
-
RE: Do IP and/or DNS changes impact Paid search (Adwords, Bing, etc.)posted in Paid Search Marketing
I have never seen any impact on PPC when migrating a site to a new server relating to the IP/DNS changes.
If the migration didn't go as planned and the new server returned 404, 500 or similar, however, you would find Google may stop serving ads if it noticed. You would receive a notification in Google Ads if that was the case tho.
Why do you believe that your ads were being suppressed?
-
RE: Is there a way to get a list of all pages of your website that are indexed in Google?posted in Technical SEO
There are a number of different ways, but I'd always start with Google Search Console Coverage reports.
If you haven't already, sign up and configure Search Console.
Then, go to your property > Coverage.
Then select the Valid "tab". You'll then be able to click on the two types of valid ("Submitted and indexed" and "Indexed, not submitted in sitemap"). Within each of these categories, you'll be able to download a CSV of the pages lists.
In the same area you'll be able to see lists of pages that Google knows about but hasn't indexed for one reason or another.