Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Should 301-ed links be removed from sitemap?
-
In an effort to do some housekeeping on our site we are wanting to change the URL format for a couple thousand links on our site. Those links will all been 301 redirected to corresponding links in the new URL format. For example, old URL format: /tag/flowers as well as search/flowerswill be 301-ed to, new URL format: /content/flowers**Question:**Since the old links also exist in our sitemap, should we add the new links to our sitemap in addition to the old links, or replace the old links with new ones in our sitemap? Just want to make sure we don’t lose the ranking we currently have for the old links.Any help would be appreciated. Thanks!
-
I'm going to disagree a little bit with the other commenters. I've done quite a few large scale redirect projects and I'm not 100% opposed to using a "dirty sitemap" for a short duration. The better option is to leave some internal links pointed at the old URLs. I know what the search engines say, but I also know what I've experienced when it comes to getting 301'd links crawled again.
Read this post by Everett Sizemore for more info at what I'm describing:
http://a-moz.groupbuyseo.org/blog/uncrawled-301s-a-quick-fix-for-when-relaunches-go-too-well
-
"A sitemap should only contain links to active pages."
Hi shawn81
Alex is absolutely correct there.
In fact, Duane Forrester has said repeatedly that Bing absolutely does not like to find such pages in a sitemap and that you should make sure there are never 3XX, 4XX or 5XX status pages included because it will stop Bingbot from crawling your site.
While Googlebot is not so sensitive, the reality is that all search engines allocate a certain amount of crawl capacity for your site...if your sitemaps include a load of pages that are not likely to be indexed, the result is twofold:
- you are wasting capacity on useless pages and the crawler may never get to the stuff you really want indexed
- if the crawler encounters a lot of non-active pages when it crawls, future crawl capacity (not to mention trust) is likely to be reduced
Replace the old URLs with the new and give the bots a little thrill of adventure
Hope that helps,
Sha
- you are wasting capacity on useless pages and the crawler may never get to the stuff you really want indexed
-
There shouldn't be any 301 links in a sitemap. A sitemap should only contain links to active pages. So in your case, you should remove all the 301 links and replace them with the new links.
Couple notes - Having 301 links in your sitemap won't hurt your site or SEO unless the sitemap is so huge that you need to split it up into multiple files. But you should really only have the final links in the sitemap, neither people nor bots want to be redirected around. If you properly 301'd the crawlers will automatically update their links.
Changing links around in the sitemap generally won't hurt your site. Especially if the links no longer exist and you're improving the list. There are very few cases where making changes will hurt the site.
-
We have had a problem with this ourselves. We put a 301 redirect on our domain when we were building a new site (went from new. to www.) and search engines are still crawling the new. domain. Bing webmaster tools registers it as an error because they can't find the old site. I would lean toward removing it just because your users are probably being redirected somewhere they wouldn't necessarily want to go.
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
Automate XML Sitemaps
Quick question, which is the best method that people have for automating sitemaps. We publish around 200 times a day and I would like to make sure as soon as we publish it gets updated in the site map. What is the best method of updating a sitemap so it gets updated immediately after it is published.
Technical SEO | | mattdinbrooklyn0 -
Good alternatives to Xenu's Link Sleuth and AuditMyPc.com Sitemap Generator
I am working on scraping title tags from websites with 1-5 million pages. Xenu's Link Sleuth seems to be the best option for this, at this point. Sitemap Generator from AuditMyPc.com seems to be working too, but it starts handing up, when a sitemap file, the tools is working on,becomes too large. So basically, the second one looks like it wont be good for websites of this size. I know that Scrapebox can scrape title tags from list of url, but this is not needed, since this comes with both of the above mentioned tools. I know about DeepCrawl.com also, but this one is paid, and it would be very expensive with this amount of pages and websites too (5 million ulrs is $1750 per month, I could get a better deal on multiple websites, but this obvioulsy does not make sense to me, it needs to be free, more or less). Seo Spider from Screaming Frog is not good for large websites. So, in general, what is the best way to work on something like this, also time efficient. Are there any other options for this? Thanks.
Technical SEO | | blrs120 -
Correct linking to the /index of a site and subfolders: what's the best practice? link to: domain.com/ or domain.com/index.html ?
Dear all, starting with my .htaccess file: RewriteEngine On
Technical SEO | | inlinear
RewriteCond %{HTTP_HOST} ^www.inlinear.com$ [NC]
RewriteRule ^(.*)$ http://inlinear.com/$1 [R=301,L] RewriteCond %{THE_REQUEST} ^./index.html
RewriteRule ^(.)index.html$ http://inlinear.com/ [R=301,L] 1. I redirect all URL-requests with www. to the non www-version...
2. all requests with "index.html" will be redirected to "domain.com/" My questions are: A) When linking from a page to my frontpage (home) the best practice is?: "http://domain.com/" the best and NOT: "http://domain.com/index.php" B) When linking to the index of a subfolder "http://domain.com/products/index.php" I should link also to: "http://domain.com/products/" and not put also the index.php..., right? C) When I define the canonical ULR, should I also define it just: "http://domain.com/products/" or in this case I should link to the definite file: "http://domain.com/products**/index.php**" Is A) B) the best practice? and C) ? Thanks for all replies! 🙂
Holger0 -
Dofollow and Nofollow links
What is the difference between dofollow and nofollow links? I know that some sites/blogs only let you post nofollow links. In such a case how do I know if a comment I posted on a certain site will be a nofollow or dofollow? How about big traffic sites such as Huff Post. Do they only allow nofollow links?
Technical SEO | | greenfoxone0 -
301 redirect from Blogger
Hello, I have a client with a Wordpress network of blogs, each blog is owned by a different blogger. Many of them were migrated time ago from Blogger. I have seen that the way used to redirect them is a meta refresh, so no authority is being passed. I cannot find any reliable way of making a 301 from Blogger, There are some plugins, but I'm afraid of using them. Any of you have experience with this situation please? I have even thought about placing a global rel canonical before the meta refresh, but I think that here the problem is the meta refresh itself.... Thank you in advance
Technical SEO | | Juandbbam0 -
Links from the same server has value or not
Hi Guys, Sometime ago one of the SEO experts said to me if I get links from the same IP address, Google doesn't count them as with much value. For an example, I am a web devleoper and I host all my clients websites on one server and link them back to me. Im wondering whether those links have any value when it comes to seo or should I consider getting different hosting providers? Regards Uds
Technical SEO | | Uds0 -
Do we need to manually submit a sitemap every time, or can we host it on our site as /sitemap and Google will see & crawl it?
I realized we don't have a sitemap in place, so we're going to get one built. Once we do, I'll submit it manually to Google via Webmaster tools. However, we have a very dynamic site with content constantly being added. Will I need to keep manually re-submitting the sitemap to Google? Or could we have the continually updating sitemap live on our site at /sitemap and the crawlers will just pick it up from there? I noticed this is what SEOmoz does at http://www.seomoz.org/sitemap.
Technical SEO | | askotzko0 -
Is link cloaking bad?
I have a couple of affiliate gaming sites and have been cloaking the links, the reason I do this is to stop have so many external links on my sites. In the robot.txt I tell the bots not to index my cloaked links. Is this bad, or doesnt it really matter? Thanks for your help.
Technical SEO | | jwdesign0