Moz Q&A is closed.
After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.
Screaming Frog returning both HTTP and HTTPS results...
-
Hi,
About 10 months I switched from HTTP to HTTPS. I then switched back (long story).
I noticed that Screaming Frog is picking up the HTTP and HTTPS version of the site. Maybe this doesn't matter, but I'd like to know why SF is doing that. The URL is: www.aerlawgroup.com
Any feedback, including how to remove the HTTPS version, is greatly appreciated.
Thanks.
-
No problem! I'm not too familiar with secure migrations in WP, but I bet there is some kind of plugin for that.
-
Hi,
Thank you! Fixing that link prevents SF from finding the HTTPS pages.
I use WP for this site. Is there a quick way to redirect HTTPS to HTTP? As opposed to forwarding page by page? Ideally, there is a way in WP (either in WP directly or via some plugin) that I can tell WP to automatically 301 redirect all HTTPS to HTTP. I use WPEngine for my hosting, if that's at all relevant.
Thank you again for your excellent answer.
-
Hi,
Found the source of your issue: this URL (https://www.aerlawgroup.com/sex-crimes.html) has a link pointing to it from a HTTP URL (http://www.aerlawgroup.com/sex-crimes/new-allegations-emerge-in-sexual-misconduct-case.html), which is probably how SF found all the other secure URLs.
All of your HTTPS page return 200. If you want to go back to non-secure, those secure URLs will need to be redirected back to HTTP, the same as you probably did when you went to secure.
Also, looks like you've still got a couple HTTPS pages indexed: https://www.google.com/search?q=site%3Aaerlawgroup.com+inurl%3Ahttps&oq=site%3Aaerlawgroup.com+inurl%3Ahttps&gs_l=serp.3...5577.15088.0.15655.24.24.0.0.0.0.150.1864.19j5.24.0....0...1.1.64.serp..3.0.0.bf5ZiosfHMI
Browse Questions
Explore more categories
-
Moz Tools
Chat with the community about the Moz tools.
-
SEO Tactics
Discuss the SEO process with fellow marketers
-
Community
Discuss industry events, jobs, and news!
-
Digital Marketing
Chat about tactics outside of SEO
-
Research & Trends
Dive into research and trends in the search industry.
-
Support
Connect on product support and feature requests.
Related Questions
-
How get google reviews on search results?
Hi, We have good google reviews. (4,8) Can we get this rating stars also on our organic search results ? Best remco
Intermediate & Advanced SEO | | remcoz0 -
Google Indexing Of Pages As HTTPS vs HTTP
We recently updated our site to be mobile optimized. As part of the update, we had also planned on adding SSL security to the site. However, we use an iframe on a lot of our site pages from a third party vendor for real estate listings and that iframe was not SSL friendly and the vendor does not have that solution yet. So, those iframes weren't displaying the content. As a result, we had to shift gears and go back to just being http and not the new https that we were hoping for. However, google seems to have indexed a lot of our pages as https and gives a security error to any visitors. The new site was launched about a week ago and there was code in the htaccess file that was pushing to www and https. I have fixed the htaccess file to no longer have https. My questions is will google "reindex" the site once it recognizes the new htaccess commands in the next couple weeks?
Intermediate & Advanced SEO | | vikasnwu1 -
Setting A Custom User Agent in Screaming Frog
Hi all, Probably a dumb question, but I wanted to make sure I get this right. How do we set a custom user agent in Screaming Frog? I know its in the configuration settings, but what do I have to do to create a custom user agent specifically for a website? Thanks much! Malika
Intermediate & Advanced SEO | | Malika10 -
Redirect Search Results to Category Pages
I am planning redirect the search results to it's matching category page to avoid having two indexed pages of essentially the same content. Example http://www.example.com/search/?kw=sunglasses
Intermediate & Advanced SEO | | WizardOfMoz
wil be redirected to
http://www.example.com/category/sunglasses/ Is this a good idea? What are the possible negative effect if I go this route? Thanks.0 -
Why would our server return a 301 status code when Googlebot visits from one IP, but a 200 from a different IP?
I have begun a daily process of analyzing a site's Web server log files and have noticed something that seems odd. There are several IP addresses from which Googlebot crawls that our server returns a 301 status code for every request, consistently, day after day. In nearly all cases, these are not URLs that should 301. When Googlebot visits from other IP addresses, the exact same pages are returned with a 200 status code. Is this normal? If so, why? If not, why not? I am concerned that our server returning an inaccurate status code is interfering with the site being effectively crawled as quickly and as often as it might be if this weren't happening. Thanks guys!
Intermediate & Advanced SEO | | danatanseo0 -
Google Said "Repeat the search with the omitted results included."
We have some pages targeting the different countries but with the Near to Similar content/products, just distinguished with the country name etc. one of the page was assigned to me for optimizing. two or three Similar pages are ranked with in top 50 for the main keyword. I updated some on page content to make it more distinguish from others. After some link building, I found that this page still not showing in Google result, even I found the following message on the google. "In order to show you the most relevant results, we have omitted some entries very similar to the 698 already displayed.
Intermediate & Advanced SEO | | alexgray
If you like, you can repeat the search with the omitted results included." I clicked to repeat omitted result and found that my targeted url on 450th place in google (before link building this was not) My questions are Is google consider this page low quality or duplicate content? Is there any role of internal linking to give importance a page on other (when they are near to similar)? Like these pages can hurt the whole site rankings? How to handle this issue?0 -
How long until you see results?
How long does it typically take for SEO efforts to materialize? We recently performed a complete website redesign (new site, and am wondering how long we should wait until we analyze the results and possibly change our seo/keyword strategy?
Intermediate & Advanced SEO | | tdeboer0
