• majorAlexa

        See all notifications

        Skip to content
        Moz logo Menu open Menu close
        • Products
          • Moz Pro
          • Moz Pro Home
          • Moz Local
          • Moz Local Home
          • STAT
          • Moz API
          • Moz API Home
          • Compare SEO Products
          • Moz Data
        • Free SEO Tools
          • Domain Analysis
          • Keyword Explorer
          • Link Explorer
          • Competitive Research
          • MozBar
          • More Free SEO Tools
        • Learn SEO
          • Beginner's Guide to SEO
          • SEO Learning Center
          • Moz Academy
          • MozCon
          • Webinars, Whitepapers, & Guides
        • Blog
        • Why Moz
          • Digital Marketers
          • Agency Solutions
          • Enterprise Solutions
          • Small Business Solutions
          • The Moz Story
          • New Releases
        • Log in
        • Log out
        • Products
          • Moz Pro

            Your all-in-one suite of SEO essentials.

          • Moz Local

            Raise your local SEO visibility with complete local SEO management.

          • STAT

            SERP tracking and analytics for enterprise SEO experts.

          • Moz API

            Power your SEO with our index of over 44 trillion links.

          • Compare SEO Products

            See which Moz SEO solution best meets your business needs.

          • Moz Data

            Power your SEO strategy & AI models with custom data solutions.

          Let your business shine with Listings AI
          Moz Local

          Let your business shine with Listings AI

          Learn more
        • Free SEO Tools
          • Domain Analysis

            Get top competitive SEO metrics like DA, top pages and more.

          • Keyword Explorer

            Find traffic-driving keywords with our 1.25 billion+ keyword index.

          • Link Explorer

            Explore over 40 trillion links for powerful backlink data.

          • Competitive Research

            Uncover valuable insights on your organic search competitors.

          • MozBar

            See top SEO metrics for free as you browse the web.

          • More Free SEO Tools

            Explore all the free SEO tools Moz has to offer.

          NEW Keyword Suggestions by Topic
          Moz Pro

          NEW Keyword Suggestions by Topic

          Learn more
        • Learn SEO
          • Beginner's Guide to SEO

            The #1 most popular introduction to SEO, trusted by millions.

          • SEO Learning Center

            Broaden your knowledge with SEO resources for all skill levels.

          • On-Demand Webinars

            Learn modern SEO best practices from industry experts.

          • How-To Guides

            Step-by-step guides to search success from the authority on SEO.

          • Moz Academy

            Upskill and get certified with on-demand courses & certifications.

          • MozCon

            Save on Early Bird tickets and join us in London or New York City

          Unlock flexible pricing & new endpoints
          Moz API

          Unlock flexible pricing & new endpoints

          Find your plan
        • Blog
        • Why Moz
          • Digital Marketers

            Simplify SEO tasks to save time and grow your traffic.

          • Small Business Solutions

            Uncover insights to make smarter marketing decisions in less time.

          • Agency Solutions

            Earn & keep valuable clients with unparalleled data & insights.

          • Enterprise Solutions

            Gain a competitive edge in the ever-changing world of search.

          • The Moz Story

            Moz was the first & remains the most trusted SEO company.

          • New Releases

            Get the scoop on the latest and greatest from Moz.

          Surface actionable competitive intel
          New Feature

          Surface actionable competitive intel

          Learn More
        • Log in
          • Moz Pro
          • Moz Local
          • Moz Local Dashboard
          • Moz API
          • Moz API Dashboard
          • Moz Academy
        • Avatar
          • Moz Home
          • Notifications
          • Account & Billing
          • Manage Users
          • Community Profile
          • My Q&A
          • My Videos
          • Log Out

        The Moz Q&A Forum

        • Forum
        • Questions
        • My Q&A
        • Users
        • Ask the Community

        Welcome to the Q&A Forum

        Browse the forum for helpful insights and fresh discussions about all things SEO.

        1. Home
        2. SEO Tactics
        3. How can I make a list of all URLs indexed by Google?

        Moz Q&A is closed.

        After more than 13 years, and tens of thousands of questions, Moz Q&A closed on 12th December 2024. Whilst we’re not completely removing the content - many posts will still be possible to view - we have locked both new posts and new replies. More details here.

        How can I make a list of all URLs indexed by Google?

        SEO Tactics
        redirect indexed urls google url index google broken links
        6
        7
        4532
        Loading More Posts
        • Watching

          Notify me of new replies.
          Show question in unread.

        • Not Watching

          Do not notify me of new replies.
          Show question in unread if category is not ignored.

        • Ignoring

          Do not notify me of new replies.
          Do not show question in unread.

        • Oldest to Newest
        • Newest to Oldest
        • Most Votes
        Reply
        • Reply as question
        Locked
        This topic has been deleted. Only users with question management privileges can see it.
        • aplusnetsolutions
          aplusnetsolutions Subscriber last edited by

          I have a large site with over 6000 pages indexed but only 600 actual pages and need to clean up with 301 redirects. Haven't had this need since Google stopped displaying the url's in the results.

          BlueprintMarketing SHIJOTACS 3 Replies Last reply Reply Quote 0
          • Kaushalwpandas
            Kaushalwpandas last edited by

            very depressed of being not indexing URL - https://www.wrightleatherworks.com/collections/gun-holsters-for-women please someone help

            1 Reply Last reply Reply Quote 0
            • Natalie305
              Natalie305 last edited by

              Open the sitemap in a text editor or XML viewer. Look at the <loc> tags in the sitemap file. These tags contain the URL of each page on the website. You can copy the URLs from the <loc> tags into a spreadsheet or text document.

              1 Reply Last reply Reply Quote 1
              • Luna204
                Luna204 last edited by

                Open the sitemap in a text editor or XML viewer. Look at the <loc> tags in the sitemap file. These tags contain the URL of each page on the website. You can copy the URLs from the <loc> tags into a spreadsheet or text document.

                1 Reply Last reply Reply Quote 0
                • SHIJOTACS
                  SHIJOTACS @aplusnetsolutions last edited by

                  @aplusnetsolutions create a seperate sitemap for not indexed pages and upload to google web master tools. i had a website had same issues. Also remember it will take some time also

                  1 Reply Last reply Reply Quote 0
                  • BlueprintMarketing
                    BlueprintMarketing @aplusnetsolutions last edited by

                    @aplusnetsolutions
                    Make sure all these things are selected in screaming frog and crawl analysis selected

                    1 Reply Last reply Reply Quote 1
                    • BlueprintMarketing
                      BlueprintMarketing @aplusnetsolutions last edited by

                      @aplusnetsolutions

                      Going to Google search console it will tell you how many inches are inside Google search index there are a few other ways to do this if you want

                      #1
                      By far the easiest is to going to search console and select coverage and all known pages
                      alt text

                      number 2

                      For the quick and dirty method, simply perform a simple site search in your Google search bar with “site:yourdomain.com”

                      #3

                      You can crawl your site using screaming frog (it will 500 pages for free but every SEO should have it) use it to index your site and connect it to Google search console API as well as Google analytics and you will have a easy-to-read CSV export of your index and non-index pages
                      more on how this is done

                      ![ you can see the non-indexable URLs on the the right click that](77e61d47-2172-491c-8070-9a6cd5905fff-image.png image url)

                      video https://youtu.be/iYeXSdUt_hg

                      #4
                      or indexer site with any thing that can export a CSV download the CSV uploaded here and it will tell you which pages are in Google's index
                      https://www.rankwatch.com/free-tools/google-index-checker

                      https://searchengineland.com/check-urls-indexed-without-upsetting-google-follow-267472

                      https://seo.co/google-index/

                      1 Reply Last reply Reply Quote 0
                      • 1 / 1
                      • First post
                        Last post

                      Browse Questions

                      Explore more categories

                      • Moz Tools

                        Chat with the community about the Moz tools.

                      • SEO Tactics

                        Discuss the SEO process with fellow marketers

                      • Community

                        Discuss industry events, jobs, and news!

                      • Digital Marketing

                        Chat about tactics outside of SEO

                      • Research & Trends

                        Dive into research and trends in the search industry.

                      • Support

                        Connect on product support and feature requests.

                      • See all categories

                      Related Questions

                      • laurentjb

                        How do I carry out a redirect? Is there a code I need to use?

                        redirect redirect chain

                        How do I carry out a redirect? Is there a code I need to use? Thank you in advance.

                        On-Page Optimization | | laurentjb
                        0
                      • Roycd

                        Google not listing website correctly

                        google listing seo website submission

                        have a domain that was unused for a while and displayed the "parked by Gandi" domain message. Recently, I decided to set up a fully working website on that domain, but I'm struggling to get it to rank correctly on Google. Google isn't updating the http://domain.com to the correct https://www.domain.com, and I have tried everything. The http://domain.com redirects to https://www.domain.com if you click on it. I managed to get the correct version, https://www.domain.com, to show up, but after a couple of days, it reverted to the old version and has stayed that way. This has been going on for a few weeks now. Additionally, I had an old version of the site using .com, and I forwarded that to the .co.uk, but the old .com will not be removed from Google. it was just one page without much content. At the moment, Google's search results look very messy and out of date. I have submitted the URL in the Google Search Console using the URL inspection tool and requested indexing and reindexing. I have also submitted sitemaps, but nothing is really working. Any ideas?

                        SEO Tactics | | Roycd
                        0
                      • Sert_Sibiu

                        URL with .html or not

                        url with .html

                        Hello,
                        I have an online shop for printer consumables
                        Categories url's are like that https://www.sertit.ro/consumabile-imprimante.html
                        And products https://www.sertit.ro/cartuse-toner/xerox/cartus-toner-xerox-phaser-3020-workcentre-3025-1500-pagini.html
                        Is better to have .html or not?

                        SEO Tactics | | Sert_Sibiu
                        0
                      • sasansasyino

                        google webmaster tools Indexing request rejected

                        indexing request rejected webmaster indexing

                        when i try to index my posts in google webmaster tools i see this eror : Indexing request rejected
                        During live testing, indexing issues were detected with the URL
                        Crawl
                        Time
                        Sep 23, 2023, 11:05:05 PM
                        Crawled as
                        Google Inspection Tool desktop
                        Crawl allowed?
                        Yes
                        Page fetch
                        error
                        Failed: Hostload exceeded
                        Indexing allowed?
                        N/A
                        Indexing
                        User-declared canonical
                        N/A
                        Google-selected canonical
                        
                        Only determined after indexing my website : http://123select.ir/

                        SEO Tactics | | sasansasyino
                        0
                      • xShams

                        Google is ranking only home page not even considering to rank other pages!

                        ranking homepage sub-pages not ranking

                        Hey Community! I am facing an ranking issue, I am trying to rank on multiple quires with my sub-pages. The issue is homepage is ranking on every single query. Even the homepage's content is not relevant to that query! I have made sure to remove any sort of content relevancy from homepage which i trying to rank with my sub-pages. Business/Website Details: We are a recruitment agency based Pakistan providing manpower around the globe and other recruiting services. We are targeting each country with a sub-page. Even after I have done some necessary things but still there is not effect on rankings. Let me share some examples: Query: recruitment agency for gulf in pakistan Home Page Showing
                        1.jpg Page i want to ranking:
                        2.jpg This issue is not only with this page, it's appearing on multiple quires but I think it should clear the issue I am facing. Now: Any Possible Solutions to this technical or ranking error. Backlinks on this query don't matter because you can check the other search results don't have any backlink data on them. Please share some quick thoughts on-page content. Even I make a page that have the word recruiting/recruitment in it google will automatically rank homepage and not the page i want! Thanks in advance for help 🙂

                        SEO Tactics | | xShams
                        0
                      • BakeryTech

                        Unsolved Using NoIndex Tag instead of 410 Gone Code on Discontinued products?

                        ecommerce noindex shopify indexed urls

                        Hello everyone, I am very new to SEO and I wanted to get some input & second opinions on a workaround I am planning to implement on our Shopify store. Any suggestions, thoughts, or insight you have are welcome & appreciated! For those who aren't aware, Shopify as a platform doesn't allow us to send a 410 Gone Code/Error under any circumstance. When you delete or archive a product/page, it becomes unavailable on the storefront. Unfortunately, the only thing Shopify natively allows me to do is set up a 301 redirect. So when we are forced to discontinue a product, customers currently get a 404 error when trying to go to that old URL. My planned workaround is to automatically detect when a product has been discontinued and add the NoIndex meta tag to the product page. The product page will stay up but be unavailable for purchase. I am also adjusting the LD+JSON to list the products availability as Discontinued instead of InStock/OutOfStock.
                        Then I let the page sit for a few months so that crawlers have a chance to recrawl and remove the page from their indexes. I think that is how that works?
                        Once 3 or 6 months have passed, I plan on archiving the product followed by setting up a 301 redirect pointing to our internal search results page. The redirect will send the to search with a query aimed towards similar products. That should prevent people with open tabs, bookmarks and direct links to that page from receiving a 404 error. I do have Google Search Console setup and integrated with our site, but manually telling google to remove a page obviously only impacts their index. Will this work the way I think it will?
                        Will search engines remove the page from their indexes if I add the NoIndex meta tag after they have already been index?
                        Is there a better way I should implement this? P.S. For those wondering why I am not disallowing the page URL to the Robots.txt, Shopify won't allow me to call collection or product data from within the template that assembles the Robots.txt. So I can't automatically add product URLs to the list.

                        Technical SEO | | BakeryTech
                        0
                      • Leviiii

                        slug Link redirect to subdomain?

                        redirect subdomain link building

                        Hi !
                        Im Levi new here and new in the world of SEO, please dont judge if my questions are silly. Back on the days when the site was built we thought it is a good ideea to have subdomains that together with the domain name represent our main keywords.
                        ex. https://stansted.tonorwich.uk, https://heathrow.tonorwich.uk, https://luton.tonorwich.uk, https://gatwick.tonorwich.uk. There is content on this subdomains, would it make any difference from SEO perspective if we create slugs that redirect to these subdomains? for example creating https://tonorwich.uk/taxi-minibus-vip-tesla-norwich-to-stansted that redirects to https://stansted.tonorwich.uk ? Or better create these slugs with slightly different content?
                        Any ideeas would be appreciated.
                        Thanks in advance!

                        Link Building | | Leviiii
                        0
                      • Tunnel7

                        Unsolved Site showing up in Google search results for irrelevant keywords

                        serp keyword rankings

                        Hi there, one of my client's sites is showing up in Google search results / getting a lot of site traffic from keywords that while very close to words we're actually trying to target on the site, are irrelevant for the client and their site content. Does anyone have ideas of how to address this?

                        SEO Tactics | | Tunnel7
                        0

                      Get started with Moz Pro!

                      Unlock the power of advanced SEO tools and data-driven insights.

                      Start my free trial
                      Products
                      • Moz Pro
                      • Moz Local
                      • Moz API
                      • Moz Data
                      • STAT
                      • Product Updates
                      Moz Solutions
                      • SMB Solutions
                      • Agency Solutions
                      • Enterprise Solutions
                      • Digital Marketers
                      Free SEO Tools
                      • Domain Authority Checker
                      • Link Explorer
                      • Keyword Explorer
                      • Competitive Research
                      • Brand Authority Checker
                      • Local Citation Checker
                      • MozBar Extension
                      • MozCast
                      Resources
                      • Blog
                      • SEO Learning Center
                      • Help Hub
                      • Beginner's Guide to SEO
                      • How-to Guides
                      • Moz Academy
                      • API Docs
                      About Moz
                      • About
                      • Team
                      • Careers
                      • Contact
                      Why Moz
                      • Case Studies
                      • Testimonials
                      Get Involved
                      • Become an Affiliate
                      • MozCon
                      • Webinars
                      • Practical Marketer Series
                      • MozPod
                      Connect with us

                      Contact the Help team

                      Join our newsletter
                      Moz logo
                      © 2021 - 2025 SEOMoz, Inc., a Ziff Davis company. All rights reserved. Moz is a registered trademark of SEOMoz, Inc.
                      • Accessibility
                      • Terms of Use
                      • Privacy

                      Looks like your connection to Moz was lost, please wait while we try to reconnect.