Find archived sitemap of a website that no longer exists

[EdKim]

I am trying to figure out the site structure of a website and the urls of all the pages. Normally this would be easy but a couple of months ago the website went down and I don't think it will ever come back. Any help would be appreciated.

[seoelevated]

Use the internet archive (wayback machine) which effectdigital mentioned above, to find the /robots.txt file from the desired date. In that file you should find the referenced sitemap file (assuming the site properly included its sitemap reference in its robot.txt file). Then you can use the same process to request the sitemap file which was referenced in the robots.txt file.

[NickSamuel]

Hi Effect,

Does your second link automatically provider the sitemaps available, or does the user still need to "know" or be able to guess where they might be e.g /sitemap.xml?

Nick

[effectdigital]

You can use this site to see legacy site-maps for some websites (though they may be partial or incomplete):

• https://archive.org/web/

For example, check these sitemap results:

• https://web.archive.org/web/sitemap/toyota.com

For smaller sites, the results are much easier to look at.