Screaming Frog’s "XML sitemap generator" is lowkey OP. way better than some paid tools.
also, the "URL parameters" tab helps spot duplicate content from tracking tags.
if crawls take forever, try splitting the site into sections and crawling separately.
also, the "URL parameters" tab helps spot duplicate content from tracking tags.
if crawls take forever, try splitting the site into sections and crawling separately.
