OP here—wow, thanks for all the recs! Didn’t expect so many options.
Tried Screaming Frog and it worked like a charm for my needs. The 500 URL limit was fine, and it was way easier than I thought.
Might give Scrapy a shot next since so many of you vouched for it.
Anyone got a favorite tutorial for beginners? The official docs look... intense.
Also, lol at the "sketchy crawler" warnings—almost fell for one of those "unlimited" scams. Thanks for the heads up!
Tried Screaming Frog and it worked like a charm for my needs. The 500 URL limit was fine, and it was way easier than I thought.
Might give Scrapy a shot next since so many of you vouched for it.
Anyone got a favorite tutorial for beginners? The official docs look... intense.
Also, lol at the "sketchy crawler" warnings—almost fell for one of those "unlimited" scams. Thanks for the heads up!
