Best Practices for Using wget to Scrape URLs: Any Tips or Pitfalls to Avoid?

Wow, thanks everyone for the awesome tips! I tried using `--wait` and `--limit-rate` together, and it worked like a charm. No blocks so far!

I also set up a folder structure using `-P` and it’s way more organized now.

One quick follow-up: has anyone used `--mirror` for large sites? Does it handle nested directories well, or should I stick with `-r`?

Thanks again, you all rock!