Yo, Scrapy is def a beast for scraping apartments, but have you tried Selenium? It’s a bit slower but handles JS-heavy sites like a champ.
Also, for proxies, I’ve been using Bright Data (formerly Luminati). It’s pricey but worth it if you’re scraping at scale.
And yeah, anti-scraping measures are getting wild. Rotating user agents and adding random delays helps a ton.
Also, for proxies, I’ve been using Bright Data (formerly Luminati). It’s pricey but worth it if you’re scraping at scale.
And yeah, anti-scraping measures are getting wild. Rotating user agents and adding random delays helps a ton.
