Dynamic content is a nightmare, but here’s what I do: I use WebDriverWait with custom conditions. Like, I’ll wait for a specific class or attribute to appear.
For AJAX, I sometimes inject JavaScript to check if the page is fully loaded. Something like `return document.readyState === 'complete'`. It’s not perfect, but it helps.
Also, have you looked into Scrapy? It’s not Selenium, but it’s great for scraping and handles dynamic content pretty well with middleware.
For AJAX, I sometimes inject JavaScript to check if the page is fully loaded. Something like `return document.readyState === 'complete'`. It’s not perfect, but it helps.
Also, have you looked into Scrapy? It’s not Selenium, but it’s great for scraping and handles dynamic content pretty well with middleware.