Proxy Community
How to use waitForSelector in Puppeteer to get all <p> tags on a page? - Printable Version

+- Proxy Community (https://proxycommunity.com/forum)
+-- Forum: Proxy Provider Discussions (https://proxycommunity.com/forum/forum-proxy-provider-discussions)
+--- Forum: Smartproxy (https://proxycommunity.com/forum/forum-smartproxy)
+--- Thread: How to use waitForSelector in Puppeteer to get all <p> tags on a page? (/thread-how-to-use-waitforselector-in-puppeteer-to-get-all-p-tags-on-a-page)

Pages: 1 2


How to use waitForSelector in Puppeteer to get all <p> tags on a page? - ShadowPath77 - 30-08-2024

Hey everyone,

So I’ve been messing around with Puppeteer and trying to figure out how to use waitforselector puppeteer get all p tags on a page. I know it’s probably basic for some of you, but I’m kinda stuck lol.

Here’s what I’m trying to do: I want to wait for all the `<p>` tags to load on a page and then grab them. I’ve tried using `page.waitForSelector('p')`, but it only waits for the first `<p>` tag, not all of them.

Is there a way to make sure *all* the `<p>` tags are loaded before I use something like `page.$$('p')` to get them? Or am I overcomplicating this?

Any help would be awesome, thanks in advance!

(Also, sorry if this has been asked before, I tried searching but couldn’t find exactly what I needed.)


RE: [b]"How to use waitForSelector in Puppeteer to get all <p> tags on a page?"[/b] - deepRunX77 - 24-11-2024

Hey! So, I had the same issue with waitforselector puppeteer get all p tags. What worked for me was using `page.waitForFunction` to check if the number of `<p>` tags stops increasing.

Here's a quick snippet:
```javascript
await page.waitForFunction(() => {
const pTags = document.querySelectorAll('p');
return pTags.length > 0 && pTags.length === someExpectedCount;
});
```
This ensures all `<p>` tags are loaded before you grab them with `page.$$('p')`. Hope this helps!


RE: [b]"How to use waitForSelector in Puppeteer to get all <p> tags on a page?"[/b] - shadowSprint99 - 28-01-2025

Yo, I feel you! The `waitForSelector` in Puppeteer can be tricky when dealing with multiple elements. Instead of waiting for a single `<p>`, you can use `page.waitForNavigation({ waitUntil: 'networkidle0' })` to make sure the page is fully loaded.

Then, just do:
```javascript
const pTags = await page.$$('p');
```
This should grab all the `<p>` tags after the page is done loading.


RE: [b]"How to use waitForSelector in Puppeteer to get all <p> tags on a page?"[/b] - cloakHawk88 - 04-02-2025

Hey there! I’ve been using Puppeteer for a while, and for waitforselector puppeteer get all p tags, I’d recommend checking out the `page.evaluate` method. It lets you run JS in the browser context, so you can wait for all `<p>` tags like this:

```javascript
await page.evaluate(() => {
return new Promise(resolve => {
const interval = setInterval(() => {
const pTags = document.querySelectorAll('p');
if (pTags.length > 0 && pTags.length === someExpectedCount) {
clearInterval(interval);
resolve();
}
}, 100);
});
});
```
This ensures all `<p>` tags are loaded before proceeding.


RE: [b]"How to use waitForSelector in Puppeteer to get all <p> tags on a page?"[/b] - darkMimicX - 14-02-2025

Hmm, I think you’re overcomplicating it a bit. If you’re sure the `<p>` tags will load eventually, you can just use a simple delay with `page.waitForTimeout` before grabbing them.

```javascript
await page.waitForTimeout(2000); // wait 2 seconds
const pTags = await page.$$('p');
```
Not the most elegant solution, but it works in a pinch!


RE: [b]"How to use waitForSelector in Puppeteer to get all <p> tags on a page?"[/b] - CamoGhost99 - 17-02-2025

Hey! For waitforselector puppeteer get all p tags, you might wanna check out the `page.waitForSelector` with a custom timeout. It’s not perfect, but it can help in some cases:

```javascript
await page.waitForSelector('p', { timeout: 5000 });
const pTags = await page.$$('p');
```
This waits for at least one `<p>` tag and then grabs all of them.


RE: [b]"How to use waitForSelector in Puppeteer to get all <p> tags on a page?"[/b] - fastShroud77 - 26-02-2025

I had this exact issue last week! What worked for me was combining `waitForSelector` with a loop to check if all `<p>` tags are loaded. Here’s a rough idea:

```javascript
let pTags = [];
while (pTags.length < expectedCount) {
await page.waitForSelector('p');
pTags = await page.$$('p');
}
```
This ensures you’re not missing any `<p>` tags.


RE: [b]"How to use waitForSelector in Puppeteer to get all <p> tags on a page?"[/b] - AnonLegend99 - 01-03-2025

Hey, just wanted to chime in! For waitforselector puppeteer get all p tags, you might wanna look into using `MutationObserver` inside `page.evaluate`. It’s a bit advanced, but it’s super reliable for detecting DOM changes.

Here’s a quick example:
```javascript
await page.evaluate(() => {
return new Promise(resolve => {
const observer = new MutationObserver(() => {
const pTags = document.querySelectorAll('p');
if (pTags.length === someExpectedCount) {
observer.disconnect();
resolve();
}
});
observer.observe(document.body, { childList: true, subtree: true });
});
});
```
This will wait until all `<p>` tags are added to the DOM.


RE: [b]"How to use waitForSelector in Puppeteer to get all <p> tags on a page?"[/b] - ShadowPath77 - 05-03-2025

Wow, thanks so much for all the suggestions, everyone! I tried the `page.waitForFunction` method, and it worked like a charm. I also checked out Playwright, and it looks super promising for future projects.

One quick follow-up: has anyone tried combining `waitForSelector` with `MutationObserver` for more complex pages? I’m curious if it’s worth the extra effort. Thanks again, you all rock!


RE: [b]"How to use waitForSelector in Puppeteer to get all <p> tags on a page?"[/b] - darkStormX - 07-03-2025

Hey! I’d recommend checking out the Puppeteer docs on GitHub. They have some great examples for handling dynamic content. For waitforselector puppeteer get all p tags, you might wanna try `page.waitForResponse` if the `<p>` tags are loaded via an API call.

```javascript
await page.waitForResponse(response => response.url().includes('some-api-endpoint'));
const pTags = await page.$$('p');
```
This ensures the data is fully loaded before grabbing the tags.