[b]"How to use waitForSelector in Puppeteer to get all <p> tags on a page?"[/b]

10 Replies, 1526 Views

Hey everyone,

So I’ve been messing around with Puppeteer and trying to figure out how to use waitforselector puppeteer get all p tags on a page. I know it’s probably basic for some of you, but I’m kinda stuck lol.

Here’s what I’m trying to do: I want to wait for all the `<p>` tags to load on a page and then grab them. I’ve tried using `page.waitForSelector('p')`, but it only waits for the first `<p>` tag, not all of them.

Is there a way to make sure *all* the `<p>` tags are loaded before I use something like `page.$$('p')` to get them? Or am I overcomplicating this?

Any help would be awesome, thanks in advance!

(Also, sorry if this has been asked before, I tried searching but couldn’t find exactly what I needed.)
Hey! So, I had the same issue with waitforselector puppeteer get all p tags. What worked for me was using `page.waitForFunction` to check if the number of `<p>` tags stops increasing.

Here's a quick snippet:
```javascript
await page.waitForFunction(() => {
const pTags = document.querySelectorAll('p');
return pTags.length > 0 && pTags.length === someExpectedCount;
});
```
This ensures all `<p>` tags are loaded before you grab them with `page.$$('p')`. Hope this helps!
Yo, I feel you! The `waitForSelector` in Puppeteer can be tricky when dealing with multiple elements. Instead of waiting for a single `<p>`, you can use `page.waitForNavigation({ waitUntil: 'networkidle0' })` to make sure the page is fully loaded.

Then, just do:
```javascript
const pTags = await page.$$('p');
```
This should grab all the `<p>` tags after the page is done loading.
Hey there! I’ve been using Puppeteer for a while, and for waitforselector puppeteer get all p tags, I’d recommend checking out the `page.evaluate` method. It lets you run JS in the browser context, so you can wait for all `<p>` tags like this:

```javascript
await page.evaluate(() => {
return new Promise(resolve => {
const interval = setInterval(() => {
const pTags = document.querySelectorAll('p');
if (pTags.length > 0 && pTags.length === someExpectedCount) {
clearInterval(interval);
resolve();
}
}, 100);
});
});
```
This ensures all `<p>` tags are loaded before proceeding.
Hmm, I think you’re overcomplicating it a bit. If you’re sure the `<p>` tags will load eventually, you can just use a simple delay with `page.waitForTimeout` before grabbing them.

```javascript
await page.waitForTimeout(2000); // wait 2 seconds
const pTags = await page.$$('p');
```
Not the most elegant solution, but it works in a pinch!
Hey! For waitforselector puppeteer get all p tags, you might wanna check out the `page.waitForSelector` with a custom timeout. It’s not perfect, but it can help in some cases:

```javascript
await page.waitForSelector('p', { timeout: 5000 });
const pTags = await page.$$('p');
```
This waits for at least one `<p>` tag and then grabs all of them.
I had this exact issue last week! What worked for me was combining `waitForSelector` with a loop to check if all `<p>` tags are loaded. Here’s a rough idea:

```javascript
let pTags = [];
while (pTags.length < expectedCount) {
await page.waitForSelector('p');
pTags = await page.$$('p');
}
```
This ensures you’re not missing any `<p>` tags.
Hey, just wanted to chime in! For waitforselector puppeteer get all p tags, you might wanna look into using `MutationObserver` inside `page.evaluate`. It’s a bit advanced, but it’s super reliable for detecting DOM changes.

Here’s a quick example:
```javascript
await page.evaluate(() => {
return new Promise(resolve => {
const observer = new MutationObserver(() => {
const pTags = document.querySelectorAll('p');
if (pTags.length === someExpectedCount) {
observer.disconnect();
resolve();
}
});
observer.observe(document.body, { childList: true, subtree: true });
});
});
```
This will wait until all `<p>` tags are added to the DOM.
Wow, thanks so much for all the suggestions, everyone! I tried the `page.waitForFunction` method, and it worked like a charm. I also checked out Playwright, and it looks super promising for future projects.

One quick follow-up: has anyone tried combining `waitForSelector` with `MutationObserver` for more complex pages? I’m curious if it’s worth the extra effort. Thanks again, you all rock!
Hey! I’d recommend checking out the Puppeteer docs on GitHub. They have some great examples for handling dynamic content. For waitforselector puppeteer get all p tags, you might wanna try `page.waitForResponse` if the `<p>` tags are loaded via an API call.

```javascript
await page.waitForResponse(response => response.url().includes('some-api-endpoint'));
const pTags = await page.$$('p');
```
This ensures the data is fully loaded before grabbing the tags.



Users browsing this thread: 1 Guest(s)