Latest Threads


Best way to JSON Python get all values for key? Need help! or How to JSON Python get all values for k


API and Development    18 Replies

MaskedWalkerX, 26-08-2024, 06:29 PM

"Struggling to json python get all values for key – any tips?"

Hey everyone,

I’ve been working with some nested JSON data in Python and hitting a wall.

Need to json python get all values for a specific key, but some are buried deep. Tried looping and recursive approaches, but it feels clunky.

Is there a cleaner or built-in way to do this? Maybe a one-liner or a library I’m missing?

Example:
```python
data = {"a": 1, "b": {"a": 2}, "c": [{"a": 3}]}
# Need all values for key "a" → [1, 2, 3]
```

Any help or shortcuts would be awesome! Thanks in advance.

(Also, if you’ve got a fav tool for this, lmk!)


  Open Thread

Looking for the best automated chrome extension scraper – any recommendations? or How reliable is an


Web Scraping    16 Replies

StealthShroudX, 26-08-2024, 05:09 PM

"Looking for the best automated chrome extension scraper – any recommendations?"

Hey folks!

I’ve been digging around for a solid *automated chrome extension scraper* but can’t seem to find one that’s both fast and reliable.

Anybody got a favorite tool they’ve used? Free or paid, doesn’t matter—just needs to handle bulk scraping without craashing every 5 mins lol.

Bonus points if it’s easy to set up. Thanks in advance!

---

Or:

"How reliable is an automated chrome extension scraper for bulk data extraction?"

Yo,

Thinking of using an *automated chrome extension scraper* for a big project, but worried it’ll choke on 10k+ entries.

Anyone tested these tools at scale? Do they actually work or just *look* good in demos?

Would love some real-world feedback before I commit. Cheers!

---

Or:

"Has anyone built a custom automated chrome extension scraper? Share your experience!"

Hey devs,

Built my own *automated chrome extension scraper* (typo intended, lol) and it’s… kinda janky.

If you’ve rolled your own, what stack did you use? Any major pitfalls to avoid?

Mine keeps missing dynamic content—super annoying. Tips appreciated!

---

(Let me know if you want tweaks!)


  Open Thread

Anyone Using fpool Proxifier? How Reliable Is It for Managing Proxies?


Troubleshooting    9 Replies

secureTunneler99, 26-08-2024, 04:46 PM

Hey everyone,

So, I’ve been looking into fpool proxifier for managing my proxies, and I’m kinda curious if anyone here has actually used it?

Like, how reliable is it? Does it handle multiple proxies smoothly, or does it get glitchy? I’ve seen some mixed reviews, but I wanna hear from real users.

Also, is it easy to set up? I’m not super techy, so if it’s a headache to configure, I might pass lol.

And does it work well with different types of proxies (residential, datacenter, etc.)? Or is it picky?

Any tips or warnings before I dive in? Thanks in advance!

Cheers!


  Open Thread

What Are the Best Data Scraping Tools for Efficient Web Scraping? or Which Data Scraping Tools Do You


Web Scraping    20 Replies

deepXchangeX, 26-08-2024, 04:35 PM

"What’s Your Experience With Data Scraping Tools? Any Hidden Gems?"

Hey folks! 👋 Been diving deep into data scraping tools lately and wanted to hear your thoughts.

I’ve tried the usual suspects like Scrapy and BeautifulSoup, but curious if anyone’s stumbled upon lesser-known tools that just *work*.

Like, is there a tool you swear by for messy websites? Or something that handles CAPTCHAs without costing an arm and a leg?

Also, how do you even choose between all these options? Speed? Ease of use? Or just whatever gets the job done?

And hey, are free data scraping tools actually reliable, or is it worth shelling out for paid ones?

Spill your secrets—any hidden gems out there? 🚀


  Open Thread

Getting HTTP 415 Errors – What’s the Best Way to Fix Unsupported Media Type Issues? or HTTP 415 Unsup


Troubleshooting    20 Replies

secureTunneler99, 26-08-2024, 04:14 PM

"HTTP 415 Unsupported Media Type – Why Is My API Rejecting My Request?"

Hey folks,

So I keep hitting this http 415 error when I try to POST to my API. The request *looks* fine to me, but the server keeps yelling about "unsupported media type."

I’m sending JSON with `Content-Type: application/json` in the headers... or at least I *think* I am. Could it be a typo? Or maybe the server expects something else?

Anyone else run into this? How’d you fix it?

(Also, is it just me or does http 415 sound like some kinda HTTP police code? 😅)

Thanks in advance!


  Open Thread

How can I smoothly scroll to an element using javascript scrollintoview? or Why isn't javascript scro


API and Development    14 Replies

shadowSurferX, 26-08-2024, 02:55 PM

"Why isn't javascript scrollintoview working as expected?"

Hey folks!

So I’m trying to use javascript scrollintoview to smoothly scroll to a section on my page, but it’s just snapping there like it’s in a hurry or something. 😅

I’m using the basic `element.scrollIntoView()` but it feels kinda jarring. Anyone else run into this?

Also, does javascript scrollintoview even support smooth scrolling by default, or do I need to tweak it? Tried adding `{ behavior: 'smooth' }` but it’s still hit or miss, especially on mobile.

Speaking of mobile—does javascript scrollintoview work consistently there? Or am I better off using a polyfill?

Thanks in advance! 🙏


  Open Thread

How do I remove proxy settings from my computer? Need help! or How do I remove proxy settings? Can't


Troubleshooting    18 Replies

AnonVortex99, 26-08-2024, 02:28 PM

Title: how do i remove proxy settings from my computer? need help!

hey guys, i’m kinda stuck here. somehow my pc got these weird proxy settings enabled, and now my internet’s acting up.

tried googling but all the guides are way too technical. anyone know how do i remove proxy settings without messing anything else up?

i’m on windows 10 btw, but mac tips are welcome too for others!

thanks in advance, y’all are lifesavers 🙏

---

Title: how do i remove proxy settings? can’t seem to figure it out

ugh, this is so annoying. my browser keeps redirecting, and i think it’s because of some leftover proxy settings.

how do i remove proxy settings for good? i checked the internet options, but it’s still not working.

pls help a noob out!

(p.s. if it matters, i’m on windows 11)

---

Title: stuck with unwanted proxy settings—how do i remove them?

help! my wifi’s fine, but my browser’s acting like it’s stuck behind a proxy.

how do i remove proxy settings? i swear i didn’t even turn them on lol.

tried resetting stuff, no luck. any quick fixes?

thx!!


  Open Thread

Can Someone Explain How to Find the Craigslist Ad Number for a Specific Listing?


Others    18 Replies

AnonyGamer88, 26-08-2024, 02:27 PM

Hey y’all,
So I’m trying to find the Craigslist ad number for a specific listing, but I’m kinda stuck. Like, where is it even located? I’ve been clicking around but no luck.

Is it in the URL or somewhere in the ad itself? I’ve seen ppl mention it, but I can’t figure it out.

Also, does the Craigslist ad number change if the listing gets reposted? Or is it always the same?

Any help would be awesome, thx!

(Also, sorry if this is a dumb question lol, I’m new to this whole Craigslist thing.)


  Open Thread

Has Anyone Found a Reliable Instagram Unblocker That Actually Works? or What’s the Best Instagram Unb


Instagram    22 Replies

fastLurkX99, 26-08-2024, 01:39 PM

"Has Anyone Found a Reliable Instagram Unblocker That Actually Works?"

Hey guys, so I got blocked (again 🙄) and I’m desperate to get back into my account. Tried a few "Instagram unblocker" tools but they either didn’t work or looked super sketchy.

Anyone here had luck with one that’s legit? Free or paid, idc at this point lol. Just need something that *actually* works.

Also, pls no scams—I’ve already wasted enough time on those. 😅

Thanks in advance!


  Open Thread

Having issues with npm install scrapegraphai? Any solutions or tips? or What’s the best way to use np


API and Development    18 Replies

GhostRunnerX, 26-08-2024, 01:24 PM

"Having issues with npm install scrapegraphai? Any solutions or tips?"

Hey folks,

So I tried running `npm install scrapegraphai` for a web scraping project, but hit a few snags. First, it took forever to install, and then I got some weird dependency errors.

Anyone else run into this? Did you find a workaround?

Also, is there a specific Node version that works best with it? I’m on v16 but wondering if I should switch.

Thanks in advance!

---

OR

"What’s the best way to use npm install scrapegraphai for web scraping?"

Hey everyone,

Just discovered `npm install scrapegraphai` and wanna use it for scraping some dynamic sites.

But the docs are kinda sparse? Like, how do you handle login pages or CAPTCHAs with this thing?

Any pro tips or example configs you’ve got working?

Also, does it play nice with Puppeteer/Playwright, or is it standalone?

Appreciate any advice!

---

OR

"Does npm install scrapegraphai work smoothly for you, or are there common pitfalls?"

Yo,

Tried `npm install scrapegraphai` yesterday and it *mostly* works, but man, the error messages are cryptic.

Had to fiddle with proxy settings forever to get past blocks.

Anyone else notice it’s kinda slow on heavy JS sites? Or am I just doing it wrong?

Also, does it auto-retry failed requests, or do I need to handle that myself?

---

OR

"How reliable is npm install scrapegraphai for scraping dynamic content?"

Hey,

Need to scrape a site that’s all React/Angular. `npm install scrapegraphai` claims it handles dynamic stuff, but my results are hit-or-miss.

Sometimes it grabs everything, other times it’s just... blank?

Is there a trick to waiting for elements to load, or should I just ditch it for Playwright?

Kinda wanna make it work tho—seems lightweight if configured right.

---

OR

"Any alternatives to npm install scrapegraphai, or is it the best option right now?"

Sup,

Been using `npm install scrapegraphai` for a bit, but it’s kinda janky with anti-bot systems.

Before I deep-dive into fixing it—are there better libs out there? Heard of Cheerio/Puppeteer combos, but dunno if they’re overkill.

Or is scrapegraphai still the move if you tweak it enough?

Lemme know what y’all think!


  Open Thread