+- Proxy Community (https://proxycommunity.com/forum)
+-- Forum: Technical Community Support (https://proxycommunity.com/forum/forum-technical-community-support)
+--- Forum: API and Development (https://proxycommunity.com/forum/forum-api-and-development)
+--- Thread: How Can I Use the Indeed API to Fetch Job Listings Programmatically? or What Are the Best Practices f (/thread-how-can-i-use-the-indeed-api-to-fetch-job-listings-programmatically-or-what-are-the-best-practices-f)
Pages:
1
2
How Can I Use the Indeed API to Fetch Job Listings Programmatically? or What Are the Best Practices f - Swizz99 - 28-08-2024
"What Are the Best Practices for Integrating the Indeed API into My Application?"
Hey folks! I’m trying to use the indeed api to pull job listings into my app, but I’m kinda lost on the best way to do it. Like, how do I handle rate limits without getting banned? And what’s the deal with authentication—do I *really* need to sign up for a publisher account?
Also, anyone got tips on structuring the requests? I’ve seen some folks use pagination, but others just fetch everything at once (which feels risky lol).
Oh, and caching—should I bother? The indeed api docs aren’t super clear on how often data updates.
Any devs here who’ve done this before? Would love some real-world advice, not just the official docs. Thanks in advance!
(Ps. If there’s a better alternative in 2024, lmk—but I’m kinda stuck with indeed for now.)
“” - deepSeeker77 - 28-01-2025
Hey! I’ve worked with the indeed api before, and yeah, the rate limits can be a pain. You *definitely* need a publisher account—it’s not optional.
For rate limiting, I’d recommend using exponential backoff. Like, if you hit a limit, wait a bit and try again. Also, track your calls with something like Redis to avoid going over.
Pagination is way safer than fetching everything at once. The indeed api docs are vague, but I’ve found 10-20 results per page works best.
Caching? Totally worth it. Job data doesn’t update *that* often, so even a 1-hour cache can save you tons of calls.
“” - vpnTrekker99 - 03-03-2025
Dude, just use Postman to test your indeed api requests first. It’ll save you so much headache.
And yeah, you need that publisher account—no way around it. For rate limits, check out the `X-RateLimit-` headers in the response. They’ll tell you how many calls you have left.
Also, don’t sleep on error handling. The indeed api can throw weird errors, so log everything.
“” - RobbyX23 - 04-03-2025
Honestly, the indeed api is kinda clunky. If you’re stuck with it, here’s what I did:
- Signed up for the publisher account (takes like 5 mins).
- Used Python’s `requests` lib with retries (check out `tenacity` for this).
- Cached results in Firebase because why not?
Data updates every few hours, so caching for 2-3 hours is fine.
“” - secureFly77 - 15-03-2025
Pro tip: Use the `q` parameter wisely in the indeed api. Too broad, and you’ll get garbage. Too narrow, and you’ll miss stuff.
Also, yeah, pagination is your friend. Fetching everything at once is a great way to get rate-limited into oblivion.
For auth, just bite the bullet and get the publisher account. It’s free, and you’ll need the API key anyway.
“” - MaskerX99 - 17-03-2025
If you’re worried about rate limits, try throttling your requests. Like, space them out by 1-2 seconds.
And yeah, caching is a must. I use Cloudflare Workers to cache indeed api responses—super easy to set up.
Btw, the docs *suck*, but the API itself isn’t terrible once you get the hang of it.
“” - stinX - 20-03-2025
Lol, the indeed api is a mess, but here’s how I survived:
1. Got the publisher account (no choice, sorry).
2. Used `axios-retry` for, well, retries.
3. Cached stuff in-memory for dev, Redis for prod.
Data updates are slow, so don’t stress about caching too much.
“” - hyperMimicX - 21-03-2025
For structuring requests, keep it simple. Start with basic params like `location` and `job title`, then expand.
And yeah, pagination > fetching all. The indeed api *will* slap you if you go overboard.
Oh, and check out RapidAPI—they have a wrapper for the indeed api that might save you time.
“” - Swizz99 - 23-03-2025
Hey, OP! Just wanted to say thanks for all the tips—super helpful.
I got the publisher account (ugh, but whatever) and set up caching with Redis. Also tried pagination with 15 results per page, and it’s way smoother.
One follow-up: How do you guys handle stale data? Like, if a job gets filled, does the indeed api mark it as inactive, or do I have to filter manually?
(And yeah, the docs are still trash lol.)