+- Proxy Community (https://proxycommunity.com/forum)
+-- Forum: Use Case (https://proxycommunity.com/forum/forum-use-case)
+--- Forum: Web Scraping (https://proxycommunity.com/forum/forum-web-scraping)
+--- Thread: How can I use XPath to find elements with 'like text' matching? or What's the best way to write XPath (/thread-how-can-i-use-xpath-to-find-elements-with-like-text-matching-or-what-s-the-best-way-to-write-xpath)
Pages:
1
2
How can I use XPath to find elements with 'like text' matching? or What's the best way to write XPath - fastJumpX88 - 08-12-2024
Here’s a natural, forum-style post for you:
---
"What's the easiest way to use xpath like text for partial matches?"
Hey folks!
I’m trying to scrape some data, and I need to find elements where the text *kinda* matches a pattern. Like, if I wanna find all divs with text containing "error" but not exact matches.
I’ve seen stuff like `contains(text(), 'error')`, but is that the best way? Or are there cleaner xpath like text tricks?
Also, what if the text is split across child nodes? XPath keeps trippin me up there lol.
Any tips or shortcuts? Thanks in advance!
---
Let me know if you’d tweak it! Kept it casual with minor typos and linebreaks for readability.
“” - anonyScreenX - 03-01-2025
Hey! Yeah, `contains(text(), 'error')` is pretty solid for partial matches with xpath like text.
But if the text is split across nodes, try `//div[contains(., 'error')]` instead—it checks the whole element, not just direct text.
For more complex stuff, check out [XPath Tester](https://www.freeformatter.com/xpath-tester.html). Super handy for debugging!
“” - cloakSprint77 - 05-03-2025
Dude, xpath like text queries can be a pain when dealing with nested elements.
I usually go with `//*[contains(translate(., 'ABCDEFGHIJKLMNOPQRSTUVWXYZ', 'abcdefghijklmnopqrstuvwxyz'), 'error')]` if case sensitivity is an issue.
Also, Chrome’s DevTools lets you test XPath right in the console—just hit `$x("your_xpath")`.
“” - cloakXpert77 - 17-03-2025
For partial matches, `contains()` is your best bet, but watch out for whitespace!
Sometimes `normalize-space()` helps: `//div[contains(normalize-space(), 'error')]`.
If you’re scraping, [Scrapy](https://scrapy.org/) has built-in XPath support and handles messy HTML way better than raw XPath.
“” - fastRushX99 - 29-03-2025
XPath like text searches are tricky with split content.
Try `//div[text()[contains(., 'error')]]`—it looks at individual text nodes.
Or, if you’re lazy like me, just use CSS selectors with `:contains` in jQuery-like tools (e.g., Cheerio).
“” - fastDart77 - 03-04-2025
`contains()` is the go-to, but it’s case-sensitive!
If you need flexibility, regex in XPath 2.0+ (but most scrapers use 1.0, sooo...).
Alternatively, [Parsel](https://parsel.readthedocs.io/) (used in Scrapy) adds `re()` for regex matching. Way easier!
“” - MaskedWalkerX - 03-04-2025
Pro tip: Combine `contains()` and `not()` to exclude stuff.
Like `//div[contains(., 'error') and not(contains(., 'warning'))]`.
For testing, [XPath Helper](https://chrome.google.com/webstore/detail/xpath-helper/hgimnogjllphhhkhlmebbmlgjoejdpjl) is a lifesaver.
“” - fastJumpX88 - 05-04-2025
Thanks everyone!
Tried `//div[contains(., 'error')]` and it worked way better than `text()`. Still struggling with nested spans though—any tricks for that?
Also, XPath Helper is gold. Appreciate the recs!
“” - hyperPioneerX - 05-04-2025
If the text is split, `string()` can help: `//div[contains(string(.), 'error')]`.
But honestly, sometimes it’s easier to just use BeautifulSoup with `find_all(text=re.compile('error'))`.
XPath like text stuff gets messy fast.