It's a neat idea, but the internet is not very tolerant to things that don't appear to be human traffic. That's why browsers used by web automation infrastructure are often Chromium-derived. Using a browser like this would almost certainly trigger all kinds of captcha requirements or worse.
ICIC. Indeed, our next step after completing all the JS rendering would be to pretend to be human, i.e., pass the CAPTCHA. Of course, we have to respect robots.txt."
Genuine question, not sarcasm. How does this work? As in, how is it different from an agent reading the code and inferring the result?
I had to install Playwright to get Claude Code to acknowledge a rendering issue when it verified that the data was in the DOM. Can this detect something from the served content in some new and novel way?
I'm dumb, might have missed it, only scanned the page looking for a heading that would explain and didn't see one, apologies if the page explains it and I missed it.
It can save tons of tokens compared with screen capture or reading the HTML directly. The downside is that we are still not able to handle complex JavaScript and CAPTCHA checks.
How does this compare to Lightpanda, which appears to be trying to solve the same issue? https://github.com/lightpanda-io/browser (personally super happy to see Chromium-free solutions in this space!)
Just wanted to add some clarifications to your list:
> 1. I use the MIT license, so you don't have to pay. Lightpanda requires payment if you use it for business. :)
Lightpanda uses the AGPL license, you can use it for business for free. Your only obligation is to distribute any modified version of Lightpanda's code + the license to your users.
> 3. Lightpanda isn't in Markdown format; it's more like a curl format.
I'm not sure what you mean, but Lightpanda can dump a rendered page in Markdown format via the CLI, CDP (using the custom LP domain), and the native MCP. This is a feature added recently.
BTW, Pardus looks nice, congrats! I'll follow your progress.
And I agree, it's great to see more players in this space!
To the author: ask your AI "what percentage of websites will this be expected to work well on, or better than just reading the HTML? What portion of websites do we need DOM, JS and maybe CSS at this point?"
It's a neat idea, but the internet is not very tolerant to things that don't appear to be human traffic. That's why browsers used by web automation infrastructure are often Chromium-derived. Using a browser like this would almost certainly trigger all kinds of captcha requirements or worse.
ICIC. Indeed, our next step after completing all the JS rendering would be to pretend to be human, i.e., pass the CAPTCHA. Of course, we have to respect robots.txt."
Genuine question, not sarcasm. How does this work? As in, how is it different from an agent reading the code and inferring the result?
I had to install Playwright to get Claude Code to acknowledge a rendering issue when it verified that the data was in the DOM. Can this detect something from the served content in some new and novel way?
I'm dumb, might have missed it, only scanned the page looking for a heading that would explain and didn't see one, apologies if the page explains it and I missed it.
It can save tons of tokens compared with screen capture or reading the HTML directly. The downside is that we are still not able to handle complex JavaScript and CAPTCHA checks.
How does this compare to Lightpanda, which appears to be trying to solve the same issue? https://github.com/lightpanda-io/browser (personally super happy to see Chromium-free solutions in this space!)
3 differences:
1. I use the MIT license, so you don't have to pay. Lightpanda requires payment if you use it for business. :)
2. I use Rust to try to further optimize.
3. Lightpanda isn't in Markdown format; it's more like a curl format.
Disclaimer: I'm a Lightpanda co-author
Just wanted to add some clarifications to your list:
> 1. I use the MIT license, so you don't have to pay. Lightpanda requires payment if you use it for business. :)
Lightpanda uses the AGPL license, you can use it for business for free. Your only obligation is to distribute any modified version of Lightpanda's code + the license to your users.
> 3. Lightpanda isn't in Markdown format; it's more like a curl format.
I'm not sure what you mean, but Lightpanda can dump a rendered page in Markdown format via the CLI, CDP (using the custom LP domain), and the native MCP. This is a feature added recently.
BTW, Pardus looks nice, congrats! I'll follow your progress. And I agree, it's great to see more players in this space!
To the author: ask your AI "what percentage of websites will this be expected to work well on, or better than just reading the HTML? What portion of websites do we need DOM, JS and maybe CSS at this point?"