JavaScript Capable Webpage Download Tool?

Karl Semich 0xloem at
Sun Sep 5 03:55:04 PDT 2021

> Maybe a local http server that provides access to a selenium session as if
> it is non-js html?

I found

Headless browser based on WebKit

This tool will help you make your AJAX applications crawlable.

Webpages based on JavaScript MVC libraries can't be positioned
by default because search engines can't run (yet) all the
JavaSript code that your page needs to execute in order to *show*
anything. That's why you need a headless browser that will fetch
the page, run the JavaSript and output the resulting HTML to the
crawler, which will then be able to index your page.

from , a list which
attempts to enumerate all headless browsers and apis.

I'm on mobile and haven't tried it.

I'm interested if anyone finds a free-beer ecommerce scraping solution or
other tools for connecting the cli to the web.

-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: text/html
Size: 1854 bytes
Desc: not available
URL: <>

More information about the cypherpunks mailing list