JavaScript Capable Webpage Download Tool?

[Karl Semich]

>
> Maybe a local http server that provides access to a selenium session as if
> it is non-js html?
>

I found https://github.com/alexandernst/headless_browser

Headless browser based on WebKit
================

This tool will help you make your AJAX applications crawlable.

Webpages based on JavaScript MVC libraries can't be positioned
by default because search engines can't run (yet) all the
JavaSript code that your page needs to execute in order to *show*
anything. That's why you need a headless browser that will fetch
the page, run the JavaSript and output the resulting HTML to the
crawler, which will then be able to index your page.


from https://github.com/dhamaniasad/HeadlessBrowsers , a list which
attempts to enumerate all headless browsers and apis.

I'm on mobile and haven't tried it.

I'm interested if anyone finds a free-beer ecommerce scraping solution or
other tools for connecting the cli to the web.

>
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Type: text/html
Size: 1854 bytes
Desc: not available
URL: <https://lists.cpunks.org/pipermail/cypherpunks/attachments/20210905/6cdeb637/attachment.txt>