0xloem at gmail.com
Sun Sep 5 03:55:04 PDT 2021
> Maybe a local http server that provides access to a selenium session as if
> it is non-js html?
I found https://github.com/alexandernst/headless_browser
Headless browser based on WebKit
This tool will help you make your AJAX applications crawlable.
by default because search engines can't run (yet) all the
JavaSript code that your page needs to execute in order to *show*
anything. That's why you need a headless browser that will fetch
the page, run the JavaSript and output the resulting HTML to the
crawler, which will then be able to index your page.
from https://github.com/dhamaniasad/HeadlessBrowsers , a list which
attempts to enumerate all headless browsers and apis.
I'm on mobile and haven't tried it.
I'm interested if anyone finds a free-beer ecommerce scraping solution or
other tools for connecting the cli to the web.
-------------- next part --------------
A non-text attachment was scrubbed...
Name: not available
Size: 1854 bytes
Desc: not available
More information about the cypherpunks