grarpamp at gmail.com
Sat Sep 4 22:39:31 PDT 2021
On 9/3/21, coderman <coderman at protonmail.com> wrote:
> try selenium webdriver:
Wasn't really wanting to write some new tool,
from a library, that someone out there has probably
already done better.
Wget / wget2 / curl / fetch / elinks / * old CLI tools
save to disk are 'source', not pulled out of the webserver
via the pages embedded js yielding the text and links one
would see in a GUI browser. Then there are those pages that
do crazy js based variables mathfuscation, bloatfeaturing, etc.
One of the old tools used to compile against ecma/js, but
it was disclaimed as broken and capable of only basic js.
Maybe can run ./firefox -<options> <url> > <file>
but is inefficient, no spidering options, etc.
In return, people should know that 'gron' and similar tools exist.
More information about the cypherpunks