[ot] scraping websites undetectably in python

Undescribed Horrific Abuse, One Victim & Survivor of Many gmkarl at gmail.com
Thu May 18 17:14:00 PDT 2023


may not address captchas and behavior profiling

https://github.com/ultrafunkamsterdam/undetected-chromedriver/issues/998#issuecomment-1386475005

spending time adjacent to that free-gpt community i learned the modern
way to do this is a package called “undetected chromedriver” which
purportedly bypasses server detection of selenium, and comes inside
seleniumbase which purportedly has some autoinstall procedures

haven’t tried it, but it’s interesting to learn after selenium
displaced phantomjs in a vendor-directed manner for so long

some of the people in these communities are quite experienced at web
scraping; there’s a movement to go back to pure http requests,
deobfuscating and replicating browser javascript to make speedy
clients


More information about the cypherpunks mailing list