go-rod / stealth Goto Github PK
View Code? Open in Web Editor NEWanti-bot-detection with rod
Home Page: https://pkg.go.dev/github.com/go-rod/stealth
License: MIT License
anti-bot-detection with rod
Home Page: https://pkg.go.dev/github.com/go-rod/stealth
License: MIT License
I'm trying to use bypass with a remote launcher in a docker image, but it fails the WebDriver test. Any chance I can make it work this way?
func main() {
client := launcher.MustNewRemote("ws://localhost:9222")
client.Headless(true)
browser := rod.New().Timeout(time.Minute*2).Client(client.Client()).MustConnect()
...
Hello :),
There are many things that need to be spoofed in order to pass bot protection nowadays.
Most important things are custom fingerprints, CPU, GPU and several other things which can't be emulated yet in stealth.
Is it possible to add it somehow, because I do get fingerprinted easily on some sites with heave bot protection?
If not is there any way to emulate this whole fingerprint I posted down here?
{
"navigator": {
"appCodeName": "Mozilla",
"appName": "Netscape",
"appVersion": "5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Mobile Safari/537.36",
"language": "fr-FR",
"languages": [
"fr-FR",
"fr",
"en-US",
"en"
],
"userAgent": "Mozilla/5.0 (Linux; Android 6.0; Nexus 5 Build/MRA58N) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/108.0.0.0 Mobile Safari/537.36",
"vendor": "Google Inc.",
"product": "Gecko",
"productSub": "20030107",
"platform": "Win32",
"vibrate": true,
"getBattery": true,
"credentials": true,
"appMinorVersion": false,
"bluetooth": true,
"storage": true,
"getGamepads": true,
"getStorageUpdates": false,
"hardwareConcurrency": true,
"mediaDevices": true,
"mozAlarms": false,
"mozConnection": false,
"mozIsLocallyAvailable": false,
"mozPhoneNumberService": false,
"msManipulationViewsEnabled": false,
"permissions": true,
"registerProtocolHandler": true,
"requestMediaKeySystemAccess": true,
"requestWakeLock": false,
"sendBeacon": true,
"serviceWorker": true,
"storeWebWideTrackingException": false,
"webkitGetGamepads": false,
"webkitTemporaryStorage": true,
"cookieEnabled": true,
"javaEnabled": false,
"doNotTrack": -1,
"plugins": []
},
"window": {
"ActiveXObject": false,
"innerHeight": 888,
"innerWidth": 1862,
"outerWidth": 1862,
"outerHeight": 888,
"devicePixelRatio": 2,
"addEventListener": true,
"XMLHttpRequest": true,
"XDomainRequest": false,
"orientation": "undefined",
"DeviceOrientationEvent": true,
"DeviceMotionEvent": true,
"TouchEvent": true,
"chrome": {
"app": {
"isInstalled": false,
"InstallState": {
"DISABLED": "disabled",
"INSTALLED": "installed",
"NOT_INSTALLED": "not_installed"
},
"RunningState": {
"CANNOT_RUN": "cannot_run",
"READY_TO_RUN": "ready_to_run",
"RUNNING": "running"
}
}
},
"prototype_bind": true,
"PointerEvent": true,
"sessionStorage": true,
"localStorage": true,
"indexedDB": true,
"FileReader": true,
"HTMLElement": false,
"webRTC": true,
"mozInnerScreenY": 0
},
"document": {
"documentMode": "undefined",
"webdriver": false,
"driver": false,
"selenium": false,
"hidden": false,
"webkitHidden": false
},
"other": {
"CC_ON": false,
"InstallTrigger": false,
"prototype_forEach": true,
"imul": true,
"parseInt": true,
"hypot": true,
"value1": false,
"XPathResult": true
},
"performance": {
"timeOrigin": 1672836946172.5,
"timing": {
"connectStart": 1672836946239,
"navigationStart": 1672836946171,
"secureConnectionStart": 1672836946283,
"fetchStart": 1672836946179,
"domContentLoadedEventStart": 1672836946872,
"responseStart": 1672836946588,
"domInteractive": 1672836946872,
"domainLookupEnd": 1672836946239,
"responseEnd": 1672836946589,
"redirectStart": 0,
"requestStart": 1672836946553,
"unloadEventEnd": 0,
"unloadEventStart": 0,
"domLoading": 1672836946594,
"domComplete": 1672836947114,
"domainLookupStart": 1672836946239,
"loadEventStart": 1672836947114,
"domContentLoadedEventEnd": 1672836946872,
"loadEventEnd": 1672836947115,
"redirectEnd": 0,
"connectEnd": 1672836946553
},
"navigation": {
"type": 0,
"redirectCount": 0
}
},
"canvas": {
"value1": "420217769",
"value2": "1243744842"
},
"fonts_optm": "12,14,15,16,17,27,3,38,40,44,7,9",
"fonts": "12,14,15,16,17,27,3,38,40,44,7,9",
"rCFP": "-158760887",
"ssh": "e54e4dcbd557a1fe86616568abcd0650295cf8d2f959bd8c84fec71923f1a445",
"mr": "20,20,20,20,20,20,20,20,20,0,0,0,0,100,",
"brave": "0",
"wv": "Google Inc. (NVIDIA)",
"wr": "ANGLE (NVIDIA, NVIDIA GeForce RTX 3060 Ti Direct3D11 vs_5_0 ps_5_0, D3D11)",
"weh": "fab37783e5c3bc89f876774bac442ad3e613f7dc5b769b19a8d4d8c6b80ef077",
"wl": 29,
"fmh": "082f3e287baebaac8dfa0e53be71bed004ee57a16b3dffd269f318915d405c81",
"fpValstr": "420217769;1243744842;dis;;true;true;true;-60;true;24;24;true;false;-1",
"np": "11321144241322243122",
"screen": {
"availHeight": 888,
"availLeft": 0,
"availTop": 0,
"availWidth": 1862,
"colorDepth": 24,
"height": 888,
"pixelDepth": 24,
"width": 1862
},
"__v": 0
}
FYI there's https://www.npmjs.com/package/extract-stealth-evasions now, which is part of the puppeteer-extra repo.
Is here any complete example code?
复现代码如下
package main
import (
"crypto/md5"
"fmt"
"github.com/go-rod/rod/lib/launcher"
"time"
"github.com/go-rod/rod"
"github.com/go-rod/stealth"
)
func main() {
path, _ := launcher.LookPath()
url := launcher.New().Bin(path).Headless(false).MustLaunch()
browser := rod.New().
ControlURL(url).Timeout(time.Minute).MustConnect()
defer browser.MustClose()
// You can also use stealth.JS directly without rod
fmt.Printf("js: %x\n\n", md5.Sum([]byte(stealth.JS)))
page := stealth.MustPage(browser)
page.MustNavigate("http://lib.cqvip.com/")
time.Sleep(10 * time.Minute)
}
比如打开:https://tly38.com/
怎么绕过Cloudflare,example没看明白怎么使用的
hello, I'm trying to bypass cloudflare with your example_file but i'm not finding any documentation to wait until the pages is fully loaded.
As for example on aw-lab.com you have to wait like 5-6 seconds before been redirect in the home page.
I tried to use page.Timeout(10 * time.Second).MustWaitLoad()
but it didnt wait 10 seconds
When I visit https://amiunique.org/fp and https://coveryourtracks.eff.org/ using:
page := stealth.MustPage(browser)
page.MustEmulate(devices.Device{
Title: "Custom Linux Desktop",
UserAgent: "Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:24.0) Gecko/20100101 Firefox/24.0)",
AcceptLanguage: "en",
})
page.MustNavigate("https://amiunique.org/fp")
AmIUnique always shows Intel Iris as a graphics card and EFF says my browser's fingerprint is unique and detectable. If I visit using my regular desktop Brave browser it says: your browser has a randomized fingerprint. So, is there any way to customize those through go-rod/stealth?
It seems there might be ways to change or randomize those using JavaScript here and here for example. But, I couldn't find a way in the API to allow me to do that.
So relevant issue on berstend/puppeteer-extra that might explain the issue better.
In a regular browser either Firefox or Chrome if you look for Fingerprint extensions there are some extensions that do a good job at randomizing the Fingerprint. They even notify you when an attempt is made by the website you are visiting Fingerprint you. If we could load these extensions, using rod they might be able to do the job. Sadly, it seems load-extension not working in my Firefox tests, haven't tested with chrome yet. But, I remember chromedp had no way to do that. Especially if I load an existing user data-dir. If I recall correctly, I saw some comments in the rod doc that NewUserMode might not be usable in such scenarios as well.
I appreciate it if you head me in the right direction. Thank you!
Try to automate login with google process
it gives me error message:
"This browser or app may not be secure"
Here's my code so far:
package main
import (
"github.com/go-rod/rod"
"github.com/go-rod/rod/lib/utils"
"github.com/go-rod/stealth"
)
func main() {
const email = "<my_gmail_account>"
browser := rod.New().MustConnect()
page := stealth.MustPage(browser)
page.MustNavigate("https://accounts.google.com/ServiceLogin")
page.MustElement("#identifierId").MustInput(email).MustType()
page.MustElement("#identifierNext").MustClick()
utils.Pause()
}
someone report the same issue here:
berstend/puppeteer-extra#668
anybody, can give me directions?
Large js may cause cdp crash on some machines.
Help beginners to understand it to reduce misuse.
Click on a label to open a new page, and webdriver is true on the new page.
What methods are there to deal with?
Blocking fingerprinting is more like a "cat and mouse" game, I won't say there could be a common way, so it'll be a long-term thing to do.
We need to figure out the ways they collect unique information from the browser, then use js to hook them.
Fortunately, we can get all source codes from the client-side, even if they're obfuscated.
Some information:
main imports
github.com/go-rod/stealth: github.com/go-rod/[email protected]: verifying module: checksum mismatch
downloaded: h1:kKjZSB6osxJInG/2ywC8vt1Y0Oig/UnUfkcd/IRMGDM=
sum.golang.org: h1:JU5MgM1TZ1t1r+JLkKyf3EhnxMNCLHjcfQUFe/Vxta8=
Hi
We spoke on: https://github.com/go-rod/rod/issues/162
A. Do you plan on adding canvas evasion?
B. JA3 fingerprinting evasion (it becomes easier and easier to validate it, and going forward I think this fingerprinting will be used by a lot of sites)
C. What else is different in this repo from: https://github.com/berstend/puppeteer-extra/tree/master/packages/extract-stealth-evasions#readme
使用stealth库,headless设置为true,就无法获取网页信息,但是设置为false,页面返回是403,这是啥原因。
Run `npm install` to download the correct Chromium revision
Remove the cache of the npx will resolve it rm ~/.npm/_npx
Hello, I'm trying to bypass cloudflare with code:
func main() {
lc := launcher.New().
Set(flags.UserDataDir, ".chrome").
Headless(false)
ctlURL := lc.MustLaunch()
browser := rod.New().ControlURL(ctlURL).MustConnect()
defer browser.MustClose()
page := stealth.MustPage(browser)
page.MustNavigate("https://nowsecure.nl/").MustWaitLoad()
utils.Pause()
}
Not passed whether to enable Headless
or not.
can you add multi parameter to page.EvalOnNewDocument(JS)
something like page.Eval(js string, jsArgs ...interface{}).
it can better pass parameters to JS
And bypass can use this code for handle all new open target
wait := browser.EachEvent(func(e *proto.TargetTargetCreated) {
// if it's not a page return
if e.TargetInfo.Type != proto.TargetTargetInfoTypePage {
fmt.Println("not a page", e.TargetInfo.Type)
return
}
fmt.Println("page created")
// create a page from the page id
page := browser.MustPageFromTargetID(e.TargetInfo.TargetID)
_, err = page.EvalOnNewDocument(JS)
if err != nil {
fmt.Println(err)
}
})
go wait()
When using this stealth plugin, it seems its not able to open/show the browser window - which is possible when not using this.
I am using the launcher like
l := launcher.New().
Headless(false).
Devtools(true)
defer l.Cleanup() // remove launcher.FlagUserDataDir
url := l.MustLaunch()
however with stealth it seems to ignore the Headless(false) parameter?
current version has old version of navigator.plugins and navigator.mimeTypes, that is the problem, i think...
current stealth version have:
but new version chromium browser have default values in different object structure:
i was test it in different detect sites (https://bot.sannysoft.com/ and other) and i think, just need to update navigator.plugins object to new version and navigator.mimeTypes in this package and after it will work fine becouse everything else is still fine
Looks like the PageSetBypassCSP no longer works, see below
But looks like there is also a bypass in the puppeteer-extra plugin
berstend/puppeteer-extra#668 (comment)
If I am wrong and this function is already available, how can we use ?
Thanks
I want to use stealth to bypass Cloudflare's latest bot detection. This feature was great last year, but recently when I wanted to use stealth, it didn't work like it used to. Stealth cannot bypass Cloudflare's bot detection mechanism.
package main
import (
"context"
"os"
"time"
"github.com/go-rod/rod"
"github.com/go-rod/rod/lib/launcher"
"github.com/go-rod/stealth"
)
func main() {
l := launcher.NewUserMode().
Leakless(true).
UserDataDir("tmp/t").
Set("disable-default-apps").
Set("no-first-run").
Headless(false)
defer l.Kill()
browser := rod.New().ControlURL(l.MustLaunch()).MustConnect()
defer browser.MustClose()
page := stealth.MustPage(browser)
page.MustNavigate("https://nowsecure.nl").MustWaitLoad()
time.Sleep(time.Minute)
}
I want to see the actual pages behind Cloudflare protection.
The page is stuck on the Cloudflare bot detection page.
I think stealth
is a better name for this project.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.