davincible / chromedp-undetected Goto Github PK
View Code? Open in Web Editor NEWChromedp undetected
License: MIT License
Chromedp undetected
License: MIT License
This package is nice, but configuration item ChromePath
seems to be missing, and setting it does not take effect.
chrome_test.go
file with the following contentspackage chromedpundetected
import (
"context"
"fmt"
"testing"
"time"
"github.com/chromedp/chromedp"
"github.com/hashicorp/go-multierror"
)
var n = 3
func TestChromedpundetected(t *testing.T) {
testRun(t,
n,
NewConfig(
WithTimeout(time.Hour),
WithHeadless(),
),
func(ctx context.Context) error {
return chromedp.Run(ctx,
chromedp.Navigate("https://nowsecure.nl"),
chromedp.WaitVisible(`//div[@class="notfound"]`),
)
},
)
t.Logf("Undetected!")
}
func testRun(t *testing.T, n int, cfg Config, run func(ctx context.Context) error) {
var gerr error
var success bool
// Attempt to run the tests multiple times, as in CI they are unstable.
for i := 0; i < n; i++ {
t.Logf("Attempt %d/%d", i+1, n)
ctx, cancel, err := New(cfg)
if err != nil {
gerr = multierror.Append(gerr, fmt.Errorf("create context: %w", err))
continue
}
if err = run(ctx); err != nil {
gerr = multierror.Append(gerr, fmt.Errorf("chromedp run: %w", err))
// Close Chrome instance.
ctxN, cancelN := context.WithTimeout(ctx, time.Second*10)
if err := chromedp.Cancel(ctxN); err != nil {
t.Logf("failed to cancel: %v", err)
}
cancelN()
cancel()
continue
}
success = true
if gerr != nil {
t.Logf("attempt %d, errors: %s", i+1, gerr.Error())
}
cancel()
break
}
if !success {
t.Fatal(gerr)
}
}
FROM golang:1.19
RUN apt-get update && apt-get -y upgrade && apt-get -y install gcc g++ ca-certificates chromium xvfb
WORKDIR /app
COPY . .
RUN go mod download
ENTRYPOINT [ "go", "test", "-run", "TestChromedpundetected" ]
Wait a few minutes, then go into the container and look at it, you will see that some core files were created.
Hi there,
First of all, thanks for making this package, it works perfectly for the protected site(s) I want to scrape!
My issue is that headless mode does not work on Mac. You already mentioned this in your README:
The headless option only works on linux, and requires Xvfb to be installed. Could theoretically work on Mac OS with xquartz but I don't have a Mac to test with, so feel free to PR.
I did install xquartz and your example only works by removing the headless option.
You mentioned you can't test it on here so I'm willing to do so but I would need some pointers on where to start if possible.
Thanks!
Are there any plans to update to playwright-go?
Hi Davincible
I made an error when compiling windows or macos. According to the error message, it seems that I am using a linux-specific field?
../../go/pkg/mod/github.com/!davincible/[email protected]/chrome.go:153:21: cmd.SysProcAttr.Pdeathsig undefined (type *syscall.SysProcAttr has no field or method Pdeathsig)
../../go/pkg/mod/github.com/!davincible/[email protected]/display.go:82:19: xvfb.SysProcAttr.Pdeathsig undefined (type *syscall.SysProcAttr has no field or method Pdeathsig)
Is there a way to be compatible with windows or macos
Hello, can you release a usable package for the latest version of the code? When I pull it up, it's still 1.3.4
Thanks for the project, can we support rod?
https://github.com/go-rod/rod
A declarative, efficient, and flexible JavaScript library for building user interfaces.
๐ Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. ๐๐๐
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google โค๏ธ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.