Comments (5)
Please add a valid Rod Version: v0.0.0
to your issue. Current version is v0.116.0
Please fix the format of your markdown:
2 MD031/blanks-around-fences Fenced code blocks should be surrounded by blank lines [Context: "```"]
2 MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
6:1 MD033/no-inline-html Inline HTML [Element: img]
10 MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
generated by check-issue
from rod.
尝试不加载这些资源,都将导致无法继续后续的,也不知道为什么
ext := filepath.Ext(ctx.Request.URL().Path)
szT := util.Any2Str(ctx.Request.Req().Header["Content-Type"]) // stylesheet,font, image
// 阻止请求 strings.Contains(szT, "stylesheet") ||
if strings.Contains(szT, "stylesheet") || strings.Contains(szT, "font") || strings.Contains(szT, "image") || ext == ".png" || ext == ".jpg" || ext == ".jpeg" || ext == ".gif" || ext == ".css" || ext == ".woff" || ext == ".woff2" || ext == ".ttf" || ext == ".eot" {
//ctx.Skip = true // ctx.Request.Req().Header.Set("My-Header", "test")
//ctx.Response.Fail(proto.NetworkErrorReasonAborted)
return false
}
from rod.
抱歉,我看不太懂你描述
from rod.
1、MustAdd 添加了拦截器,我因为是做爬虫,所以想阻止图片、字体、css文件的加载,始终没有成功
2、我希望在页面加载完了、添加的拦截事件都运行完了得到一个事件,及时关闭page,似乎爬虫处理的页面在
page.MustNavigate(szUrl.Url).MustWaitLoad() 之后立刻关闭page的话,很多拦截器就无法得到运行
我当前是基于延时70秒,但是这不太友好
您可以尝试下爬虫:https://www.alipan.com/s/4iQjiU9KUmG
1、抓取"/adrive/v2/file/list_by_share" 的数据
3、阻止图片、字体、css文件的加载
我不确定:
wait := page.MustWaitNavigation()
page.MustNavigate(szUrl.Url).MustWaitLoad()
wait()
我不确定这里关闭page是否合适
from rod.
这个错误是由于爬虫处理过程中这文件返回的是图片,然后就这样了
router.MustAdd(k1, func(hj *rod.Hijack) {
hj.OnError = func(err error) {
log.Println("HijackRequests error", err, hj.Request.URL().String())
}
......
HijackRequests error {-32602 Invalid parameters Failed to deserialize params.body - BINDINGS: binary value expected at position 64} https://cn-beijing-data.aliyundrive.net/pxcIoOce%2F951346711%2F65cf1bc23d2e20b1bf52475a955ec4e247b8049a%2F65cf1bc2c77c291d35294a6db78d47e7ee32c861?security-token=CAISvgJ1q6Ft5B2yfSjIr5bDfNXu3qxT5fOJW1fej0wFb8tLqZH5izz2IHhMf3NpBOkZvvQ1lGlU6%2Fcalq5rR4QAXlDfNRjYXxzCqFHPWZHInuDox55m4cTXNAr%2BIhr%2F29CoEIedZdjBe%2FCrRknZnytou9XTfimjWFrXWv%2Fgy%2BQQDLItUxK%2FcCBNCfpPOwJms7V6D3bKMuu3OROY6Qi5TmgQ41Uh1jgjtPzkkpfFtkGF1GeXkLFF%2B97DRbG%2FdNRpMZtFVNO44fd7bKKp0lQLs0ARrv4r1fMUqW2X543AUgFLhy2KKMPY99xpFgh9a7j0iCbSGyUu%2FhcRm5sw9%2Byfo34lVYne4%2FFR30p3LuHwufJ7FxfIREfquk63pvSlHLcLPe0Kjzzleo2k1XRPVFF%2B535IaHXuToXDnvSihKwGRvXtuMkagAFeAve5ns3ex0wAOSBalZvD0klFJxbg3effir9a7vEkZDzJOXxAlR7yiQXj1lcSxPnrCbdziYangfTPvr46KuhWuTkdD5%2FUwDg56Lz6r2Wg3faIFmc0r%2FWequSF52z0R8QmwZKpkPR2Sfd4RTWk4DjG5fjgsLVIVYs8GjhiY5uBySAA&x-oss-access-key-id=STS.NUv7oZ3srR1bYqokLPcDgFTRi&x-oss-expires=1717268108&x-oss-process=video%2Fsnapshot%2Ct_1000%2Cf_jpg%2Car_auto%2Cw_256&x-oss-signature=T%2B6BUXZy%2BJ4PhXJYxxsl3o9UipOJ%2BwaGn1SfzVganmw%3D&x-oss-signature-version=OSS2
from rod.
Related Issues (20)
- launcher docker 是否支持常驻且直接连接 HOT 4
- Examples fail on Ubuntu 24.04 with "No usable sandbox!" HOT 8
- Open the webpage, the page size is changed HOT 1
- Limit the number of redirects and store all accessed URLs HOT 1
- NewUserMode 无法关闭浏览器 HOT 1
- NewUserMode 和 NewManaged 可以同时使用吗 HOT 1
- How do I get Ifram without using--disable-site-isolation-trials? HOT 4
- navigation failed: net::ERR_TUNNEL_CONNECTION_FAILED HOT 1
- using "MustPage" to open the page will simulate a device by default HOT 2
- Any idea about cdp hang at syscall? HOT 1
- Panic on .HTML() HOT 8
- 找不到元素,点击按钮超时等问题 HOT 3
- 总是“panic: context deadline exceeded”,任何一步都有可能 HOT 2
- MustWaitStable 在特定网站会始终阻塞 HOT 2
- how can I set up a proxy for this request after hijacking the domain name? HOT 2
- Trying to get elements from a date input in a google form, but the element cannot be retrieved HOT 1
- Should I Install Chrome or Chromium In AWS For Production On Linux? HOT 4
- Support connecting to BrightData Scraping Browsers HOT 4
- Why the default UserAgent's platform is "Intel Mac" even in windows? HOT 1
- Get cookies json unmarshal error `page.MustCookies()` HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from rod.