GithubHelp home page GithubHelp logo

Comments (5)

github-actions avatar github-actions commented on July 20, 2024

Please add a valid Rod Version: v0.0.0 to your issue. Current version is v0.116.0

Please fix the format of your markdown:

2 MD031/blanks-around-fences Fenced code blocks should be surrounded by blank lines [Context: "```"]
2 MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]
6:1 MD033/no-inline-html Inline HTML [Element: img]
10 MD040/fenced-code-language Fenced code blocks should have a language specified [Context: "```"]

generated by check-issue

from rod.

hktalent avatar hktalent commented on July 20, 2024

尝试不加载这些资源,都将导致无法继续后续的,也不知道为什么

ext := filepath.Ext(ctx.Request.URL().Path)
		szT := util.Any2Str(ctx.Request.Req().Header["Content-Type"]) // stylesheet,font, image
		// 阻止请求 strings.Contains(szT, "stylesheet") ||
		if strings.Contains(szT, "stylesheet") || strings.Contains(szT, "font") || strings.Contains(szT, "image") || ext == ".png" || ext == ".jpg" || ext == ".jpeg" || ext == ".gif" || ext == ".css" || ext == ".woff" || ext == ".woff2" || ext == ".ttf" || ext == ".eot" {
			//ctx.Skip = true // ctx.Request.Req().Header.Set("My-Header", "test")
			//ctx.Response.Fail(proto.NetworkErrorReasonAborted)
			return false
		}

from rod.

ysmood avatar ysmood commented on July 20, 2024

抱歉,我看不太懂你描述

from rod.

hktalent avatar hktalent commented on July 20, 2024

1、MustAdd 添加了拦截器,我因为是做爬虫,所以想阻止图片、字体、css文件的加载,始终没有成功
2、我希望在页面加载完了、添加的拦截事件都运行完了得到一个事件,及时关闭page,似乎爬虫处理的页面在
page.MustNavigate(szUrl.Url).MustWaitLoad() 之后立刻关闭page的话,很多拦截器就无法得到运行
我当前是基于延时70秒,但是这不太友好

您可以尝试下爬虫:https://www.alipan.com/s/4iQjiU9KUmG
1、抓取"/adrive/v2/file/list_by_share" 的数据
3、阻止图片、字体、css文件的加载

我不确定:

wait := page.MustWaitNavigation()
	page.MustNavigate(szUrl.Url).MustWaitLoad()
	wait()
我不确定这里关闭page是否合适

from rod.

hktalent avatar hktalent commented on July 20, 2024

这个错误是由于爬虫处理过程中这文件返回的是图片,然后就这样了

router.MustAdd(k1, func(hj *rod.Hijack) {
					hj.OnError = func(err error) {
						log.Println("HijackRequests error", err, hj.Request.URL().String())
					}
......

HijackRequests error {-32602 Invalid parameters Failed to deserialize params.body - BINDINGS: binary value expected at position 64} https://cn-beijing-data.aliyundrive.net/pxcIoOce%2F951346711%2F65cf1bc23d2e20b1bf52475a955ec4e247b8049a%2F65cf1bc2c77c291d35294a6db78d47e7ee32c861?security-token=CAISvgJ1q6Ft5B2yfSjIr5bDfNXu3qxT5fOJW1fej0wFb8tLqZH5izz2IHhMf3NpBOkZvvQ1lGlU6%2Fcalq5rR4QAXlDfNRjYXxzCqFHPWZHInuDox55m4cTXNAr%2BIhr%2F29CoEIedZdjBe%2FCrRknZnytou9XTfimjWFrXWv%2Fgy%2BQQDLItUxK%2FcCBNCfpPOwJms7V6D3bKMuu3OROY6Qi5TmgQ41Uh1jgjtPzkkpfFtkGF1GeXkLFF%2B97DRbG%2FdNRpMZtFVNO44fd7bKKp0lQLs0ARrv4r1fMUqW2X543AUgFLhy2KKMPY99xpFgh9a7j0iCbSGyUu%2FhcRm5sw9%2Byfo34lVYne4%2FFR30p3LuHwufJ7FxfIREfquk63pvSlHLcLPe0Kjzzleo2k1XRPVFF%2B535IaHXuToXDnvSihKwGRvXtuMkagAFeAve5ns3ex0wAOSBalZvD0klFJxbg3effir9a7vEkZDzJOXxAlR7yiQXj1lcSxPnrCbdziYangfTPvr46KuhWuTkdD5%2FUwDg56Lz6r2Wg3faIFmc0r%2FWequSF52z0R8QmwZKpkPR2Sfd4RTWk4DjG5fjgsLVIVYs8GjhiY5uBySAA&x-oss-access-key-id=STS.NUv7oZ3srR1bYqokLPcDgFTRi&x-oss-expires=1717268108&x-oss-process=video%2Fsnapshot%2Ct_1000%2Cf_jpg%2Car_auto%2Cw_256&x-oss-signature=T%2B6BUXZy%2BJ4PhXJYxxsl3o9UipOJ%2BwaGn1SfzVganmw%3D&x-oss-signature-version=OSS2

from rod.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.