chromedp / pdlgen

Generates templated code using Chrome DevTools PDL definitions.

About chromedp

Package chromedp is a faster, simpler way to drive browsers supporting the Chrome DevTools Protocol in Go without external dependencies.

Installing

Install in the usual Go way:

$ go get -u github.com/chromedp/chromedp

Examples

Refer to the Go reference for the documentation and examples. Additionally, the examples repository contains more examples on complex actions, and other common high-level tasks such as taking full page screenshots.

Frequently Asked Questions

I can't see any Chrome browser window

By default, Chrome is run in headless mode. See DefaultExecAllocatorOptions, and an example to override the default options.

I'm seeing "context canceled" errors

When the connection to the browser is lost, chromedp cancels the context, and it may result in this error. This occurs, for example, if the browser is closed manually, or if the browser process has been killed or otherwise terminated.

Chrome exits as soon as my Go program finishes

On Linux, chromedp is configured to avoid leaking resources by force-killing any started Chrome child processes. If you need to launch a long-running Chrome instance, manually start Chrome and connect using RemoteAllocator.

Executing an action without Run results in "invalid context"

By default, a chromedp context does not have an executor, however one can be specified manually if necessary; see issue #326 for an example.

I can't use an Action with Run because it returns many values

Wrap it with an ActionFunc:

ctx, cancel := chromedp.NewContext(context.Background())
defer cancel()
chromedp.Run(ctx, chromedp.ActionFunc(func(ctx context.Context) error {
	_, err := domain.SomeAction().Do(ctx)
	return err
}))

I want to use chromedp on a headless environment

The simplest way is to run the Go program that uses chromedp inside the chromedp/headless-shell image. That image contains headless-shell, a smaller headless build of Chrome, which chromedp is able to find out of the box.

Resources

• headless-shell - A build of headless-shell that is used for testing chromedp
• chromedp: A New Way to Drive the Web - GopherCon SG 2017 talk
• Chrome DevTools Protocol - Chrome DevTools Protocol reference
• chromedp examples - More complicated examples for chromedp
• github.com/chromedp/cdproto - Go reference for the generated Chrome DevTools Protocol API
• github.com/chromedp/pdlgen - tool used to generate cdproto
• github.com/chromedp/chromedp-proxy - a simple CDP proxy for logging CDP clients and browsers

pdlgen's Issues

Page.setBypassCSP missing?

See https://chromedevtools.github.io/devtools-protocol/tot/Page/#method-setBypassCSP

easyjson generated marshalling methods use value receiver on type containing mutex

Hi! This might ultimately be an easyjson bug, but running go vet on the generated cdproto package showed issues of mutex copies in the cdp sub-package.

Seen on latest master bfeac1f

$ go vet ./...
# github.com/chromedp/cdproto/cdp
cdp/easyjson.go:353:81: easyjsonC5a4559bEncodeGithubComChromedpCdprotoCdp1 passes lock by value: github.com/chromedp/cdproto/cdp.Node
cdp/easyjson.go:562:9: MarshalJSON passes lock by value: github.com/chromedp/cdproto/cdp.Node
cdp/easyjson.go:564:57: call of easyjsonC5a4559bEncodeGithubComChromedpCdprotoCdp1 copies lock value: github.com/chromedp/cdproto/cdp.Node
cdp/easyjson.go:569:9: MarshalEasyJSON passes lock by value: github.com/chromedp/cdproto/cdp.Node
cdp/easyjson.go:570:56: call of easyjsonC5a4559bEncodeGithubComChromedpCdprotoCdp1 copies lock value: github.com/chromedp/cdproto/cdp.Node
cdp/easyjson.go:631:81: easyjsonC5a4559bEncodeGithubComChromedpCdprotoCdp2 passes lock by value: github.com/chromedp/cdproto/cdp.Frame
cdp/easyjson.go:684:9: MarshalJSON passes lock by value: github.com/chromedp/cdproto/cdp.Frame
cdp/easyjson.go:686:57: call of easyjsonC5a4559bEncodeGithubComChromedpCdprotoCdp2 copies lock value: github.com/chromedp/cdproto/cdp.Frame
cdp/easyjson.go:691:9: MarshalEasyJSON passes lock by value: github.com/chromedp/cdproto/cdp.Frame
cdp/easyjson.go:692:56: call of easyjsonC5a4559bEncodeGithubComChromedpCdprotoCdp2 copies lock value: github.com/chromedp/cdproto/cdp.Frame

I'm new to this project and don't quite understand how all the code is generated, but I'm happy keep digging if it's not a straight forward fix.

Fixup broken

https://github.com/chromedp/cdproto-gen/blob/master/fixup/fixup.go#L216-L220

see chromedp/chromedp#560 as reference

jsonUnmarshal fail from a jsonMarshal result of *network.Cookie

follow code will panic:

package main

import (
   	"github.com/chromedp/cdproto/network"
	"encoding/json"
)
func main() {
	b,err:=json.Marshal(&network.Cookie{})
	if err!=nil{
		panic(err)
	}
	var c *network.Cookie
	err=json.Unmarshal(b,&c)
	if err!=nil{
		panic(err)
	}
	return
}

same issue from chromedp/cdproto#7

Protocol definition files no longer available

The URL to download these files (as in the README and found from other sources) is no longer valid. Do you know an updated URL?

mmand

DispatchMouseEvent does not support mouseWheel along one axis

chromedp/chromedp#491

What versions are you running?

$ go list -m github.com/chromedp/chromedp
$ google-chrome --version
$ go version
github.com/chromedp/chromedp
Version 77.0.3865.90
go version go1.12.7 darwin/amd64

What did you do? Include clear steps.
I'm trying to implement the mouseWheel event,

p := input.DispatchMouseEvent(input.MouseWheel, 0, 0)
p = p.WithDeltaX(0)
p = p.WithDeltaY(float64(y))
err = p.Do(ctx)
without scrolling along the x-axis p.WithDeltaX(0).

What did you expect to see?
MouseWheel event triggered and webpage scrolls to y, without scrolling x.

What did you see instead?
Error 'deltaX' and 'deltaY' are expected for mouseWheel event (-32602)

Confusion?
Can we not scroll y without x?

type DispatchMouseEventParams struct {
	Type        MouseType                     `json:"type"`                  // Type of the mouse event.
	X           float64                       `json:"x"`                     // X coordinate of the event relative to the main frame's viewport in CSS pixels.
	Y           float64                       `json:"y"`                     // Y coordinate of the event relative to the main frame's viewport in CSS pixels. 0 refers to the top of the viewport and Y increases as it proceeds towards the bottom of the viewport.
	Modifiers   Modifier                      `json:"modifiers"`             // Bit field representing pressed modifier keys. Alt=1, Ctrl=2, Meta/Command=4, Shift=8 (default: 0).
	Timestamp   *TimeSinceEpoch               `json:"timestamp,omitempty"`   // Time at which the event occurred.
	Button      ButtonType                    `json:"button,omitempty"`      // Mouse button (default: "none").
	Buttons     int64                         `json:"buttons,omitempty"`     // A number indicating which buttons are pressed on the mouse when a mouse event is triggered. Left=1, Right=2, Middle=4, Back=8, Forward=16, None=0.
	ClickCount  int64                         `json:"clickCount,omitempty"`  // Number of times the mouse button was clicked (default: 0).
	DeltaX      float64                       `json:"deltaX,omitempty"`      // X delta in CSS pixels for mouse wheel event (default: 0).
	DeltaY      float64                       `json:"deltaY,omitempty"`      // Y delta in CSS pixels for mouse wheel event (default: 0).
	PointerType DispatchMouseEventPointerType `json:"pointerType,omitempty"` // Pointer type (default: "mouse").
}

During the encoding they are ignored, not sure if this is the part causing the error?

	if in.DeltaX != 0 {
		const prefix string = ",\"deltaX\":"
		out.RawString(prefix)
		out.Float64(float64(in.DeltaX))
	}
	if in.DeltaY != 0 {
		const prefix string = ",\"deltaY\":"
		out.RawString(prefix)
		out.Float64(float64(in.DeltaY))
	}

Failure with chromium/97.0.4673.1.pdl

panic: could not resolve type cdp.compatibilitymode in domain cdp

goroutine 1 [running]:
github.com/chromedp/cdproto-gen/gen/gotpl.ResolveRef(0xc00419b2b8, 0xc00183fe60, {0xc0047d7c00, 0x2d, 0xc00419b258})
        github.com/chromedp/cdproto-gen/gen/gotpl/util.go:143 +0x25d
github.com/chromedp/cdproto-gen/gen/gotpl.ResolveType(0xc004542b89, 0xc00183fe60, {0xc0047d7c00, 0x14034c6, 0xc000050900})
        github.com/chromedp/cdproto-gen/gen/gotpl/util.go:160 +0x1ff

TimeSinceEpoch is not correct

It seems that the time should multiply time.Millisecond (1e6) rather than time.Second (1e9)

The corresponding code in cdproto are https://github.com/chromedp/cdproto/blob/8501c74e8228308e0dbd4aedb1f24ea1c103dd6e/network/types.go#LL1027
and
https://github.com/chromedp/cdproto/blob/8501c74e8228308e0dbd4aedb1f24ea1c103dd6e/cdp/types.go#L742

The following code snapshot is what I tried

package main

import (
	"fmt"
	"time"
)

func main() {
	// a response time from cdp log
	t := 1.685446691349539e+12

	// correct time 2023-05-30 19:38:11.349539072 +0800 CST
	fmt.Println(time.Unix(0, int64(t*float64(time.Millisecond))))

	// wrong time 1677-09-21 08:12:43.145224192 +0800 CST
	fmt.Println(time.Unix(0, int64(t*float64(time.Second))))
}