proposal: Improve ergonomics of handling multiple events about cdp

[mafredri]

One way to improve this would be to not return an error when creating a client, just: frameNavigated := c.Page.FrameNavigated(ctx). I've entertained this possibility before but I dislike it for two reasons:

1. It does not instantly inform the user if the connection is already closed when they are trying to create the client
2. It might promote incorrect usage ev, err := c.Page.FrameNavigated(ctx).Recv() which would leak memory (leaves the event client open, buffering any future events of this kind)

For these two reasons I think it's best to find alternate solutions (like above). It's a bit unfortunate, since not returning the error would allow us to write the above as:

events := []cdp.Event{
	c.Page.FrameNavigated(ctx),
	c.Domain.SomeOther(ctx),
}

Which doesn't introduce an alternate way to reference the events.

[mafredri]

The more I think about this issue, perhaps the right API for this kind of functionality is a cdp.RecvAll(ctx, c) (name and signature TBD).

This would create a client that automatically subscribes to all possible events.

We could introduce state into cdp.Client so that only domains that have been enabled (via client) are subscribed to. Disable would unsubscribe. Don't know if this is useful or not. This might warrant a (*cdp.Client).Close() method. If so, does it also close *rpcc.Conn?

[matthewmueller]

So far this hasn't been a big issue for me. Despite starting out with a bunch of events, I generally only use maybe 2-4 now.

The copying and pasting is a little annoying, but I've been sticking them in goroutines to clean it up a bit. To me, that's been a little nicer than using a switch statement.

One nice thing about something like cdp.RecvAll(ctx, c) would be for logging / debugging though

[mafredri]

Thanks for the input!

The copying and pasting is a little annoying, but I've been sticking them in goroutines to clean it up a bit. To me, that's been a little nicer than using a switch statement.

I can relate to the annoying part, but I'm glad you like it anyway! One reason for this proposal is use cases that aren't possible with the current API, for example, syncing a local representation of the DOM, consider the following:

switch ev := ev.(type) {
case *dom.AttributeModifiedReply:
	data.updateAttribute(ev.NodeID, ev.Name, ev.Value)
case *dom.AttributeRemovedReply:
	data.removeAttribute(ev.NodeID, ev.Name)
case *dom.ChildNodeCountUpdatedReply:
	data.updateNodeCount(ev.NodeID, ev.ChildNodeCount)
case *dom.ChildNodeInsertedReply:
	data.insertNode(ev.ParentNodeID, ev.PreviousNodeID, ev.Node)
case *dom.ChildNodeRemovedReply:
	data.removeNode(ev.ParentNodeID, ev.NodeID)
}

With the proposed API we can be sure that all events arrive in the correct order, allowing us to keep an accurate representation as long as we handle all events. This also reduces the number of goroutines we need to have running (1 in this case vs at least 5 by launching a goroutine for each client). This also allows us to keep all the syncing logic in one place.

One nice thing about something like cdp.RecvAll(ctx, c) would be for logging / debugging though

That's a good point. On that note, did you know there's a logging example in the repo/godoc? It's a bit verbose (custom codec) but quite flexible :-).

I think a potentially valid option (in addition to cdp.RecvAll) would be cdp.Merge, it could look like this:

attributeModified, err := c.DOM.AttributeModified(ctx)
if err != nil {
	return err
}
attributeRemoved, err := c.DOM.AttributeRemoved(ctx)
if err != nil {
	return err
}
mergedClient := cdp.Merge(attributeModified, attributeRemoved)
mergedClient.Recv() // (interface{}, error)

[mafredri]

I believe I finally found a satisfactory (alternative) solution to merging events. It avoids the need to switch on event types entirely.

It doesn't solve the sync problem though, but I think that could be solvable via an option passed to the client (e.g. c.DOM.AttributeModified(ctx, syncOption)).

Here's what the alternative solution looks like:

attributeModified, err := c.DOM.AttributeModified(ctx)
if err != nil {
	return err
}
attributeRemoved, err := c.DOM.AttributeRemoved(ctx)
if err != nil {
	return err
}

go func() {
	defer func() {
		attributeModified.Close()
		attributeRemoved.Close()
	}

	select {
		case <-attributeModified.Ready():
			ev, err := attributeModified.Recv() // Does not block.
			if err != nil {
				// Handle error.
			}
			_ = ev
		case <-attributeRemoved.Ready():
			// attributeRemoved.Recv() ...
		case <-timeout:
			log.Println("timed out")
			return
	}
}

[mafredri]

Regarding synchronization, I think I have it down to two alternatives (although I'm open to better ideas!)

1. Create event clients like before, then call cdp.Sync(client1, client2, client3), the clients are now synced
   • cpd.Sync could return a function unsync()
   • cdp.Sync discards all previous (unread) events on client(s)
   • + Hard to use incorrectly
2. Create a new option syncOpt := cdp.Sync() and use it when creating clients: c.DOM.AttributeModified(ctx, syncOpt)
   • + Can be a bit smarter when creating the client (resource wise, quite insignificant though)
   • - Same sync option must be used for all clients to be synced
   • - Might encourage bad usage: c.DOM.AttributeModified(ctx, cdp.Sync()) will not work as intended

In both cases, all calls to RecvMsg will block unless the next message belongs to that event client.

Sync behavior example:

• client1 and client2 receive events (in that order)
• client1 Ready-channel closes
• Calls to RecvMsg will block for both client2 and client3, but not client1
• client1.RecvMsg is called
• client2 Ready-channel closes
• Calls to RecvMsg will block for both client1 and client3, but not client2
• client2.RecvMsg is called
• Now all will block again (no events pending)