presence of an interface value always panics about gofuzz

[howardjohn]

Any update? This makes this infeasible to use for any testing using protobuf enums

[jackgene]

BTW go has built-in fuzzing now: https://go.dev/doc/fuzz/

It does, but the fuzzing it does is limited, and I still rely on gofuzz. For instance, since Go 1.18's fuzzer doesn't fuzz structs, I still do:

func FuzzWithStruct(f *testing.F) {
	f.Add([]byte("seed"))
	f.Fuzz(func(t *testing.T, data []byte) {
		// Set up
		fuzzer := fuzz.NewFromGoFuzz(data)
		testSet := map[PIIField]struct{}{}
		fuzzer.Fuzz(&testSet)

		t.Logf("Set: %+v", testSet)

		// Test
		_ = doThingWithSet(testSet)

		// Verify
		// The point of the fuzz is just to ensure method does not panic
	})
}

[lavalamp]

Hi, sorry, I must have missed this in my email-- can you not register a custom fuzzing function for an interface{} type? Alternatively, register one for the enclosing struct. Let me know if that doesn't work.

[lavalamp]

It's unfortunately not generally possible to know what makes sense for interfaces, functions, or channels. :(

The panic message is not very helpful, though--that should be improved.

[howardjohn]

type Test struct {
	ErrorType TestInterface
}

type TestInterface interface {
	Size() int
}

func TestFuzzE(t *testing.T) {
	x := Test{}
	f := fuzz.New().NilChance(.5).Funcs(
		func(e *TestInterface, c fuzz.Continue) {
		})
	f.Fuzz(&x)
	f1 := fuzz.New().NilChance(.5).Funcs(
		func(e *interface{}, c fuzz.Continue) {
		})
	f1.Fuzz(&x)
}

First one works, second one doesn't. It doesn't seem to get called as a custom function. I'll play around with it a bit more to see if I can get something to stick. Basically my plan was to have a function called for every type, and filter out the interfaces or similar

[lavalamp]

Yeah, I'd expect your second one to get called for this struct:

type Test struct {
	ErrorType interface{}
}

It kinda has to work this way, since only you know how to construct reasonable random implementations of your interface.

[lavalamp]

Or are you asking for a second lookup so you can set the behavior for custom interfaces without a specific custom fuzzing function?

[howardjohn]

Basically my understanding is right now the presence of any interface type leads to a panic. The only way to avoid this is to define a custom function (which does work).

Ideally I would not need to define custom functions as I have a lot of interfaces and I would rather be lazy 🙂. Instead I would expect to basically be able to skip any interface types. So instead of panicking, we would just do nothing to that field. Then if you do want to add custom logic for that type, you can define your own function. This could be a configurable option if that is not generally desirable?

[lavalamp]

A goal is to not give false confidence, by e.g. silently not fuzzing things. I guess it makes sense to have an opt-out. A second, fallback call like I just suggested doesn't actually make sense, since the interfaces are of different types.

What if the panic message listed the types that are missing fuzzing functions, in a form that you could copy-paste that into your test file with your custom fuzzing functions?

[howardjohn]

A helpful error message + some doc on resolving it seems reasonable to me

[lavalamp]

OK, great. I'm totally open to a PR fixing this. I probably won't have time to write one anytime soon, though.

[howardjohn]

I was thinking about

It kinda has to work this way, since only you know how to construct reasonable random implementations of your interface.

a bit more. One common case for this to occur is using protobuf oneof. I wonder if its feasible to implement awareness of protobuf and automatically handle these cases.

[lavalamp]

How could one programmatically identify that arrangement? E.g. is it represented in struct tags? I'm open to ideas.

[howardjohn]

	// Types that are valid to be assigned to HealthCheckMethod:
	//	*ReadinessProbe_HttpGet
	//	*ReadinessProbe_TcpSocket
	//	*ReadinessProbe_Exec
	HealthCheckMethod    isReadinessProbe_HealthCheckMethod `protobuf_oneof:"health_check_method"`

this is the top level struct, it has a protobuf_oneof tag.

Then there is a helper function:

func (*ReadinessProbe) XXX_OneofWrappers() []interface{} {
	return []interface{}{
		(*ReadinessProbe_HttpGet)(nil),
		(*ReadinessProbe_TcpSocket)(nil),
		(*ReadinessProbe_Exec)(nil),
	}
}

which seems to return all possible types

I am not sure how much these are meant to be implementation details though, it might be better to use https://godoc.org/google.golang.org/protobuf/reflect/protoreflect

[lavalamp]

gofuzz operates strictly on go types, so doing something proto specific doesn't seem feasible.

Given the setup you posted, I can imagine a fuzz function that operates on interface types, looks at the name of the type, somehow looks for a corresponding XXX_something() []interface{} method, calls it, picks one of the choices (or nil, according to the nil chance).

[tzachshabtay]

@lavalamp maybe I'm missing something, but I have the same issue with fuzzing proto oneof.
I can't do the suggested workaround to solve this, i.e I can't do a custom fuzz function on that interface because it is non-exported (i.e see the example posted above by @howardjohn , isReadinessProbe_HealthCheckMethod is private).

I tried declaring my own interface with the same shape but that didn't work.

Any ideas?

[freman]

I've just run into this without the protobuf problem, would a struct tag, say nofuzz, feasible?

[lavalamp]

BTW go has built-in fuzzing now: https://go.dev/doc/fuzz/

[lavalamp]

@jackgene you are right, that's more limited than I realized.

@freman Yeah if there's a better idea than struct tags I'm not seeing it. Not sure what to do about e.g. map[string]interface{} though.