knqyf263 / go-plugin Goto Github PK

Since the *.proto files can change, as well as the logic of the plugin, the files that were generated at the previous stage according to the template must be deleted beforehand.

Template for removing: <package>*.pb.go

I just updated my code snippets to latest go-plugin and wazero, and I start seeing this message in host stdout:

panic: free: invalid pointer

It seems it does not affect the logic, just says that something bad (?) happened?

It's with "Cat" example which outputs the contents of a host file by reading it from guest.

I suspect that's because of this change to autogenerated stubs:

	// We don't need the memory after deserialization: make sure it is freed.
	if resPtr != 0 {
		defer p.free.Call(ctx, uint64(resPtr))
	}

https://github.com/inkeliz/karmem

Supports many languages and fast .

not say protocol buffers are not nice, but just wanted to let you know about another approach.

there is no tie into Wazero in that repo.

I have five methods defined in the .proto file, and only one of them works when calling them in the host. I tried to find out what happened, but it seems like after calling from the host, the default result struct is returned without value.

here are the proto files and generated codes: https://github.com/wshops/wshop-plugin-common

host code

func TestPluginFunction(t *testing.T) {
	ctx := context.Background()
	p, err := pcaptcha.NewCaptchaPlugin(ctx, pcaptcha.CaptchaPluginOption{
		Stdout: os.Stdout,
		Stderr: os.Stderr,
	})
	if err != nil {
		t.Error(err)
	}

	pluginInstance, err := p.Load(ctx, "./plugins/captcha/hcaptcha.wasm")
	if err != nil {
		t.Error(err)
	}

	pi, err := pluginInstance.GetPluginInfo(ctx, emptypb.Empty{})
	if err != nil {
		t.Error(err)
	}
	fmt.Println("plugin version: ", pi.GetVersion())

	res, err := pluginInstance.VerifyCaptcha(ctx, pcaptcha.VerifyCaptchaRequest{Captcha: "123"})
	if err != nil {
		t.Error(err)
	}
	fmt.Println("verify result: ", res.GetIsValid())

	htmlInput, err := pluginInstance.GetCustomHtmlInputField(ctx, emptypb.Empty{})
	if err != nil {
		t.Error(err)
	}
	fmt.Println("input html: ", htmlInput.GetHtml())

	htmlHead, err := pluginInstance.GetCustomHtmlHead(ctx, emptypb.Empty{})
	if err != nil {
		t.Error(err)
	}
	fmt.Println("head html: ", htmlHead.GetHtml())

	htmlBody, err := pluginInstance.GetCustomHtmlBodyEnd(ctx, emptypb.Empty{})
	if err != nil {
		t.Error(err)
	}
	fmt.Println("body html: ", htmlBody.GetHtml())

	t.Cleanup(func() {
		p.Close(ctx)
	})
}

result:

=== RUN   TestPluginFunction
plugin version:  
validating captcha...
verify result:  true
input html:  
head html:  
body html:  

version
go: go1.19.2
go-plugin: v0.3.0
sys: MacOS 13.0.1
chip: Apple M1 Max

wazero Memory is not using contexts anymore. tetratelabs/wazero@126bd90

I just have tried to compile with go tool, since support for wasip1 added in go1.21, and got this error:

$ GOOS=wasip1 GOARCH=wasm go build -tags=tinygo.wasm -o wasm/plugin.wasm ./wasm
package gitlab.com/***/test-wasm-plugin/wasm
        imports gitlab.com/***/test-wasm-plugin/api
        imports github.com/knqyf263/go-plugin/wasm: build constraints exclude all Go files in C:\Users\***\go\pkg\mod\github.com\knqyf263\[email protected]\wasm

Note that I'm passing -tags=tinygo.wasm, because my plugin code is also protected with this build constraint. However, error message sounds like if go-plugin is not intended for use with official go compiler at all?

It would be nice to add a linter in CI pipeline with basics golangci-lint run (on current stage) setup

A really great feature would be able to load a plugin from binary, instead of from filepath. This would be done by simply adding a method similar to Load on the generated plugin (and probably refactoring Load as well). Here's what it would look like in the example given in the README:

func (p *GreeterPlugin) Load(ctx context.Context, pluginPath string) (greeter, error) {
	b, err := os.ReadFile(pluginPath)
	if err != nil {
		return nil, err
	}

	return p.LoadBinary(ctx, b)
}

func (p *GreeterPlugin) LoadBinary(ctx context.Context, pluginBinary []byte) (greeter, error) {
	// Create a new runtime so that multiple modules will not conflict
	r, err := p.newRuntime(ctx)
	if err != nil {
		return nil, err
	}

	// Compile the WebAssembly module using the default configuration.
	code, err := r.CompileModule(ctx, pluginBinary)
	if err != nil {
		return nil, err
	}

	// InstantiateModule runs the "_start" function, WASI's "main".
	module, err := r.InstantiateModule(ctx, code, p.moduleConfig)
	if err != nil {
		// Note: Most compilers do not exit the module after running "_start",
		// unless there was an Error. This allows you to call exported functions.
		if exitErr, ok := err.(*sys.ExitError); ok && exitErr.ExitCode() != 0 {
			return nil, fmt.Errorf("unexpected exit_code: %d", exitErr.ExitCode())
		} else if !ok {
			return nil, err
		}
	}

	// Compare API versions with the loading plugin
	apiVersion := module.ExportedFunction("greeter_api_version")
	if apiVersion == nil {
		return nil, errors.New("greeter_api_version is not exported")
	}
	results, err := apiVersion.Call(ctx)
	if err != nil {
		return nil, err
	} else if len(results) != 1 {
		return nil, errors.New("invalid greeter_api_version signature")
	}
	if results[0] != GreeterPluginAPIVersion {
		return nil, fmt.Errorf("API version mismatch, host: %d, plugin: %d", GreeterPluginAPIVersion, results[0])
	}

	sayhello := module.ExportedFunction("greeter_say_hello")
	if sayhello == nil {
		return nil, errors.New("greeter_say_hello is not exported")
	}

	malloc := module.ExportedFunction("malloc")
	if malloc == nil {
		return nil, errors.New("malloc is not exported")
	}

	free := module.ExportedFunction("free")
	if free == nil {
		return nil, errors.New("free is not exported")
	}
	return &greeterPlugin{
		runtime:  r,
		module:   module,
		malloc:   malloc,
		free:     free,
		sayhello: sayhello,
	}, nil
}

I'd be more than happy to submit a PR for this if you all would be ok with it.

Hi,
I have

// The host functions embedded into the plugin
// go:plugin type=host
service HostService {
  rpc Log(LogRequest) returns (google.protobuf.Empty) {}
}
message LogRequest {
  string message = 1;
}

and on compilation with

	protoc -I pkg/plugins/interop/service --go-plugin_out=pkg/plugins/interop --go-plugin_opt=paths=source_relative pkg/plugins/interop/service/interop.proto

and generating my plugin with

	tinygo build -o pkg/plugins/examples/basic.wasm -scheduler=none -target=wasi --no-debug pkg/plugins/examples/basic.go

I then have in my plugin

func main() {
	interop.RegisterPluginService(MyPlugin{})
}

type MyPlugin struct{}

func (m MyPlugin) SendMessage(ctx context.Context, request *interop.DataMessage) (*interop.DataMessage, error) {

	hostFunctions := interop.NewHostService()
	hostFunctions.Log(ctx, &interop.LogRequest{
		Message: "Sending a log...",
	})

where SendMessage is one of the plugin functions that the host calls, I then want to call the logger on the host as you can see.

In my host I have

func (PluginFunctions) Log(ctx context.Context, request *interop.LogRequest) (*emptypb.Empty, error) {
	// Use the host logger
	log.Println("logging ", request.GetMessage())
	return &emptypb.Empty{}, nil
}

where the plugin was registered with

		if this, err := p.Load(ctx, path, PluginFunctions{}); err != nil {

now, If I remove in the plugin the `hostFunctions.Log` call everything works as expected. However when I add this back and compile the plugin I get

tinygo build -o pkg/plugins/examples/basic.wasm -scheduler=none -target=wasi --no-debug pkg/plugins/examples/basic.go
tinygo:wasm-ld: warning: function signature mismatch: log

defined as (i32, i32) -> i64 in lto.tmp
defined as (f64) -> f64 in /opt/homebrew/Cellar/tinygo/0.30.0/lib/wasi-libc/sysroot/lib/wasm32-wasi/libc.a(log.o)

its a warning, however when i run the host I get

2024/01/11 12:04:49 error starting plugin service wasm error: unreachable
wasm stack trace:
.(main.MyPlugin).SendMessage(i32)
.plugin_service_send_message(i32,i32) i64

- this is hard to debug. Could it be the version of wasm is wasm32 and for some reason the host function needs 64 or something? 
thanks

Is it possible to get plugin id before a plugin is loaded?
I have to ensure the plugin that have the same id (but maybe different version) only exists one instance in the memory.

There are two alternative solutions:

• I can define a method, and call the method to get plugin's id, but it's not a good design.
• I can packed the metadata as json with the wasm file into a zip, and read the metadata file before load the plugin. However, go-plugin not support load plugin from memory, so I must extract the wasm file to somewhere before I can load it

In the example here it is outlined that the host can pass an embedded FS to the plugin and the plugin will only be able to access content within that embedded filesystem.

Is it possible to "pass" a filesystem the other way - i.e from plugin to host?

I have embedded some assets in the plugin with embed.FS and I want the host to be able to access those. Is there a way to expose the FS to the host?

Running the host-functions example I get the following error:

2022/11/12 14:42:05 Sending a HTTP request...
2022/11/12 14:42:05 wasm error: unreachable
wasm stack trace:
	.runtime.runtimePanic(i32,i32)
	.runtime.lookupPanic()
	.malloc(i32) i32 (recovered by wazero)
wasm stack trace:
	env.log(i32,i32) i64
	.greeter_greet(i32,i32) i64

• go version go1.18.7 darwin/arm64
• macOS 12.6.1
• Chip M1 Max

I noticed that gen-go-pugin generates code which uses values, so the code looks like:

// The greeting service definition.
// go:plugin type=plugin version=1
type Greeter interface {
	SayHello(context.Context, GreetReq) (GreetResp, error)
}

While gRPC plugin generates this code from the same *.proto definition:

// GreeterClient is the client API for Greeter service.
//
// For semantics around ctx use and closing/ending streaming RPCs, please refer to https://pkg.go.dev/google.golang.org/grpc/?tab=doc#ClientConn.NewStream.
type GreeterClient interface {
	SayHello(ctx context.Context, in *GreetReq, opts ...grpc.CallOption) (*GreetResp, error)
}

This also makes gopls complain about copying of sync.Mutex:

If it's by design, I would like to learn more about it.

I have a list of web assembly modules written in go that provide extensibility to my program.

I don't want to compile the plug-in each time i use it though because the program is a long running multi goroutine application where 15ms+ for plugin loading each time would be very bad.

Ideally I'd like the ability to compile module beforehand and use it later by instantiating new instances per goroutine.

I am sending data from the host to the plugin and i'm getting a huge crash every so often - after about 4-5 'sends' of data but can be 1 or 2 sometimes.

Anything glaringly obvious?

I have

  rpc PluginEvent(DataMessage) returns (google.protobuf.Empty) {}

in my service (.proto)
and I have a function that is creating a listener for each of my plugins, so that they will fire on an event meant for them.

	for _, v := range a.pluginManager.Plugins {
		eventName := fmt.Sprintf("plugin_frontend_event_%s", v.Info.PluginId)
		runtime.EventsOn(ctx, eventName, func(optionalData ...interface{}) {
			///codes,,,
			byt, err := json.Marshal(optionalData)
			if err != nil {
				fmt.Println("error marhsalling data ", err)
				return
			}
			var receivedDataMessages []interop.DataMessage
			if err := json.Unmarshal(byt, &receivedDataMessages); err == nil {
				fmt.Printf("optional data from frontend %d - %+v", len(receivedDataMessages), receivedDataMessages)
			} else {
				fmt.Println("error unmarshal ", err)
			}
			_, err = v.PluginEvent(ctx, &receivedDataMessages[0])

Where i call PluginEvent is where the crash ultimately happens. I am not sure if its possible to get debugging logs from a plugin at this point to see if anything happened in there:
(thx)

unexpected fault address 0xfdd8
fatal error: fault
[signal SIGSEGV: segmentation violation code=0x2 addr=0xfdd8 pc=0xfdd8]

goroutine 212 gp=0x140005d0a80 m=14 mp=0x14000680008 [running]:
runtime.throw({0x1050f05df?, 0x140001d05b0?})
        /usr/local/go/src/runtime/panic.go:1023 +0x40 fp=0x140002cf3d0 sp=0x140002cf3a0 pc=0x10434b4a0
runtime.sigpanic()
        /usr/local/go/src/runtime/signal_unix.go:895 +0x198 fp=0x140002cf400 sp=0x140002cf3d0 pc=0x104362d08
github.com/tetratelabs/wazero/internal/engine/compiler.(*callEngine).execWasmFunction(0x14000268c60, {0x10559ff90, 0x1400031b320}, 0x140001d0500)
        /Users/user/go/pkg/mod/github.com/tetratelabs/[email protected]/internal/engine/compiler/engine.go:1043 +0x78 fp=0x140002cf580 sp=0x140002cf410 pc=0x104b321f8
github.com/tetratelabs/wazero/internal/engine/compiler.(*callEngine).call(0x14000268c60, {0x10559ff90, 0x1400031b320}, {0x14000129540, 0x2, 0x2}, {0x0, 0x0, 0x0})
        /Users/user/go/pkg/mod/github.com/tetratelabs/[email protected]/internal/engine/compiler/engine.go:784 +0x2f0 fp=0x140002cf710 sp=0x140002cf580 pc=0x104b30d60
github.com/tetratelabs/wazero/internal/engine/compiler.(*callEngine).Call(0x14000268c60, {0x10559ff90, 0x1400031b320}, {0x14000129540, 0x2, 0x2})
        /Users/user/go/pkg/mod/github.com/tetratelabs/[email protected]/internal/engine/compiler/engine.go:736 +0x1c8 fp=0x140002cf850 sp=0x140002cf710 pc=0x104b307b8
github.com/user/gui/pkg/plugins/interop.(*pluginServicePlugin).PluginEvent(0x140001bc280, {0x10559ff90, 0x1400031b320}, 0x140007e2480)
        /Users/user/go/src/github.com/user/gui/pkg/plugins/interop/interop_host.pb.go:546 +0x5e4 fp=0x140002cfce0 sp=0x140002cf850 pc=0x104c16564
main.(*Model).startup.func1({0x14000a1a370, 0x1, 0x1})
        /Users/user/go/src/github.com/user/gui/model.go:79 +0x61c fp=0x140002cff80 sp=0x140002cfce0 pc=0x1050dc74c
github.com/wailsapp/wails/v2/internal/frontend/runtime.(*Events).notifyBackend.gowrap2()
        /Users/user/go/pkg/mod/github.com/wailsapp/wails/[email protected]/internal/frontend/runtime/events.go:137 +0x4c fp=0x140002cffd0 sp=0x140002cff80 pc=0x1048da4bc
runtime.goexit({})
        /usr/local/go/src/runtime/asm_arm64.s:1222 +0x4 fp=0x140002cffd0 sp=0x140002cffd0 pc=0x104380484
created by github.com/wailsapp/wails/v2/internal/frontend/runtime.(*Events).notifyBackend in goroutine 107
        /Users/user/go/pkg/mod/github.com/wailsapp/wails/[email protected]/internal/frontend/runtime/events.go:137 +0x318

goroutine 1 gp=0x140000021c0 m=0 mp=0x105dc9080 [syscall, locked to thread]:
runtime.cgocall(0x1050df608, 0x140006d7d18)
        /usr/local/go/src/runtime/cgocall.go:157 +0x44 fp=0x140006d7ce0 sp=0x140006d7ca0 pc=0x104315844
github.com/wailsapp/wails/v2/internal/frontend/desktop/darwin._Cfunc_RunMainLoop()
        _cgo_gotypes.go:568 +0x30 fp=0x140006d7d10 sp=0x140006d7ce0 pc=0x1048dc390
github.com/wailsapp/wails/v2/internal/frontend/desktop/darwin.(*Frontend).RunMainLoop(0x1400032a300)
        /Users/user/go/pkg/mod/github.com/wailsapp/wails/[email protected]/internal/frontend/desktop/darwin/frontend.go:69 +0x20 fp=0x140006d7d20 sp=0x140006d7d10 pc=0x1048dfa70
github.com/wailsapp/wails/v2/internal/frontend/devserver.(*DevWebServer).RunMainLoop(0x14000342a20)
        <autogenerated>:1 +0x3c fp=0x140006d7d40 sp=0x140006d7d20 pc=0x1049918dc
github.com/wailsapp/wails/v2/internal/app.(*App).Run(0x1400032e5a0)
        /Users/user/go/pkg/mod/github.com/wailsapp/wails/[email protected]/internal/app/app_dev.go:33 +0x78 fp=0x140006d7db0 sp=0x140006d7d40 pc=0x10499aee8
github.com/wailsapp/wails/v2/pkg/application.(*Application).Run(0x140002508c0)
        /Users/user/go/pkg/mod/github.com/wailsapp/wails/[email protected]/pkg/application/application.go:73 +0x17c fp=0x140006d7e50 sp=0x140006d7db0 pc=0x10499e05c
github.com/wailsapp/wails/v2.Run(0x140001b31e0)
        /Users/user/go/pkg/mod/github.com/wailsapp/wails/[email protected]/wails.go:14 +0x30 fp=0x140006d7e90 sp=0x140006d7e50 pc=0x10499e250
main.main()
        /Users/user/go/src/github.com/user/gui/main.go:18 +0x3d4 fp=0x140006d7f60 sp=0x140006d7e90 pc=0x1050db924
runtime.main()
        /usr/local/go/src/runtime/proc.go:271 +0x228 fp=0x140006d7fd0 sp=0x140006d7f60 pc=0x10434dc28
runtime.goexit({})
        /usr/local/go/src/runtime/asm_arm64.s:1222 +0x4 fp=0x140006d7fd0 sp=0x140006d7fd0 pc=0x104380484

goroutine 2 gp=0x14000002c40 m=nil [force gc (idle)]:
runtime.gopark(0x10558b528, 0x105dc5950, 0x11, 0xa, 0x1)
        /usr/local/go/src/runtime/proc.go:402 +0xe0 fp=0x14000078f70 sp=0x14000078f40 pc=0x10434e080
runtime.goparkunlock(0x105dc5950?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:408 +0x34 fp=0x14000078fa0 sp=0x14000078f70 pc=0x10434e114
runtime.forcegchelper()
        /usr/local/go/src/runtime/proc.go:326 +0xb4 fp=0x14000078fd0 sp=0x14000078fa0 pc=0x10434dea4
runtime.goexit({})
        /usr/local/go/src/runtime/asm_arm64.s:1222 +0x4 fp=0x14000078fd0 sp=0x14000078fd0 pc=0x104380484
created by runtime.init.6 in goroutine 1
        /usr/local/go/src/runtime/proc.go:314 +0x24

goroutine 18 gp=0x14000104380 m=nil [GC sweep wait]:
runtime.gopark(0x10558b528, 0x105dc6760, 0xc, 0x9, 0x1)
        /usr/local/go/src/runtime/proc.go:402 +0xe0 fp=0x14000074740 sp=0x14000074710 pc=0x10434e080
runtime.goparkunlock(0x105dc6760?, 0x0?, 0x0?, 0x0?)
        /usr/local/go/src/runtime/proc.go:408 +0x34 fp=0x14000074770 sp=0x14000074740 pc=0x10434e114
runtime.bgsweep(0x14000110000)
        /usr/local/go/src/runtime/mgcsweep.go:318 +0xdc fp=0x140000747b0 sp=0x14000074770 pc=0x104338abc
runtime.gcenable.gowrap1()
        /usr/local/go/src/runtime/mgc.go:203 +0x28 fp=0x140000747d0 sp=0x140000747b0 pc=0x10432d428
runtime.goexit({})
        /usr/local/go/src/runtime/asm_arm64.s:1222 +0x4 fp=0x140000747d0 sp=0x140000747d0 pc=0x104380484
created by runtime.gcenable in goroutine 1
        /usr/local/go/src/runtime/mgc.go:203 +0x6c

goroutine 19 gp=0x14000104540 m=nil [GC scavenge wait]:
runtime.gopark(0x10558b528, 0x105dc77e0, 0xd, 0xa, 0x2)
        /usr/local/go/src/runtime/proc.go:402 +0xe0 fp=0x14000074f20 sp=0x14000074ef0 pc=0x10434e080
runtime.goparkunlock(0x105dc77e0?, 0x68?, 0x1?, 0x0?)
        /usr/local/go/src/runtime/proc.go:408 +0x34 fp=0x14000074f50 sp=0x14000074f20 pc=0x10434e114
runtime.(*scavengerState).park(0x105dc77e0)
        /usr/local/go/src/runtime/mgcscavenge.go:425 +0x4c fp=0x14000074f80 sp=0x14000074f50 pc=0x1043360cc
runtime.bgscavenge(0x14000110000)
        /usr/local/go/src/runtime/mgcscavenge.go:658 +0x60 fp=0x14000074fb0 sp=0x14000074f80 pc=0x104336620
runtime.gcenable.gowrap2()
        /usr/local/go/src/runtime/mgc.go:204 +0x28 fp=0x14000074fd0 sp=0x14000074fb0 pc=0x10432d3c8
runtime.goexit({})
        /usr/local/go/src/runtime/asm_arm64.s:1222 +0x4 fp=0x14000074fd0 sp=0x14000074fd0 pc=0x104380484
created by runtime.gcenable in goroutine 1
        /usr/local/go/src/runtime/mgc.go:204 +0xac

goroutine 20 gp=0x14000104a80 m=nil [finalizer wait]:
runtime.gopark(0x10558b278, 0x105e2ed28, 0x10, 0xa, 0x1)
        /usr/local/go/src/runtime/proc.go:402 +0xe0 fp=0x14000078590 sp=0x14000078560 pc=0x10434e080
runtime.runfinq()
        /usr/local/go/src/runtime/mfinal.go:194 +0xf0 fp=0x140000787d0 sp=0x14000078590 pc=0x10432c5c0
runtime.goexit({})
        /usr/local/go/src/runtime/asm_arm64.s:1222 +0x4 fp=0x140000787d0 sp=0x140000787d0 pc=0x104380484
created by runtime.createfing in goroutine 1
        /usr/local/go/src/runtime/mfinal.go:164 +0x4c

When a plugin function returns an error, it seems like that error never makes it's way back to the host.

Version: f7d9444
Reproducible Test: https://github.com/cchamplin/plugin-bug

Host

	_, err = myPlugin.PassDataError(context.Background(), plugin.PassDataRequest{})
	if err == nil {
		t.Error("expected error")
		return
	}

Plugin

func (p TestPlugin) PassDataError(ctx context.Context, request host.PassDataRequest) (result host.PassDataReply, err error) {
	return host.PassDataReply{}, fmt.Errorf("an error from the plugin")
}

Expected: Test to pass
Actual:

=== RUN   TestPlugin_Error
    plugin_test.go:184: expected error
--- FAIL: TestPlugin_Error (0.10s)

FAIL

Js runtimes like node, deno, bun

So they are the client and get strongly typed code to call in.

I posted an issue in tinygo repo, as I believe it's related to compiler, at least when it comes to Windows. However I'm not sure about Linux.

So I just cross-post original issue here as you guys may know better the reason which causing it:

I have some code using knqyf263/go-plugin which compiles with no problem on both Windows and Linux platforms using tinygo v0.28.1. However it stopped working after upgrade to v0.29.0:

On Windows:

task: [plugin] tinygo build -o wasm/plugin.wasm -scheduler=none -target=wasi --no-debug ./wasm
# os
c:\tinygo\src\os\stat_linuxlike.go:18:6: fillFileStatFromSys redeclared in this block
c:\tinygo\src\os\stat_linux.go:14:6:    other declaration of fillFileStatFromSys
c:\tinygo\src\os\stat_linuxlike.go:50:6: timespecToTime redeclared in this block
c:\tinygo\src\os\stat_linux.go:46:6:    other declaration of timespecToTime
c:\tinygo\src\os\stat_linuxlike.go:55:6: atime redeclared in this block
c:\tinygo\src\os\stat_linux.go:51:6:    other declaration of atime
exit status 1~~

On Linux:

task: [plugin] tinygo build -o wasm/plugin.wasm -scheduler=none -target=wasi --no-debug ./wasm
tinygo:wasm-ld: warning: function signature mismatch: log
>>> defined as (i32, i32) -> i64 in lto.tmp
>>> defined as (f64) -> f64 in /usr/local/lib/tinygo/lib/wasi-libc/sysroot/lib/wasm32-wasi/libc.a(log.o)
task: [run] go build && ./test-wasm-plugin
Caught panic as *fmt.wrapError: wasm error: unreachable
wasm stack trace:
        .interface:{HttpGet:func:{named:context.Context,pointer:named:gitlab.com/***/test-wasm-plugin/api.HttpGetReq}{pointer:named:gitlab.com/***/test-wasm-plugin/api.HttpGetResp,named:error},Log:func:{named:context.Context,pointer:named:gitlab.com/***/test-wasm-plugin/api.LogReq}{pointer:named:github.com/knqyf263/go-plugin/types/known/emptypb.Empty,named:error}}.Log$invoke(i32,i32)
        .(main.MyPlugin).SendRequest(i32,i32)
        .greeter_send_request(i32,i32) i64

Upd.: problem on Windows was due to broken installation please disregard this part.

Hey, I got the following errors when loading my plugin.

wasm error: unreachable
wasm stack trace:
	.runtime.runtimePanic(i32,i32)
	.runtime.lookupPanic()
	.malloc(i32) i32

• MacOS: 13.0.1
• Chip: Apple M1 Max
• Go: go1.19.2 darwin/arm64
• go-plugin: 0.2.0

When some message contains field of google.protobuf.FieldMask type, protoc-gen-go-plugin generates code with this import:

fieldmaskpb "github.com/knqyf263/go-plugin/types/known/fieldmaskpb"

However this package is not present in go-plugin repo.

This makes it not possible to use google.protobuf.FieldMask.

Hey, I might have missed something. I am loading plugins like so

        pluginMap := make(map[string]*MyPlugin)
	filepath.Walk(root, func(path string, info os.FileInfo, err error) error  {

		if !info.IsDir() && filepath.Ext(path) == EXT {
			if this, err := p.Load(ctx, path, PluginFunctions{}); err != nil {
				return err
			} else {
				pluginMap[uuid.New().String()] = &this
			}
		}
		return nil
	})
	m.plugins = pluginMap

then from "somewhere else" I am calling

func (p PluginManager) CallContentOnPlugins(ctx context.Context) error {
	for _, v := range p.plugins {
		content, err := (*v).Content(ctx, emptypb.Empty{})
		if err != nil {
			return err
		}
		fmt.Println("render content ", content.GetContent())
	}
	return nil
}

If I were to do this directly, in the first code snippet above where I am adding the plugin to the map, it works fine. However when I try to do this from this function, I get the error

error module "" closed with exit_code(0)

My guess is that I would have to keep calling .Load on the plugin however I would have thought all would be OK as I am storing the plugin in the map as a reference, but for some reason that doesnt' work? Any tips as to why?

(on another note, is there ways to expose functions in the plugin to the 'host' at runtime - i.e functions that are not known about in the proto file. Perhaps with reflection, or perhaps with a map of functions in the plugin that are exposed in the proto file?)

Thanks for your help!

A new latest release (v0.6.0) was downloading, but after generating files from scratch, the first hard-coded protoc-gen-go-plugin v0.1.0 version was added.

// Code generated by protoc-gen-go-plugin. DO NOT EDIT.
// versions:
// 	protoc-gen-go-plugin v0.1.0
// 	protoc               v3.21.12

There are compile errors when building plugin with (latest) tinygo v0.32.0:

$ tinygo build -target=wasip1 -scheduler=none --no-d
ebug
# github.com/knqyf263/go-plugin/wasm
../../../vendor/github.com/knqyf263/go-plugin/wasm/plugin.go:18:10: cannot use uintptr(size) (value of type uintptr) as int value in assignment
../../../vendor/github.com/knqyf263/go-plugin/wasm/plugin.go:19:10: cannot use uintptr(size) (value of type uintptr) as int value in assignment

Everything is fine with tinygo v0.31.*.

This is a non-featured proposal, without adding new features or fixing bugs.

The ideas is utilizing golang text template engine and have a single file with code template per functional area (plugin, host, etc.) instead of using tons of the g.P() outputs with fmt.Sprintf which is not so easy to read and maintain.

We could probably to add a plugin option to generate following stubs

• plugin empty code
• main function for plugin calling
• host functions

Hi,
I am trying to get the helloworld example working with the evening greeter. My app structure is currently

.
├── build
├── main.go
└── plugins
    ├── greeter
    │   └── protos
    └── plugin-evening

My final app ends up in build, main.go is loading the plugin, pluginsholds all my plugins and the greeter interface, the proto file and the generated pb.go files.plugin-evening` is the example plugin.

I set option go_package = "plugins/greeter/protos"; in the protos file so that the path is the same as where the protos are and the generated go files. I have updated this path in the plugin-evening.go file - however (limited knowledge here) my IDE recommended I set the import to greeting "changeme/plugins/greeter/protos" - I'm not sure why the changeme

I have set the same import in my main.go greeting "changeme/plugins/greeter/protos"

I'm then calling

	ctx := context.Background()
	p, err := greeting.NewGreeterPlugin(ctx, greeting.GreeterPluginOption{})
	if err != nil {
		return err
	}
	defer p.Close(ctx)
	eveningPlugin, err := p.Load(ctx, "plugins/plugin-evening/evening.wasm")
	if err != nil {
		return err
	}
	reply, err := eveningPlugin.Greet(ctx, greeting.GreetRequest{
		Name: "go-plugin",
	})
	if err != nil {
		return err
	}

	fmt.Println(reply.GetMessage())

I'm getting an error:

  ERROR
          # changeme/plugins/greeter/protos
          plugins/greeter/protos/greet_host.pb.go:154:53: too many arguments in call to p.module.Memory().Write
          	have (context.Context, uint32, []byte)
          	want (uint32, []byte)
          plugins/greeter/protos/greet_host.pb.go:155:129: too many arguments in call to p.module.Memory().Size
          	have (context.Context)
          	want ()
          plugins/greeter/protos/greet_host.pb.go:169:51: too many arguments in call to p.module.Memory().Read
          	have (context.Context, uint32, uint32)
          	want (uint32, uint32)
          plugins/greeter/protos/greet_host.pb.go:172:44: too many arguments in call to p.module.Memory().Size
          	have (context.Context)
          	want ()

          exit status 2

any idea as to why that would happen?

When calling into plugins the behavior is not consistent depending on how much data is being passed. For small amounts of data everything seems fine. For larger amounts of data each successive call gets slower. For even larger amounts of data only one or two calls is possible before the callee hangs indefinitely. When the hang occurs it seems like we never make it into the plugin function, so it appears to be an issue with memory allocation or garbage collection within in the wasm boundary.

Version: f7d9444
Reproducible Test: https://github.com/cchamplin/plugin-bug

=== RUN   TestPlugin_MemorySmall
2022/12/09 15:14:13 Size of data being passed 1000 (139890 bytes)
2022/12/09 15:14:13 calling plugin 0
2022/12/09 15:14:13 call execution time: 61.440708ms
2022/12/09 15:14:13 calling plugin 1
2022/12/09 15:14:13 call execution time: 61.888625ms
...
2022/12/09 15:14:13 calling plugin 18
2022/12/09 15:14:14 call execution time: 252.296958ms
2022/12/09 15:14:14 calling plugin 19
2022/12/09 15:14:14 call execution time: 18.755334ms
--- PASS: TestPlugin_MemorySmall (1.17s)

=== RUN   TestPlugin_MemoryMedium
2022/12/09 15:14:14 Size of data being passed 5000 (703890 bytes)
2022/12/09 15:14:14 calling plugin 0
2022/12/09 15:14:16 call execution time: 2.158440416s
2022/12/09 15:14:16 calling plugin 1
2022/12/09 15:14:18 call execution time: 1.677055583s
2022/12/09 15:14:18 calling plugin 2
2022/12/09 15:14:20 call execution time: 2.145829459s
2022/12/09 15:14:20 calling plugin 3
2022/12/09 15:14:20 call execution time: 100.430083ms
...
022/12/09 15:15:10 call execution time: 11.31319075s
2022/12/09 15:15:10 calling plugin 18
2022/12/09 15:15:10 call execution time: 122.91525ms
2022/12/09 15:15:10 calling plugin 19
2022/12/09 15:15:10 call execution time: 132.877958ms
--- PASS: TestPlugin_MemoryMedium (56.58s)

=== RUN   TestPlugin_MemoryBig
2022/12/09 15:15:10 Size of data being passed 10000 (1408890 bytes)
2022/12/09 15:15:10 calling plugin 0
...
Never completes

It should auto-generate Close() as below.
https://github.com/tetratelabs/wazero/blob/b1b9b6d40486ca2302de80ae7d3cde05085a76e1/examples/allocation/tinygo/greet.go#L30