glycerine / guestdll

go building reloadable (dlclose supported) dlls with shared runtime

guestdll

go building reloadable (dlclose() supported) dlls with shared runtime

go install -buildmode=shared runtime sync/atomic
go install -buildmode=shared std  # may or may not be needed
go install -buildmode=shared -linkshared github.com/glycerine/guestdll/guest

# the last 'go install' puts the .so shared library into
cd ${GOPATH}/pkg/linux_amd64_dynlink/libgithub.com-glycerine-guestdll-guest.so

# the runtime .so got built and saved here:
${GOROOT}/pkg/linux_amd64_dynlink/libruntime,sync-atomic.so


Current sticking point:

Trying to load the runtime (or stdlib) references a main.init function, which
is not found, when loading is attempted (here via python)

from ctypes import *
import _ctypes
fn = '/root/vol/go/pkg/linux_amd64_dynlink/libgithub.com-glycerine-guestdll-guest.so'
runtime = '/usr/local/go/pkg/linux_amd64_dynlink/libruntime,sync-atomic.so'
CDLL(runtime)
## OSError: /usr/local/go/pkg/linux_amd64_dynlink/libruntime,sync-atomic.so: undefined symbol: main.init

## update: we hacked around this temporarily by commenting out the
## references to main_init and main_main in runtime/proc.go and rebuilding
## both runtime and stdlib.  We will probably have to inject a small
## stub DLL that just has these two functions or otherwise allow
## these references to be satisfied.

next issue: how to we initialize the Go runtime?  When we
examined a Go binary on linux, the ELF header tells us that
the _rt0_amd64_linux function is the initializer, or first
run entry point function. So we constructed guestdll/host/host.c
which dynamically loads the runtime .so and then tries
to call _rt0_amd64_linux(). However this currently segfaults
and we are investigating why.

here's our little host.c loader program:

#include "stdlib.h"
#include "stdio.h"
#include "dlfcn.h"

// NB change this to reflect the path to your runtime library.
const char* runtimeFn = "/usr/local/go/pkg/linux_amd64_dynlink/libruntime,sync-atomic.so";

int main() {
  void *rtHandle;
  void (*runtimeStart)();
  char *error;
  
  printf("host.c main has begun.\n");
  rtHandle = dlopen(runtimeFn, RTLD_LAZY | RTLD_GLOBAL | RTLD_NODELETE);
  if (!rtHandle) {
    fprintf(stderr, "%s\n", dlerror());
    exit(EXIT_FAILURE);
  }
  dlerror();    /* Clear any existing error */  
  runtimeStart = dlsym(rtHandle, "_rt0_amd64_linux");

  error = dlerror();
  if (error != NULL) {
    fprintf(stderr, "%s\n", error);
    exit(EXIT_FAILURE);
  }
  printf("good: about to invoke runtimeStart().\n");    
  runtimeStart();
  printf("host.c main is done.\n");
  return 0;
}

here's how I looked for a stack trace from the segfault:

root@eceba7590181:~/go/src/github.com/glycerine/guestdll/host# ulimit -c unlimited
root@eceba7590181:~/go/src/github.com/glycerine/guestdll/host# ./host
host.c main has begun.
good: about to invoke runtimeStart().
Segmentation fault (core dumped)
root@eceba7590181:~/go/src/github.com/glycerine/guestdll/host# gdb host core
GNU gdb (Ubuntu 7.11.1-0ubuntu1~16.5) 7.11.1
Copyright (C) 2016 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.  Type "show copying"
and "show warranty" for details.
This GDB was configured as "x86_64-linux-gnu".
Type "show configuration" for configuration details.
For bug reporting instructions, please see:
<http://www.gnu.org/software/gdb/bugs/>.
Find the GDB manual and other documentation resources online at:
<http://www.gnu.org/software/gdb/documentation/>.
For help, type "help".
Type "apropos word" to search for commands related to "word"...
Reading symbols from host...done.
[New LWP 6714]
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
Core was generated by `./host'.
Program terminated with signal SIGSEGV, Segmentation fault.
#0  0x00007f740a5970f0 in runtime.sysargs ()
   from /usr/local/go/pkg/linux_amd64_dynlink/libruntime,sync-atomic.so
(gdb) bt
#0  0x00007f740a5970f0 in runtime.sysargs ()
   from /usr/local/go/pkg/linux_amd64_dynlink/libruntime,sync-atomic.so
#1  0x00007f740a4c6ac0 in ?? ()
   from /usr/local/go/pkg/linux_amd64_dynlink/libruntime,sync-atomic.so
#2  0x000000000184c470 in ?? ()
#3  0x00007ffd02577128 in ?? ()
#4  0x00007ffd02577124 in ?? ()
#5  0x00007f740a47f028 in _GLOBAL_OFFSET_TABLE_ () from /lib/x86_64-linux-gnu/libpthread.so.0
#6  0x00000000004006a0 in ?? ()
#7  0x00007ffd02577620 in ?? ()
#8  0x0000000000000000 in ?? ()
(gdb)

So the runtime seems to be crashing in runtime.sysargs(). println() added to runtime/os_linux.go shows that the argc passed in is 4196440. The argv is probably garbage too.

So let's just hack in and force argc to be 0, and see how far we get. Here's the next problem:

root@eceba7590181:~/go/src/github.com/glycerine/guestdll/host# ./host
host.c main has begun.
good: about to invoke runtimeStart().
sysargs starting!
argc =  4196440  # forcing this to zero
fatal error: failed to get system page size
runtime: panic before malloc heap initialized

runtime stack:
runtime.throw(0x7ffb34d77bdc, 0x1e)
	/usr/local/go/src/runtime/panic.go:616 +0x83 fp=0x7fffb11e1c48 sp=0x7fffb11e1c28 pc=0x7ffb34d347d3
runtime.mallocinit()
	/usr/local/go/src/runtime/malloc.go:232 +0x526 fp=0x7fffb11e1ce8 sp=0x7fffb11e1c48 pc=0x7ffb34d14a86
runtime.schedinit()
	/usr/local/go/src/runtime/proc.go:491 +0x8b fp=0x7fffb11e1d50 sp=0x7fffb11e1ce8 pc=0x7ffb34d3814b
runtime.rt0_go(0x7fffb11e1d80, 0x400858, 0x7fffb11e1d80, 0x4006a0, 0x400858, 0x400880, 0x135f470, 0x7ffb34d6d0e0, 0x0, 0x400880, ...)
	/usr/local/go/src/runtime/asm_amd64.s:252 +0x251 fp=0x7fffb11e1d58 sp=0x7fffb11e1d50 pc=0x7ffb34d693c1
root@eceba7590181:~/go/src/github.com/glycerine/guestdll/host#

But this is very good progress! It looks like enough of the runtime was initialized that a panic actually worked! The error is coming from src/runtime/malloc.go:232 (in go1.10.2 anyway). It looks like some of the OS init isn't being done, based on the comment there:

    ## malloc.go:229-233 in go1.10.2 on linux:
    // Check physPageSize.
    if physPageSize == 0 {
         // The OS init code failed to fetch the physical page size.
         throw("failed to get system page size")  ## where we are current crashing
    }

It looks like physPageSize variable is supposed to also get set in the sysargs() call in runtime/os_linux.go, but apparently isn't at the moment, when starting the runtime after loading it as a DLL.