wkz / kmemd

Explore a live Linux kernel's memory using GDB

kmemd

Explore a live Linux kernel's memory using GDB

For more background, see this blog entry

Building and Installing

kmem uses Autotools, so the procdure is hopefully familiar to many.

If you are building from a cloned GIT repo (as opposed to a release tarball), you have to start by generating the configure script:

~/kmemd$ ./autogen.sh

To build and install with the default settings:

~/kmemd$ ./configure && make && sudo make install

Using kmemd

BEWARE: You are about to serve up your kernel's memory over a file or socket, i.e. basically Hearbleed as a service. Anyone with access to that interface will be able to read anything in there, including crypto keys and whatnot. Consider yourself warned!

In is simplest form, kmem can be started without any arguments. As we are going to completely root the box, we need superuser permissions.

~$ sudo kmemd

Without arguments, kmemd will listen for connections on the named UNIX socket /run/kmemd.sock, which works well in scenarios where you want to inspect the kernel running on your local machine.

In cases where GDB is run on a different system than the one being inspected (which is often the case when debugging embedded systems, for example), you will most likely want to bind to a TCP socket instead:

~$ sudo kmemd -s :1234

At this point you should be able to attach to kmemd using GDB's remote debugging facility in the normal way:

~/linux$ gdb vmlinux
(gdb) target remote the-system:1234

KASLR

If your kernel is running with address layout randomization (KASLR), the debug symbols in your vmlinux won't match the addresses used by the running kernel.

You can use this GDB Python extension to compensate for it: gdb-linux-kaslr.py. Because it needs to parse /proc/kallsyms to figure out the current base address, GDB needs to be run as root (which it most likely needs to connect to the default UNIX socket anyway):

~/linux$ sudo gdb
(gdb) add-vmlinux vmlinux
(gdb) target remote /run/kmemd.sock

kmemd's Issues

gdb server does not work for me

GNU gdb (Ubuntu 12.0.90-0ubuntu1) 12.0.90
...
(gdb) target remote /run/kmemd.sock
Remote 'g' packet reply is too long (expected 312 bytes, got 560 bytes): 0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000

# kmemd -d
Using 'g' size of 560 bytes
Listening on /run/kmemd.sock
Session started
RECV:"qSupported:multiprocess+;swbreak+;hwbreak+;qRelocInsn+;fork-events+;vfork-events+;exec-events+;vContSupported+;QThreadEvents+;no-resumed+;memory-tagging+;xmlRegisters=i386"
SEND:""
RECV:"vMustReplyEmpty"
SEND:""
RECV:"Hg0"
SEND:""
RECV:"qTStatus"
SEND:""
RECV:"?"
SEND:"S05"
RECV:"qfThreadInfo"
SEND:""
RECV:"qL1200000000000000000"
SEND:""
RECV:"Hc-1"
SEND:""
RECV:"qC"
SEND:""
RECV:"qAttached"
SEND:""
RECV:"g"
SEND:"0000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000"
Session ended (len:-9 err:0)
^C

gdb_send_iter reports -EIO when client ack too early

kmemd/src/gdb.c

Lines 145 to 151 in eb3f1d9

	fflush(fp);
	c = fgetc(fp);
	if (c == '+')
	return 0;
	return -EIO;

It seems that if the client send ack(+) too fast, the session may broken.