wenzel / libmicrovmi Goto Github PK
View Code? Open in Web Editor NEWA cross-platform unified Virtual Machine Introspection API library
Home Page: https://wenzel.github.io/libmicrovmi/
License: GNU General Public License v3.0
A cross-platform unified Virtual Machine Introspection API library
Home Page: https://wenzel.github.io/libmicrovmi/
License: GNU General Public License v3.0
https://github.com/Wenzel/libmicrovmi/pull/115/checks?check_run_id=1000813980#step:10:87
The tests are failing if we use kvmi master
branch (or the one I uploaded on crates.io)
This should be a simple fix.
Provide default implementation for Introspectable
so that the driver doesn't have to define and implement everything to compile
use log
crate instead of println!
macro
Another goal is to translate a virtual address to a physical address.
This implies parsing the page tables.
Now either the hypervisor supports this already and can offer us an API, or we have to implement this ourselves.
Check from existing crates like x86_64 from rust-osdev if they can share page tables structures definition, and even the translation functions.
Xen breaks the API between 4.9.2
and 4.11.2
4.9.2: xc_monitor_enable(domid_t domid)
4.11.2: xc_monitor_enable(uint32_t domid)
How to deal with this in Bindgen ?
When doing a physical write the input byte buffer that is being written to the target is usually not mutated directly. It will just cause the memory of the target to be mutated. Currently, the libmicrovmi API exposes a physical write function that enforces mutability of the input byte buffer though.
fn write_physical(&self, _paddr: u64, _buf: &mut [u8]) -> Result<(), Box<dyn Error>> {
unimplemented!();
}
From #161 , Volatility3 is performing huge read operations, far beyong the usual 4K.
However, in KVM invalid_page_access
returns true if the size requested is superior to the PAGE_SIZE
, which is 4K.
This means that the libmicrovmi
should split the read into 4K chunks.
Cargo already knows how to deal with examples binaries with the examples
folder.
Move it there.
According to #125 (comment)
When reading xenstore from a dedicated virtual machine, the access is more limited, and reading some entries may fail with a permission denied error.
In Libvmi, NULL entries are ignored.
We should implement the same behavior, handling explicitely the permission denied error, and continue the loop.
At the moment we are triggering a dual build for internal pull requests (a PR originating from this repo, to master branch for example).
After a quick search, it's possible to avoid this. examples:
I recommend to limit the push
to master
branch and let pull_request
trigger on any branches.
Libmicrovmi's api needs to explicitely specify the hypervisor that we want to use.
We could improve that behavior by detecting if a given hypervisor is present, and test all possible hypervisors (the ones compiled in the library).
As we are only compiling a specific hypervisor, there will be only 2 hypervisors available:
How can we modify the init
function, so that it takes an optional driver type ?
pub fn init(driver_type: Option<DriverType>, domain_name: &String) -> Box<Introspectable> {
So we could pass None and let the function decide.
What do you think @tathanhdinh, how to implement this feature ?
Thanks.
This issue tracks the integration status of libmicrovmi with LibVMI
➡️ A fork of libvmi is available with a basic integration of libmicrovmi:
https://github.com/Wenzel/libvmi on the libmicrovmi
branch
A pull request has been opened (only on the forked repo) to track the changes:
Wenzel/libvmi#1
A github actions has been configured to test the build:
https://github.com/Wenzel/libvmi/runs/1527128223?check_suite_focus=true
Each driver initialization through new
should return a Result
instead of panicking.
PR #152 is working toward this goal by refactoring error handling.
PR #151 attempts to have default drivers enabled.
The issue raised by this system is that it requires to have
all installed and detected to compile, unless you explicitely disable a driver.
An improvment would be to do like libvmi's cmake build system, and have opportunistic features, where we try to build the crates on which the driver depend upon, and on failure we simply disable the feature and continue.
I don't know how to implement this with cargo, build.rs
.
Next step would be to ask on Rust language forums.
A similar issue is opened on actions-rs repo:
actions-rs/meta#6
Let's see where it leads
TODO:
crates.io
Libmicrovmi aims to be compatible with LibVMI and other already existing C projects.
We need to build and expose a C API.
Support was added in the kvmi crate to init using a VSock socket:
Wenzel/kvmi#39
changes required in libmicrovmi:
DriverInitParam
's KVMiSocket field should use a
SocketType`: https://github.com/Wenzel/kvmi/blob/master/src/lib.rs#L129Writing a new address space for volatility3, to run vol plugins directly on the VM physical memory.
From Ikelos:
Sure, so if you define an appropriate handler (using the python handlers system) similar to the JarHandler we have at https://github.com/volatilityfoundation/volatility3/blob/develop/volatility3/framework/layers/resources.py#L202, then people will be able to use python vol.py --single-location vmi://whatever
to run against the VMI image directly.
In volatility 3, we don't pass parameters, but instead every "Configurable" (some that needs information) provides requirements through the get_requirements method, which contains simple types like int, str, etc, to act as parameters.
You can see this at work for every plugin, but you can also see it in the intel layer here: https://github.com/volatilityfoundation/volatility3/blob/develop/volatility3/framework/layers/intel.py#L231 which can then be accessed via self.config[<requirement name>]
. Hopefully that makes a bit more sense? I'd probably go for the handler method, if you've already done something similar for rekall...
TODO:
The init
function should fail instead of panicking
When trying to setup multi distro testing under Github Actions, the Xen driver failed to compiled,
as it relied on xenforeignmemory
, which is an API which appeared recently in Xen.
Add more compatibility for the Xen driver.
#45
While trying to integrate libmicrovmi into LibVMI, I noticed that one of the KPCR
search algorithm required the IDTR_BASE
LibVMI register.
Even though there are fallback algorithms, they failed for unclear reasons.
So I decided to hardcode my IDTR_BASE
value:
And vmi-process-list
example works, on top of libmicrovmi
🎉
cc @rageagainsthepc
the current implementation (without IDTR_BASE
hardcoded) is here:
https://github.com/Wenzel/libvmi/tree/libmicrovmi
When using libmicrovmi in an external project it is rather unfortunate to have the init function just panic if something goes wrong. Imho it would be better to have it return result objects instead so the caller can decide how to handle the situation.
One example is libvmi which tests hypervisor connections by initializing them. If one fails, libvmi just moves on to the next one.
Rust has an official guide to write llibraries and APIs:
https://rust-lang.github.io/api-guidelines/checklist.html
we should strive to follow it.
Icebox
is a Virtual Machine Introspection solution that enable you to stealthily trace and debug any process (kernel or user).
The project is based on the Fast Debugging Protocol
library, which ties it to VirtualBox for now.
A libmicrovmi integration could be handy for the Icebox project to support KVM or Xen
difference is unmaintained
Details | |
---|---|
Status | unmaintained |
Package | difference |
Version | 2.0.0 |
URL | johannhof/difference.rs#45 |
Date | 2020-12-20 |
The author of the difference
crate is unresponsive.
Maintained alternatives:
See advisory page for additional details.
get_page_access
is actually not implemented by hypervisor. the library is supposed to maintain the state of page whose permissions have been modified, and return this state, via this API.
So it's a higher-level than we expected.
Example in Libvmi:
https://github.com/libvmi/libvmi/blob/master/libvmi/events.c#L667
While reviewing the code:
https://github.com/Wenzel/libmicrovmi/blob/master/src/driver/kvm.rs#L122
We shouldn't be intercepting INT3 by default, unless we are explicitely asked to do so.
if a debugger is running, the INT3 will be catched and put the in the queue, (and if the user never listens, never processed).
Libvmi implementation:
https://github.com/libvmi/libvmi/blob/master/libvmi/driver/kvm/kvm_events.c#L615
The KVM driver needs more arguments than the domain name.
It also needs a socket, which is hardcoded in the KVM driver ATM:
https://github.com/Wenzel/libmicrovmi/blob/master/src/driver/kvm.rs#L32
Redesign the API to allow specifying a set of parameters, or passing a dictionary for example.
Libvmi has been built with (too) strong compatibility in mind, and used a VMI_SUCCESS/FAILURE
as a return type when it could have included stdbool.h
and returned a boolean.
stdbool.h
only exists since C99
:
https://stackoverflow.com/a/1608350/3017219
I think we should not try to go lower than this standard.
Following #49 I'm opening this issue to start the discussion on how the event API should be implemented.
I'm reposting an idea from @kylerky, where each driver event (KVMEvent
, XenEvent
...) should implement a set of traits, and Event
should contain generic type:
pub struct Event {
common: EventCommon,
extra: EventExtra,
}
pub enum EventExtra<CR>
where CR: CREvent
{
CR(CR),
Pause,
}
pub trait CREvent {
fn get_old() -> u64;
fn get_new() -> u64;
}
pub struct KVMCrEvent {
pub fn construct_reply(...) -> KVMCrReply {
}
}
impl CREvent for KVMCrEvent {...}
struct KVMCrReply {...}
impl Reply for KVMCrReply {...}
fn main() {
let kvm_cr_event = ...;
let reply = kvm_cr_event.construct_reply(Continue, new);
dom.send(reply);
}
fn send<T: Reply>(reply: T) { ... }
Integrate Clippy in Travis CI to catch errors early
libkvmi v6 offers a method to get max physical address:
https://github.com/KVM-VMI/kvm/blob/kvmi/tools/kvm/kvmi/include/kvmi/libkvmi.h#L125
The Xen events ABI has changed a lot along the Xen releases.
Right now, we are following the ABI from Xen 4.11 and onwards.
If we want to be compatible with all release, we need to follow what LibVMI implemented with xen_events_abi.h
And deduce the vm_event version from the Xen version:
https://github.com/libvmi/libvmi/blob/master/libvmi/driver/xen/xen_events.c#L3213
Create Python bindings to call libmicrovmi from Python
TODO (updated):
DriverType
and DriverInitParam
read_physical
setup.py
to accept --features
parameter, and set features in RustExtension
quay.io/pypa/manylinux2014_x86_64
and install all the drivers required development headersbuild-wheels.sh
to accept --features
and pass them to setup.py
python/Cargo.toml
to fill setup.py
metadataWe should use the rust-cpython
project
rust-cpython will help us to generate a Python extension module, wrapping the libmicrovmi Rust API.
@rageagainsthepc if you have an opinion
Volatility was unable to read a requested page: Page error 0xffff81e13858 in layer primary (Page Fault at entry 0x0 in table page directory)
Strange, are you using Volatility3 ?
Maybe open an issue on their repo, they will get back to you.
The problem is that mem-dump.rs does not create a valid memory dump, in the sense that it simply skips non-accessible pages, and thus physical address and offset in dump file will not match. Volatility depends on both being the same.
Something like this results in a working dump file (as sparse file, so not wasting disk space). Tested on Linux only, I am not familiar with Windows, if it supports sparse file that way as well.
--- a/examples/mem-dump.rs
+++ b/examples/mem-dump.rs
@@ -1,6 +1,9 @@
use std::fs::File;
use std::io::Write;
use std::path::Path;
+use std::io::SeekFrom;
+use std::io::Seek;
+use std::convert::TryInto;
use clap::{App, Arg, ArgMatches};
use env_logger;
@@ -56,7 +59,9 @@ fn main() {
.write_all(&buffer)
.expect("failed to write to file");
}
- Err(_error) => (),
+ Err(_error) => {
+ dump_file.seek(SeekFrom::Current(PAGE_SIZE.try_into().unwrap())).expect("failed to seek in file");
+ },
}
}
On feature that I would like on the API is when an Event
is being Dropped from the scope, it would automatically send a default response, which would be EventReplyType::Continue
.
tasks:
Introspectable
in the Event
, so the event can call the driverimpl Drop for Event
Event
so that you can't reply twice to the same eventrelated: https://github.com/Wenzel/libmicrovmi/pull/49/files
thoughts @kylerky, @rageagainsthepc ?
Today libmicrovmi has 2 limitations:
We should produce a library that detects at runtime which libraries are available and initializes the compatible drivers.
However, we rely on bindgen, and this feature is not ready yet:
rust-lang/rust-bindgen#1541
libmicrovmi is using the log crate to define its log messages.
These message are enabled by the env_logger
, initialized at beginning of each example.
However, how can we initialize the env_logger
with the C API ? 🤔
Even if I export RUST_LOG=debug
now, I have no debug output when running a LibVMI example :/
failure is officially deprecated/unmaintained
Details | |
---|---|
Status | unmaintained |
Package | failure |
Version | 0.1.8 |
URL | rust-lang-deprecated/failure#347 |
Date | 2020-05-02 |
The failure
crate is officially end-of-life: it has been marked as deprecated
by the former maintainer, who has announced that there will be no updates or
maintenance work on it going forward.
The following are some suggested actively developed alternatives to switch to:
See advisory page for additional details.
As discussed on twitter (@deiderajr) i receive a segmentation fault after using the mem-dump example code (same for other examples).
This is our testing environment:
Steps to reproduce the error:
Retrieving data with libvmi works (see terminal_libvmi.png).
Getting a correct .dump file
Segmentation fault (see terminal_libmicrovmi.png)
If i missed important information please let me know. Thanks!
HyperDbg
debugger is an open-source, hypervisor-assisted user-mode, and kernel-mode Windows debugger with a focus on using modern hardware technologies. It is a debugger designed for analyzing, fuzzing and reversing.
How does it listen for VM events such as EPT pagefaults ?
What hypervisors are supported ?
Could HyperDBG make use of libmicrovmi ?
cc @SinaKarvandi here :)
It's more natural to call reply on an Event
than on the driver
:
// old
drv.reply_event(&ev, EventReplyType::Continue)
// new
ev.reply(EventReplyType::Continue)
reply_event
in the Introspectable
traitimpl Event
, so users should call this API insteadrelated: https://github.com/Wenzel/libmicrovmi/pull/49/files
thoughts @kylerky, @rageagainsthepc ?
The Hyper-V driver currenty loops over all vmwp.exe and returns when he found a partition handle.
This means that the vm_name parameter is not checked.
We need to list runnings Hyper-V VMs, and match the vm_name parameter.
Line 204 in 5c15b94
memflow
is a library that allows live memory introspection of running systems and their snapshots. Due to its modular approach it trivial to support almost any scenario where Direct Memory Access is available.
A libmicrovmi connector is already available ! 🎉
https://github.com/memflow/memflow-microvmi
Thanks to @ko1N !
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.