koute / bytehound Goto Github PK
View Code? Open in Web Editor NEWA memory profiler for Linux.
License: Other
A memory profiler for Linux.
License: Other
It has some error when i build on ubuntu 16.04
Code version is 0.1.0
Errors when build memory-profiler:
error[E0554]: #![feature] may not be used on the stable release channel
--> /home/yangz/.cargo/registry/src/github.com-1ecc6299db9ec823/sc-0.2.2/src/lib.rs:15:1
|
15 | #![feature(asm)]
| ^^^^^^^^^^^^^^^^error: aborting due to previous error
For more information about this error, try
rustc --explain E0554
.
error: Could not compilesc
.
warning: build failed, waiting for other jobs to finish...
error: build failed
error: failed to run custom build command for
server-core v0.1.0 (/mnt/e/temp/nokia-memory-profiler/memory-profiler-0.1.0/server-core)
process didn't exit successfully:/mnt/e/temp/nokia-memory-profiler/memory-profiler-0.1.0/target/release/build/server-core-eaaa58d183d73b7b/build-script-build
(exit code: 101)
--- stderr
thread 'main' panicked at 'cannot launch a child process to install the dependencies for the WebUI: Os { code: 2, kind: NotFound, message: "No such file or directory" }', src/libcore/result.rs:1009:5
note: Run withRUST_BACKTRACE=1
for a backtrace.warning: build failed, waiting for other jobs to finish...
error: build failed
`
Ubuntu 16.04 LTS
Linux XXX 4.5.1-040501-generic #201604121331 SMP Tue Apr 12 17:33:29 UTC 2016 x86_64 x86_64 x86_64 GNU/Linux
Is it possible for bytehound to analyze an already running application? It will be beneficial to diagnose memory leaks.
On debian10 with nigthly rust
thread '<unnamed>' panicked at 'cannot access a TLS value during or after it is destroyed: AccessError', src/libcore/result.rs:999:5
stack backtrace:
0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
1: std::panicking::default_hook::{{closure}}
at src/libstd/sys_common/backtrace.rs:71
at src/libstd/sys_common/backtrace.rs:59
at src/libstd/panicking.rs:197
2: std::panicking::rust_panic_with_hook
at src/libstd/panicking.rs:211
at src/libstd/panicking.rs:474
3: std::panicking::continue_panic_fmt
at src/libstd/panicking.rs:381
4: rust_begin_unwind
at src/libstd/panicking.rs:308
5: core::panicking::panic_fmt
at src/libcore/panicking.rs:85
6: core::result::unwrap_failed
at /rustc/37ff5d388f8c004ca248adb635f1cc84d347eda0/src/libcore/macros.rs:18
7: memory_profiler::unwind::grab
at /root/.cargo/git/checkouts/not-perf-e01bfa01482c86ed/9739e8b/nwind/src/local_unwinding.rs:0
at preload/src/unwind.rs:166
8: calloc
at preload/src/lib.rs:1423
at preload/src/lib.rs:1470
9: g_malloc0
10: g_slice_free_chain_with_offset
11: g_queue_free
12: __nptl_deallocate_tsd.part.8
13: start_thread
14: clone
Aborted (core dumped)
My dat file is about 6.0G, and encounter the following error:
[2021-12-16T08:26:08Z INFO server_core] Trying to load "memory-profiling_test-exe_1639640943_16894.dat"...
Killed
error: failed to run custom build command for `nwind v0.1.0 (https://github.com/koute/not-perf.git?rev=20e4ddc#20e4ddc2)`
Caused by:
process didn't exit successfully: `/home/jnelson/rust-community/bytehound/target/release/build/nwind-37356a14aefb82cf/build-script-build` (exit status: 1)
--- stdout
cargo:rerun-if-changed=src/arch/amd64_get_regs.s
cargo:rerun-if-changed=src/arch/amd64_trampoline.s
TARGET = Some("x86_64-unknown-linux-gnu")
OPT_LEVEL = Some("3")
HOST = Some("x86_64-unknown-linux-gnu")
CC_x86_64-unknown-linux-gnu = None
CC_x86_64_unknown_linux_gnu = None
HOST_CC = None
CC = None
CFLAGS_x86_64-unknown-linux-gnu = None
CFLAGS_x86_64_unknown_linux_gnu = None
HOST_CFLAGS = None
CFLAGS = None
CRATE_CC_NO_DEFAULTS = None
DEBUG = Some("true")
CARGO_CFG_TARGET_FEATURE = Some("fxsr,llvm14-builtins-abi,sse,sse2")
running: "cc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-m64" "-Wall" "-Wextra" "-o" "/home/jnelson/rust-community/bytehound/target/release/build/nwind-04d6e93b8c8527d4/out/src/arch/amd64_get_regs.o" "-c" "src/arch/amd64_get_regs.s"
cargo:warning=clang: warning: argument unused during compilation: '-ffunction-sections' [-Wunused-command-line-argument]
cargo:warning=clang: warning: argument unused during compilation: '-fdata-sections' [-Wunused-command-line-argument]
cargo:warning=clang: warning: argument unused during compilation: '-fno-omit-frame-pointer' [-Wunused-command-line-argument]
cargo:warning=src/arch/amd64_get_regs.s:38:9: error: ambiguous operand size for instruction 'pop'
cargo:warning= pop [rdi+8*49]
cargo:warning= ^~~~~~~~~
exit status: 1
--- stderr
error occurred: Command "cc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-m64" "-Wall" "-Wextra" "-o" "/home/jnelson/rust-community/bytehound/target/release/build/nwind-04d6e93b8c8527d4/out/src/arch/amd64_get_regs.o" "-c" "src/arch/amd64_get_regs.s" with args "cc" did not execute successfully (status code exit status: 1).
$ cc --version
clang version 10.0.0-4ubuntu1
Target: x86_64-pc-linux-gnu
Thread model: posix
InstalledDir: /usr/bin
GCC works fine.
Hello,
maybe the most things for this are already in place and it's easy to do.
Currently manual usage of the heaptrack GUI is needed to see the memory flamegraph. Inferno could be used to directly generate an interactive SVG by exporting a perf-compatible text format (the text format can even be without collapsed stacks, as there is the inferno-collapse-perf
command for that).
When I run "cargo build --release -p memory-profiler-cli", it failed and the error log as below:
davidwang@system-MS-7918:~/Tools/memory-profiler$ cargo build --release -p memory-profiler-cli
Compiling server-core v0.1.0 (/home/davidwang/Tools/memory-profiler/server-core)
00h00m00s 0/0: : **error: failed to run custom build command for server-core v0.1.0** (/home/davidwang/Tools/memory-profiler/server-core)
Caused by:
process didn't exit successfully: /home/davidwang/Tools/memory-profiler/target/release/build/server-core-5dfb1577abf53aa3/build-script-build
(exit code: 101)
--- stderr
ERROR: [Errno 2] No such file or directory: 'install'
thread 'main' panicked at 'Failed to install the dependencies for the WebUI; child process exited with error code Some(1)! You might want to try to run 'rm -Rf ~/.cache/yarn' and try again.', server-core/build.rs:61:21
stack backtrace:
0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
1: std::sys_common::backtrace::_print
at src/libstd/sys_common/backtrace.rs:71
2: std::panicking::default_hook::{{closure}}
at src/libstd/sys_common/backtrace.rs:59
at src/libstd/panicking.rs:197
3: std::panicking::default_hook
at src/libstd/panicking.rs:211
4: std::panicking::rust_panic_with_hook
at src/libstd/panicking.rs:474
5: std::panicking::continue_panic_fmt
at src/libstd/panicking.rs:381
6: std::panicking::begin_panic_fmt
at src/libstd/panicking.rs:336
7: semalock::Semalock::with
at server-core/build.rs:61
at /home/davidwang/.cargo/registry/src/github.com-1ecc6299db9ec823/semalock-0.2.0/src/lib.rs:99
at /rustc/8869ee03d7f258e1b76a11c6fbb01b5708a9f504/src/libcore/result.rs:639
at /home/davidwang/.cargo/registry/src/github.com-1ecc6299db9ec823/semalock-0.2.0/src/lib.rs:96
8: build_script_build::main
at server-core/build.rs:46
9: std::rt::lang_start::{{closure}}
at /rustc/8869ee03d7f258e1b76a11c6fbb01b5708a9f504/src/libstd/rt.rs:64
10: std::panicking::try::do_call
at src/libstd/rt.rs:49
at src/libstd/panicking.rs:293
11: __rust_maybe_catch_panic
at src/libpanic_unwind/lib.rs:85
12: std::rt::lang_start_internal
at src/libstd/panicking.rs:272
at src/libstd/panic.rs:388
at src/libstd/rt.rs:48
13: main
14: __libc_start_main
15: _start
It seems to find install dir failed, but where should the dir be?
bytehound: 591a 591a INF Version: 0.8.0
bytehound: 591a 591a INF Options:
bytehound: 591a 591a INF MEMORY_PROFILER_BASE_SERVER_PORT = 8100
bytehound: 591a 591a INF MEMORY_PROFILER_CHOWN_OUTPUT_TO = None
bytehound: 591a 591a INF MEMORY_PROFILER_DISABLE_BY_DEFAULT = false
bytehound: 591a 591a INF MEMORY_PROFILER_ENABLE_BROADCAST = false
bytehound: 591a 591a INF MEMORY_PROFILER_ENABLE_SERVER = false
bytehound: 591a 591a INF MEMORY_PROFILER_GRAB_BACKTRACES_ON_FREE = false
bytehound: 591a 591a INF MEMORY_PROFILER_INCLUDE_FILE = None
bytehound: 591a 591a INF MEMORY_PROFILER_OUTPUT = "memory-profiling_%e_%t_%p.dat"
bytehound: 591a 591a INF MEMORY_PROFILER_REGISTER_SIGUSR1 = true
bytehound: 591a 591a INF MEMORY_PROFILER_REGISTER_SIGUSR2 = true
bytehound: 591a 591a INF MEMORY_PROFILER_USE_PERF_EVENT_OPEN = true
bytehound: 591a 591a INF MEMORY_PROFILER_USE_SHADOW_STACK = true
bytehound: 591a 591a INF MEMORY_PROFILER_WRITE_BINARIES_TO_OUTPUT = true
bytehound: 591a 591a INF MEMORY_PROFILER_ZERO_MEMORY = false
bytehound: 591a 591a INF MEMORY_PROFILER_GATHER_MMAP_CALLS = false
bytehound: 591a 591a INF MEMORY_PROFILER_BACKTRACE_CACHE_SIZE_LEVEL_1 = 16384
bytehound: 591a 591a INF MEMORY_PROFILER_BACKTRACE_CACHE_SIZE_LEVEL_2 = 327680
bytehound: 591a 591a INF MEMORY_PROFILER_CULL_TEMPORARY_ALLOCATIONS = false
bytehound: 591a 591a INF MEMORY_PROFILER_TEMPORARY_ALLOCATION_LIFETIME_THRESHOLD = 10000
bytehound: 591a 591a INF MEMORY_PROFILER_TEMPORARY_ALLOCATION_PENDING_THRESHOLD = None
bytehound: 591a 591a INF Setting atexit hook...
bytehound: 591a 591a INF Tracing will be toggled ON (for the first time)
bytehound: 591a 591a INF Registering SIGUSR1 handler...
bytehound: 591a 591a INF Registering SIGUSR2 handler...
bytehound: 591a 591a INF Startup initialization finished
bytehound: 591a 591a DBG Opening perf events; pid=0, cpu=-1, frequency=0, stack_size=0, reg_mask=0x0000000000000000, event_source=SwDummy, inherit=false, start_disabled=false...
bytehound: 591a 591a DBG Maximum sample rate: 100000
bytehound: 591a 591a DBG Allocating 16 + 1 pages for the ring buffer for PID 0 on CPU -1
bytehound: 591a 591a DBG Perf events open with fd=3
bytehound: 591a 591a INF Spawning event processing thread...
I tried to profile my application. It allocates several gigabytes of memory at peak.
Recording process went well and a 5.2 Gb trace was generated. I tried to open the dump using "memory-profiler-cli server" command, but it segfaulted with "memory allocation of 21299200000 bytes failed" error. The same happened when i tried to export data using export-heaptrack.
I tried to use "memory-profiler-cli squeeze", and it did shrank the trace to 124M. I was able to open the new trace, but it seems to contain no sensible data. Accoding to the squeezed trace, only 72kb was allocated.
Could you please provide any advice on what to do next? Can I do anything besides running my program with less input data?
I think, at least we should add git tags and gh releases. To make for users alignments more straightforward. Each release could hold short description of the changes in exposed modules and what alignments are required.
I think it could be uploaded to crates.io, but I do not if there are some cons?
(sorry, for duplication with issue on not-perf, but it is different repo, so duplication is needed I think).
I'm seeing the following deadlock at program start:
Thread 2 (Thread 0x7f9fafbff640 (LWP 520330) "mem-prof"):
#0 0x00007f9fb0c41600 in __lll_lock_wait () from /usr/lib/libpthread.so.0
#1 0x00007f9fb0c3a580 in pthread_mutex_lock () from /usr/lib/libpthread.so.0
#2 0x00007f9fb095eca4 in __cxa_thread_atexit_impl () from /usr/lib/libc.so.6
#3 0x00007f9fb0e1dce5 in std::thread::local::fast::Key::register_dtor<core::cell::RefCell<core::option::Option<std::sys_common::thread_info::ThreadInfo>>> () at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b//library/std/src/thread/local.rs:577
#4 std::sys_common::thread_info::THREAD_INFO::__getit () at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b//library/std/src/thread/local.rs:220
#5 std::thread::local::LocalKey::try_with<core::cell::RefCell<core::option::Option<std::sys_common::thread_info::ThreadInfo>>, std::sys_common::thread_info::set::{closure#0}, ()> () at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b//library/std/src/thread/local.rs:398
#6 std::thread::local::LocalKey::with<core::cell::RefCell<core::option::Option<std::sys_common::thread_info::ThreadInfo>>, std::sys_common::thread_info::set::{closure#0}, ()> () at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b//library/std/src/thread/local.rs:375
#7 std::sys_common::thread_info::set () at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b//library/std/src/sys_common/thread_info.rs:42
#8 std::thread::{impl#0}::spawn_unchecked::{closure#1}<bytehound::global::spawn_processing_thread::{closure#0}, ()> () at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b/library/std/src/thread/mod.rs:482
#9 core::ops::function::FnOnce::call_once<std::thread::{impl#0}::spawn_unchecked::{closure#1}, ()> () at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b/library/core/src/ops/function.rs:227
#10 0x00007f9fb0e9acd5 in alloc::boxed::{impl#44}::call_once<(), dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global> () at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b/library/alloc/src/boxed.rs:1694
#11 alloc::boxed::{impl#44}::call_once<(), alloc::boxed::Box<dyn core::ops::function::FnOnce<(), Output=()>, alloc::alloc::Global>, alloc::alloc::Global> () at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b/library/alloc/src/boxed.rs:1694
#12 std::sys::unix::thread::{impl#2}::new::thread_start () at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b//library/std/src/sys/unix/thread.rs:106
#13 0x00007f9fb0c38259 in start_thread () from /usr/lib/libpthread.so.0
#14 0x00007f9fb0a1d5e3 in clone () from /usr/lib/libc.so.6
Thread 1 (Thread 0x7f9fb05f6300 (LWP 520329) "i4"):
#0 0x00007f9fb0a0453b in sched_yield () from /usr/lib/libc.so.6
#1 0x00007f9fb0df19b2 in std::sys::unix::thread::Thread::yield_now () at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b//library/std/src/sys/unix/thread.rs:113
#2 std::thread::yield_now () at /rustc/db9d1b20bba1968c1ec1fc49616d4742c1725b4b//library/std/src/thread/mod.rs:696
#3 bytehound::global::spawn_processing_thread () at preload/src/global.rs:228
#4 bytehound::global::try_enable (state=<optimized out>) at preload/src/global.rs:445
#5 0x00007f9fb0e18ee2 in bytehound::global::StrongThreadHandle::acquire () at preload/src/global.rs:536
#6 bytehound::api::allocate (requested_size=32, kind=...) at preload/src/api.rs:185
#7 bytehound::api::calloc (count=<optimized out>, element_size=140735463581872) at preload/src/api.rs:254
#8 0x00007f9fb0c6dc48 in ?? () from /usr/lib/libdl.so.2
#9 0x00007f9fb0c6d544 in dlsym () from /usr/lib/libdl.so.2
#10 0x00007f9fb064906d in ?? () from /usr/lib/libGLdispatch.so.0
#11 0x00007f9fb1164e2e in call_init () from /lib64/ld-linux-x86-64.so.2
#12 0x00007f9fb1164f1c in _dl_init () from /lib64/ld-linux-x86-64.so.2
#13 0x00007f9fb11560ca in _dl_start_user () from /lib64/ld-linux-x86-64.so.2
#14 0x0000000000000001 in ?? ()
#15 0x00007fff87509551 in ?? ()
#16 0x0000000000000000 in ?? ()
Notably __cxa_thread_atexit_impl
tries to acquire the lock held by dlsym
(or one of the functions calling dlsym
).
The following problem occurred today, it didn't exist before:
thread '' panicked at 'not implemented: 'memalign' is unimplemented!', preload/src/api.rs:831:5
I have a big memory profiler data file that size is 15G.
It crash when I run like this : memory-profiler-cli server memory-profiling_xxx-cli_1561961665_5949.dat
It seems has no memory to allocate.(It used more than 59G).
Can we load it partially?
see the stack bellow:
#0 0x00007ffff6e81428 in __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:54
#1 0x00007ffff6e8302a in __GI_abort () at abort.c:89
#2 0x000055555596d747 in std::sys::unix::abort_internal::h7c2e3e81a8081686 () at src/libstd/sys/unix/mod.rs:155
#3 0x000055555597b102 in rust_oom () at src/libstd/alloc.rs:217
#4 0x0000555555983db6 in alloc::alloc::handle_alloc_error::hd2bda6ca6d32a8b8 () at src/liballoc/alloc.rs:246
#5 0x00005555558673c5 in cli_core::loader::Loader::process::hf4e9fe002a9f499e (self=<optimized out>, event=...) at cli-core/src/loader.rs:424
#6 0x000055555569a229 in server_core::main::h5ae3a963a1f0c9e8 (inputs=..., debug_symbols=..., load_in_parallel=<optimized out>, interface=..., port=<optimized out>)
at /home/zds/tmp/memory-profiler/cli-core/src/loader.rs:270
#7 0x00005555559bb459 in memory_profiler_cli::run::haa3badce4805cbb3 (opt=...) at cli/src/main.rs:114
#8 memory_profiler_cli::main::h25e367f23a3bb32d () at cli/src/main.rs:143
#9 0x0000555555996633 in std::rt::lang_start::_$u7b$$u7b$closure$u7d$$u7d$::h12a8d2f2fadc57b2 () at /rustc/2887008e0ce0824be4e0e9562c22ea397b165c97/src/libstd/rt.rs:64
#10 0x00005555559b1d97 in main ()
the web GUI is awansome! it's easy to find which func alloc more mems by flamegraph.
but it hard to find which func alloc more than dealloc...
Is there a way to make a diff between start and end? like gperftools, compare two profiles
My app can run with libmemory_profiler.so very well but it crashes when it throws an exception.
GCC : 7.4.0
Compile : g++ -Wall -Wextra -g -ggdb -O0 test.cpp -pthread
Run : sudo LD_PRELOAD=./libmemory_profiler.so ./a.out
Code : (I don't know how to make the code pretty, it looks suck.)
#include <unistd.h>
#include <stdio.h>
#include <sys/syscall.h>
#include <array>
#include <vector>
#include <thread>
#include <random>
using namespace std;
// Init random generator
mt19937 rand_generator(chrono::high_resolution_clock::now().time_since_epoch().count());
uniform_int_distribution<int> dist(100,1000);
class A {
public:
A() {
data_.push_back(0);
data_.push_back(1);
data_.push_back(2);
}
private:
vector<int> data_;
};
// Set a global value and it may alloca memory before main()
A a;
void ThreadEntry() {
while(true) {
char*p = new char[128];
*p += 1;
try {
if(0 == dist(rand_generator) % 2) {
//throw "Throw a exception"; //crashes when it runs
delete [] p;
}
} catch (const char* msg) {
printf("catch exception [%s]\n", msg);
}
this_thread::sleep_for(chrono::milliseconds(10));
}
}
int main() {
array<thread, 1> thread_pool;
for(auto &th : thread_pool)
th = thread(ThreadEntry);
for(auto &th : thread_pool)
th.join();
}
Backtrace:
(gdb) bt
#0 __GI_raise (sig=sig@entry=6) at ../sysdeps/unix/sysv/linux/raise.c:51
#1 0x00007ffff6cca801 in __GI_abort () at abort.c:79
#2 0x00007ffff753e957 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#3 0x00007ffff7544ab6 in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#4 0x00007ffff7544af1 in std::terminate() () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#5 0x00007ffff7544d24 in __cxa_throw () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#6 0x00005555555551c9 in ThreadEntry () at test.cpp:42
#7 0x00005555555563b3 in std::__invoke_impl<void, void (*)()> (__f=@0x55555576e098: 0x55555555514a <ThreadEntry()>) at /usr/include/c++/7/bits/invoke.h:60
#8 0x0000555555555dd1 in std::__invoke<void (*)()> (__fn=@0x55555576e098: 0x55555555514a <ThreadEntry()>) at /usr/include/c++/7/bits/invoke.h:95
#9 0x000055555555764a in std::thread::_Invoker<std::tuple<void (*)()> >::_M_invoke<0ul> (this=0x55555576e098) at /usr/include/c++/7/thread:234
#10 0x0000555555557606 in std::thread::_Invoker<std::tuple<void (*)()> >::operator() (this=0x55555576e098) at /usr/include/c++/7/thread:243
#11 0x00005555555575d6 in std::thread::_State_impl<std::thread::_Invoker<std::tuple<void (*)()> > >::_M_run (this=0x55555576e090) at /usr/include/c++/7/thread:186
#12 0x00007ffff756f66f in ?? () from /usr/lib/x86_64-linux-gnu/libstdc++.so.6
#13 0x00007ffff70826db in start_thread (arg=0x7fffeffff700) at pthread_create.c:463
#14 0x00007ffff6dab88f in clone () at ../sysdeps/unix/sysv/linux/x86_64/clone.S:95
I'd like to hunt memcpy and memmoves the same way allocations are tracked, how hard would be to add this feature?
It looks like some build script panics when RUSTC_WRAPPER
is set to sccache
.
Resulting error:
error: failed to run `rustc` to learn about target-specific information
Caused by:
could not execute process `sccache rustc - --crate-name ___ --print=file-names --crate-type bin --crate-type rlib --crate-type dylib --crate-type cdylib --crate-type staticlib --crate-type proc-macro --print=sysroot --print=cfg` (never executed)
Caused by:
No such file or directory (os error 2)
If yes, email me [email protected]
Hey,
Ran into this panic when building memory-profiler-cli on Linux with rust nightly.
error: failed to run custom build command for `server-core v0.1.0 (/home/harshasrisri/Code/memory-profiler/server-core)`
process didn't exit successfully: `/home/harshasrisri/Code/memory-profiler/target/release/build/server-core-436551ca2cd1c07d/build-script-build` (exit code: 101)
--- stderr
thread 'main' panicked at 'cannot launch a child process to install the dependencies for the WebUI: Os { code: 2, kind: NotFound, message: "No such file or directory" }', src/libcore/result.rs:999:5
stack backtrace:
0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
1: std::sys_common::backtrace::_print
at src/libstd/sys_common/backtrace.rs:71
2: std::panicking::default_hook::{{closure}}
at src/libstd/sys_common/backtrace.rs:59
at src/libstd/panicking.rs:197
3: std::panicking::default_hook
at src/libstd/panicking.rs:211
4: std::panicking::rust_panic_with_hook
at src/libstd/panicking.rs:474
5: std::panicking::continue_panic_fmt
at src/libstd/panicking.rs:381
6: rust_begin_unwind
at src/libstd/panicking.rs:308
7: core::panicking::panic_fmt
at src/libcore/panicking.rs:85
8: core::result::unwrap_failed
at /rustc/a9ec99f4201ec33026a468ef1289f98a95b4d71a/src/libcore/macros.rs:18
9: semalock::Semalock::with
at /rustc/a9ec99f4201ec33026a468ef1289f98a95b4d71a/src/libcore/result.rs:827
at server-core/build.rs:50
at /home/harshasrisri/.cargo/registry/src/github.com-1ecc6299db9ec823/semalock-0.2.0/src/lib.rs:99
at /rustc/a9ec99f4201ec33026a468ef1289f98a95b4d71a/src/libcore/result.rs:639
at /home/harshasrisri/.cargo/registry/src/github.com-1ecc6299db9ec823/semalock-0.2.0/src/lib.rs:96
10: build_script_build::main
at server-core/build.rs:46
11: std::rt::lang_start::{{closure}}
at /rustc/a9ec99f4201ec33026a468ef1289f98a95b4d71a/src/libstd/rt.rs:64
12: std::panicking::try::do_call
at src/libstd/rt.rs:49
at src/libstd/panicking.rs:293
13: __rust_maybe_catch_panic
at src/libpanic_unwind/lib.rs:85
14: std::rt::lang_start_internal
at src/libstd/panicking.rs:272
at src/libstd/panic.rs:388
at src/libstd/rt.rs:48
15: main
16: __libc_start_main
17: _start
warning: build failed, waiting for other jobs to finish...
error: build failed
I tried to use bytehound on my application and ran into this disappointing panic:
Line 835 in 7d34d88
Are there any plans to support aligned_alloc
?
Repo with bug reproduction
Tried all suggestions from #10, nothing helped.
Rust version:
stable-x86_64-unknown-linux-gnu (default)
rustc 1.58.1 (db9d1b20b 2022-01-20)
Kernel is
5.15.16-1
UPD.
Checked all versions up to 0.6.0
Profiling was working this summer, so something in our code is affecting it.
In section 3, https://koute.github.io/bytehound/memory_leak_analysis.html, I try to follow your script
let remaining = allocations().only_not_matching_backtraces(groups);
Throw ERROR: Variable not found: groups (line 1, position 60)
error
So I try to filter out all of allocations previous section. But I'm not sure what is all of allocation? I have 2 options
let groups = allocations()
.only_leaked();
let remaining = allocations().only_not_matching_backtraces(groups);
or
let groups = allocations()
let remaining = allocations().only_not_matching_backtraces(groups);
But I don't know which one is correct.
What are the benefits of your tool over them?
For example big limitation of Asan + Msan is that they can't be used simultaneously.
So I can't check invalid use of uninitialised memory and invalid use of adresses.
Can this tool check both at the same time?
What about performance?
Or am I misunderstanding and this is only a profiler not a memory debugger?
If so how does it compare to DHAT and others?
Hey, is it possible to build & release a static binary? Perhaps with musl?
I was trying to run bytehound on an server which doesn't have newer GLIBC:
$ bytehound/bytehound
bytehound/bytehound: /lib64/libm.so.6: version `GLIBC_2.27' not found (required by bytehound/bytehound)
bytehound/bytehound: /lib64/libm.so.6: version `GLIBC_2.29' not found (required by bytehound/bytehound)
bytehound/bytehound: /lib64/libc.so.6: version `GLIBC_2.14' not found (required by bytehound/bytehound)
bytehound/bytehound: /lib64/libc.so.6: version `GLIBC_2.17' not found (required by bytehound/bytehound)
bytehound/bytehound: /lib64/libc.so.6: version `GLIBC_2.18' not found (required by bytehound/bytehound)
Hi,
On some systems (like debian SID), the packaged yarn is too old and compilation fails with the following error
--- stdout
0.32+git
--- stderr
ERROR: [Errno 2] No such file or directory: 'install'
thread 'main' panicked at 'Failed to install the dependencies for the WebUI; child process exited with error code Some(1)! You might want to try to run 'rm -Rf ~/.cache/yarn' and try again.', server-core/build.rs:85:21
stack backtrace:
0: rust_begin_unwind
at /rustc/0d1754e8bf6942b4c1d24d7c923438782129ba5a/library/std/src/panicking.rs:498:5
1: core::panicking::panic_fmt
at /rustc/0d1754e8bf6942b4c1d24d7c923438782129ba5a/library/core/src/panicking.rs:106:14
2: build_script_build::main::{{closure}}
at ./build.rs:85:21
3: semalock::Semalock::with::{{closure}}
at /home/sam/.cargo/registry/src/github.com-1ecc6299db9ec823/semalock-0.2.0/src/lib.rs:99:30
4: core::result::Result<T,E>::and_then
at /rustc/0d1754e8bf6942b4c1d24d7c923438782129ba5a/library/core/src/result.rs:966:22
5: semalock::Semalock::with
at /home/sam/.cargo/registry/src/github.com-1ecc6299db9ec823/semalock-0.2.0/src/lib.rs:96:9
6: build_script_build::main
at ./build.rs:62:5
7: core::ops::function::FnOnce::call_once
at /rustc/0d1754e8bf6942b4c1d24d7c923438782129ba5a/library/core/src/ops/function.rs:227:5
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
warning: build failed, waiting for other jobs to finish...
error: build failed
The solution is to use yarn installed from npm and not from the system's repositories. Could we have a version check and a message telling the using to install yarn from https://classic.yarnpkg.com/en/ ?
rustc 1.60.0-nightly
gcc (Ubuntu 9.4.0-1ubuntu1~16.04) 9.4.0
yarn 1.22.17
Error message:
The following warnings were emitted during compilation:
warning: src/arch/aarch64_trampoline.s: Assembler messages:
warning: src/arch/aarch64_trampoline.s:13: Error: bad register expression
error: failed to run custom build command for `nwind v0.1.0 (https://github.com/koute/not-perf.git?rev=51003a4#51003a40)`
Caused by:
process didn't exit successfully: `/SD_32GB/home/cappadokesm/code/bytehound/target/release/build/nwind-85e5cfcf1e9018ce/build-script-build` (exit status: 1)
--- stdout
cargo:rerun-if-changed=src/arch/aarch64_get_regs.s
cargo:rerun-if-changed=src/arch/aarch64_trampoline.s
TARGET = Some("aarch64-unknown-linux-gnu")
OPT_LEVEL = Some("3")
HOST = Some("aarch64-unknown-linux-gnu")
CC_aarch64-unknown-linux-gnu = None
CC_aarch64_unknown_linux_gnu = None
HOST_CC = None
CC = None
CFLAGS_aarch64-unknown-linux-gnu = None
CFLAGS_aarch64_unknown_linux_gnu = None
HOST_CFLAGS = None
CFLAGS = None
CRATE_CC_NO_DEFAULTS = None
DEBUG = Some("true")
CARGO_CFG_TARGET_FEATURE = Some("fp,neon,pmuv3")
running: "cc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-Wall" "-Wextra" "-o" "/SD_32GB/home/cappadokesm/code/bytehound/target/release/build/nwind-bbe614d31d8d25f4/out/src/arch/aarch64_get_regs.o" "-c" "src/arch/aarch64_get_regs.s"
exit status: 0
running: "cc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-Wall" "-Wextra" "-o" "/SD_32GB/home/cappadokesm/code/bytehound/target/release/build/nwind-bbe614d31d8d25f4/out/src/arch/aarch64_trampoline.o" "-c" "src/arch/aarch64_trampoline.s"
cargo:warning=src/arch/aarch64_trampoline.s: Assembler messages:
cargo:warning=src/arch/aarch64_trampoline.s:13: Error: bad register expression
exit status: 1
--- stderr
error occurred: Command "cc" "-O3" "-ffunction-sections" "-fdata-sections" "-fPIC" "-g" "-fno-omit-frame-pointer" "-Wall" "-Wextra" "-o" "/SD_32GB/home/cappadokesm/code/bytehound/target/release/build/nwind-bbe614d31d8d25f4/out/src/arch/aarch64_trampoline.o" "-c" "src/arch/aarch64_trampoline.s" with args "cc" did not execute successfully (status code exit status: 1).
I remain at your disposal in case you need any additional information.
Thanks a lot,
X
Edit: the above came up upon running cargo build --release -p bytehound-preload
.
I am on Mac and we may need libbytehound.dylib rather than libbytehound.so. Could we build libbytehound.dylib?
When attempting to load my application with libmemory_profiler.so
in LD_PRELOAD
, it immediately segfaults.
Built with:
rustc 1.36.0-nightly (6afcb5628 2019-05-19)
Captured from GDB:
gdb --args env LD_PRELOAD=/usr/local/lib/libmemory_profiler.so ./build/myapp
Program received signal SIGSEGV, Segmentation fault.
0x00007ffff7881e75 in memory_profiler::initialize () at preload/src/lib.rs:1239
1239 fn initialize() {
(gdb) bt
#0 0x00007ffff7881e75 in memory_profiler::initialize () at preload/src/lib.rs:1239
#1 0x00007ffff7883cc9 in memory_profiler::allocate (size=32, is_calloc=<optimized out>) at preload/src/lib.rs:1417
#2 malloc (size=32) at preload/src/lib.rs:1459
#3 0x00007ffff7883df6 in memory_profiler::allocate (size=32, is_calloc=<optimized out>) at preload/src/lib.rs:1429
#4 malloc (size=32) at preload/src/lib.rs:1459
...
I was thinking about using this project to compare the memory usage of a Go application that I have rewritten in Rust, but when testing the memory-profiler with a simple go example it generated no result.
Heres the go example I've tested:
package main
import "fmt"
import "os"
import "strconv"
func fib(n uint64) uint64 {
switch n {
case 0:
return 0
case 1:
return 1
default:
return fib(n-1) + fib(n-2)
}
}
func main() {
input, _ := strconv.Atoi(os.Args[1])
n := uint64(input)
fmt.Println(fib(n))
}
And running:
go build test.go
LD_PRELOAD=/home/jonathas/Repositories/memory-profiler/target/release/libmemory_profiler.so ./test 40
Didn't generate any dat
file at all. I have even tried adding a make
or new
calls to the fib func
in order to introduce more allocations but I still got no results.
I've been trying to generate a framegraph from the cli server, but I always results in a panic. Here's a semple of one of the outputs with rustś bracktrace on:
$RUST_BACKTRACE=1 memory-profiler-cli server --port 8081 *.dat
[2020-09-15T15:06:20Z INFO server_core] Trying to load "memory-profiling_integration_test-365ad0caa04e4325_1600177642_7839.dat"...
[2020-09-15T15:06:21Z INFO cli_core::loader] Loaded data in 0s 169
[2020-09-15T15:06:21Z INFO server_core] Trying to load "memory-profiling_uhupkg_full_extraction-5627470b7f3790bb_1600177283_6643.dat"...
[2020-09-15T15:06:21Z INFO cli_core::loader] Loaded data in 0s 235
[2020-09-15T15:06:21Z INFO server_core] Trying to load "memory-profiling_uhupkg_full_extraction-5627470b7f3790bb_1600178040_8425.dat"...
[2020-09-15T15:06:25Z INFO cli_core::loader] Loaded data in 3s 829
[2020-09-15T15:06:25Z INFO actix_server::builder] Starting 8 workers
[2020-09-15T15:06:25Z INFO actix_server::builder] Starting server on 127.0.0.1:8081
thread '<unnamed>' panicked at 'attempted to leave type `&cli_core::Frame` uninitialized, which is invalid', /rustc/397b390cc76ba1d98f80b2a24a371f708dcc9169/library/core/src/mem/mod.rs:666:9
stack backtrace:
0: rust_begin_unwind
at /rustc/397b390cc76ba1d98f80b2a24a371f708dcc9169/library/std/src/panicking.rs:475
1: core::panicking::panic_fmt
at /rustc/397b390cc76ba1d98f80b2a24a371f708dcc9169/library/core/src/panicking.rs:85
2: core::panicking::panic
at /rustc/397b390cc76ba1d98f80b2a24a371f708dcc9169/library/core/src/panicking.rs:50
3: core::mem::uninitialized
at /rustc/397b390cc76ba1d98f80b2a24a371f708dcc9169/library/core/src/mem/mod.rs:666
4: cli_core::tree::Tree<K,V>::new
at /home/jonathas/Repositories/memory-profiler/cli-core/src/tree.rs:60
5: cli_core::exporter_flamegraph_pl::dump_collation
at /home/jonathas/Repositories/memory-profiler/cli-core/src/exporter_flamegraph_pl.rs:62
6: cli_core::exporter_flamegraph::export_as_flamegraph
at /home/jonathas/Repositories/memory-profiler/cli-core/src/exporter_flamegraph.rs:19
7: server_core::handler_export_flamegraph::{{closure}}
at /home/jonathas/Repositories/memory-profiler/server-core/src/lib.rs:1214
8: server_core::async_data_handler::{{closure}}
at /home/jonathas/Repositories/memory-profiler/server-core/src/lib.rs:208
note: Some details are omitted, run with `RUST_BACKTRACE=full` for a verbose backtrace.
Aborted (core dumped)
I'm getting this:
Compiling goblin v0.0.21
error: failed to run custom build command for `nwind v0.1.0 (https://github.com/nokia/not-perf.git?rev=9739e8b#9739e8b6)`
Caused by:
process didn't exit successfully: `/Users/andrewbanchich/Downloads/memory-profiler/target/release/build/nwind-a1a37c92bd103957/build-script-build` (exit code: 101)
--- stderr
thread 'main' panicked at 'unsupported target: x86_64-apple-darwin', /Users/andrewbanchich/.cargo/git/checkouts/not-perf-e01bfa01482c86ed/9739e8b/nwind/build.rs:13:19
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
warning: build failed, waiting for other jobs to finish...
Can't wait to try it out!
I've set my global cargo target dir to a tmp
directory to avoid thrashing my SSD and get faster builds. This is the first project I've encountered any issues with while building. I previously didn't have issues building this project, before I changed that setting. Setting CARGO_TARGET_DIR=target
for just the build doesn't seem to fix it.
This is a build with .cargo/config
set to
[build]
rustc-wrapper = "/home/mumu/.cargo/bin/sccache"
rustflags = ["-Clink-args=-fuse-ld=lld"]
target-dir = "/tmp/rust-builds"
λ ~/code/oss/memory-profiler/ master cargo +nightly build --release -p memory-profiler-cli
Compiling serde v1.0.126
Compiling actix-threadpool v0.1.2
Compiling common v0.6.1 (/home/mumu/code/oss/memory-profiler/common)
Compiling nwind v0.1.0 (https://github.com/koute/not-perf.git?rev=2d8a5ee#2d8a5ee3)
Compiling phf v0.8.0
Compiling server-core v0.6.1 (/home/mumu/code/oss/memory-profiler/server-core)
error: failed to run custom build command for `server-core v0.6.1 (/home/mumu/code/oss/memory-profiler/server-core)`
Caused by:
process didn't exit successfully: `/tmp/rust-builds/release/build/server-core-39c38d9af2396655/build-script-build` (exit status: 101)
--- stderr
thread 'main' panicked at 'failed to acquire a semaphore: "OpenOptions::open failed: entity not found"', server-core/build.rs:57:64
note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace
warning: build failed, waiting for other jobs to finish...
error: build failed
Commenting out the target-dir
line results in a successful server-core
build, though I seem to get build errors in the yarn stuff too now, that's likely unrelated to this issue. #16 (comment) is another instance of someone seeming to run into this problem.
I got problem with linker for i686-pc-windows-gnu target. (I assume that, the linker is managed by rustup/cargo, not from system).
after cargo build --target=i686-pc-windows-gnu
(form /gather), I got:
= note: /usr/bin/ld: unrecognized option '--enable-long-section-names'
/usr/bin/ld: use the --help option for usage information
collect2: error: ld returned 1 exit status
error: aborting due to previous error
error: could not compile `memory-profiler-gather`.
Here goes cargo build --target=i686-pc-windows-gnu --verbose
(second time)
output.txt
Hi,
Currently (on 1.59-nightly), bytehound doesn't compile because of the stabilization of thread_local_const_init
feature:
error[E0635]: unknown feature `thread_local_const_init`
--> preload/src/lib.rs:1:12
|
1 | #![feature(thread_local_const_init)]
| ^^^^^^^^^^^^^^^^^^^^^^^
For more information about this error, try `rustc --explain E0635`.
error: could not compile `bytehound-preload` due to previous error
The feature line should be removed from the code from now on I think.
Cheers,
Gerry
Hi! I made a simple program to try this out with the downloadable binaries for 0.1.0 but I ended up with this on my ubuntu 19.04 x86_64 machine:
$ RUST_BACKTRACE=1 LD_PRELOAD=./libmemory_profiler.so ./memleak
thread '' panicked at 'failed to initialize perf_event_open: Os { code: 1, kind: PermissionDenied, message: "Operation not permitted" }', src/libcore/result.rs:999:5
stack backtrace:
0: std::sys::unix::backtrace::tracing::imp::unwind_backtrace
at src/libstd/sys/unix/backtrace/tracing/gcc_s.rs:39
1: std::panicking::default_hook::{{closure}}
at src/libstd/sys_common/backtrace.rs:71
at src/libstd/sys_common/backtrace.rs:59
at src/libstd/panicking.rs:197
2: std::panicking::rust_panic_with_hook
at src/libstd/panicking.rs:211
at src/libstd/panicking.rs:474
3: std::panicking::continue_panic_fmt
at src/libstd/panicking.rs:381
4: rust_begin_unwind
at src/libstd/panicking.rs:308
5: core::panicking::panic_fmt
at src/libcore/panicking.rs:85
6: core::result::unwrap_failed
at /rustc/00859e3e653973120006aaf3227823062dde1ba7/src/libcore/macros.rs:18
7: std::sync::once::Once::call_once::{{closure}}
at /rustc/00859e3e653973120006aaf3227823062dde1ba7/src/libcore/result.rs:827
at preload/src/unwind.rs:108
at /rustc/00859e3e653973120006aaf3227823062dde1ba7/src/libcore/ops/function.rs:231
at /home/travis/.cargo/registry/src/github.com-1ecc6299db9ec823/lazy_static-1.3.0/src/inline_lazy.rs:28
at /rustc/00859e3e653973120006aaf3227823062dde1ba7/src/libstd/sync/once.rs:220
8: std::sync::once::Once::call_inner
at src/libstd/sync/once.rs:387
9: memory_profiler::unwind::grab
at /rustc/00859e3e653973120006aaf3227823062dde1ba7/src/libstd/sync/once.rs:220
at /home/travis/.cargo/registry/src/github.com-1ecc6299db9ec823/lazy_static-1.3.0/src/inline_lazy.rs:27
at /home/travis/build/nokia/memory-profiler/<::lazy_static::__lazy_static_internal macros>:12
at /home/travis/build/nokia/memory-profiler/<::lazy_static::__lazy_static_internal macros>:13
at preload/src/unwind.rs:132
10: malloc
at preload/src/lib.rs:1423
at preload/src/lib.rs:1459
11:
12:
13:
14:
Aborted (core dumped)
My simple program which I made for the test purpose:
`
#include <stdio.h>
#include <malloc.h>
int main()
{
printf("lets leak some memory\n");
while (1) {
char *p = malloc(1024);
sleep(1);
}
return 0;
}
`
Hi. I want to use bytehound to profile a long running cloud server.
MEMORY_PROFILER_CULL_TEMPORARY_ALLOCATIONS=1 \
MEMORY_PROFILER_TEMPORARY_ALLOCATION_LIFETIME_THRESHOLD=60000 \
MEMORY_PROFILER_OUTPUT=/mnt/data/memprof_%e_%t_%p_%n.mem \
MEMORY_PROFILER_DISABLE_BY_DEFAULT=1 \
MEMORY_PROFILER_LOG=info \
MEMORY_PROFILER_LOGFILE=/mnt/data/bytehound_%e_%p.log \
LD_PRELOAD=/usr/local/lib/libbytehound.so \
/path/to/nemo
pkill --signal SIGUSR1 app
pkill --signal SIGUSR1 app
Logged events shows that's what is happening, however the created profile /mnt/data/memprof_nemo_1644512060_7_0.mem
continues to grow, long after having disabled profiling (15 minutes after).
What am I doing wrong?
I was under the impression I could start/stop profiling multiple times, at different moments of my application execution, and have a different profile created for each of them.
0007 0007 INF Version: 0.8.0
0007 0007 INF Options:
0007 0007 INF MEMORY_PROFILER_BASE_SERVER_PORT = 8100
0007 0007 INF MEMORY_PROFILER_CHOWN_OUTPUT_TO = None
0007 0007 INF MEMORY_PROFILER_DISABLE_BY_DEFAULT = true
0007 0007 INF MEMORY_PROFILER_ENABLE_BROADCAST = false
0007 0007 INF MEMORY_PROFILER_ENABLE_SERVER = false
0007 0007 INF MEMORY_PROFILER_GRAB_BACKTRACES_ON_FREE = false
0007 0007 INF MEMORY_PROFILER_INCLUDE_FILE = None
0007 0007 INF MEMORY_PROFILER_OUTPUT = "/mnt/data/memprof_%e_%t_%p_%n.mem"
0007 0007 INF MEMORY_PROFILER_REGISTER_SIGUSR1 = true
0007 0007 INF MEMORY_PROFILER_REGISTER_SIGUSR2 = true
0007 0007 INF MEMORY_PROFILER_USE_PERF_EVENT_OPEN = true
0007 0007 INF MEMORY_PROFILER_USE_SHADOW_STACK = true
0007 0007 INF MEMORY_PROFILER_WRITE_BINARIES_TO_OUTPUT = true
0007 0007 INF MEMORY_PROFILER_ZERO_MEMORY = false
0007 0007 INF MEMORY_PROFILER_GATHER_MMAP_CALLS = false
0007 0007 INF MEMORY_PROFILER_BACKTRACE_CACHE_SIZE_LEVEL_1 = 16384
0007 0007 INF MEMORY_PROFILER_BACKTRACE_CACHE_SIZE_LEVEL_2 = 327680
0007 0007 INF MEMORY_PROFILER_CULL_TEMPORARY_ALLOCATIONS = true
0007 0007 INF MEMORY_PROFILER_TEMPORARY_ALLOCATION_LIFETIME_THRESHOLD = 60000
0007 0007 INF MEMORY_PROFILER_TEMPORARY_ALLOCATION_PENDING_THRESHOLD = None
0007 0007 INF Setting atexit hook...
0007 0007 INF Registering SIGUSR1 handler...
0007 0007 INF Registering SIGUSR2 handler...
0007 0007 INF Startup initialization finished
0007 0007 INF Signal handler triggered with signal: SIGUSR1 (10)
0007 0007 INF Tracing will be toggled ON (for the first time)
0007 0018 INF Spawning event processing thread...
0007 0130 INF Starting event thread...
0007 0130 INF Data ID: b3e2da0ddf32c424bd7414740d025a80
0007 0130 INF File '/mnt/data/memprof_nemo_1644512060_7_0.mem' opened for writing
0007 0130 INF Writing initial header...
0007 0130 INF Writing wall clock...
0007 0130 INF Writing uptime...
0007 0130 INF Writing environ...
0007 0130 INF Writing maps...
0007 0130 INF Writing binaries...
0007 0018 INF Couldn't find jemalloc in the executable's address space
0007 0018 INF Tracing was enabled
0007 0130 INF Flushing...
0007 0007 INF Signal handler triggered with signal: SIGUSR1 (10)
0007 0007 INF Tracing will be toggled OFF
0007 0026 INF Reloading address space
0007 0026 INF Reloading address space
0007 0026 INF Reloading address space
0007 0026 INF Reloading address space
0007 000a INF Reloading address space
0007 0023 INF Reloading address space
0007 0020 INF Reloading address space
0007 0020 INF Reloading address space
0007 0012 INF Reloading address space
0007 0007 INF Signal handler triggered with signal: SIGUSR1 (10)
0007 0007 INF Tracing will be toggled ON
0007 0007 INF Signal handler triggered with signal: SIGUSR1 (10)
0007 0007 INF Tracing will be toggled OFF
When filtering a timestamp from something like 2022-06-21 22:16:33.701
to 2022-06-21 22:16:33.900
the time is truncated to the nearest second, meaning all data is lost and nothing is returned.
This error occurs on Ubuntu 18.04.2 LTS.
After profiling my application with
LD_PRELOAD=./libmemory_profiler.so ./haystack --haystack .. --needle test -e -b
and starting the server with
./memory-profiler-cli server memory-profiling_haystack_1561391660_18942.dat
the following error appears:
ERROR memory_profiler_cli failed to fill whole buffer
Any ideas on that issue? :)
is there a way to use this tool with jemalloc? I set it up according to https://crates.io/crates/jemallocator. My profile file only includes a few allocations at the very beginning of the applications though. It looks like everything that was allocated with jemalloc is not captured. I am mainly interested in the memory fragmentation graph, which is why I can't just switch to the system allocator for profiling.
It'd be nice to have a quick comparison in the readme about why you would want to use one or the other.
bytehound: 60cb 60cb ERR Failed to fully unwind; some of the stack traces will be incomplete. (Are you using a JIT compiler?)
and crash problem:
#0 0x00007f91115ce7ff in raise () from /usr/lib64/libc.so.6
#1 0x00007f91115b8c35 in abort () from /usr/lib64/libc.so.6
#2 0x00007f91125085e6 in nwind::local_unwinding::abort () at /root/.cargo/git/checkouts/not-perf-af1a46759dd83df9/51003a4/nwind/src/local_unwinding.rs:62
#3 0x00007f91124a7937 in nwind::local_unwinding::on_shadow_stack_no_entry_found (stack_pointer=, expected_index=4, tls=0x7f9110fbf000) at /root/.cargo/git/checkouts/not-perf-af1a46759dd83df9/51003a4/nwind/src/local_unwinding.rs:89
#4 0x00007f91124e98b2 in nwind::local_unwinding::ShadowStack::push (self=, stack_pointer=2, address_location=) at /root/.cargo/git/checkouts/not-perf-af1a46759dd83df9/51003a4/nwind/src/local_unwinding.rs:472
#5 nwind::local_unwinding::LocalAddressSpace::unwind_through_fresh_frames (self=, ctx=..., callback=...) at /root/.cargo/git/checkouts/not-perf-af1a46759dd83df9/51003a4/nwind/src/local_unwinding.rs:905
#6 bytehound::unwind::grab (tls=) at preload/src/unwind.rs:361
#7 0x00007f91125ace41 in nwind_ret_trampoline_start () at src/arch/amd64_trampoline.s:17
Hello,
I have nodejs v11.15.0
and yarn v1.16.0
on ArchLinux.
yarn install fails:
% cargo build --release -p memory-profiler-cli
Compiling server-core v0.1.0 (/opt/memory-profiler/server-core)
error: failed to run custom build command for `server-core v0.1.0 (/opt/memory-profiler/server-core)`
Caused by:
process didn't exit successfully: `/opt/memory-profiler/target/release/build/server-core-f10adbcb2301933b/build-script-build` (exit code: 101)
--- stdout
yarn install v1.16.0
[1/4] Resolving packages...
[2/4] Fetching packages...
info [email protected]: The platform "linux" is incompatible with this module.
info "[email protected]" is an optional dependency and failed compatibility check. Excluding it from installation.
[3/4] Linking dependencies...
[4/4] Building fresh packages...
info Visit https://yarnpkg.com/en/docs/cli/install for documentation about this command.
--- stderr
error /opt/memory-profiler/webui/node_modules/deasync: Command failed.
Exit code: 1
Command: node ./build.js
Arguments:
Directory: /opt/memory-profiler/webui/node_modules/deasync
Output:
events.js:170
throw er; // Unhandled 'error' event
^
Error: spawn node-gyp ENOENT
at Process.ChildProcess._handle.onexit (internal/child_process.js:247:19)
at onErrorNT (internal/child_process.js:429:16)
at processTicksAndRejections (internal/process/task_queues.js:81:17)
at process.runNextTicks [as _tickCallback] (internal/process/task_queues.js:56:3)
at Function.Module.runMain (internal/modules/cjs/loader.js:880:11)
at internal/main/run_main_module.js:21:11
Emitted 'error' event at:
at Process.ChildProcess._handle.onexit (internal/child_process.js:253:12)
at onErrorNT (internal/child_process.js:429:16)
[... lines matching original stack trace ...]
at internal/main/run_main_module.js:21:11
thread 'main' panicked at 'Failed to install the dependencies for the WebUI; child process exited with error code Some(1)! You might want to try to run 'rm -Rf ~/.cache/yarn' and try again.', server-core/build.rs:61:21
note: Run with `RUST_BACKTRACE=1` environment variable to display a backtrace.
I've checked deasync
repository for related issues and tried upgrading it to 0.1.15
by following changes
diff --git a/webui/package.json b/webui/package.json
index a955aee..7640d78 100644
--- a/webui/package.json
+++ b/webui/package.json
@@ -17,6 +17,7 @@
"dependencies": {
"bootstrap": "^4.1.1",
"bootswatch": "^4.1.1",
+ "deasync": "^0.1.15",
"dygraphs": "^2.1.0",
"feather-icons": "^4.7.3",
"font-awesome": "^4.7.0",
diff --git a/webui/yarn.lock b/webui/yarn.lock
index d4b879a..7728097 100644
--- a/webui/yarn.lock
+++ b/webui/yarn.lock
@@ -1386,12 +1386,13 @@ date-now@^0.1.4:
version "0.1.4"
resolved "https://registry.yarnpkg.com/date-now/-/date-now-0.1.4.tgz#eaf439fd4d4848ad74e5cc7dbef200672b9e345b"
-deasync@^0.1.13:
- version "0.1.13"
- resolved "https://registry.yarnpkg.com/deasync/-/deasync-0.1.13.tgz#815c2b69bbd1117cae570152cd895661c09f20ea"
+deasync@^0.1.13, deasync@^0.1.15:
+ version "0.1.15"
+ resolved "https://registry.yarnpkg.com/deasync/-/deasync-0.1.15.tgz#788c4bbe6d32521233b28d23936de1bbadd2e112"
+ integrity sha512-pxMaCYu8cQIbGkA4Y1R0PLSooPIpH1WgFBLeJ+zLxQgHfkZG86ViJSmZmONSjZJ/R3NjwkMcIWZAzpLB2G9/CA==
dependencies:
bindings "~1.2.1"
- nan "^2.0.7"
+ node-addon-api "^1.6.0"
[email protected], debug@^2.1.2, debug@^2.2.0, debug@^2.3.3, debug@^2.6.8, debug@^2.6.9:
version "2.6.9"
@@ -2575,7 +2576,7 @@ [email protected]:
version "2.0.0"
resolved "https://registry.yarnpkg.com/ms/-/ms-2.0.0.tgz#5608aeadfc00be6c2901df5f9861788de0d597c8"
-nan@^2.0.7, nan@^2.9.2:
+nan@^2.9.2:
version "2.11.0"
resolved "https://registry.yarnpkg.com/nan/-/nan-2.11.0.tgz#574e360e4d954ab16966ec102c0c049fd961a099"
@@ -2607,6 +2608,11 @@ nice-try@^1.0.4:
version "1.0.5"
resolved "https://registry.yarnpkg.com/nice-try/-/nice-try-1.0.5.tgz#a3378a7696ce7d223e88fc9b764bd7ef1089e366"
+node-addon-api@^1.6.0:
+ version "1.6.3"
+ resolved "https://registry.yarnpkg.com/node-addon-api/-/node-addon-api-1.6.3.tgz#3998d4593e2dca2ea82114670a4eb003386a9fe1"
+ integrity sha512-FXWH6mqjWgU8ewuahp4spec8LkroFZK2NicOv6bNwZC3kcwZUI8LeZdG80UzTSLLhK4T7MsgNwlYDVRlDdfTDg==
+
node-fetch@^1.0.1:
version "1.7.3"
resolved "https://registry.yarnpkg.com/node-fetch/-/node-fetch-1.7.3.tgz#980f6f72d85211a5347c6b2bc18c5b84c3eb47ef"
After changes in yarn.lock
and packages.json
, yarn add deasync
succeeds.
Now the dependency installation finishes without exceptions.
LD_PRELOAD=./libbytehound.so ./your_application 这个./your_application是什么,如果我是一个apk怎么指向这个程序,我想分析apk软件怎么去分析它,或者是一个linux程序怎么分析,可以具体说一下吗
the Linux man page says:
If ptr is NULL, 0 is returned. [...] This function is a GNU extension.
since this is a glibc extension, bytehound should conform to the glibc specification, as other Linux replacement allocators, such as tcmalloc, jemalloc, mimalloc, and musl already do.
No description, website, or topics provided.
I suggest something like this:
Memory profiler for Linux with leak detection, stack traces collection and web GUI for multiple CPU architectures and ability to use on resource-constrained systems.
With the following topics:
memory-profiler linux embedded mips64 aarch64 arm x86-64 web-gui rust
Hey there,
super cool project! I wonder, would it be possible to make your shadow-stack based unwinding trick reusable for other projects? It would definitely benefit me in heaptrack, and I bet it would benefit others too.
When trying to open a generated profile, the cli panics with an "Address already in use" error.
Both the profiler library and the cli have been build from a fresh checkout from less then an hour old using today's rust nightly.
The profile causing the issue is attached below
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.