llogiq / flame Goto Github PK

Minimal test case:

use rayon::prelude::*;
use flame;
use std:: fs::File;

fn something(i: usize) -> f64 {
    flame::start("Something");
    std::thread::sleep(std::time::Duration::from_millis(10));
    flame::end("Something");
    0.56
}
fn main() {
    let a = (0..200).collect::<Vec<_>>()
        .par_iter()
        .map(|i| something(*i))
        .collect::<Vec<_>>();

    flame::dump_html(&mut File::create("flames.html").unwrap()).unwrap();
    println!("{:?}", flame::threads());
}

Dependencies:

rayon = "1.0.3"
flame = "0.2.2"

The written file has no graphics.
The output is: [Thread { id: 140470522639296, name: Some("main"), spans: [], _priv: () }], so no data is available.

As the 0.1.12 update yesterday, I noticed that building lib unit tests is failing in unicode-bidi with flame_it feature. I went back and minimized the changes for the break, which actually reduced to only extern crate flame;, and only happening for =0.1.12.

Here's the commit causing error on a stable unicode-bidi build: behnam/rust-unicode-bidi@6e6fc78

Here's the error:

error[E0283]: type annotations required: cannot resolve `level::Level: std::convert::Into<_>`
   --> src/level.rs:331:31
    |
331 |         assert_eq!(1u8, level.into());
    |                               ^^^^

error: aborting due to previous error

from https://travis-ci.org/behnam/rust-unicode-bidi/jobs/243806493#L259

I have no idea how flame can cause this in a module that doesn't use any of the traits. What do you think, @TyOverby ?

https://github.com/TyOverby/flame/blob/master/Cargo.toml#L10

I use flame for a ray tracer, which roughly correspond to

for i in 0..x {
  for j in 0..y {
    let _guard = flame::start_guard("compute color");
    out[i][j] = compute_color(i, j);
  }
}

The trouble is that I get x*y "compute color" frames, each taking 0.001% of running time. Is there a way to combine these in a single frame, aside from doing everything manually with flame::frames()?

It would be great if you could could see the time in milliseconds spent on different events in the HTML view. When you hover over a slice it will show you the percentage and the number of samples, but it doesn't show you milliseconds like it does if you do a dump_stdout().

I am not writing a game, but numerical code, such that algorithms have loops with some interesting parts inside the loop and some outside.

So I want a span before the loop starts, a span on the whole loop, inside the loop each iteration is a frame, and more spans inside the loop, partitioning each frame.

Currently this results in errors like thread 'logisticregression3' panicked at 'flame::end("SublinearAveragingSolver") called without a currently running span!', /home/danielv/.cargo/registry/src/github.com-88ac128001ac3a9a/flame-0.1.5/src/lib.rs:257
(presumably because my next_frame inside the loop is hiding the loop scope flame::start("Sublinear...");

Does this make sense?

To allow easy insertion of frames without having to rely on the guard interface, having a method like flame::end_with<T>(frame: &str, result: T) -> T could make insertion on exit easier. Otherwise we'd have to create a temporary to store a return value.

Feature request: This would be much more ergonomic if we had a procedural macro to insert guarded events on all function scopes, optionally also in loops or even on every scoped block. This could be an annotation macro like #[flame(loops)] and work on single functions or even whole modules (including submodules). Also perhaps allow a #[noflame] annotation to carve out un-instrumented code parts.

What do you think?

I'm running into an issue where when I call flame::start("repl") and then later flame::end("repl") from one of my functions, the end call results in this panic:

error: thread 'main' panicked at 'flame::end(repl) attempted to end end'

I don't ever call start/end with "end" as an argument that I know of, unless flamer is doing that for some reason. When I have a few minutes to kill I'm gonna go through my project and comment out all of the start/end calls until I find the minimal repro case, but I figured I'd post here in the meantime in case anyone else has any intuition about what would cause this.

Hi!

This looks great and easy to use. Is there any API planned for folding the results (like https://github.com/brendangregg/FlameGraph)? I want to profile some tight loops and only a few seconds of running my program generates 50+MB html file...

#25 was closed, but workaround mentioned there didn't work...

+ /usr/bin/cargo test --release -j1
       Fresh serde v1.0.19
       Fresh num-traits v0.1.40
       Fresh dtoa v0.4.2
       Fresh lazy_static v0.2.9
       Fresh itoa v0.3.4
       Fresh unicode-xid v0.1.0
       Fresh libc v0.2.33
       Fresh quote v0.3.15
       Fresh serde_json v1.0.6
       Fresh synom v0.11.3
       Fresh thread-id v3.2.0
       Fresh syn v0.11.11
       Fresh serde_derive_internals v0.17.0
       Fresh serde_derive v1.0.19
   Compiling flame v0.2.0 (file:///builddir/build/BUILD/flame-0.2.0)
     Running `/usr/bin/rustc --crate-name flame src/lib.rs --emit=dep-info,link -C opt-level=3 --test --cfg 'feature="default"' --cfg 'feature="json"' --cfg 'feature="serde"' --cfg 'feature="serde_derive"' --cfg 'feature="serde_json"' -C metadata=55a8dd78fadf7079 -C extra-filename=-55a8dd78fadf7079 --out-dir /builddir/build/BUILD/flame-0.2.0/target/release/deps -L dependency=/builddir/build/BUILD/flame-0.2.0/target/release/deps --extern serde=/builddir/build/BUILD/flame-0.2.0/target/release/deps/libserde-9be79e3bf217c101.rlib --extern serde_json=/builddir/build/BUILD/flame-0.2.0/target/release/deps/libserde_json-c4e6d1533382957b.rlib --extern serde_derive=/builddir/build/BUILD/flame-0.2.0/target/release/deps/libserde_derive-b8e3961dc7f06b8e.so --extern lazy_static=/builddir/build/BUILD/flame-0.2.0/target/release/deps/liblazy_static-81a739ef1bc96bbc.rlib --extern thread_id=/builddir/build/BUILD/flame-0.2.0/target/release/deps/libthread_id-4879e83dca0b7ebf.rlib -Copt-level=3 -Cdebuginfo=2 -Clink-arg=-Wl,-z,relro,-z,now`
     Running `/usr/bin/rustc --crate-name demo examples/demo.rs --crate-type bin --emit=dep-info,link -C opt-level=3 --cfg 'feature="default"' --cfg 'feature="json"' --cfg 'feature="serde"' --cfg 'feature="serde_derive"' --cfg 'feature="serde_json"' -C metadata=1a0c0dff5daa6f29 -C extra-filename=-1a0c0dff5daa6f29 --out-dir /builddir/build/BUILD/flame-0.2.0/target/release/examples -L dependency=/builddir/build/BUILD/flame-0.2.0/target/release/deps --extern serde=/builddir/build/BUILD/flame-0.2.0/target/release/deps/libserde-9be79e3bf217c101.rlib --extern serde_json=/builddir/build/BUILD/flame-0.2.0/target/release/deps/libserde_json-c4e6d1533382957b.rlib --extern serde_derive=/builddir/build/BUILD/flame-0.2.0/target/release/deps/libserde_derive-b8e3961dc7f06b8e.so --extern lazy_static=/builddir/build/BUILD/flame-0.2.0/target/release/deps/liblazy_static-81a739ef1bc96bbc.rlib --extern thread_id=/builddir/build/BUILD/flame-0.2.0/target/release/deps/libthread_id-4879e83dca0b7ebf.rlib --extern flame=/builddir/build/BUILD/flame-0.2.0/target/release/deps/libflame-1016207c5c092409.rlib -Copt-level=3 -Cdebuginfo=2 -Clink-arg=-Wl,-z,relro,-z,now`
     Running `/usr/bin/rustc --crate-name tests tests/tests.rs --emit=dep-info,link -C opt-level=3 --test --cfg 'feature="default"' --cfg 'feature="json"' --cfg 'feature="serde"' --cfg 'feature="serde_derive"' --cfg 'feature="serde_json"' -C metadata=8b7dc1aeb3c717dd -C extra-filename=-8b7dc1aeb3c717dd --out-dir /builddir/build/BUILD/flame-0.2.0/target/release/deps -L dependency=/builddir/build/BUILD/flame-0.2.0/target/release/deps --extern serde=/builddir/build/BUILD/flame-0.2.0/target/release/deps/libserde-9be79e3bf217c101.rlib --extern serde_json=/builddir/build/BUILD/flame-0.2.0/target/release/deps/libserde_json-c4e6d1533382957b.rlib --extern serde_derive=/builddir/build/BUILD/flame-0.2.0/target/release/deps/libserde_derive-b8e3961dc7f06b8e.so --extern lazy_static=/builddir/build/BUILD/flame-0.2.0/target/release/deps/liblazy_static-81a739ef1bc96bbc.rlib --extern thread_id=/builddir/build/BUILD/flame-0.2.0/target/release/deps/libthread_id-4879e83dca0b7ebf.rlib --extern flame=/builddir/build/BUILD/flame-0.2.0/target/release/deps/libflame-1016207c5c092409.rlib -Copt-level=3 -Cdebuginfo=2 -Clink-arg=-Wl,-z,relro,-z,now`
    Finished release [optimized] target(s) in 5.23 secs
     Running `/builddir/build/BUILD/flame-0.2.0/target/release/deps/flame-55a8dd78fadf7079`
running 0 tests
test result: ok. 0 passed; 0 failed; 0 ignored; 0 measured; 0 filtered out
     Running `/builddir/build/BUILD/flame-0.2.0/target/release/deps/tests-8b7dc1aeb3c717dd`
running 11 tests
test dropped_guarded_event ... ok
test double_nested ... ok
test cant_note ... ok
test implicit_guarded_event ... ok
test end_with ... ok
test named_guarded_event ... ok
test single_event ... ok
test single_nested ... ok
test multiple_guard_early_return ... ok
test wrong_name ... ok
test threads ... FAILED
failures:
---- threads stdout ----
	thread 'threads' panicked at 'assertion failed: `(left == right)`
  left: `2`,
 right: `6`', tests/tests.rs:111:4
failures:
    threads
test result: FAILED. 10 passed; 1 failed; 0 ignored; 0 measured; 0 filtered out

I think at least, that this is how it works? I spent some time trying to figure out why I got an empty graph when all profiling happens in other threads.

I noticed that some methods in the online doc has been removed from the current published version of flame, for example the dump_svg. Also some newly added methods (threads() mentioned in #11 , e.g.) haven't been included there.

I tried to cargo doc the whole thing locally but it took extremely long time ( which I suspect was because of documenting some crate-dependencies of thread_id).

We do have the choice of fallback to reading source code directly though, but maybe it would be nice to update the online documentation? Is this the appropriate timing to do so, or should we be waiting for some further rounds of iterations?

It appears like flame::dump_html writes json data to a script tag in the generated html file, without doing json or html escaping. That means that the generated page could be unable to load due to syntax errors or get incorrect values.

Trying out the flamegraph in https://github.com/RustPython/RustPython, I got syntax errors in the generated data such as:

{
name: "init VirtualMachine",
value: 307664657,
start: 80780829,
end: 388445486,
children: [
{
name: "call_method("__setattr__")",
value: 195873,
start: 80813067,
end: 81008940,
children: [
{

The flamegraph hade spans named call_method("__setattr__"), where the quotes would have to be escaped in order to be put in a javascript/json string.

As noted on http://www.brendangregg.com/FlameGraphs/cpuflamegraphs.html#description

The x-axis spans the sample population. It does not show the passing of time from left to right, as most graphs do. The left to right ordering has no meaning (it's sorted alphabetically to maximize frame merging).

Looks like rust flame is not doing any merging, which results in:

• technically not being a CPU Flame Graph;
• hard to see where the actual CPU usage is, as where the number of calls are high, the names don't fit in the small lines, and even the lines become invisible.
• too many SVG objects on the screen that slow down the rendering and digging in.

I am currently profiling some graphics code and I saw that currently open spans are not plotted in the output and this produces misleading stack plots.

I wrote a counter for a short number of frames and after that, I plot the flamegraph (html) and I shout down the application. This call happens inside of the function that renders the current frame and therefore is being profiled. the span is not present in the plot, although the nested ones (already finished) are.

I know that this is not the most pretty coding style, but since it can be written: flame should do the right job here... the question is how.

Anyhow, great job here. It is a very nice tool, especially when used with flamer.

	thread 'threads' panicked at 'assertion failed: `(left == right)` (left: `5`, right: `6`)', tests/tests.rs:111

Not sure which information to provide.

Currently, a thread-local library is used to collect frames. However, this fails when using multiple threads; the report will only ever contain frames for the thread that started the reporting. So how to report for multiple threads?

I think using a thread local library for collection is the only viable choice (otherwise we'd synchronize cores and thus destroy the performance we intend to measure). Using this for multiple threads would probably require some kind of global library registry (where each thread local would need to be registered on construction, and would also require moving out of the thread on drop). Also we'd need a way to display data from multiple threads without thoroughly confusing users.

Since your lib requires modifications to the source code, it's not easy to disable it for a true release. I see two ways to fix this:

• Add a compile time feature which prevents data collection. I'm thinking global const with a compile time value, and a if !STATIC_ENABLED { return } in every function. Since the condition is known at compile time, it should be inlined and removed by rustc.
• Add a runtime switch, like flame::disable() for example. It could set an AtomicBool. It would be a bit slower but would allow the activation without recompiling. Now that I thing of it, it would be a very nice feature to have!

These two options could be implemented at the same time, like if !STATIC_ENABLED || !DYNAMIC_ENABLED ....

The crate's code uses extern crate serde_derive without #[cfg(feature = "json")], meaning that it doesn't compile if the json feature is disabled.

The published documentation lists methods which do not exist in the latest version on Crates.io.

I also can’t find code in the repo which matches the documentation which is currently published.

Could you clarify what’s gone on here?

Right now, notes are only in the in-memory data structures, they don't get output to stdout or HTML.

The README still says .frames() whereas the lib.rs already says .spans().

Apache-2.0 requires text license to be presented along with source... Having MIT text license would not hurt as well.

Following up the problem we faced in servo/unicode-bidi#40, I'm able to repro the problem consistently on Ubuntu 12.04.5 with latest nightly rust.

Here's the repro:

~/code/rust-unicode-bidi$ RUST_BACKTRACE=1 RUST_TEST_THREADS=1 cargo +nightly test --verbose --lib --features flame_it -- --nocapture >/dev/null
       Fresh matches v0.1.6
       Fresh serde v1.0.8
       Fresh libc v0.2.23
       Fresh winapi-build v0.1.1
       Fresh winapi v0.2.8
       Fresh lazy_static v0.2.8
       Fresh serde_test v1.0.8
       Fresh kernel32-sys v0.2.2
       Fresh thread-id v2.0.0
       Fresh flame v0.1.11
       Fresh flamer v0.1.4
       Fresh unicode-bidi v0.3.3 (file:///home/behnam/code/rust-unicode-bidi)
    Finished dev [unoptimized + debuginfo] target(s) in 0.0 secs
     Running `/home/behnam/code/rust-unicode-bidi/target/debug/deps/unicode_bidi-f20b46f3613e37fb --nocapture`
thread '<unnamed>' panicked at 'use of std::thread::current() is not possible after the thread's local data has been destroyed', /checkout/src/libcore/option.rs:823
stack backtrace:
   0: std::sys::imp::backtrace::tracing::imp::unwind_backtrace

The failure is caused by the std::thread::current() expecting thread dat to not be destroyed: https://doc.rust-lang.org/1.3.0/src/std/thread/mod.rs.html#409

Basically, when flame (and flamer) are enabled, a thread panics at the ends of the test. (The tests themselves all pass with ok message.) Without flame/flamer crates loaded, there's no panic.

Crate dependency tree is:

├── flame v0.1.11
│   ├── lazy_static v0.2.8
│   └── thread-id v2.0.0
│       ├── kernel32-sys v0.2.2
│       │   └── winapi v0.2.8
│       └── libc v0.2.23

I'm guessing that the bug is somewhere under thread-id. (cc @ruuda)

I haven't been able to repro it in any later Ubuntu version. I'll report if and when I can.

I found that you have a threads() function, from which I can get a Vec<Span> for each thread. But Span only has a function to write to json, not html. How can I get HTML output from different threads? Or alternatively, what can I do with a JSON file to display the flame graph?

Now that inferno is relatively stable, and exposes separate APIs for both stack folding and producing flamegraphs, I wonder if it'd be interesting to see if we could make flame use inferno to generate the flamegraphs. That way you wouldn't need to vendor all the JS code and such into this project. You'd also get stack folding for free if you wished!

Hello, I'm trying to use this across multiple crates, is this possible?