backtrace can infinte-loop. about probe-run

[Lotterleben]

reporting back for the record and also as a note for myself for when I pick this back up–
the problem is indeed that stack_corrupted doesan't paint the full picture, but this one is tricky to detect – after 9:, the program counter doesn't change (checked by lr & !THUMB_BIT == pc & !THUMB_BIT), but the CFA (Canonical Frame Address) keep shifting, as the HardFault occurs in a loop https://github.com/embassy-rs/embassy/blob/f8c434400393d74fb51acb130bbaaf835e7edf38/embassy/src/executor/mod.rs#L202 .
We can't just make the && a || because in other cases like for example.... several recursive function calls, this is a perfectly legal state that should not interrupt the backtrace printing.
We've checked if gdb does some detection magic, but it seems to run into the same problem:

(paging prevents the endless scrolling here though, which is nice)

As a band-aid, I'll implement a backtrace line limit after which the backtrace is cut off (length reconfigurable by flag) to prevent it from scrolling forever. If we're feeling fancy we could additionally cut out the middle and only print, say, the top 10 and bottom 10 lines separated by a [...] by default for better glance-ability.

[Lotterleben]

Closing this now due to #179 being merged– @Dirbaio , feel free to re-open if you disagree or have new input

[japaric]

I would have expected infinite loops like these to be caught by this check. Would be interesting to log LR, PC and other registers as probe-run unwinds the stack.

I'm seeing this behavior with -C force-frame-pointers=no.

did you compile the whole Rust code with -C force-frame-pointers=no or is it just the assembly (asm.rs?) what was compiled w/o frame pointers? The backtrace looks fine for the first 17 frames or so.

[Urhengulas]

Hi @Dirbaio, Could you please provide us with some code to reproduce this error? Thanks in advance! 😄

[Dirbaio]

Attached ELF: repro.tar.gz
Built from this commit akiles/embassy@f8c4344

[dirbaio@mars embassy-nrf-examples]$ probe-run --version
0.2.1
supported defmt version: 0.2
[dirbaio@mars embassy-nrf-examples]$ rustc --version
rustc 1.52.0-nightly (a15f484b9 2021-02-22)
[dirbaio@mars embassy-nrf-examples]$ cargo build --bin rtc_async
    Finished dev [optimized + debuginfo] target(s) in 0.05s
rustc 1.52.0-nightly (a15f484b9 2021-02-22)
[dirbaio@mars embassy-nrf-examples]$ probe-run --chip nRF52840_xxAA ../target/thumbv7em-none-eabi/debug/rtc_async
  (HOST) INFO  flashing program (11.52 KiB)
  (HOST) INFO  success!
────────────────────────────────────────────────────────────────────────────────
       0 INFO  Hello World!
└─ rtc_async::__cortex_m_rt_main @ src/bin/rtc_async.rs:42
       1 INFO  tick
└─ rtc_async::run2::task::{{closure}} @ src/bin/rtc_async.rs:31
       2 INFO  BIG INFREQUENT TICK
└─ rtc_async::run1::task::{{closure}} @ src/bin/rtc_async.rs:22
       3 INFO  tick
└─ rtc_async::run2::task::{{closure}} @ src/bin/rtc_async.rs:31
       4 INFO  tick
└─ rtc_async::run2::task::{{closure}} @ src/bin/rtc_async.rs:31
       5 INFO  tick
└─ rtc_async::run2::task::{{closure}} @ src/bin/rtc_async.rs:31
       6 INFO  tick
└─ rtc_async::run2::task::{{closure}} @ src/bin/rtc_async.rs:31
stack backtrace:
   0: HardFaultTrampoline
      <exception entry>
   1: rtc_async::run1::task::{{closure}}
        at src/bin/rtc_async.rs:24
   2: <core::future::from_generator::GenFuture<T> as core::future::future::Future>::poll
        at /home/dirbaio/.rustup/toolchains/nightly-x86_64-unknown-linux-gnu/lib/rustlib/src/rust/library/core/src/future/mod.rs:80
   3: embassy::executor::Task<F>::poll
        at /home/dirbaio/akiles/embassy/embassy/src/executor/mod.rs:71
   4: embassy::executor::timer_queue::TimerQueue::update
        at /home/dirbaio/akiles/embassy/embassy/src/executor/timer_queue.rs:35
   5: embassy::executor::raw::Executor::run_queued::{{closure}}
        at /home/dirbaio/akiles/embassy/embassy/src/executor/raw.rs:135
   6: embassy::executor::run_queue::RunQueue::dequeue_all
        at /home/dirbaio/akiles/embassy/embassy/src/executor/run_queue.rs:66
   7: embassy::executor::raw::Executor::run_queued
        at /home/dirbaio/akiles/embassy/embassy/src/executor/raw.rs:117
   8: cortex_m::asm::wfe
        at /home/dirbaio/.cargo/registry/src/github.com-1ecc6299db9ec823/cortex-m-0.7.1/src/asm.rs:49
   9: embassy::executor::Executor::run
        at /home/dirbaio/akiles/embassy/embassy/src/executor/mod.rs:202
  10: cortex_m::asm::wfe
        at /home/dirbaio/.cargo/registry/src/github.com-1ecc6299db9ec823/cortex-m-0.7.1/src/asm.rs:49
  11: embassy::executor::Executor::run
        at /home/dirbaio/akiles/embassy/embassy/src/executor/mod.rs:202
  12: cortex_m::asm::wfe
        at /home/dirbaio/.cargo/registry/src/github.com-1ecc6299db9ec823/cortex-m-0.7.1/src/asm.rs:49
  13: embassy::executor::Executor::run
        at /home/dirbaio/akiles/embassy/embassy/src/executor/mod.rs:202
  14: cortex_m::asm::wfe
        at /home/dirbaio/.cargo/registry/src/github.com-1ecc6299db9ec823/cortex-m-0.7.1/src/asm.rs:49
  15: embassy::executor::Executor::run
        at /home/dirbaio/akiles/embassy/embassy/src/executor/mod.rs:202
  16: cortex_m::asm::wfe
        at /home/dirbaio/.cargo/registry/src/github.com-1ecc6299db9ec823/cortex-m-0.7.1/src/asm.rs:49
.... infinite loop

[Urhengulas]

Thank you @Dirbaio for providing the reproducable!

---

Jonas mentioned in #163 that following condition is likely the cause for this bug:

probe-run/src/main.rs

Line 801 in d702944

let stack_corrupted = !cfa_changed && lr & !THUMB_BIT == pc & !THUMB_BIT;