waitUntil and "serial" execution of XCTests about nimble

[nikita-leonov]

Disregard request. Apparently we had "dispatch_async(dispatch_get_main_queue()) {" in a parallel test that was not wrapped into waitUntil, so test case was "finished" buts its code still was working and affected tests close to it.

[nikita-leonov]

I actually would like to reopen bug, as it is not clearly some specific project implementation issue but more fundamental Nimble issue. Here is a test that clearly shows how issue looks like — https://github.com/nikita-leonov/NimbleWaitUntilError

with some pre-steps following "it" will fail:

it("will fail.") {
  waitUntil { done in
    done()
  }
}

I didn't investigate yet on alternative for waitUntil implementation, but hopefully there is a way that will allow to provide better isolation.

[jeffh]

Hey @nikita-leonov thanks for the bug report.

I looked into this, but I'm also not sure how to resolve this. Due to the nature of the run loop, there is no way to advance the run loop without potentially hitting expensive code on the main thread. It is currently intended behavior, but not ideal.

As a reference note, XCTest's async expectations also exhibits this failure (although it bails out). The best I can do is bailout early when seeing a stalled main thread.

[nikita-leonov]

@jeffh thanks for an update. Indeed solution of such task is questionable, while having some additional log information may at least give an understanding that there is a "third-party" effects affecting test results and require additional care. Indeed solution proposed by you may save my day if it was in code before :)