Renamed/Deleted test cases appear in Navigator about quick

[karapigeon]

I can't seem to replicate the problem. Closing for now.

[modocache]

Thanks for the issue! I agree that it may be more of an Xcode issue, but I'll keep my peeled for a way to "clear the cache".

[dmeehan1968]

I did discover that if you close the project and reload it clears the navigator.

On 9 Jun 2014, at 11:46, Brian Gesiak [email protected] wrote:

Thanks for the issue! I agree that it may be more of an Xcode issue, but I'll keep my peeled for a way to "clear the cache".

—
Reply to this email directly or view it on GitHub.

[dmeehan1968]

Must admit I'm finding this to be a real pain, the test navigator list is so unreliable. I can't find anyway of resetting its idea of things other than closing the project and reopening. Clearing the build products doesn't seem to help.

I found a reference on StackOverflow where someone had the same problem and suggested switching from the App to Test Target scheme, but I guess that was outdated info as you use the same scheme for both in recent Xcode versions. As clearing the build products doesn't affect its behaviour, this sort of suggests that the test results are kept in memory, and not loaded from disk. Perhaps done this way to support the run single test functionality.

I couldn't find much other mention of the issue, so perhaps doesn't affect people using XCTest (or OCUnit) natively, so suspect that this might be something between Quick's use of the XCTest calls and the IDE not finding the code structure that it thinks should exist, so might be very difficult to resolve, unless someone knows how to game the IDE itself (I believe there is an API for this, but not well documented).

[modocache]

Do you know off the top of your head whether similar issues occur on Xcode 5 with Kiwi or Specta? I'll double-check myself soon.

I found a reference on StackOverflow where someone had the same problem

Link? Maybe we can file a radar.

[dmeehan1968]

I’ve been trying to get to bugreport.apple.com all day to report the earlier swift compiler issue but its not loading for me.

Yes, I think I had similar issues in Xcode 5 and Kiwi, but its been some months since I did much work with Kiwi due to other projects.

This was the SO issue that I found, but relates to Xcode 4

http://stackoverflow.com/questions/6397150/how-do-i-clear-unit-test-results-in-xcode-4

On 10 Jun 2014, at 18:31, Brian Gesiak [email protected] wrote:

Do you know off the top of your head whether similar issues occur on Xcode 5 with Kiwi or Specta? I'll double-check myself soon.

I found a reference on StackOverflow where someone had the same problem

Link? Maybe we can file a radar.

—
Reply to this email directly or view it on GitHub.

[dmeehan1968]

I did a bit more investigation on this. Whilst working on the support for Any, I was writing unit tests in XCTest cases, not Quick. These all worked fine and as can be expected.

Looking at the invocation support that Quick implements (and likewise Kiwi of Obj-C), I think the problem stems from the dynamic selectors that are created to support the tests. Because these are ephemeral (not appearing in the source files) the Test Navigator/IDE can't keep track of them as its designed to do.

Given that the names are mangled from the expectation descriptions, it might not be that relevant to keep them. Related is the proposed renaming on exampleGroups(), #30, which perhaps suggests that we should just have a method called test...() (suffix optional) and allow XCTest to find its by itself. Failing tests should then all be reported under a single test title. Not as nice as XCTest results I know, but to be honest the only use I find for the test navigator is to quickly locate the failing assertion, which it should still do.

[modocache]

Hmm, I'm not sure I agree. My two objections:

1. Grouping all examples under a single test doesn't just cause them to not be displayed in the test navigator; the red line indicating failures is also not displayed in Xcode's editor panes. See the 606d77a commit message for details.
2. It sounds like you're proposing that, since our "ephemeral invocations" are a little buggy in the test navigator, we should just abandon them altogether. I'd like to take a more ambitious approach, attempting to solve the problem if possible.

You have inspired me to file a radar, however.

I think the problem stems from the dynamic selectors that are created to support the tests. Because these are ephemeral (not appearing in the source files) the Test Navigator/IDE can't keep track of them as its designed to do.

I'm not sure if the developer tools team will care about supporting "ephemeral invocations", but it's worth a try!

[dmeehan1968]

I agree its less than perfect. I’d like the Any support to work better, but its just not there at the moment, and so I’d rather leave the functionality out than have something that causes confusion.

The Test Navigator issues I feel the same way - support what works, not whats buggy if it adds little.

I was vaguely aware that there was a problem with jumping to failing tests if they were lumped under one, so thanks for the pointer to the commit that deals with it.

Just feeling a little disappointed that Apple haven’t done a better job with Swift - it might have been nice if they’d announced which of their own apps they had used it on, but I guess the omission suggests that they have not used it for anything other than samples and experiments. If they’d said ‘look at all these Extensions and Continuity features that are written in Swift’, you’d have more confidence that this is a real Beta and not an Alpha.

On 12 Jun 2014, at 10:13, Brian Gesiak [email protected] wrote:

Hmm, I'm not sure I agree. My two objections:

• Grouping all examples under a single test doesn't just cause them to not be displayed in the test navigator; the red line indicating failures is also not displayed in Xcode's editor panes. See the 606d77a commit message for details.
• It sounds like you're proposing that, since our "ephemeral invocations" are a little buggy in the test navigator, we should just abandon them altogether. I'd like to take a more ambitious approach, attempting to solve the problem if possible.
You have inspired me to file a radar, however.

I think the problem stems from the dynamic selectors that are created to support the tests. Because these are ephemeral (not appearing in the source files) the Test Navigator/IDE can't keep track of them as its designed to do.

I'm not sure if the developer tools team will care about supporting "ephemeral invocations", but it's worth a try!

—
Reply to this email directly or view it on GitHub.

[modocache]

Hopefully someone at Apple will be interested in this "bug": http://openradar.appspot.com/radar?id=5793293614972928

I'll keep this ticket open until I get a response. I'm not expecting much, but who knows?

I confirm this issue is ongoing. I'm seeing it in Xcode 6.3 and XCTestCase. The project doesn't contain Quick at all.

The only workaround I found was quitting Xcode, restarting it, then rebuilding all the tests.

[modocache]

Yes, this is an issue with Xcode and XCTest integration. Please file radars!

Sent from my iPhone

On Apr 13, 2015, at 2:47 PM, SixArm [email protected] wrote:

I confirm this issue is ongoing. I'm seeing it in Xcode 6.3 and XCTestCase. The project doesn't contain Quick at all.

―
Reply to this email directly or view it on GitHub.

[kunass2]

http://stackoverflow.com/questions/32264603/my-test-navigator-copies-some-test-cases-from-one-test-class-to-another

I asked about this on stack overflow. it is still not resolved in Xcode7beta6

[karapigeon]

Since this is was a problem with XCTest and Xcode, I think it's safe to close this issue as we've already filed radars with Apple. With that said, I'm going to confirm that the issue has been fixed in the latest non-beta version of Xcode.

[modocache]

Awesome, thanks @istx25!

It might be fun to do "radar drives", where we periodically encourage Quick users to keep filing radars about this issue. But I agree that since there's not much maintainers can do, keeping this issue open is not super valuable.

[karapigeon]

Totally agree. Alternatively, we could possibly make a wiki page where we list out this issue and others in similar situations where we want to get the attention of Apple. I tend to try to keep issue numbers low to reduce confusion/clutter.