Wrong cross-check status shown for Model.CONNECT about fmi-cross-check

[t-sommer]

The results are calculated by this script. Can you step through the code and try find to find the issue?

[klausschuch]

I checked the script and I think I found the reason for the confusion:
The numbers shown in the detailed view are the sum of all passed FMUs of ALL versions of an exporting tool.
This script checks if there are >= 3 passed FMUs of ONE version of an exporting tool.

The corresponding sentence of the cross-check-rules does not mention the tool version:

FMU importing tools must successfully import at least 3 FMUs for at least one supported platform of at least 3 exporting tools (3×3 FMUs per platform) by other vendors and run for as long as the {Model_Name}_ref.opt states and supply a CSV file of the solution they computed to receive a green status (Filtering – e.g. by platform – will lead to limiting the display of tools).

My interpretation was that all version count and this is what the detailed view indicates.
(With the old system, the user could see the results per version and could figure out why the importing tool got green/blue labels.)

@andreas-junghanns
However, this should be clarified in the rules.

[t-sommer]

@klausschuch, can we close this issue?

[klausschuch]

@t-sommer The description in the cross check rules document should be clarified first.
I guess this needs a decision by the fmi steering committee.

@andreas-junghanns Could you please comment on this topic?

[andreas-junghanns]

OK, this apparently needs clarification in the rules. (Please create a PR with a clarification that you would understand!)
We need to release new rules soon anyway.

I would clarify it in the stricter sense - we should encourage proper testing.
Also because the current scripts seem to be implemented that way and only require fixing the rules.

I am also not worried about AVL being on the list: AVL is tough enough to supply sufficient proof of compliance.

(Don´t complain now: you asked me to comment.)

[klausschuch]

@andreas-junghanns
I am not complaining at all. I just want to have this clarified in the rules.
We uploaded new results a while ago but unfortunately to the wrong directory. To get this stupid mistake fixed, I'm waiting for #93 to get merged.

[andreas-junghanns]

@andreas-junghanns
I am not complaining at all. I just want to have this clarified in the rules.
We uploaded new results a while ago but unfortunately to the wrong directory. To get this stupid mistake fixed, I'm waiting for #93 to get merged.

Can you not merge this yourself?

@t-sommer : if you are the only one that can merge tickets, please merge this, or explain why you cannot merge yet

[chrbertsch]

I propose to close this.
#93 is already merged, #119 shlall clarify the counting