Opening .xdf generated by LabRecorder about pyxdf

[nejemia]

Dear Clemens. I do appreciate your support very much indeed.
I am now trying to get the LabRecorder to work with its GUI, I was able to get only the CLI version...
Will try to recompile everything form the source codes instead from the Apps contained in the "labstreaminglayer" package.
The CLI version is not self explanatory in its use, So I can not be sure if closed the file correctly or not...
Take a lot of care !
N

[cboulay]

@cbrnr @agricolab
Is there any reason this line casts the srate to an int?
nominal_srate=int(chunk["nominal_srate"])

[nejemia]

Hi. Eliminated the cast to int() in line 731.
Now it opens .xdf happy.
May be this cast remains from old Python 2 version ...
Will continue testings.
Rgds

[cboulay]

There should still be a numeric cast to make sure it doesn't stay a string.

If there's any advantage to keeping it an int when the nominal_srate is declared as an int, then we can try

nominal_srate = ast.literal_eval(chunk["nominal_srate"])
or

try:
    nominal_srate = int(chunk["nominal_srate"])
except ValueError:
    nominal_srate = float(chunk["nominal_srate"])

Of course, if there's no benefit to keeping it an int when appropriate, then we can just replace int with float.

[agricolab]

If i read the code correctly, the regular loading of the file within load_xdf uses StreamData to initialize information about the stream in L299. There, self.srate = round(float(xml['info']['nominal_srate'][0])).

The requirement for an int comes from parse_chunks in L749, which is only used in resolve_streams, which is only called if specific streams were selected during loading (L213).

Both implementations parse the string from f.read with ET.fromstring, either in L777 or in L293. It should be noted that load_xdf decodes the string with errors='replace' in L293, while the other implementation L777 uses the decode default, which is strict.

I can't see a logical reason why it needs to be an int, a float seems also feasible, especially as this is the type of the classical implementation in load_xdf. For consistency with the original parser, i suggest to use round(float(chunk["nominal_srate"])) or similar.

[cbrnr]

I don't really remember why I used int(...) instead of round(float(...)) - maybe it was just a copy and paste error, but I vaguely remember some edge case where this needed to be an int. Let me do some testing with my test files locally before we make this change.

[cbrnr]

Looking at the problematic file, it seems this was generated with the (latest?) LabRecorder which already saves files as XDF 1.1. I'm not sure if this is intentional, because AFAIR we haven't converged on whether or not to bump the XDF version number yet. Furthermore, this version is stored as a floating point number with 15 decimal places in the XML, i.e. <version>1.100000000000000</version>. This should definitely be changed to just 1 decimal place (regardless of whether or not it remains 1.1 or gets reverted to 1.0).

The same floating point format seems to be applied to nominal_srate; I suggest that LabRecorded should use a sensible format for this value (i.e. if it is an integer write no decimal places, if it is a float use the minimum number of decimal places necessary).

[agricolab]

According to XDFBrowser, the header reads <?xml version="1.0"?><info><version>1.0</version></info>, so i guess the LabRecorder Bump was not the culprit.

I agree that LabRecorder should save floats with less decimal places. This would prevent the issue at hand. But non-integer sampling rates are always a possibility, and these are always invalid literals for int. So i guess we should fix the parsing, too.

[cbrnr]

According to XDFBrowser, the header reads <?xml version="1.0"?><info><version>1.0</version></info>, so i guess the LabRecorder Bump was not the culprit.

I'm sorry, you're right, the file header version is 1.0 - I accidentally looked at a StreamHeader chunk, which for this file includes the fields <nominal_srate>1000.000000000000</nominal_srate> and <version>1.100000000000000</version>. I guess the version tag refers to something else, but I still think it's not necessary to include 15 decimal places.

I agree that LabRecorder should save floats with less decimal places. This would prevent the issue at hand. But non-integer sampling rates are always a possibility, and these are always invalid literals for int. So i guess we should fix the parsing, too.

Yes, definitely. I'm currently looking into that to find out if the cast to int was necessary for some file. I don't think so, but it's better to make sure before we change it to round(float(...)). However, this means that every non-integer sampling rate will be converted to integer (which I think is OK).

[cbrnr]

I didn't find any negative effects of changing that value to a rounded float, so let's go ahead and resolve this issue in #65 (I made some minor comments).

[cboulay]

Non-integer sampling rates are rare but they are definitely a thing. We shouldn't be rounding at all.

[cbrnr]

But we're talking about nominal_srate as claimed by the recording device. We can always measure the true sampling rate with effective_srate (where it's OK and expected that this is a float).

In any case, we have always rounded nominal_srate in load_xdf. If you think we shouldn't be doing that please open a new issue and we can discuss.

[cbrnr]

Also, BTW this XDF file does not have StreamFooter chunks for both streams. These chunks are mandatory, so technically this is not a valid XDF file. @nejemia you said that you used LabRecorder to record the file - was the recording properly closed?

[nejemia]

LabRecorder used was compiled locally on Raspberry PI running latest Debian image. I just made only one attempt and I was not careful in opening nor closing the file. Shame on me.
I am now focused on getting mnelab to run on Windows 10... So far, does not work.
More testings to come.
Keep healthy and thanks a million.
Hut off for your incredible and fantastic work, geniuses.
N

[cbrnr]

Thanks @nejemia. Please report any bugs you find with MNELAB, I'm happy to help getting it to run on your machine.