Comments (19)
For anyone looking for a python wrapper for an xdf writer, @agricolab has done a great job with pyLieSL, following the example for the LabRecorderCLI here.
from pyxdf.
Hi @cbrnr,
Indeed, we have written an XDF writer for our needs at our workplace. Sorry for the delay in responding to you, I have taken some time to look at both code bases and look at potential barriers to submitting the code here. I will describe the code we have and the potential barriers to merging. Then we can decide how to proceed together.
- pxydf only requires
numpy
. Our code requirespydantic
. Would you be open to adding pydantic as a requirement? (I understand that adding requirements is a big decision, so let's decide this up front) - Our writer is not a streaming writer as described in this issue. It simply writes an entire file in one shot. Would this be acceptable?
Thanks for considering these issues. It would be great to clarify these issues up front. The next step would be to decide on an API, then I could open a PR. Thanks 🙏
from pyxdf.
Thank you @cbrnr, @agricolab, and @cboulay for your considered responses.
From the feedback above I understand that adding pydantic as a dependency is not ideal, I understand the reasons for this. Unfortunately with our current workload, we can not commit to rewriting our xdf writer to remove pydantic. Thanks for your consideration. We will keep this on our internal project list, and if our development capacity increases we can revisit this in the future.
Thanks for maintaining such a great project, we use it regularly and hope to contribute back when we can.
from pyxdf.
I have a rudimentary exporter nearly complete in pure python using numpy and such at the moment. Needs some adjustments for the XDF to be readable in sigviewer. I was also planning to add a few features not included in the LabRecorder / listed here. I will post here when I am done but I'm going to be delayed until after my MCAT in Mid-Jan. I'll be in touch.
After I get the python basic functionality up and running, I'll toss my code onto GitHub and post a link here. I also plan to re-implement it in C or C++ afterward.
from pyxdf.
I've also started a prototype, could you upload your not-yet-ready code so I could take a look at it?
from pyxdf.
I don't suppose there's been any progress on this front from anyone? @tstenner @Argzero
from pyxdf.
Hi, any progress on this? I'm looking for a way of loading an XDF, modifying it and saving it back. I have checked pyLieSL but it doesn't seem to be able to do that (correct me if I'm wonrg).
from pyxdf.
No, it can't.
from pyxdf.
Would be great to have this feature
from pyxdf.
@rob-luke, you mentioned elsewhere that you (or someone from your group) might be interested in contributing. Is this still true? Do you want to discuss your ideas here? I don't think that anyone else currently has the resources to implement a writer.
from pyxdf.
Regarding the second point, I don't think we intended to implement a streaming writer. This is already done by LSL, and the purpose of an XDF writer here would be to dump data (which is already available in some form TBD) into XDF.
Re pydantic, would it be possible to remove the type checks? Or does your writer critically rely on this functionality so it doesn't work without it?
from pyxdf.
Re pydantic, would it be possible to remove the type checks? Or does your writer critically rely on this functionality so it doesn't work without it?
Unfortunately, it critically relies on this and won't work without it. My primary concern here is that this will change the versions of python that pyxdf would work with. Plus the additional changes to maintenance etc.
from pyxdf.
This would require Python 3.7+, which is fine with me. I've never worked with pydantic, but I assume it could be replaced by data classes (and removing all type checks) if we really don't want it?
from pyxdf.
I assume it could be replaced by data classes (and removing all type checks) if we really don't want it?
Correct. Pydantic handles lots of checking of types and conversion too, to ensure you pass in the right data. So this would need to be all written manually instead.
from pyxdf.
OK, so it could be done, and the least effort solution would be to not check and convert at all, but assume that the input is in the desired format.
Let's wait what others have to say though. @cboulay @tstenner @agricolab
from pyxdf.
Thanks for the contribtution, @rob-luke!
My five cents: Although i personally am a fan of type hints and their benefits for static code analysis and readability, their enforcement during runtime is not pythonic, and prevents a lot of things i love in python (like easy monkey patching). I would prefer to drop type checking.
As i understand it, you use the type for control flow, don't you? If the type of an argument is relevant for control flow, overloading could be alternative solution which is easy to extend, and would even work for init: E.g. https://github.com/dabeaz/python-cookbook/blob/master/src/9/multiple_dispatch_with_function_annotations/example1.py#L94-L121
from pyxdf.
FYI, LabRecorder was originally a Python app so you can find some inspiration in that old code.
https://code.google.com/archive/p/labstreaminglayer/source/default/source
from pyxdf.
Thank you @cbrnr, @agricolab, and @cboulay for your considered responses.
From the feedback above I understand that adding pydantic as a dependency is not ideal, I understand the reasons for this. Unfortunately with our current workload, we can not commit to rewriting our xdf writer to remove pydantic. Thanks for your consideration. We will keep this on our internal project list, and if our development capacity increases we can revisit this in the future.
Thanks for maintaining such a great project, we use it regularly and hope to contribute back when we can.
Hi @rob-luke , is it possible to find a link to your code anywhere which uses pydantic for other to use in the mean time?
from pyxdf.
I have a writer in the works that creates .xdf files that is equivalent to the implementation of the LabRecorder's XDFwriter. Currently, it:
-
- Written for normal lists or numpy arrays.
-
- Thread-safe, for multi-streaming (assuming that headers and footers are written at the start and end of the writing).
Cython could be included. Should I request a PR when it's done?
from pyxdf.
Related Issues (20)
- Bump Python to >= 3.9 HOT 1
- Replace Azure Pipelines with GitHub Actions HOT 2
- We shouldn't be rounding nominal_srate in load_xdf HOT 6
- Clean up branches HOT 1
- XDF.jl (Julia importer) HOT 4
- xdf_load does not synchronize clocks HOT 8
- Rename master -> main HOT 2
- jooc: why are some values in stream meta-data lists of dicts and some just dicts? HOT 2
- unpack requires a buffer of 8 bytes HOT 7
- Merge streams HOT 2
- Nominal vs. effective sampling rate HOT 25
- Error when dejitter_timestamps=False on stream without samples HOT 6
- "Reading chunk length error" after a Python crash at the end of the recording, any solution ? HOT 5
- OSError: Invalid XDF file HOT 5
- Make progress bar HOT 2
- Incompatible with numpy 1.24
- Azure Pipelines broken? HOT 6
- Single PC clock offset vs. latency and jitter HOT 10
- Possible bug with _clock_sync and clock reset detection HOT 2
- 2-n order drift correction
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from pyxdf.