Comments (3)
ok I see that checker raises an error in case of unknown mime type, we should extend that list and add a flag to allow for unknown mime types, we'll try to add that shortly.
from croissant.
Thanks for reaching out!
Is the bug about the Croissant spec or about the mlcroissant python library?
One should be able to describe a dataset containing videos using Croissant, similarly as what is done in https://github.com/mlcommons/croissant/blob/main/datasets/1.0/audio_test/metadata.json (replacing sc:AudioObject
by sc:VideoObject
and audio/mpeg
by video/mpeg
for example).
It is however possible that libraries (including mlcroissant library) might not support videos atm.
Similarly as #696, when the data is stored in a file format which is not supported, we advise to create a Croissant dataset that specifies the dataset level information and the resources, while omitting RecordSets that contain data stored in files with an unsupported format.
This would unblock you, and it would allow tools that can work with only such metadata to already support your dataset (eg: index dataset, download raw data), while providing a signal for the Croissant contributors on which formats to support first, in the spec and/or various implementations.
Please let us know if there are problems with defining such an incomplete croissant definition and we will look into this.
from croissant.
OK, so I did run validation (eg: mlcroissant validate --jsonld ../../datasets/1.0/titanic/metadata.json
) on a croissant config containing an unknown encoding format, and it did not raise an error
(
And looking at the code, it seems to me like validation should work fine.
Do you have a command line that would reproduce failure to validate a croissant file due to unknown encodingFormat
?
from croissant.
Related Issues (20)
- Joins as described by Croissant Format Specification are not supported by mlcroissant python library. HOT 1
- Add containedIn fileProperty to the Extract class
- [NeurIPS] How do i create a Croissant File using this library HOT 2
- [NeurIPS] Fileld has no source HOT 8
- Feature Request: support for CoNLL format HOT 2
- [NeurIPS] How do i create a Croissant File using this library for my dataset? HOT 1
- [NeurIPS] Compatibility with HuggingFace Image dataset HOT 1
- [NeurIPS] URL to dataset metedata
- Example for audio dataset HOT 2
- [NeurIPS] How to "ingest" multiple datasets made of .xz files as data/samples and space-separated .txt files as ground truth
- Please improve documentation
- Intro example not working HOT 3
- [NeurIPS] Support for (Sequential) Dataset Stored in TDMS Files HOT 3
- [Neurips]: any support for large h5 files? tried various encodings but no luck. HOT 1
- JSON LD parsing fails when multiple strings given to FileSet.includes HOT 1
- [NeurIPS] Join fails with reference if read from fileObject HOT 2
- Feature request: loading a record set as a pandas dataframe HOT 4
- Expose the underlying pd.DataFrames instead of an iterator
- Feature request: Add a mapping from FieldId to RecordSetId to Metadata.
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from croissant.