curating high-quality video data about open-sora-plan

[bhack]

You could also check
https://laion.ai/blog/video2dataset/

[cxh0519]

Our preliminary goal is to achieve impressive results in a specific data domain to verify the effectiveness of our pipeline and then extend our plan for generalization. Please keep attention on our project and feel free to contact us for discussion and potential cooperation.

[pengzhangzhi]

A random thought. As a community, would be great to have a discord channel for discussions and updates.

[aluo-x]

I should be able to help on the high quality video data aspect (along with transcripts), although the source of captions is a more difficult problem.

[sayakpaul]

Is the domain of the data fixed? If so, do you have more public information available on that?

Regardless, I think having some sort of a data curation pipeline (like the one used in the Stable Video Diffusion paper) would be really nice.

[cxh0519]

Agree. After the preliminary validation, we will construct a data curation pipeline following successful projects and pay more attention to data.

[sayakpaul]

Cool, sounds like a plan. FWIW, I put together a simple repository that walks through the primary steps of the Stable Video Diffusion curation pipeline: https://github.com/sayakpaul/single-video-curation-svd.

[cxh0519]

We will check it and thanks for your effort!

[yuxuan-li92]

You could also check data-juicer, it seems to be beneficial for video data curation.

[bhack]

https://arxiv.org/abs/2403.06098