veykril / cubism-rs Goto Github PK

A big question regarding the api is how controllers should be applied. What I mean with controllers are basically all objects that modify model values like the EyeBlinking effects or motions for example. There are two ways I can think of which both advantages and disadvantates.

1. Tie controllers to models/mocs(see UserModel)
   • Advantages
     • Reduces performance overhead due to having to name lookup parts which currently involves a lot of string comparing
   • Disadvantages
     • Will most likely cause a lot of duplication at runtime, since each model most likely needs their own controller copy.
2. Have controllers be seperate and apply them manually do models each tick.
   • Advantages
     • Allows library consumers to have better control over things
     • Reuse of controllers for different models as long as they have the correct parts
   • Disadvantages
     • Will incur a performance penalty due to part and parameter lookup through strings(maybe this can be acted against by creating a part and parameter system using hashes and other things?)

Controllers are now registered to models through their TypeId meaning that a controller can at most exist once in each UserModel.

Potential controllers that have yet to be implemented:

• [] Posing (#19)
• [] Breathing (#20)
• [] Expression (already done partially, #21)
• [] Motion (#22)
• [] Physics (#23)

There are some options to implement these:

• Re-implement the whole feature in Rust 🦀
  • Rust/cargo-native
  • Hard way to go
• Use FFI and call the original framework
  • Fairly easy
  • No modification required
  • Needs to compile the framework separately
• Convert the framework into Rust
  • Black magic, sort of...
  • Able to use rustc/cargo to compile
  • Potential license issues (cannot be re-licensed as MIT/Apache because of "5.7 No Combination with Incompatible License" in Live2D Open Software License Agreement)

Currently working on re-implementing some features of Cubism 3 Native SDK Framework (Such as loading .model3.json, 3c1u:cubism-fw-port) and I found out that there is no well-documented information for files it handles (Even in Japanese!).

All the(so far) known json files cubism uses:

• model3(has been partially added in 7c5db8b, completed in 842d509)
• physics3 (added in #17)
• pose3 (added in #12)
• userdata3 (added in 842d509)
• expr3 (added in #12)
• motion3 (added in #10)
• cdi3 (added in #17)

Some of these are just a simple struct definition, nevertheless each probably deserves its own module in the json module. The json module is solely for the definitions, since those basically only act as configuration. Any proper functionality(aside from maybe validation) should go into their own modules.

Discussion: Regarding structure definitions, which fields should get a default de/serialization instead of Option wrappers? This can probably get resolved once the use case for each field is clearer and whether the field has an appropriate default or not.

Is there any chance this will continue to get worked on, or is this an abandoned project?

For the following backends:

• glium(priority right now, since glium has an imgui renderer ready which helps with debugging)
• gfx-hal
• vulkano?
• piston2d

Glium and piston2d now got very rough renderers for testing. A proper renderer api has yet to be thought of though.

Cubism 4 SDK has been released. I replaced the third-party folder with the new SDK and it worked fine because of its backward compability.

However, some features have been added to Core/Framework.

Live2D/CubismNativeFramework@aa57257

Quick and dirty implementation has already been done here. It would probably benefit from having some randomness attached to it later on.

The current api of the (two) implemented renderers has been made quite hastily and I'm uncertain whether these are good enough or if they can be improved.

In general it seems best to have one renderer per moc(all models that use the same moc) since these always have the same drawable count meaning there is no need to recreate a vertex buffer then. Even better, the index buffer only has to be uploaded once since the index data is static(this is now implemented in the glium renderer. It creates the index buffers once for a given moc and then only allows rendering models from the same moc).

This can probably be enforced by checking the underlying moc, for example by cloning the Arc into the renderer and then comparing.

It's probably best to have one renderer per model after all so that the renderer can cache the vertexbuffers if necessary by making use of the DynamicFlags::VERTEX_POSITIONS_CHANGED flags.

Expressions have already been partially implemented, but the implementation is very rough and has to be improved. See here for the current implementation. It is implemented as a controller now but it only allows to set a single active expression a time and doesnt allow for any transitions which is obviously not the greatest feature set.