Consider Using GLTF as the interchange format rather than bevy's scene format. about blender_bevy_toolkit

[sdfgeoff]

So I had a bit of a fiddle this evening: trying to use the gltf-master-asset approach linked above. However, this block of code:

pub fn spawn_gltfs(
    mut commands: Commands,
    mut gltf_loader: ResMut<BlenderGltfLoader>,
    assets_gltf: Res<Assets<Gltf>>,
    assets_scene: Res<Assets<Scene>>,
) {
    // if the GLTF has loaded, we can navigate its contents

    gltf_loader.gltfs.retain(|gltf| {
        if let Some(mut raw_asset) = assets_gltf.get_mut(gltf) {
            // Loaded already
            for raw_scene in raw_asset.scenes.iter_mut() {
                if let Some(mut scene) = assets_scene.get_mut(raw_scene) {
                    let world = &mut scene.world;
                    let mut nodes_with_extras = world.query::<(Entity, &GltfExtras)>();
                    for (entity, extras) in nodes_with_extras.iter(&world) {
                        println!("{:?}. {:?}", entity, extras);
                    }
                } else {
                    println!("FATAL: GLTF loaded but scenes not loaded?!")
                }
            }

Fails because you can't DerefMut something in the asset server (ie once an asset is loaded, it is immutable as far as I can tell - and I was hoping to mutate the asset).

One solution is to add the components at spawn time. There is a small crate here: https://github.com/nicopap/bevy-scene-hook that demo's how to do this. This will have a slight spawn-time cost, but we'll see if this is an issue - I doubt it will be.

[sdfgeoff]

I fiddled with the bevy_scene_hook this evening with great success: I can attach components after using serde to parse JSON in the GltfExtras field, and it works fairly well.

A couple challenges:

• Generics. I can't find a good way of getting serde to deserialize a map of:

{
    "enumVariantName": {"details":"thing"...}
    "enumVariantName2": {"other":"thing2"...}
    "enumVariantName3": {"new":"thing3"...}
}

I think this is what serde_with::EnumMap is meant to do, but combining that with the fact that blender exports non-existent components as {} this makes things a bit challenging.
This is totally solvable by writing a custom serde deserializer, but I'll finish derisking first.

At this point I can successfully import/export data from blender. Yay!

In the current plugin, physics data (eg convex hull shape) is encoded into the scene at export time, but if we are using GLTF as the interchange, then we need to grab it from the objects Mesh it at load time. For whatever reason, the GLTF Loader creates extra entities. All meshes are added to the object as children of blank transform-entities. So the blank-transform-entities get the GltfExtra component and they have a child with the Handle<Mesh> component. This makes finding what mesh to use for physics shapes rather awkward.
I'll have a bit of a think about this, but I wonder if I should just have a "build physics shape" button in blender and send it via the extra properties..... There's probably some design work to do here.

But anyway, the proof of concept suggests this can be made to work. Now I need to decide if I want to make it work!

---

There are really two parts to this project, and using GLTF as the interchange really clearly distinguishes them:

1. A blender UI that encodes extra data into a GLTF file
2. A bevy plugin that loads components from data in a GLTF file.

The cool thing is that these are completely separable so long as the format of the extra components is well defined. I haven't yet found if anyone has defined a GLTF extension for this, but if someone has, then we should totally use it. It would make the both parts useful even in absense of each other. I can imagine a future where multiple content tools (eg blender, wings3d, max, maya) all can export components, and multiple game engines (eg bevy, godot, unity) can all import the components. This would greatly broaden the ability for scenes containing logic to be described in a cross-tool way

[sdfgeoff]

If I redefine how components are stored in the blender file, then I can avoid a whole stack of complexity in parsing (ie no more serde-mashing required. So that's what I'm planning to do.

I've decided to split this project into two (or three):

1. The specification of how to represent ECS components in GLTF. (Complete)
2. The blender addon to make this easy. (In Progress)
3. The bevy plugin to make loading them easy.

This repository will turn into #3
I imagine that #2 will require relatively little development after initially being created (hopefully not much until blender 4.0....) and because we will be using bevy's GLTF importer, hopefully there won't be much to change in this repo between bevy versions either.

[kulkalkul]

Hey, I was exploring the options about defining components using GLTF and saw this repository and issue. Any updates on this?

Quick update, I saw this now:
sdfgeoff/gltf_ecs_components#9