Improve Godot Engine's boolean operator engine (CSG) with `elalish/manifold` about manifold

[Calinou]

To give some context on Godot's CSG usage, Godot features CSG nodes (with both primitive and user-supplied custom meshes) that can be used to block out levels. CSG nodes can also have one or more material slots configured (and assigned on a per-face basis), which are preserved by the CSG algorithm and can be assigned by the user. The documentation can be found here.

Custom code had to be written for this back in 2018, since CSG libraries available at that time were generally GPL-licensed. (Godot is MIT-licensed and therefore needs to avoid (L)GPL-licensed components.)

This custom code is relatively small, but it also suffers from many bugs, especially with coplanar faces:

• godotengine/godot#41140
• godotengine/godot#43755
• godotengine/godot#58637

Mesh generation performance is good enough for static level design, but performing real-time changes should be avoided to prevent stuttering during gameplay. (That said, I doubt any CSG library can be fast enough to guarantee that mesh generation happens in < 16 milliseconds with typical low-poly game meshes. Moving mesh generation to a separate thread is probably a good idea.)

[x] Tech lead of Godot Engine is ok with requiring manifold geometry inputs for CSG

For reference, the current custom CSG implementation already has this as a requirement:

Any mesh can be used for CSG, this makes it easier to implement some types of custom shapes.
Even multiple materials will be properly supported.¹ There are some restrictions for geometry, though:

• It must be closed
• It must not self-intersect
• Is must not contain internal faces
• Every edge must connect to only two other faces

—Godot gets CSG support

In parallel, we'd like to rework the CSG editor for better usability, but this can be done separately from the underlying mesh generation algorithm.

Footnotes

1. Multiple materials per CSG node are already supported since Godot 3.1. ↩

[fire]

Some fun shapes.

[fire]

Some interesting new development. https://github.com/SarahWeiii/CoACD was able to turn arbitrary meshes into a manifold meshes.

Not sure how, but something to do with openvdb.

Useful for supporting arbitrary meshes as input

[elalish]

Assimp is only linked for building meshIO. It's separated for exactly this reason; I figured most users would use their own input/output code.

[fire]

I am currently trying to release Godot Engine 4.0 so things have been hectic. Sorry for the lack of communication.

[fire]

I updated to the latest manifold sources.

Still having trouble with original mappings from the triangles, but many things have been improved in the manifold code.

[fire]

Can you evaluate https://github.com/haasdo95/graphlite? I wasn't able to find many substitutes.

What operations are you using?

[elalish]

All I use is connected_components (in two places):

manifold/manifold/src/impl.cu

Line 427 in 3e92f47

boost::connected_components(graph, components.data());

graphlite doesn't technically include it, but it basically has it sketched out here: https://github.com/haasdo95/graphlite/blob/dde08e41c75b5fe42f7e3e0c6b02c4883b23155f/src/graph_lite/serialize.h#L156

Shouldn't be too hard to switch over. I like the idea of keeping my dependencies light.

[elalish]

Shouldn't be more than a few hours, ideally. However, I'm leaving on vacation tomorrow for the rest of the week. If you want to take a stab at it, you're welcome to. Otherwise I'll take a look next week.

As far as taking a dependency on graphlite, should I just copy in graph_lite.h and note the git hash it's from? I can't think of a better way at the moment...

[Calinou]

So @fire, if this is working, does that mean the Boost dependency is okay, or you still need that removed?

The Boost dependency still needs to be removed if this is to be integrated in Godot core. We can't integrate such a massive dependency in Godot's repository (as we include the source of all dependencies within https://github.com/godotengine/godot). Small binary size and fast build times are important to us, so we had to make this decision.

[elalish]

@Calinou Thank you for the explanation, that's very helpful! I've never written with error codes, so if someone wants to take a stab at ifdeffing one of the current exceptions to show how you'd handle it, that would be very helpful. Obviously we could just remove them all, but then it'll crash instead of throw, which seems worse?

Anyway, this library is already using C++17 and lots of modern style. I want to eventually move forward onto the latest C++ parallel algorithms, which certainly isn't supported by all the old compilers, but I think is the right move for clean GPU and multi-core parallelized code.

[fire]

I upgraded the manifold library https://github.com/V-Sekai/godot/tree/csg-manifold-06.

[pca006132]

I guess the problem you are facing is #148, we can already reproduce it and just need some time in fixing it.

[Calinou]

Could this feature be improved by having an option to not export the CSG nodes themselves to the exported game, but say "replace the combiner with a meshinstance".

This is being tracked in godotengine/godot-proposals#182.

[elalish]

I wouldn't say it's a questionable design; it's what graphics drivers require, but yes it causes major headaches for manifolds. I believe I now have a good solution for a small amount of extra data (mergeFromVert and mergeToVert vectors) that will allow the manifold to be easily reconstructed from graphics-friendly structures: #290 (comment)

I'm planning to make it as simple as possible to write out a GLB this way (ideally with an extension for this manifoldness data), so that engines can just use their GLB importers instead of having to write a special one for Manifold. It's not done yet, but it would be great to get your feedback on that PR as it progresses.

[elalish]

@fire Yeah, I have quite a bit of familiarity with algorithms to make meshes manifold - I managed Microsoft's relationship with Netfabb when we were integrating their watertighting algorithms into our software. If you want to do it generically (like they did) you need a lot of heuristics and there's no avoiding having some very surprising results occasionally.

This library isn't about heuristics, but reliability. That said, I would like to help with meshes that are at least attempting to be manifold to get over the hump. I'm considering a function like enum MeshGL::Merge(float precision) that would generate the mergeFromVertex and mergeToVertex vectors by welding verts on open edges that are separated by less than precision. It would return a status of either AlreadyManifold, NowManifold, or NotManifold. This would make imported models with properties much easier to deal with.

However, it would help my development greatly if you could supply some example models that you were expecting to work and reported as NotManifold, so I can test this flow.

[yetigit]

@fire did a thing here https://github.com/yetigit/force-manifold

that uses just the openvdb part without the convex hull decomposition (which is still quality stuff); basically this is raymarching looking meshes being generated, guaranteeing that the mesh is a manifold. however often times the resolution is high

LEFT is the result RIGHT is the input

[fire]

Merge() helped significantly.

https://github.com/V-Sekai/godot/blob/549b24ca9b5fcd5be5b9b3c1923cb21816ba0e7c/modules/csg/csg_shape.cpp#L190

Now I am stuck on associating ??? ReservedIDs with each csg shape's Vector<Ref<Materials>> because each shape has it's own material index and it's unique to the vector of materials.

The above is a blender monkey and a mesh.

Edited:

Use the branch due to bugfixes.

[elalish]

Yes, so just sort your vector of faces by material. Then find the indices of that vector where the material changes. These values become meshGL.runIndex. Then get a sequence of originalIDs with int first = Manifold::ReserveIDs(numMaterials);. Make a mapping between these originalIDs and your materials. Then fill meshGL.runOriginalID with the ID that maps to your material for each sorted run.

When you get a MeshGL result after operations, the triangles are already sorted into these runs of consistent ID that you can use your map to return to material. Does that make sense?

[elalish]

That would be awesome! Technically it should be possible to build Thrust with the C++ backend without using nvcc (from the CUDA toolkit) since Thrust is just a C++ header library. However, I haven't tried it yet. I'd like to migrate from Thrust to C++17 parallel algorithms partially to simplify the code and make it run in parallel on more than just CUDA GPUs, but also to make standard compiler chains work better.

Are you interested in helping to try some of these options?

[fire]

Can you briefly describe a guide?

1. where inputting manifold meshes
2. apply union, intersect and merge
3. output another mesh?

[elalish]

I think this test and the following one demonstrate all of that: https://github.com/elalish/manifold/blob/master/test/mesh_test.cpp#L698

And of course the docs: https://elalish.github.io/manifold/classmanifold_1_1_manifold.html

[fire]

Is it difficult to disable assimp?

[fire]

@elalish Can you remove boost graph dependency?

https://github.com/haasdo95/graphlite was the closest I can find.

[fire]

I was able to extract https://github.com/V-Sekai/godot/tree/csg-manifold [does not compile] and did a build system trick to enable .cu, but the boost dependency was too much after the 8th Boost folder...

[elalish]

Interesting, I'm still relatively new to the world of C++ libraries and didn't imagine Boost would be a big problem dependency-wise. I'm only using it for a connected components algorithm - figured better not to reinvent the wheel. Still, that's a generic algorithm and I'm sure it's available elsewhere, or we could just write our own. Thoughts?

[fire]

Can you do an estimate how difficult the work is, and if you need help?

[fire]

I'm swamped with pending work. I can't promise anything.

It'll mostly like be next week.

Regarding dependencies, Godot Engine has a readme and a licenses document.

[fire]

Here's my thoughts on the two Material Case.

1. mesh 1 mat 1 and mesh 2 mat 2 need to be converted to mesh 1 mat 3 and mesh 2 mat 3
2. mat 3 uses xatlas to combine or a simple algorithm that uses double the uv space. (like 4x)
3. Use manifold on the mesh 1 and mesh 2.

[elalish]

Yeah, pretty much. You can do steps 1 and 2 after 3 (which means you can do it after many boolean operations and just deal with the UV coords that are left if you want). This is what the MeshRelation is for.

[elalish]

I've got a WIP #92 to change the graph dependency.

[fire]

Debugging winding order and conventions.

Do you have an example cube with the positions and the triangle vertex indices?

[elalish]

CCW. Manifold cube = Manifold::Cube(); should give you what you need, yeah?

[fire]

If I used the meshes from the manifold library I was able to do csg operations, but still working on the mesh import. It crashes on non-manifoldness.

[ksuprynowicz]

Works perfectly now :)

[elalish]

Nice work! So @fire, if this is working, does that mean the Boost dependency is okay, or you still need that removed?

[fire]

I posted the changes I used here #93.

Here's a Godot Engine master patched branch https://github.com/V-Sekai/godot/tree/csg-manifold-03.

[fire]

Notes:

1. Generating CSG nodes is slow on a full character mesh added to a box, but did not run a profiler
2. Can start providing test cases that crash the godot engine patched with manifold
3. Can we remove exceptions from elalish/manifold ?
4. The results of boolean operations look great. No gaps.

[Calinou]

Can we remove exceptions from elalish/manifold ?

To clarify, we want Godot to remain buildable without exceptions, so they need to be disableable at build-time. It's fine to keep exception code for those who want it, but it needs to be optional.

[fire]

logs_28140.zip

7_Compilation (armv7).txt

Here are the logs that fail on exceptions.

[fire]

Posting my test godot project so I can clean up my desktop.

CSG Game Project.zip

[elalish]

When you say remove exceptions, what exactly do you mean? Are you hoping for old-school error return codes or something?

[fire]

Something like this documentation. https://docs.godotengine.org/en/stable/development/cpp/common_engine_methods_and_macros.html#error-macros

Also the exceptions must be physically removed from the code (like ifdefs) because it's checked by the c++ compiler and not godot engine.

[elalish]

I still don't quite understand what you mean; are you specifically talking about how non-manifold input produces an exception? I suppose that could generate an empty mesh instead? I also have assertions that will throw, the idea being it's better to catch an exception and stop gracefully than to crash. Perhaps you can make a PR to change one or two so I can see what you're thinking of replacing them with?

[fire]

Let's say you're simulating a game world.

1. It's better that manifold returns a valid result like the original two meshes not joined, than a crash that stops the entire simulation for like a 100 people.
2. The other use case is that Android bans exceptions according to the github actions. Not sure about the details.
3. For debugging time I can make test case that would crash because the new code I'm working is buggy with dividing meshes (the boost rewrite) and restarting the editor slows my iteration times.

You can achieve returning a valid result both by exceptions and error codes. Error codes and exceptions are a tool.

The problem I had was defining was what valid results can be returned after a boolean merge error that creates an invalid manifold..

Edited:

An empty mesh is obvious and it handles the boolean intersection case.

[elalish]

Okay, then I think this library throwing exceptions isn't really the problem (I want to notify people when operations didn't or can't work). You'll just need to try/catch them to handle them properly. It sounds like you're alright with requiring manifold input, so I guess you won't hit that exception? If a Boolean operation fails for some reason, you could fall back to using Compose, which will put the meshes together but without any intersecting.

And then there's obviously something the MS compiler doesn't like, so we'll have to debug and fix that.

[fire]

I would like to enable the Android platform, but it requires probably a define for the code that throws so that c++ compiler doesn't see it.

There may also be an exception disable on c++ compiler but I'm not sure what that entails.

[Calinou]

You'll just need to try/catch them to handle them properly.

Unfortunately, this can't be done if the code is built without support for exceptions. Old-school error codes are the only solution here. I know they're not as convenient, but they're far more universal – including for users of third-party language bindings.

It's fairly common for C++ libraries to support building without exceptions, as many users will not even consider a library if it requires building with exceptions. Some exotic platforms and compilers do not support any form of exceptions. For instance, this was a problem on WebAssembly for a long time. Even if this is no longer a problem on some platforms, exceptions can have a significant cost on performance and binary size depending on how well the compiler implements exceptions.

The further step for universal usage would be to use plain C instead of C++ for libraries, but I won't go that far here 🙂

[elalish]

I was reading up on this and found an interesting SO question with several good responses. Seems like most of the no-exceptions camp folks also avoid the standard library containers. That seems pretty extreme, but they do mention games specifically. Is that how Godot works?

[ksuprynowicz]

Maybe exceptions could be inside #ifdef ?
Then it would be possible to disable them.

[Calinou]

I was reading up on this and found an interesting SO question with several good responses. Seems like most of the no-exceptions camp folks also avoid the standard library containers. That seems pretty extreme, but they do mention games specifically. Is that how Godot works?

Godot itself doesn't use STL containers, but we still build with STL enabled to allow linking against libraries that use it. You can keep using STL on your end 🙂

[elalish]

Thanks! I have this little ALWAYS_ASSERT function I use to wrap my exceptions, so it should be a simple place to #ifdef everything. There are a few other exceptions sprinkled around, but I'd be happy to convert those to use this wrapper if someone would demonstrate how these should be disabled.

[fire]

Can we define what are good results for failure?

Here's some

1. for boolean add. returning the originals is a ok result
2. for boolean subtract. empty mesh?
3. for boolean intersect empty mesh?
4. for manifold input failure returning the empty mesh is ok
5. for manifold output error returning the empty mesh is ok

[elalish]

Good idea, but does this mean you want me to catch any possible failures and recover from them gracefully, and all without using exceptions? Or can I use exceptions internally so long as I catch them all? After all, my STL containers can still throw. My Boolean failures are basically asserts; compiling them out entirely seems reasonable. It really shouldn't fail unless something is broken (which apparently happens when compiled by MS), and in that case it's just as likely something I'm not asserting will seg-fault anyway.

The main API-level exception is around pre-conditions like input manifoldness, which agreed, we could just make it return an empty Manifold instead.

[fire]

Makes sense. I'll try to type what I can, but starting to get busy.

[fire]

I tried patching csg. What's a good way for me to send you the error cases? They trigger when I drag the csg shapes in godot engine.

[elalish]

Wow, are you running a full boolean per frame? I kind of figured that would be more of a mouse-up operation. Ideally you could save the current state of the manifold inputs, dump them out as GLB or OBJ or something and verify they have the same behavior in a tiny C++ script. If the models aren't copyrighted, you could even add them as a test case in mesh_test.

However, before you go too far, what compiler are you using? If it's MS, then there's already #86, which indicates there's a definite problem that I won't be able to debug without a Windows machine. Do you repro that? Because I can't with gcc.

[fire]

I'm using llvm-mingw which are neither of your tested choices :D. I can also send you instructions to use the github actions build versions of godot engine.

[elalish]

Sure, widening my github actions testing sounds like a good place to start. With llvm-mingw, does the whole manifold test suite pass for you locally at least? If there are any failures, please list them here.

[fire]

It was a bit of a pain. I can show you my godot manifold branch, but wasn't able to get the manifold working on cmake. (win)

[Calinou]

Wow, are you running a full boolean per frame? I kind of figured that would be more of a mouse-up operation.

This is done to allow for quick visual output, but I agree we should move this to a debounce timer eventually (defaulting to 0.1 seconds or so). This debounce timer would probably need to be made configurable somewhere in a global place, as there are always people who will try to use CSG in real-time in their projects (even if it's usually a bad idea).

Running CSG operations on a separate thread could also help, but it adds a lot of complexity.

Edit: Proposal opened: godotengine/godot-proposals#4448

[pca006132]

FYI: #111 can probably help you get this working with cmake on Windows

[fire]

Godot Engine doesn't use cmake, but yes, that was a required fix in my manifold fork.

[fire]

@elalish What's the best way to check if the input meshes I have failed and have the test cases be recreatable on standard manifold.

I think the is manifold or the operators are faulty.

[elalish]

You're welcome to create a test that demonstrates the error you see. Are you getting a crash or an exception?

[fire]

The newer master improved the crashes.

However, I had great difficulty getting the git sub-modules to be easy to use.

[pca006132]

What's the problem with the submodules? Issue with the build script? If you don't need to run the manifold tests on your repository and don't need model import/export (e.g. when provided by another library), you can ignore the google_test and assimp submodule which should be the most complicated to build. Thrust and glm are header only and should not be hard to deal with.

[fire]

Thrust was difficult to build. They use a structure where the dependency/cub is linked to the cub (a text file with a path) in the thrust/cub.

Thrust also has cub and a cuda library submodules.

[pca006132]

I see. Wonder if it is possible to build it cloning the cub and libcuda submodules when cuda is disabled. In principle you don't need that if you don't use CUDA.

[fire]

I was a bit tired and approached the thrust library wrong.

1. I have to fork thrust
2. Change fork to use a cub dependency that points online and not to a ../cub link
3. Use this in manifold.

I was able to get this into godot with submodules. It'll be an easier task to remove submodules later.

[fire]

@elalish What's the best way to provide the test cases?

Most of the crashes that create false exceptions are done when I'm moving the merge / union / intersect. How to encode this? I can do videos and provide test scenes and an executable for whatever os you want. Godot is a single executable.

[fire]

I have updated the godot engine pull request with the overlap changes.

[fire]

Screencast.from.2022-07-22.11.01.47.AM.webm

@pca006132 @elalish Does this look like anything obvious?

Empty means is_manifold failed.

[pca006132]

No, this does not look abvious (at least to me). Can you export these models so we can add it to our tests?

[elalish]

Agreed, it would be great if you (@fire) could create a debug function to dump the inputs when the output is bad so we can repro. Also, what is empty? Is that when it's blue? What is the difference between the cube that stays, the cube that vanishes, and the head that turns blue? It's a bit hard to tell what's going on, since the Boolean only takes two input meshes.

[fire]

I'm not that familiar with the inputs. What are the input formats for the tests?

How do you prefer them?

[pca006132]

I think the usual 3D model format is fine, stl/glb etc. And please provide us with the coordinates in the scene so we can reproduce the exact coordinates.

[fire]

After the changes, I can't seem to recreate the posted errors. 🎉

Edited:

New crashes. They crash inside of manifold, but from invalid data!? I'll change the godot type from primtives to meshes. Trying clang asan and bsan.

[Frontrider]

It's just something I'm going to throw in as an idea that might help with runtime performance. Could this feature be improved by having an option to not export the CSG nodes themselves to the exported game, but say "replace the combiner with a meshinstance". Not actually replace because that could break scripts, but have it display a static mesh that was created in the editor? (with warnings that you should NOT reference child nodes if this option was selected)

IF CSG is already meant for static geometry due to speed then this could make it a more viable production tool. I see that it's meant for prototyping and not released games, but a small studio or a lone dev can still make really good use of it.

[elalish]

Agreed that's harder than it ought to be. I'm working on a new API that should simplify handling properties.

[fire]

@elalish Can you provide a way to associate an arbitrary set of data like? in the properties:

	struct Face {
		Vector3 vertices[3];
		Vector2 uvs[3];
		AABB aabb;
		bool smooth = false;
		bool invert = false;
		int material = 0;
	};

[elalish]

@fire can you be a touch more specific? We talk about how to do some of this using MeshRelation in #274. I really want to avoid prescribing what kinds of materials and properties can be associated with Manifold, so it can be used generically. I'm now thinking of expanding on the properties and triProperties vectors I have to automatically interpolate and output them as part of MeshGL. I'm hoping that will make it easier than the barycentric MeshRelation, without actually having to name any properties.

In your example, what would smooth represent? Do you really want non-indexed vertices? Why an AABB per triangle? It's pretty easy to calculate from the verts.

[fire]

The problem is Godot in my questionable design has two ways to specify the same data so if the data is not written in a manifold internal format, it becomes randomly non-manifold and the entire tree of csg nodes gets lost (empty manifold)

So I was trying to think of ways to stick arbitrary data per face, per vertex, etc in the manifold data structures.

[fire]

@elalish what are your thoughts on running the https://github.com/hjwdzh/Manifold preprocess over the input mesh

[elalish]

@fire thanks, that's an interesting paper and algorithm. All manifold fixing algorithms are fundamentally heuristic, so I'd recommend testing it on quite a few bad meshes, especially characters which are often intentionally modeled with open meshes. I'm sure it'll give poor results sometimes, but I'm not sure how often. It'll also dramatically increase your vertex count.

I think it's a great idea to give it a try, but as with all fixing algorithms, please test for manifoldness first and only apply it if it fails (or simpler fixes don't work). A huge part of the problem with current workflows is that they will automatically do things like merging vertices in an attempt to fix manifoldness, but if the input was already manifold, it often breaks it instead.

[fire]

@elalish Did you stabilize the algorithm for triangle / vertex attributes? How is that going?

[elalish]

@fire Quite well, finally. Now would be a good time for API feedback if you have any. I need to finish my open PR, probably clean up a few remaining details, then I'll work on an npm v2.0 release.

[fire]

I am having some trouble. What is a commit that has the new api and works? My meshes have all been deemed non manifold,

[elalish]

It's really hard to help with that kind of a description. Can you show an example? Did it used to work? What are you attempting to do?

[elalish]

You can see for yourself that our tests are passing, perhaps you can write a test to catch the problem you're having?

[elalish]

@fire Any more information you can give me on this? I see you're now restoring materials, which presumably means you're now using MeshGL? For anything with discontinuous vertex properties, you'll need the mergeFromVert and mergeToVert vectors to maintain the manifoldness. Next I'm planning to build a helper to auto-generate those for a MeshGL based on tolerance welding, but that'll always be a heuristic process. Any examples you can give of what's giving you difficulty would help in my design of that.

[elalish]

Thanks for sharing! Yeah, voxels are a reliable way to get manifoldness, though of course they kill the efficiency of a mesh. I'm almost done with my Merge function - curious how frequently it'll help.

[fire]

I decided to try for fun today and after vendoring thrust, glm, nanobind, quickhull subrepos I hit a roadblock with thrust requiring cuda..

I think I will have a hard time convincing the other Godot Engine maintainers that adding all these libraries is worth the load.

[pca006132]

but thrust doesn't require cuda? you can try to supply the thrust directory into manifold by setting FETCHCONTENT_SOURCE_DIR_THRUST. It should work. We are building it without cuda in the CI.

[fire]

I posted an updated branch to https://github.com/V-Sekai/godot/tree/vsk-csg-4.3.

Note that the default csg shapes aren't manifold so they tend to disappear on use in a csg tree.

Edited:

Something about the indexed meshes requirement breaks the vertex attributes if I try to add more.

[elalish]

Something about the indexed meshes requirement breaks the vertex attributes if I try to add more.

Happy to help if you can give a little more detail. When you have discontinuous vertex properties, you'll need the mergeFromVert and MergeToVert vectors in MeshGL to tell it how to stitch them to make a manifold. Alternately, you can use MeshGL.Merge() to generate these automatically, fixing gaps up to precision wide.

[fire]

This is regular Godot Engine CSG for comparison to Manifold CSG

[elalish]

Looks like great progress!

Now I am stuck on associating ??? ReservedIDs with each csg shape's Vector<Ref> because each shape has it's own material index and it's unique to the vector of materials.

Yes, that's exactly what they're for. I like to make a map of originalID to Material (or material index). If each Manifold you create for input has a single material, you can just use manifold.OriginalID() to get the new entry for the map. If it has multiple materials, then you'll want to organize the triangles into runs of consistent material (like draw calls) and call Manifold::ReserveIDs() to get however many you need and put them in the meshGL.runOriginalID vector.

Does that make sense?

[fire]

The task would be to convert TypedArray<Face> into runs of single materials from its current mixed materials. Then something.

The face struct looks like:

struct CSGBrush {
	struct Face {
		Vector3 vertices[3];
		Vector2 uvs[3];
		AABB aabb; // We calculate this directly.
		bool smooth = false;
		bool invert = false;
		int material = 0;
	};
	// ...
};

Then the task gets fuzzy after that.

[fire]

So using Merge() is incompatible with making my own runs right?