enenra / space-engineers-utilities Goto Github PK

In order to be able to deal with several scenes, we need to be able to have the collection structure in every scene, not just one. Currently the checks for collections existing cannot account for the scene in which those collections should be present. This needs to be adjusted.

right now it feels kind of weird. Need to investigate better solution for creating small grid block versions.

see this:

issue appeared when pressing export button with default values.

have a mesh in every collection, make sure subtypeid is set, export path is correct
then hit export

Need to do that everywhere because it could have become invalid since setting it.

Small safety measure would be to copy the dummies over to the LOD collections on export - or at least show a warning if they are not there.

I'll first need to investigate in which collections the dummies have to be present in the first place. LOD1 for sure but LOD2 / 3 too?

The addon also needs compression to MWM. This should be fairly straightforward as soon as HKT support is added as it would just be running MWMB on the other files that were just exported. Might be a bit weird interface-wise as disabling XML / HKT / FBX export would also prevent MWM export. Might solve that by exporting anyway but then deleting the files - possible?

• investigate how to call .exe files from blender python
• look at differences in usage between eikester's and normal MWMB, force eikester if needed

There is an issue where if you import a lot of SE FBX models without saving / reloading, orphaned materials are accumulated inside blender which then prevent replacement by the linked textures.

Not entirely sure how it happens yet but this is a must fix before release.

toggle is in UI but no functionality yet.

LOD1 should not kick in further away than LOD2 and so on. I need to see whether I can set a dynamic minimum / maximum for LOD distance properties in init.

Dummies have a lot of different functions and it would likely make a lot of sense to denote those in the tooltip descriptions when the user spawns them.
Information I'd like to have on there would be rotation axis, for example, or whether a dummy always has to be rotated 180° to point in the right direction (yes, apparently that's a thing).

This is something I'd be very happy to get input on from the people who've used them before.

Add an operator + button to export local materials to materials.xml

Make it simpler to link materials from matlibs.

Add selection somewhere in materials menu to select matlibs that should be loaded into the file. Possibly similar in style to material selection. Allow custom matlibs too.

The idea is that it's annoying to have to deal with subparts in different files. So when you'd create a subpart and assign it to an object (via the context menu), the empty would be placed as the child of the object. Then on export, the object would be exported to a separate file and the empty placed in its position instead.

This could also be done on import.

Relevant discussion:
https://discordapp.com/channels/125011928711036928/161758345856811008/664143268409507860

Current ToDo left:

• Rework SEUT Node Group generation
• possibly add material version if it doesn't exist already - use it to update node groups?

It should theoretically be possible to set up support for creating transparent materials within blender as part of the material pipeline. Might be worth investigating at some point in the future.

Mapping node to restrict what part of a texture is used:
https://discord.com/channels/@me/788474498306408449/795957083799420937

<?xml version="1.0" encoding="utf-8"?>
<TransparentMaterial>
  <Id>
    <TypeId>TransparentMaterialDefinition</TypeId>
    <SubtypeId>Glass</SubtypeId>
  </Id>
  <AlphaMistingEnable>false</AlphaMistingEnable>
  <CanBeAffectedByOtherLights>false</CanBeAffectedByOtherLights>
  <Emissivity>0</Emissivity>
  <Reflection>true</Reflection>
  <IgnoreDepth>false</IgnoreDepth>
  <SoftParticleDistanceScale>1000</SoftParticleDistanceScale>
  <Texture>Textures\Particles\SquareWindowDirt.dds</Texture>
  <UseAtlas>false</UseAtlas>
  <UVOffset>
    <X>0</X>
    <Y>0</Y>
  </UVOffset>
  <UVSize>
    <X>1</X>
    <Y>1</Y>
  </UVSize>
</TransparentMaterial>

• The TypeID should remain the same for all transparent materials
• The SubtypeID is the unique name I talked about briefly earlier, this is where that unique name goes. Each different material you define in this file will need its own unique subtypeID, just like all blocks need unique TypeIDs in the Cubeblocks.sbc.
• AlphaMisting means that the alpha disappears the closer to the trans object you get
• Emissivity is the glow given to a transparent material (be careful with this, after playing around with it a lot, anything higher than 0.05 is extremely bright)
• Reflection is exactly what it implies, be careful as this can have a very negative effect on performance.
• Ignore depth means the render engine will ignore the fact that other objects may be in front of the object, and it will always be rendered
• SoftParticleDistanceScaleis the way in which normals (of sorts) are applied to a transparent surface to make it appear to have smoother transitions between hard surfaces, I suggest you leave this option alone for now.
• This parameter is where the file you created as the transparent material needs to be referenced. It is important you use the correct file extension too
• UseAtlas refers to a transparent material that takes its texture from a different file path, Again I suggest you leave this option alone for now.
• The next 4 parameters are exactly as they seem, the UV size and scaling, If you have UV mapped your transparent material properly, you can leave all these values alone.

https://steamcommunity.com/sharedfiles/filedetails/?id=300427203

The addition to #9 is seemingly simple but quite significant code-wise: When setting up mirroring in the instances, allow the user to actually mirror the instances. If they have been mirrored, save the axis on which they were mirrored and on export, also export a mirrored version of the block, then set up SBC connection etc.

This would likely be fairly involved and I'd probably need to redo a bunch of export code to support it, but it would be quite nice to have.

I should probably go through the matlibs for a third time and redo them by grouping everything but the output and the image nodes, then naming the in- and outputs correctly. This will also require a rework of XML export to search through the node tree of the grouped nodes instead and to look for the image files inserted into specific slots of the node group.

Technically this will not affect end users at all, as their materials would be updated automatically to the new format by replacing the matlibs. But it's a source of errors when somebody doesn't do one or the other so it's better to do this before 1.0

There's a Blender auto updater plugin that one can implement into an addon to offer auto updating features via github releases. That's pretty dope so I want to implement that for sure.

Balmung has done the same for his version of Harag's plugin so I should ask him if I have any trouble with it.

Addon needs to be able to export to HKT. This will require using Harag's adjusted FBX Importer most likely, but might be worth to test without first. Also pull XML Harag uses to pass to havok from his github and use as a base.

Relevant info:

• collision / mwmb support: https://discordapp.com/channels/125011928711036928/161758345856811008/662957710874247178
• summary https://discordapp.com/channels/125011928711036928/161758345856811008/663595128115560479
• Balmung's https://github.com/Hotohori/se-blender
• Thread about HKT export by Harag https://forum.keenswh.com/threads/space-engineers-mod-toolkit-for-blender.7090652/page-3#post-1285355298

StollieToday at 7:00 AM

if you want the workflow its fairly straightforward, export out an FBX with the collision model, so in this case just the collection that contains the physics shapes, pass that to the fbx importer for conversion (you wont be able to make collisions without this conversion, its a file format thing), pass the converted FBX and the HKO scene settings file to the Havok tool to create the HKT.
this is an example HKO file (havok settings) - you'll find Harag has one in

havok_options.py

which is basically just his way of storing the XML file

How to go about this

• Determine how I can run an extenal exe
• Determine how to run the adjusted FBX exporter specifically (reference Balmung's github) (set that up as a separate function)
• Determine how to best store the HKO settings, export those via a separate function
• Separate function to then call havok content tools? How does that work with export itself taking time? Is there a risk that it would run too early, before export is finished?
• Operator calls those functions.

StollieToday at 7:03 AM

@enenra this method calls the importer

def fbx_to_hkt(settings: ExportSettings, srcfile, dstfile):
    settings.callTool(
        [settings.fbximporter, srcfile, dstfile],
        logfile=dstfile + '.convert.log'
    )
which calls this method
def callTool(self, cmdline, logfile=None, cwd=None, successfulExitCodes=[0], loglines=[], logtextInspector=None):
        try:
            out = subprocess.check_output(cmdline, cwd=cwd, stderr=subprocess.STDOUT)
            if self.isLogToolOutput and logfile:
                write_to_log(logfile, out, cmdline=cmdline, cwd=cwd, loglines=loglines)
            if not logtextInspector is None:
                logtextInspector(out)

        except subprocess.CalledProcessError as e:
            if self.isLogToolOutput and logfile:
                write_to_log(logfile, e.output, cmdline=cmdline, cwd=cwd, loglines=loglines)
            if not e.returncode in successfulExitCodes:
                raise

basically its passing the callTool method the FBXImporter.exe that is selected in the blender config
then has a couple of logs and looks for return codes (exit codes from the program) which are used by a python library to check the success of the programs execution
the whole process of exportation begins in file export.py at line 116 in

class ExportSettings:

it's complex but fairly sequential if you read through it, it goes Settings > FBXImporter > Havok > MWMBuilder
there are also more settings as part of the custom node tree in nodes.py such as

class MwmExportProperties(bpy.types.PropertyGroup):
    if bpy.app.version <= (2, 78, 0):
        rescale_factor = bpy.props.FloatProperty(name="Rescale Factor", min=0.001, max=1000, soft_min=0.01, soft_max=10,
                                                 default=1.0,
                                                 description="Instructs MwmBuilder to rescale everything by the given factor. Exporting a character seems to require a value 0.01. The armature must have the same scale.")
        rotation_y = bpy.props.FloatProperty(name="Rotation Y", min=-1000, max=1000, soft_min=-360, soft_max=360,
                                             default=0,
                                             description="Instructs MwmBuilder to rotate everything around the Y-axis. Exporting a character seems to require a value of 180°")
    else:
        rescale_factor = bpy.props.FloatProperty(name="Rescale Factor", min=0.001, max=1000, soft_min=0.01, soft_max=10,
                                                 default=0.01,
                                                 description="Instructs MwmBuilder to rescale everything by the given factor. Exporting a character seems to require a value 0.01. The armature must have the same scale.")
        rotation_y = bpy.props.FloatProperty(name="Rotation Y", min=-1000, max=1000, soft_min=-360, soft_max=360,
                                             default=0,
                                             description="Instructs MwmBuilder to rotate everything around the Y-axis. Exporting a character seems to require a value of 180°")

honesty its kinda scattered all over the damn place lol
I think for your initial step of just getting to the fbximporter though you need whats in these three only

def fbx_to_hkt(settings: ExportSettings, srcfile, dstfile):

def callTool(self, cmdline, logfile=None, cwd=None, successfulExitCodes=[0], loglines=[], logtextInspector=None):

def export_fbx(settings: ExportSettings, filepath, objects, fbx_settings=None):

StollieToday at 6:39 PM

essentially its
1.) Create "ExportSettings" class which contains a bunch of variables (most of which you probably wont need)
2.) Create "export_fbx" method to add the defaults for an FBX export (that may not be needed) with some other scaling which might not be applicable
3.) Create a 1:1 copy of the fbx.py file which hooks into Blenders exporter and extends it with custom properties and saves the FBX file and investigate whats linked to it and if its still needed.
4.) Create some method tied to a button that calls "export_fbx" to fire it all off.
THEN you should have a properly formatted FBX file to easily pass to the FBXImporter
also check the button Harag used to call "export_fbx" it shows how he names the files and sequenced the methods all the way up to MWMBuilder

This can be implemented in various complexities. The basic thing I want to have for sure is a node tree for the render output that overlays the correct blueish color etc. to match the vanilla icons. Ideally I'd also have a means of (re)aligning a camera onto the block to frame it correctly. And of course a lighting setup that kind of makes sense.
This whole shebang will likely also require me to deal with the situation where the user has deleted any parts of this and to then recreate it.

Anyway, setting up a node tree shouldn't be too hard and I can include it in the default file.

Step 1.) Open a model .blend created in 2.79 using Harag's plugin that also uses Keens materials.
Step 2.) Link Matlib libaries.
Step 3.) Export.
Step 4.) Bug - Nothing written in XML for linked libraries that match the material names.

Further testing required. Test blend file attached.

DailyNeeds_Manufacturing.zip

This should be not too hard to do if I can figure out how to get an object's parent(s).

• Ensure empties have parent objects (unparented likely doesn't work)
• Ensure highlight empties are parented to an object with no parent (always have to be second highest layer)
• Ensure highlight empties are parented to the same object as their target object
• parent empty to same parent that the selected object has when creating them

Mirroring is brain-bending. Mainly due to the insane way that the SE mirroring system actually works. But there are some ways to obscure all of that and make it look like it's simple. Because it really should be.

Not quite sure yet how to implement it however. I do like the idea of doing it in a similar way to harag:

• spawn three instances of the main collection contents in + of each axis, then denote their axi via gpu draw. (might not be necessary? could maybe visualize the necessary information differently)
• ask user to rotate the instances to make them "mirror" properly across the axi.
• save those values, and allow user to disable mirroring mode / re-enable it anytime (spawning the isntances by reading the values)
• on export to SBC, translate the values to the correct strings (use harag's code as a reference because fuck figuring those out myself) and save them to the cubeblocks entry.

Also have a text field inside the panel where a "mirroring model subtype" can be listed, in case rotating (say it ain't so!) didn't do the job.

StollieToday at 9:01 AM

Traceback (most recent call last):
  File "C:\Users\Cheyne\AppData\Roaming\Blender Foundation\Blender\2.81\scripts\addons\space_engineers_utilities\seut_ot_exportHKT.py", line 27, in execute
    collisionObjects = SEUT_OT_ExportHKT.getCollisionObjects(self, collections['Collision'])
KeyError: 'Collision'

might need an error catch for that

Need to decide on how to reorganize the project a bit more to make it more manageable.

Folders

• export
• havok
• empties

init

• move scene properties to separate class
• convert register / unregister to factories

This is an extension of #12 .
Basically, it should be possible to read simple animations within blender and then export them to a script that can just be slapped into the mod. Movement is probably going to be harder if it happens across more than one axis at a time but apart from that I think it might be possible to do. Would of course need a bunch of help on the C# script side.

This is important for the SBC setup and it would probably also make sense to put it into exportXML even though it's technically not needed. Might be needed for build stages too.

probably due to the weird way SE does directions, it needs to be rotated or sth.

see here: https://discordapp.com/channels/125011928711036928/161758345856811008/670006691601055805

I need to look at the amount of BS collections that have stuff in them, then divide 1 by that amount to get BuildPercentUpperBound . Also need to dynamically create more or less entries depending on the amount of build stages.

When doing sub-part exportation for the purposes of animation the main scene model has a dummy placed at the origin point of the sub-part and the actual mesh is moved to another scene which can then be independently rotated/transformed from the main mesh via code as a separate entity to simulate its movement in game.

The current exportation process expects certain collections to exist and will throw errors if they are not present, a further complication is that you can't create an already existing name in a subscene to adhere to collection names.

For example if the 'Main Scene' has a collection called 'Main' then the 'Subpart1Scene' will not allow a collection called 'Main' it will rename it to 'Main.001'

Another thing I want to implement into the bounding box operator is an option to display directional indicators forward, (left, right, up, down, back) in the viewport as it can become kind of confusing with SE not using the normal conventions, you know, because of course they don't.

That will require diving into the text draw options of Blender. Drawing will probably be less of the issue, considering I already have the box drawn, but determining where and with which rotation to draw text might get tricky.

This is related to #46 and #12 .

In order to make the switch from 2.79 to 2.8 for modders less painful, we want to add a button that helps convert an opened blend file over to our new structure.

It should do the following:

• remap materials
• import subparts from additional scenes to main scene
• redo collections and move objects to the correct collections from the ones they get placed in after conversion from layers

In theory, custom materials are already supported. However, in reality, it's finicky and more complicated than it should be. #4 is a prerequisite for this, but I should set up a new toolbar menu in the shader editor to "create SE material", which takes the Template_Full material from the matlib and copies / renames it for the user to then plug in new texture files.
This panel would also need an option to input more of the custom material parameters such as:

<Parameter Name="Technique">MESH</Parameter>

• Create SMAT
• Field to set technique
• Specularity
• Diffuse
• Diffuse Texture (not quite sure how to handle this yet)
• XML export (also respect toggle)
• Possibly adjust remap materials to respect toggle

It's missing still. Shouldn't be too complicated to add though.

probably integrate into export_LOD

but need additional property in toolbar to set BS LOD - limit to one for all of them though.

It's probably not needed and might actually break stuff by setting the path in stone.

Currently it's only compressing the main file.
we'll need some code to instead scan the export path for files starting with the subtypeId and then run mwm conversion on them
that will catch the _BS and _LOD ones
also, HKT needs to be copied for the buildstages

this is due to the main export calling the individual operators.

The bounding box denotes the size of the block in SE. If the collision mesh is larger than the bounding box, SE throws a hissy fit and it will most likely be replaced by block collision or something like that.

In order to prevent that, I want to color the face of the bounding box that is exceeded by a part of the collision mesh in red (transparent stripes, if possible) to signify that the collision mesh needs to be adjusted.

Another thing I want to have is proper mountpoint setup support. I have an idea in my head of how to implement it but I'm not sure it's feasible (technically and skill-wise):

Basically, when "mountpoint-mode" is activated, I take the size of the bounding box and then draw square planes flat onto the outside of the bounding box ( but slightly smaller, mostly translucent).
The squares are placed in the location of the SE blocks, so gridsize * 2 (because grid is always half a SE block). Thus you'd have one square per possible mountpoint.

I then want the user to be able to click on those mountpoints to toggle them on / off (change color, possibly outline / transparency). This is the part where I'm not sure that is actually possible - code-wise it should be, but I'm by far not as confident in my ability to realize it.

Lastly, I'd have to translate the matrices that those mountpoints would be saved in to the SBC entries, which might get complicated as well considering that I'd need to traverse the matrix to find connected mountpoints and sum them up into one line in the SBC. Could be lazy and just do one mountpoint = one entry, but that would be horribly unoptimized and Kreeg has mentioned that too many mountpoints can make SE lag out hard.

needed for MWM export and can also use this in the future to relink matlibs.

We might have to rotate the model by Z: 180° before exporting to fbx.hkt

I need to do some detailed testing with stuff that works / doesn't work and try to nail down where the problem is:

First round of tests
With SEUT export:

• Import vanilla test FBX, export, test
• Import vanilla test FBX, apply my materials, export, test

With normal export

• Import vanilla test FBX, export, test
• Import vanilla test FBX, apply my materials, export, test

Second round of tests
Provided there was no difference between exports in first round.
With SEUT export:

• Import vanilla test FBX, create custom mesh, apply vanilla material, export, test
• create custom mesh, apply my material, export, test
• create custom mesh, apply generic material, export, test
• Import vanilla test FBX, apply generic material, export, test
• Import vanilla test FBX, apply custom texture to one part, export, test

HKTs aren't affected by RescaleFactor via XML, thus the model itself needs to be scaled before it is exported to match the rescalefactor (minus the additional /100 that is usually applied when writing rescalefactor)

There's some specific stuff for it and I should look into it properly, then see what can be done.

https://forum.keenswh.com/threads/guide-importing-the-astronaut-into-blender.7382628/

Some findings about export by Stollie: https://discordapp.com/channels/125011928711036928/161758345856811008/674725223542685701

https://steamcommunity.com/sharedfiles/filedetails/?id=433069217&searchtext=character

https://forum.keenswh.com/threads/guide-custom-gesture-animations-video-tutorial.7399208/

• I probably don't account for max in negative space
• I only iterate through the children of the collection, and ignore the children of the children of the collection etc.

//Export\ will error even though it's valid.

We need to consolidate reports in one place because it's getting too messy and there's a ton of duplication.

I've learnt a bunch of things while working on this project, so going again through everything and optimizing / making code more consistent will be a good idea. This should happen once everything else is done, probably. Here I should also make sure the documentation is up to snuff.

That shouldn't be possible

It was without any of the collections, when hitting big export button.

possibly connected to not having any collections at all? - no, not that because it happens with collections too.