max_labels questions about connected-components-3d

[william-silversmith]

Hi Stephan,

The max_labels argument is a memory reduction hack that's not guaranteed to work well. Typically, I try to estimate it (for connectomics data) as a "representative volume" number of labels plus a large safety factor. In Kimimaro, right or wrong, it is set to 1/4th the number of voxels in the volume.

I might have a better way to reduce memory usage. I'm currently experimenting with removing the "union by size" feature from union-find. In some experiments, it reduces memory usage by half and improves performance ~10% on a set of connectomics labels and on random arrays. However, it doesn't make a ton of sense to me that it's faster.

If I can find a theoretical justification for it I'd be happy to release that, as it's less code and more performant.

[william-silversmith]

I added issue #16 for memory reduction discussion.

[william-silversmith]

@unidesigner Check out this PR and let me know what you think. #17

[william-silversmith]

@unidesigner I released v1.2.2, which can achieve lower than 2x the previous memory consumption and is also ~40% faster.

[unidesigner]

@william-silversmith Fantastic news - thanks for getting back to this so quickly! I will test it tomorrow and report back asap.

[unidesigner]

It works for me too, is much faster and uses less memory! The number of regions it finds is still the same, so so much for an additional datapoint to confirm that things still work. I close this is issue as I don't think there is not much else we can do about the max_label option at the moment.