recut aborts after 2 mins about recut

[k1moradi]

Sumit is testing the bt value of 100 now. So far everything is OK in as much as I know.

[kdmarrett]

This image volume is so large that it broke the implementation of finding a bk threshold from a desire foreground percentage. In the meantime you can get around this by passing an exact background threshold which someone who preprocessed the data may know.

Using an exact is bg threshold value can be problematic especially if the image pixel value are poorly normalized, so a new sort implementation that doesn't involve a copy needs to be used in the long term.

[sumit-nanda]

Used a BG threshold value (of 70) this time for the Run. The recut ran for 17 hours, but used up all 4 terabytes of RAM, and then crashed.

[kdmarrett]

I assume this is when running recut -bt 70 --output-type mask, your system ran out of memory (RAM) as you indicated. This might be caused by bt of 70 being far too large to get the desired .1% of pixels. It could be that the bt is correct but the image is so large that your system can't handle it with that many parallel threads active.

You should run a smaller "test" portion of this volume and check if bt 70 does return the right fg % when you convert.

There are alternate strategies but I do not recommend because it may take a day to get feedback:

1. you could run with a lower bt, or
2. run it with less threads with less threads in flight less peak memory is used

[sumit-nanda]

It was running ok, the RAM usage was also under control. It for a day and half. But then it threw a new error and aborted.

[kdmarrett]

what is the fg percent of run-6/mask.vdb, you can query it with vdb_print -l -m run-6/mask.vdb.

It looks like there are so many fg voxels that it maxes out the maximum number of elements allowed in c++'s std::vector

[sumit-nanda]

Name: out.vdb
Information about Tree:5062 s
Type: Tree_mask_5_4_3
Configuration:pdating within 00:00:17:27 d:h:m:s
Root(1 x 101), Internal(101 x 32^3), Internal(246,699 x 16^3), Leaf(110,644,008 x 8^3)
Background value: 0detection:
Min value: 1o sdf step
Max value: 1der morphological operations
Number of active voxels: 40,554,999,062
Number of active tiles: 36,771,163
Bounding box of active voxels: [594, 12, 0] -> [137707, 11878, 820]
Dimensions of active voxels: 137114 x 11867 x 821
Percentage of active voxels: 3.04%
Average leaf node fill ratio: 35.8%
Number of unallocated nodes: 0 (0%)
Memory footprint:
Actual: 16.036 GB
Active leaf voxels: 18.873 GB
Dense equivalent: 1.215 TBSpinal_Cord_DF_20x_1000z$
Actual footprint is 1.29% of an equivalent dense volume$ recut /data/SM220621_04_Spinal_Cord_DF_20x_1000z/ch1 - Leaf voxel footprint is 118% of actual footprint.3 .3 1 -bt 100 -pl 128 --mean-shift 3 --close-steps 24 --open-Additional metadata:
snanda@u116810:/data/SM220621_04_Spinal_Cord_DF_20x_1000z$
class: fog volumee "/data/SM220621_04_Spinal_Cord_DF_20x_1000z/ch1"
creator: recut12032, 824]
file_bbox_max: [137707, 11878, 820]
file_bbox_min: [594, 12, 0]by the anisotropic factor of voxel sizes.
file_compression: blosc + active values
file_mem_bytes: 17218240088
file_voxel_count: 40554999062
is_local_space: false
name: out.vdb
original_bounding_extent_x: 137728
original_bounding_extent_y: 12032
original_bounding_extent_z: 824:00:19:49 d:h:m:s
requested_fg_pct: -0.01/mask.vdb"
resolution_level: 0detection:
Transform:sk to sdf step
voxel size: 1er morphological operations
index to world:: open = 15
[1, 0, 0, 0] on step
[0, 1, 0, 0] ep
[0, 0, 1, 0] nt step
[0, 0, 0, 1] fter throwing an instance of 'std::bad_array_new_length'
what(): std::bad_array_new_length

[kdmarrett]

Solved, see commit 826134a