GithubHelp home page GithubHelp logo

Comments (14)

dorianps avatar dorianps commented on July 18, 2024

I got the library compilation to work by editing the bash file to point to different "include" locations, but the .cpp compilation is still failing. This is what I changed in the bash script:

# Need to install Intel OpenCL SDK first
elif [ "$OPENCL_PACKAGE" -eq "$INTEL" ] ; then
#    OPENCL_HEADER_DIRECTORY1=/opt/intel/opencl-sdk/include
#    OPENCL_HEADER_DIRECTORY2=/opt/intel/opencl-sdk/include/CL
     OPENCL_HEADER_DIRECTORY1=/opt/intel/system_studio_2020/opencl/SDK/include/
     OPENCL_HEADER_DIRECTORY2=/opt/intel/system_studio_2020/opencl/SDK/include/CL/

from broccoli.

wanderine avatar wanderine commented on July 18, 2024

from broccoli.

dorianps avatar dorianps commented on July 18, 2024

Anders, thank you for replying. I know what you mean, I have built code myself which I don't use but other users continue using and need support with it.

The compilation errors are in the first post above. Basically, when I run RegisterTwoVolumes, the pipeline tries to go and compile the .cpp files, and shows the failures in the first post. Ironically, the compilation seem to produce some binaries in the compiled folder, but I don't know if they are right, and I am suspecting they are not since there is no nonlinear registration produced.

Here are the binaries produced, my graphic card is an embedded Intel Xe

dp@Surface7Win11:/opt/brp/depend/Broccoli/compiled/Kernels$ ll
total 4912
drwxrwxrwx 2 root root    4096 Aug 17 00:07  ./
drwxrwxrwx 6 root root    4096 Aug 17 00:00  ../
-rwxrwxrwx 1 root root      54 Aug 17 00:00  README.md*
-rw-r--r-- 1 dp   dp    114128 Aug 17 00:07 'broccoli_lib_kernel_Intel_Intel(R)Graphics[0x9a49]_Clusterize.bin'
-rw-r--r-- 1 dp   dp   1751800 Aug 17 00:07 'broccoli_lib_kernel_Intel_Intel(R)Graphics[0x9a49]_Convolution.bin'
-rw-r--r-- 1 dp   dp    291848 Aug 17 00:07 'broccoli_lib_kernel_Intel_Intel(R)Graphics[0x9a49]_Registration.bin'
-rw-r--r-- 1 dp   dp    178200 Aug 17 00:07 'broccoli_lib_kernel_Intel_Intel(R)Graphics[0x9a49]_Searchlight.bin'
-rw-r--r-- 1 dp   dp    310024 Aug 17 00:07 'broccoli_lib_kernel_Intel_Intel(R)Graphics[0x9a49]_Statistics1.bin'
-rw-r--r-- 1 dp   dp    513944 Aug 17 00:07 'broccoli_lib_kernel_Intel_Intel(R)Graphics[0x9a49]_Statistics2.bin'
-rw-r--r-- 1 dp   dp    475680 Aug 17 00:07 'broccoli_lib_kernel_Intel_Intel(R)Graphics[0x9a49]_Statistics3.bin'
-rw-r--r-- 1 dp   dp    632424 Aug 17 00:07 'broccoli_lib_kernel_Intel_Intel(R)Graphics[0x9a49]_Statistics4.bin'
-rw-r--r-- 1 dp   dp    536880 Aug 17 00:07 'broccoli_lib_kernel_Intel_Intel(R)Graphics[0x9a49]_Statistics5.bin'
-rw-r--r-- 1 dp   dp    137824 Aug 17 00:07 'broccoli_lib_kernel_Intel_Intel(R)Graphics[0x9a49]_Whitening.bin'
-rw-r--r-- 1 dp   dp      4191 Aug 17 10:52 'buildInfo_Intel_Intel(R)Graphics[0x9a49]_Bayesian.txt'
-rw-r--r-- 1 dp   dp        42 Aug 17 10:52 'buildInfo_Intel_Intel(R)Graphics[0x9a49]_Clusterize.txt'
-rw-r--r-- 1 dp   dp        42 Aug 17 10:52 'buildInfo_Intel_Intel(R)Graphics[0x9a49]_Convolution.txt'
-rw-r--r-- 1 dp   dp      3714 Aug 17 10:52 'buildInfo_Intel_Intel(R)Graphics[0x9a49]_Misc.txt'
-rw-r--r-- 1 dp   dp        42 Aug 17 10:52 'buildInfo_Intel_Intel(R)Graphics[0x9a49]_Registration.txt'
-rw-r--r-- 1 dp   dp        42 Aug 17 10:52 'buildInfo_Intel_Intel(R)Graphics[0x9a49]_Searchlight.txt'
-rw-r--r-- 1 dp   dp        42 Aug 17 10:52 'buildInfo_Intel_Intel(R)Graphics[0x9a49]_Statistics1.txt'
-rw-r--r-- 1 dp   dp        42 Aug 17 10:52 'buildInfo_Intel_Intel(R)Graphics[0x9a49]_Statistics2.txt'
-rw-r--r-- 1 dp   dp        42 Aug 17 10:52 'buildInfo_Intel_Intel(R)Graphics[0x9a49]_Statistics3.txt'
-rw-r--r-- 1 dp   dp        42 Aug 17 10:52 'buildInfo_Intel_Intel(R)Graphics[0x9a49]_Statistics4.txt'
-rw-r--r-- 1 dp   dp        42 Aug 17 10:52 'buildInfo_Intel_Intel(R)Graphics[0x9a49]_Statistics5.txt'
-rw-r--r-- 1 dp   dp        42 Aug 17 10:52 'buildInfo_Intel_Intel(R)Graphics[0x9a49]_Whitening.txt'

As a background note, the reason I need Broccoli is because it is in the AutoVOI pipeline, which uses rapid registration of volumes to provide some information and voxel coordinates to an MR technologist for the Minnesota MRS sequence. I could look into other fast algorithms, like greedy or GPU based ANTs/Freesurfer but I don't want to change anything from the AutoVOI pipeline to align with the proven advantage it has on MRS scans.

from broccoli.

dorianps avatar dorianps commented on July 18, 2024

This is what the text file says about the registration binary:

dp@Surface7Win11:/opt/brp/depend/Broccoli/compiled/Kernels$ cat buildInfo_Intel_Intel\(R\)Graphics\[0x9a49\]_Registration.txt
Kernel was successfully built from binary:

So, not sure if the cpp compilation errors above are related at all with these successfully built binaries. You may know better the structure of the software.

from broccoli.

dorianps avatar dorianps commented on July 18, 2024

Anders, any thoughts on this?

I will have access to a dedicated Ubuntu machine soon, and will see if anything works better, but from the errors I am starting to suspect that this is not a WSL problem but rather incompatibility between the older Broccoli write-up and the newer paths for openCL. In that case, the new Ubuntu machine may fail the same way my WSL Ubuntu fails to satisfy Broccoli's compilation need of OpenCL.

I am happy to get in a call and troubleshoot together if it helps. This project is top priority for me since we are testing the MRS sequence with a volunteer in few weeks, and we need to decide if we can use AutoVOI in the project or not.

Thank you again.

from broccoli.

wanderine avatar wanderine commented on July 18, 2024

from broccoli.

dorianps avatar dorianps commented on July 18, 2024

Here is the content of buildInfo_Intel_Intel\(R\)Graphics\[0x9a49\]_Misc.txt:

dp@Surface7Win11:/opt/brp/depend/Broccoli$ cat compiled/Kernels/buildInfo_Intel_Intel\(R\)Graphics\[0x9a49\]_Misc.txt
1:26:26: warning: unsupported OpenCL extension 'cl_khr_fp64' - ignoring
#pragma OPENCL EXTENSION cl_khr_fp64: enable
                         ^
1:65:45: error: use of type 'double' requires cl_khr_fp64 support
__kernel void IdentityMatrixDouble(__global double* Matrix,
                                            ^
1:76:37: warning: double precision constant requires cl_khr_fp64, casting to single precision
                Matrix[Calculate2DIndex(x,y,N)] = 1.0;
                                                  ^
1:80:37: warning: double precision constant requires cl_khr_fp64, casting to single precision
                Matrix[Calculate2DIndex(x,y,N)] = 0.0;
                                                  ^
1:95:42: error: use of type 'double' requires cl_khr_fp64 support
__kernel void LogitMatrixDouble(__global double* Matrix,
                                         ^
1:103:14: warning: double precision constant requires cl_khr_fp64, casting to single precision
        Matrix[x] = 1.0 - (2.0 / (1.0 + exp(-Matrix[x] )) );
                    ^
1:103:21: warning: double precision constant requires cl_khr_fp64, casting to single precision
        Matrix[x] = 1.0 - (2.0 / (1.0 + exp(-Matrix[x] )) );
                           ^
1:103:28: warning: double precision constant requires cl_khr_fp64, casting to single precision
        Matrix[x] = 1.0 - (2.0 / (1.0 + exp(-Matrix[x] )) );
                                  ^
1:103:34: error: call to 'exp' is ambiguous
        Matrix[x] = 1.0 - (2.0 / (1.0 + exp(-Matrix[x] )) );
                                        ^~~
./opencl-c.h:6937:21: note: candidate function
float __ovld __cnfn exp(float x);
                    ^
./opencl-c.h:6952:20: note: candidate function
half __ovld __cnfn exp(half x);
                   ^
1:134:43: error: use of type 'double' requires cl_khr_fp64 support
__kernel void GetSubMatrixDouble(__global double* Small_Matrix,
                                          ^
1:135:49: error: use of type 'double' requires cl_khr_fp64 support
                                 __global const double* Matrix,
                                                ^
1:178:44: error: use of type 'double' requires cl_khr_fp64 support
__kernel void PermuteMatrixDouble(__global double* Permuted_Matrix,
                                           ^
1:179:29: error: use of type 'double' requires cl_khr_fp64 support
                                                              __global const double* Matrix,
                                                                             ^
1:459:55: error: use of type 'double' requires cl_khr_fp64 support
__kernel void SubtractVolumesOverwriteDouble(__global double* Volume1,
                                                      ^
1:460:57: error: use of type 'double' requires cl_khr_fp64 support
                                                 __global const double* Volume2,
                                                                ^
1:534:55: error: use of type 'double' requires cl_khr_fp64 support
__kernel void MultiplyVolumesOverwriteDouble(__global double* Volume1,
                                                      ^
1:535:58: error: use of type 'double' requires cl_khr_fp64 support
                                                 __global const double* Volume2,
                                                                ^
1:579:37: error: use of type 'double' requires cl_khr_fp64 support
__kernel void MemsetDouble(__global double *Data,
                                    ^
1:580:35: error: use of type 'double' requires cl_khr_fp64 support
                               __private double value,
                                         ^
dp@Surface7Win11:/opt/brp/depend/Broccoli$

Any suggested next step to make it work?

from broccoli.

dorianps avatar dorianps commented on July 18, 2024

cl_khr_fp64 seems to be on the list of supported modules listed in clinfo.

dp@Surface7Win11:/opt/brp/depend/Broccoli$ clinfo
Number of platforms                               2
  Platform Name                                   Intel(R) OpenCL Graphics
  Platform Vendor                                 Intel(R) Corporation
  Platform Version                                OpenCL 3.0
  Platform Profile                                FULL_PROFILE
  Platform Extensions                             cl_khr_byte_addressable_store cl_khr_device_uuid cl_khr_fp16 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_icd cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_intel_command_queue_families cl_intel_subgroups cl_intel_required_subgroup_size cl_intel_subgroups_short cl_khr_spir cl_intel_accelerator cl_intel_driver_diagnostics cl_khr_priority_hints cl_khr_throttle_hints cl_khr_create_command_queue cl_intel_subgroups_char cl_intel_subgroups_long cl_khr_il_program cl_intel_mem_force_host_memory cl_khr_subgroup_extended_types cl_khr_subgroup_non_uniform_vote cl_khr_subgroup_ballot cl_khr_subgroup_non_uniform_arithmetic cl_khr_subgroup_shuffle cl_khr_subgroup_shuffle_relative cl_khr_subgroup_clustered_reduce cl_intel_device_attribute_query cl_khr_suggested_local_work_size cl_intel_split_work_group_barrier cl_intel_spirv_media_block_io cl_intel_spirv_subgroups cl_khr_spirv_linkonce_odr cl_khr_spirv_no_integer_wrap_decoration cl_intel_unified_shared_memory cl_khr_mipmap_image cl_khr_mipmap_image_writes cl_ext_float_atomics cl_khr_external_memory cl_intel_planar_yuv cl_intel_packed_yuv cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_image2d_from_buffer cl_khr_depth_images cl_khr_3d_image_writes cl_intel_media_block_io cl_intel_subgroup_local_block_io cl_khr_integer_dot_product cl_khr_gl_sharing cl_khr_gl_depth_images cl_khr_gl_event cl_khr_gl_msaa_sharing cl_intel_sharing_format_query cl_khr_pci_bus_info
  Platform Extensions with Version                cl_khr_byte_addressable_store                                    0x400000 (1.0.0)
                                                  cl_khr_device_uuid                                               0x400000 (1.0.0)
                                                  cl_khr_fp16                                                      0x400000 (1.0.0)
                                                  cl_khr_global_int32_base_atomics                                 0x400000 (1.0.0)
                                                  cl_khr_global_int32_extended_atomics                             0x400000 (1.0.0)
                                                  cl_khr_icd                                                       0x400000 (1.0.0)
                                                  cl_khr_local_int32_base_atomics                                  0x400000 (1.0.0)
                                                  cl_khr_local_int32_extended_atomics                              0x400000 (1.0.0)
                                                  cl_intel_command_queue_families                                  0x400000 (1.0.0)
                                                  cl_intel_subgroups                                               0x400000 (1.0.0)
                                                  cl_intel_required_subgroup_size                                  0x400000 (1.0.0)
                                                  cl_intel_subgroups_short                                         0x400000 (1.0.0)
                                                  cl_khr_spir                                                      0x400000 (1.0.0)
                                                  cl_intel_accelerator                                             0x400000 (1.0.0)
                                                  cl_intel_driver_diagnostics                                      0x400000 (1.0.0)
                                                  cl_khr_priority_hints                                            0x400000 (1.0.0)
                                                  cl_khr_throttle_hints                                            0x400000 (1.0.0)
                                                  cl_khr_create_command_queue                                      0x400000 (1.0.0)
                                                  cl_intel_subgroups_char                                          0x400000 (1.0.0)
                                                  cl_intel_subgroups_long                                          0x400000 (1.0.0)
                                                  cl_khr_il_program                                                0x400000 (1.0.0)
                                                  cl_intel_mem_force_host_memory                                   0x400000 (1.0.0)
                                                  cl_khr_subgroup_extended_types                                   0x400000 (1.0.0)
                                                  cl_khr_subgroup_non_uniform_vote                                 0x400000 (1.0.0)
                                                  cl_khr_subgroup_ballot                                           0x400000 (1.0.0)
                                                  cl_khr_subgroup_non_uniform_arithmetic                           0x400000 (1.0.0)
                                                  cl_khr_subgroup_shuffle                                          0x400000 (1.0.0)
                                                  cl_khr_subgroup_shuffle_relative                                 0x400000 (1.0.0)
                                                  cl_khr_subgroup_clustered_reduce                                 0x400000 (1.0.0)
                                                  cl_intel_device_attribute_query                                  0x400000 (1.0.0)
                                                  cl_khr_suggested_local_work_size                                 0x400000 (1.0.0)
                                                  cl_intel_split_work_group_barrier                                0x400000 (1.0.0)
                                                  cl_intel_spirv_media_block_io                                    0x400000 (1.0.0)
                                                  cl_intel_spirv_subgroups                                         0x400000 (1.0.0)
                                                  cl_khr_spirv_linkonce_odr                                        0x400000 (1.0.0)
                                                  cl_khr_spirv_no_integer_wrap_decoration                          0x400000 (1.0.0)
                                                  cl_intel_unified_shared_memory                                   0x400000 (1.0.0)
                                                  cl_khr_mipmap_image                                              0x400000 (1.0.0)
                                                  cl_khr_mipmap_image_writes                                       0x400000 (1.0.0)
                                                  cl_ext_float_atomics                                             0x400000 (1.0.0)
                                                  cl_khr_external_memory                                           0x400000 (1.0.0)
                                                  cl_intel_planar_yuv                                              0x400000 (1.0.0)
                                                  cl_intel_packed_yuv                                              0x400000 (1.0.0)
                                                  cl_khr_int64_base_atomics                                        0x400000 (1.0.0)
                                                  cl_khr_int64_extended_atomics                                    0x400000 (1.0.0)
                                                  cl_khr_image2d_from_buffer                                       0x400000 (1.0.0)
                                                  cl_khr_depth_images                                              0x400000 (1.0.0)
                                                  cl_khr_3d_image_writes                                           0x400000 (1.0.0)
                                                  cl_intel_media_block_io                                          0x400000 (1.0.0)
                                                  cl_intel_subgroup_local_block_io                                 0x400000 (1.0.0)
                                                  cl_khr_integer_dot_product                                       0x800000 (2.0.0)
                                                  cl_khr_gl_sharing                                                0x400000 (1.0.0)
                                                  cl_khr_gl_depth_images                                           0x400000 (1.0.0)
                                                  cl_khr_gl_event                                                  0x400000 (1.0.0)
                                                  cl_khr_gl_msaa_sharing                                           0x400000 (1.0.0)
                                                  cl_intel_sharing_format_query                                    0x400000 (1.0.0)
                                                  cl_khr_pci_bus_info                                              0x400000 (1.0.0)
  Platform Numeric Version                        0xc00000 (3.0.0)
  Platform Extensions function suffix             INTEL
  Platform Host timer resolution                  1ns

  Platform Name                                   Intel(R) CPU Runtime for OpenCL(TM) Applications
  Platform Vendor                                 Intel(R) Corporation
  Platform Version                                OpenCL 2.1 LINUX
  Platform Profile                                FULL_PROFILE
  Platform Extensions                             cl_khr_icd cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_byte_addressable_store cl_khr_depth_images cl_khr_3d_image_writes cl_intel_exec_by_local_thread cl_khr_spir cl_khr_fp64 cl_khr_image2d_from_buffer cl_intel_vec_len_hint
  Platform Extensions function suffix             INTEL
  Platform Host timer resolution                  1ns

  Platform Name                                   Intel(R) OpenCL Graphics
Number of devices                                 1
  Device Name                                     Intel(R) Graphics [0x9a49]
  Device Vendor                                   Intel(R) Corporation
  Device Vendor ID                                0x8086
  Device Version                                  OpenCL 3.0 NEO
  Device UUID                                     8680499a-0100-0000-0002-000000000000
  Driver UUID                                     32332e33-332e-3000-0000-000000000000
  Valid Device LUID                               No
  Device LUID                                     709b-18affc7f0000
  Device Node Mask                                0
  Device Numeric Version                          0xc00000 (3.0.0)
  Driver Version                                  23.33.0
  Device OpenCL C Version                         OpenCL C 1.2
  Device OpenCL C all versions                    OpenCL C                                                         0x400000 (1.0.0)
                                                  OpenCL C                                                         0x401000 (1.1.0)
                                                  OpenCL C                                                         0x402000 (1.2.0)
                                                  OpenCL C                                                         0xc00000 (3.0.0)
  Device OpenCL C features                        __opencl_c_int64                                                 0xc00000 (3.0.0)
                                                  __opencl_c_3d_image_writes                                       0xc00000 (3.0.0)
                                                  __opencl_c_images                                                0xc00000 (3.0.0)
                                                  __opencl_c_read_write_images                                     0xc00000 (3.0.0)
                                                  __opencl_c_atomic_order_acq_rel                                  0xc00000 (3.0.0)
                                                  __opencl_c_atomic_order_seq_cst                                  0xc00000 (3.0.0)
                                                  __opencl_c_atomic_scope_all_devices                              0xc00000 (3.0.0)
                                                  __opencl_c_atomic_scope_device                                   0xc00000 (3.0.0)
                                                  __opencl_c_generic_address_space                                 0xc00000 (3.0.0)
                                                  __opencl_c_program_scope_global_variables                        0xc00000 (3.0.0)
                                                  __opencl_c_work_group_collective_functions                       0xc00000 (3.0.0)
                                                  __opencl_c_subgroups                                             0xc00000 (3.0.0)
                                                  __opencl_c_ext_fp32_global_atomic_add                            0xc00000 (3.0.0)
                                                  __opencl_c_ext_fp32_local_atomic_add                             0xc00000 (3.0.0)
                                                  __opencl_c_ext_fp32_global_atomic_min_max                        0xc00000 (3.0.0)
                                                  __opencl_c_ext_fp32_local_atomic_min_max                         0xc00000 (3.0.0)
                                                  __opencl_c_ext_fp16_global_atomic_load_store                     0xc00000 (3.0.0)
                                                  __opencl_c_ext_fp16_local_atomic_load_store                      0xc00000 (3.0.0)
                                                  __opencl_c_ext_fp16_global_atomic_min_max                        0xc00000 (3.0.0)
                                                  __opencl_c_ext_fp16_local_atomic_min_max                         0xc00000 (3.0.0)
                                                  __opencl_c_integer_dot_product_input_4x8bit                      0xc00000 (3.0.0)
                                                  __opencl_c_integer_dot_product_input_4x8bit_packed               0xc00000 (3.0.0)
  Latest comfornace test passed                   v2023-05-16-00
  Device Type                                     GPU
  Device Profile                                  FULL_PROFILE
  Device Available                                Yes
  Compiler Available                              Yes
  Linker Available                                Yes
  Max compute units                               96
  Max clock frequency                             1300MHz
  Device Partition                                (core)
    Max number of sub-devices                     0
    Supported partition types                     None
    Supported affinity domains                    (n/a)
  Max work item dimensions                        3
  Max work item sizes                             512x512x512
  Max work group size                             512
  Preferred work group size multiple (device)     64
  Preferred work group size multiple (kernel)     64
  Max sub-groups per work group                   64
  Sub-group sizes (Intel)                         8, 16, 32
  Preferred / native vector sizes
    char                                                16 / 16
    short                                                8 / 8
    int                                                  4 / 4
    long                                                 1 / 1
    half                                                 8 / 8        (cl_khr_fp16)
    float                                                1 / 1
    double                                               0 / 0        (n/a)
  Half-precision Floating-point support           (cl_khr_fp16)
    Denormals                                     Yes
    Infinity and NANs                             Yes
    Round to nearest                              Yes
    Round to zero                                 Yes
    Round to infinity                             Yes
    IEEE754-2008 fused multiply-add               Yes
    Support is emulated in software               No
  Single-precision Floating-point support         (core)
    Denormals                                     Yes
    Infinity and NANs                             Yes
    Round to nearest                              Yes
    Round to zero                                 Yes
    Round to infinity                             Yes
    IEEE754-2008 fused multiply-add               Yes
    Support is emulated in software               No
    Correctly-rounded divide and sqrt operations  No
  Double-precision Floating-point support         (n/a)
  Address bits                                    64, Little-Endian
  Global memory size                              6805753856 (6.338GiB)
  Error Correction support                        No
  Max memory allocation                           1073741824 (1024MiB)
  Unified memory for Host and Device              Yes
  Shared Virtual Memory (SVM) capabilities        (core)
    Coarse-grained buffer sharing                 Yes
    Fine-grained buffer sharing                   No
    Fine-grained system sharing                   No
    Atomics                                       No
  Minimum alignment for any data type             128 bytes
  Alignment of base address                       1024 bits (128 bytes)
  Preferred alignment for atomics
    SVM                                           64 bytes
    Global                                        64 bytes
    Local                                         64 bytes
  Atomic memory capabilities                      relaxed, acquire/release, sequentially-consistent, work-group scope, device scope, all-devices scope
  Atomic fence capabilities                       relaxed, acquire/release, sequentially-consistent, work-item scope, work-group scope, device scope, all-devices scope
  Max size for global variable                    65536 (64KiB)
  Preferred total size of global vars             1073741824 (1024MiB)
  Global Memory cache type                        Read/Write
  Global Memory cache size                        3932160 (3.75MiB)
  Global Memory cache line size                   64 bytes
  Image support                                   Yes
    Max number of samplers per kernel             16
    Max size for 1D images from buffer            67108864 pixels
    Max 1D or 2D image array size                 2048 images
    Base address alignment for 2D image buffers   4 bytes
    Pitch alignment for 2D image buffers          4 pixels
    Max 2D image size                             16384x16384 pixels
    Max planar YUV image size                     16384x16352 pixels
    Max 3D image size                             2048x2048x2048 pixels
    Max number of read image args                 128
    Max number of write image args                128
    Max number of read/write image args           128
  Pipe support                                    No
  Max number of pipe args                         0
  Max active pipe reservations                    0
  Max pipe packet size                            0
  Local memory type                               Local
  Local memory size                               65536 (64KiB)
  Max number of constant args                     8
  Max constant buffer size                        1073741824 (1024MiB)
  Generic address space support                   Yes
  Max size of kernel argument                     2048 (2KiB)
  Queue properties (on host)
    Out-of-order execution                        Yes
    Profiling                                     Yes
  Device enqueue capabilities                     (n/a)
  Queue properties (on device)
    Out-of-order execution                        No
    Profiling                                     No
    Preferred size                                0
    Max size                                      0
  Max queues on device                            0
  Max events on device                            0
  Prefer user sync for interop                    Yes
  Profiling timer resolution                      52ns
  Execution capabilities
    Run OpenCL kernels                            Yes
    Run native kernels                            No
    Non-uniform work-groups                       Yes
    Work-group collective functions               Yes
    Sub-group independent forward progress        No
    IL version                                    SPIR-V_1.2
    ILs with version                              SPIR-V                                                           0x402000 (1.2.0)
    SPIR versions                                 1.2
  printf() buffer size                            4194304 (4MiB)
  Built-in kernels                                (n/a)
  Built-in kernels with version                   (n/a)
  Device Extensions                               cl_khr_byte_addressable_store cl_khr_device_uuid cl_khr_fp16 cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_icd cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_intel_command_queue_families cl_intel_subgroups cl_intel_required_subgroup_size cl_intel_subgroups_short cl_khr_spir cl_intel_accelerator cl_intel_driver_diagnostics cl_khr_priority_hints cl_khr_throttle_hints cl_khr_create_command_queue cl_intel_subgroups_char cl_intel_subgroups_long cl_khr_il_program cl_intel_mem_force_host_memory cl_khr_subgroup_extended_types cl_khr_subgroup_non_uniform_vote cl_khr_subgroup_ballot cl_khr_subgroup_non_uniform_arithmetic cl_khr_subgroup_shuffle cl_khr_subgroup_shuffle_relative cl_khr_subgroup_clustered_reduce cl_intel_device_attribute_query cl_khr_suggested_local_work_size cl_intel_split_work_group_barrier cl_intel_spirv_media_block_io cl_intel_spirv_subgroups cl_khr_spirv_linkonce_odr cl_khr_spirv_no_integer_wrap_decoration cl_intel_unified_shared_memory cl_khr_mipmap_image cl_khr_mipmap_image_writes cl_ext_float_atomics cl_khr_external_memory cl_intel_planar_yuv cl_intel_packed_yuv cl_khr_int64_base_atomics cl_khr_int64_extended_atomics cl_khr_image2d_from_buffer cl_khr_depth_images cl_khr_3d_image_writes cl_intel_media_block_io cl_intel_subgroup_local_block_io cl_khr_integer_dot_product cl_khr_gl_sharing cl_khr_gl_depth_images cl_khr_gl_event cl_khr_gl_msaa_sharing cl_intel_sharing_format_query cl_khr_pci_bus_info
  Device Extensions with Version                  cl_khr_byte_addressable_store                                    0x400000 (1.0.0)
                                                  cl_khr_device_uuid                                               0x400000 (1.0.0)
                                                  cl_khr_fp16                                                      0x400000 (1.0.0)
                                                  cl_khr_global_int32_base_atomics                                 0x400000 (1.0.0)
                                                  cl_khr_global_int32_extended_atomics                             0x400000 (1.0.0)
                                                  cl_khr_icd                                                       0x400000 (1.0.0)
                                                  cl_khr_local_int32_base_atomics                                  0x400000 (1.0.0)
                                                  cl_khr_local_int32_extended_atomics                              0x400000 (1.0.0)
                                                  cl_intel_command_queue_families                                  0x400000 (1.0.0)
                                                  cl_intel_subgroups                                               0x400000 (1.0.0)
                                                  cl_intel_required_subgroup_size                                  0x400000 (1.0.0)
                                                  cl_intel_subgroups_short                                         0x400000 (1.0.0)
                                                  cl_khr_spir                                                      0x400000 (1.0.0)
                                                  cl_intel_accelerator                                             0x400000 (1.0.0)
                                                  cl_intel_driver_diagnostics                                      0x400000 (1.0.0)
                                                  cl_khr_priority_hints                                            0x400000 (1.0.0)
                                                  cl_khr_throttle_hints                                            0x400000 (1.0.0)
                                                  cl_khr_create_command_queue                                      0x400000 (1.0.0)
                                                  cl_intel_subgroups_char                                          0x400000 (1.0.0)
                                                  cl_intel_subgroups_long                                          0x400000 (1.0.0)
                                                  cl_khr_il_program                                                0x400000 (1.0.0)
                                                  cl_intel_mem_force_host_memory                                   0x400000 (1.0.0)
                                                  cl_khr_subgroup_extended_types                                   0x400000 (1.0.0)
                                                  cl_khr_subgroup_non_uniform_vote                                 0x400000 (1.0.0)
                                                  cl_khr_subgroup_ballot                                           0x400000 (1.0.0)
                                                  cl_khr_subgroup_non_uniform_arithmetic                           0x400000 (1.0.0)
                                                  cl_khr_subgroup_shuffle                                          0x400000 (1.0.0)
                                                  cl_khr_subgroup_shuffle_relative                                 0x400000 (1.0.0)
                                                  cl_khr_subgroup_clustered_reduce                                 0x400000 (1.0.0)
                                                  cl_intel_device_attribute_query                                  0x400000 (1.0.0)
                                                  cl_khr_suggested_local_work_size                                 0x400000 (1.0.0)
                                                  cl_intel_split_work_group_barrier                                0x400000 (1.0.0)
                                                  cl_intel_spirv_media_block_io                                    0x400000 (1.0.0)
                                                  cl_intel_spirv_subgroups                                         0x400000 (1.0.0)
                                                  cl_khr_spirv_linkonce_odr                                        0x400000 (1.0.0)
                                                  cl_khr_spirv_no_integer_wrap_decoration                          0x400000 (1.0.0)
                                                  cl_intel_unified_shared_memory                                   0x400000 (1.0.0)
                                                  cl_khr_mipmap_image                                              0x400000 (1.0.0)
                                                  cl_khr_mipmap_image_writes                                       0x400000 (1.0.0)
                                                  cl_ext_float_atomics                                             0x400000 (1.0.0)
                                                  cl_khr_external_memory                                           0x400000 (1.0.0)
                                                  cl_intel_planar_yuv                                              0x400000 (1.0.0)
                                                  cl_intel_packed_yuv                                              0x400000 (1.0.0)
                                                  cl_khr_int64_base_atomics                                        0x400000 (1.0.0)
                                                  cl_khr_int64_extended_atomics                                    0x400000 (1.0.0)
                                                  cl_khr_image2d_from_buffer                                       0x400000 (1.0.0)
                                                  cl_khr_depth_images                                              0x400000 (1.0.0)
                                                  cl_khr_3d_image_writes                                           0x400000 (1.0.0)
                                                  cl_intel_media_block_io                                          0x400000 (1.0.0)
                                                  cl_intel_subgroup_local_block_io                                 0x400000 (1.0.0)
                                                  cl_khr_integer_dot_product                                       0x800000 (2.0.0)
                                                  cl_khr_gl_sharing                                                0x400000 (1.0.0)
                                                  cl_khr_gl_depth_images                                           0x400000 (1.0.0)
                                                  cl_khr_gl_event                                                  0x400000 (1.0.0)
                                                  cl_khr_gl_msaa_sharing                                           0x400000 (1.0.0)
                                                  cl_intel_sharing_format_query                                    0x400000 (1.0.0)
                                                  cl_khr_pci_bus_info                                              0x400000 (1.0.0)

  Platform Name                                   Intel(R) CPU Runtime for OpenCL(TM) Applications
Number of devices                                 1
  Device Name                                     11th Gen Intel(R) Core(TM) i7-1165G7 @ 2.80GHz
  Device Vendor                                   Intel(R) Corporation
  Device Vendor ID                                0x8086
  Device Version                                  OpenCL 2.1 (Build 0)
  Driver Version                                  18.1.0.0920
  Device OpenCL C Version                         OpenCL C 2.0
  Device Type                                     CPU
  Device Profile                                  FULL_PROFILE
  Device Available                                Yes
  Compiler Available                              Yes
  Linker Available                                Yes
  Max compute units                               8
  Max clock frequency                             2800MHz
  Device Partition                                (core)
    Max number of sub-devices                     8
    Supported partition types                     by counts, equally, by names (Intel)
    Supported affinity domains                    (n/a)
  Max work item dimensions                        3
  Max work item sizes                             8192x8192x8192
  Max work group size                             8192
  Preferred work group size multiple (kernel)     <getWGsizes:1474: create context : error -2>
  Max sub-groups per work group                   1
  Preferred / native vector sizes
    char                                                 1 / 32
    short                                                1 / 16
    int                                                  1 / 8
    long                                                 1 / 4
    half                                                 0 / 0        (n/a)
    float                                                1 / 8
    double                                               1 / 4        (cl_khr_fp64)
  Half-precision Floating-point support           (n/a)
  Single-precision Floating-point support         (core)
    Denormals                                     Yes
    Infinity and NANs                             Yes
    Round to nearest                              Yes
    Round to zero                                 No
    Round to infinity                             No
    IEEE754-2008 fused multiply-add               No
    Support is emulated in software               No
    Correctly-rounded divide and sqrt operations  No
  Double-precision Floating-point support         (cl_khr_fp64)
    Denormals                                     Yes
    Infinity and NANs                             Yes
    Round to nearest                              Yes
    Round to zero                                 Yes
    Round to infinity                             Yes
    IEEE754-2008 fused multiply-add               Yes
    Support is emulated in software               No
  Address bits                                    64, Little-Endian
  Global memory size                              8248033280 (7.682GiB)
  Error Correction support                        No
  Max memory allocation                           2062008320 (1.92GiB)
  Unified memory for Host and Device              Yes
  Shared Virtual Memory (SVM) capabilities        (core)
    Coarse-grained buffer sharing                 Yes
    Fine-grained buffer sharing                   Yes
    Fine-grained system sharing                   Yes
    Atomics                                       Yes
  Minimum alignment for any data type             128 bytes
  Alignment of base address                       1024 bits (128 bytes)
  Preferred alignment for atomics
    SVM                                           64 bytes
    Global                                        64 bytes
    Local                                         0 bytes
  Max size for global variable                    65536 (64KiB)
  Preferred total size of global vars             65536 (64KiB)
  Global Memory cache type                        Read/Write
  Global Memory cache size                        262144 (256KiB)
  Global Memory cache line size                   64 bytes
  Image support                                   Yes
    Max number of samplers per kernel             480
    Max size for 1D images from buffer            128875520 pixels
    Max 1D or 2D image array size                 2048 images
    Base address alignment for 2D image buffers   64 bytes
    Pitch alignment for 2D image buffers          64 pixels
    Max 2D image size                             16384x16384 pixels
    Max 3D image size                             2048x2048x2048 pixels
    Max number of read image args                 480
    Max number of write image args                480
    Max number of read/write image args           480
  Max number of pipe args                         16
  Max active pipe reservations                    32767
  Max pipe packet size                            1024
  Local memory type                               Global
  Local memory size                               32768 (32KiB)
  Max number of constant args                     480
  Max constant buffer size                        131072 (128KiB)
  Max size of kernel argument                     3840 (3.75KiB)
  Queue properties (on host)
    Out-of-order execution                        Yes
    Profiling                                     Yes
    Local thread execution (Intel)                Yes
  Queue properties (on device)
    Out-of-order execution                        Yes
    Profiling                                     Yes
    Preferred size                                4294967295 (4GiB)
    Max size                                      4294967295 (4GiB)
  Max queues on device                            4294967295
  Max events on device                            4294967295
  Prefer user sync for interop                    No
  Profiling timer resolution                      1ns
  Execution capabilities
    Run OpenCL kernels                            Yes
    Run native kernels                            Yes
    Sub-group independent forward progress        No
    IL version                                    SPIR-V_1.0
    SPIR versions                                 1.2
  printf() buffer size                            1048576 (1024KiB)
  Built-in kernels                                (n/a)
  Device Extensions                               cl_khr_icd cl_khr_global_int32_base_atomics cl_khr_global_int32_extended_atomics cl_khr_local_int32_base_atomics cl_khr_local_int32_extended_atomics cl_khr_byte_addressable_store cl_khr_depth_images cl_khr_3d_image_writes cl_intel_exec_by_local_thread cl_khr_spir cl_khr_fp64 cl_khr_image2d_from_buffer cl_intel_vec_len_hint


NULL platform behavior
  clGetPlatformInfo(NULL, CL_PLATFORM_NAME, ...)  No platform
  clGetDeviceIDs(NULL, CL_DEVICE_TYPE_ALL, ...)   No platform
  clCreateContext(NULL, ...) [default]            No platform
  clCreateContext(NULL, ...) [other]              Success [INTEL]
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_DEFAULT)  Success (1)
    Platform Name                                 Intel(R) OpenCL Graphics
    Device Name                                   Intel(R) Graphics [0x9a49]
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_CPU)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_GPU)  Success (1)
    Platform Name                                 Intel(R) OpenCL Graphics
    Device Name                                   Intel(R) Graphics [0x9a49]
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_ACCELERATOR)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_CUSTOM)  No devices found in platform
  clCreateContextFromType(NULL, CL_DEVICE_TYPE_ALL)  Success (1)
    Platform Name                                 Intel(R) OpenCL Graphics
    Device Name                                   Intel(R) Graphics [0x9a49]
        NOTE:   your OpenCL library only supports OpenCL 2.2,
                but some installed platforms support OpenCL 3.0.
                Programs using 3.0 features may crash
                or behave unexpectedly

from broccoli.

wanderine avatar wanderine commented on July 18, 2024

from broccoli.

dorianps avatar dorianps commented on July 18, 2024

Some progress, but hit another error related to libniftiio.a.

Here is a summary. First, I tried to run compile_wrappers.sh, but it complained it could not find opencl.h. I edited the bash script to point to the right folders where opencl.h is located. I tried both the SDK intel and the /usr/include locations as you see below (the second may be installed from intel graphics compiler, not sure).

elif [ "$OPENCL_PACKAGE" -eq "$INTEL" ]; then
#    OPENCL_HEADER_DIRECTORY1=/opt/intel/opencl-sdk/include
#    OPENCL_HEADER_DIRECTORY2=/opt/intel/opencl-sdk/include/CL
#    OPENCL_LIBRARY_DIRECTORY=/opt/intel/opencl/lib64

#    OPENCL_HEADER_DIRECTORY1=/opt/intel/system_studio_2020/opencl/SDK/include
#    OPENCL_HEADER_DIRECTORY2=/opt/intel/system_studio_2020/opencl/SDK/include/CL

    OPENCL_HEADER_DIRECTORY1=/usr/include
    OPENCL_HEADER_DIRECTORY2=/usr/include/CL

    OPENCL_LIBRARY_DIRECTORY=/opt/intel/system_studio_2020/opencl/SDK/lib64

The compiling process starts, there is a ton of logs and warnings, but the problem at the end is in Broccoli/code/Bash_Wrapper/nifticlib-2.0.0/lib/libniftiio.a:

(last few lines of log)

In file included from RandomiseGroupLevel.cpp:29:
HelpFunctions.cpp: In function ‘void ReadBinaryFile(float*, int, const char*, void**, int&, nifti_image**, int)’:
HelpFunctions.cpp:230:14: warning: ignoring return value of ‘size_t fread(void*, size_t, size_t, FILE*)’ declared with attribute ‘warn_unused_result’ [-Wunused-result]
  230 |         fread(pointer,sizeof(float),size,fp);
      |         ~~~~~^~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
/usr/bin/ld: /opt/brp/depend/Broccoli/code/Bash_Wrapper/nifticlib-2.0.0/lib/libniftiio.a(nifti1_io.o): relocation R_X86_64_32S against `.data' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: failed to set dynamic section sizes: bad value
collect2: error: ld returned 1 exit status
/usr/bin/ld: /opt/brp/depend/Broccoli/code/Bash_Wrapper/nifticlib-2.0.0/lib/libniftiio.a(nifti1_io.o): relocation R_X86_64_32S against `.data' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: failed to set dynamic section sizes: bad value
collect2: error: ld returned 1 exit status
/usr/bin/ld: /opt/brp/depend/Broccoli/code/Bash_Wrapper/nifticlib-2.0.0/lib/libniftiio.a(nifti1_io.o): relocation R_X86_64_32S against `.data' can not be used when making a PIE object; recompile with -fPIE
/usr/bin/ld: failed to set dynamic section sizes: bad value
collect2: error: ld returned 1 exit status
mv: cannot stat 'MotionCorrection': No such file or directory
mv: cannot stat 'RegisterTwoVolumes': No such file or directory
mv: cannot stat 'TransformVolume': No such file or directory
mv: cannot stat 'RandomiseGroupLevel': No such file or directory
mv: cannot stat 'FirstLevelAnalysis': No such file or directory
mv: cannot stat 'SliceTimingCorrection': No such file or directory
mv: cannot stat 'Smoothing': No such file or directory
mv: cannot stat 'GLM': No such file or directory
mv: cannot stat 'ICA': No such file or directory
mv: cannot stat 'Searchlight': No such file or directory
dp@Surface7Win11:/opt/brp/depend/Broccoli/code/Bash_Wrapper$

I googled the error and the advice I can find is to recompile for the x64 platform. Not sure if this means the nifti library was compiled on a different, incompatible platform.

What can I try next?

from broccoli.

wanderine avatar wanderine commented on July 18, 2024

from broccoli.

dorianps avatar dorianps commented on July 18, 2024

I think I managed to fix all the errors and compile. The registration of a sample dataset produced both affine and non-linear (deformed) outputs.

  1. This time I worked on Ubuntu 20.04 instead of 22.04. Probably it does not matter, it only made me waste time with building cmake 3.20 from source, which is the min version needed for other compilations. As usual, a ton of various intel packages needed to be installed to expose OpenCL within WSL.
  2. I compiled de novo nifticlib-2.0.0/ with sudo make. There were errors there too, I think I was missing a library or something like that. Managed to compile successfully at the end.
  3. I found out that intel has removed cl_khr_fp64 for Gen11 processors, so the inclusion of that module and setting variables as double did not work.
  4. I replaced all double with float in kernelMisc.cpp and kernelBayesian.cpp, easy job. After that the compilation worked fine. Paths to CL folders needed to be changed both in /code/Bash_Wrapper/compile_wrappers.sh and code/BROCCOLI_LIB/compile_broccoli_library.sh.
  5. I finally ran RegisterTwoVolumes on my test data and produced affine and nonlinear files. At first, this command tried to compile binaries again. This is strange because I already had run /code/Bash_Wrapper/compile_wrappers.sh and code/BROCCOLI_LIB/compile_broccoli_library.sh to produce the needed binaries, but anyway...

Below is the final result of my test registration. I ran it as root because all my Broccoli folders are in system locations owned by root.

root@Surface7Win11:/home/dp# /opt/brp/depend/Broccoli/compiled/Bash/Linux/Release/RegisterTwoVolumes \
> /home/dp/SampleData/mprage_reorg.nii \
> /home/dp/SampleData/atlas.nii \
> -output /home/dp/SampleData/OUTPUT.nii
Authored by K.A. Eklund
Volume 1 size: 176 x 224 x 256
Volume 1 voxel size: 1.000000 x 1.000000 x 1.000000 mm
Volume 2 size: 182 x 218 x 182
Volume 2 voxel size: 1.000000 x 1.000000 x 1.000000 mm

Affine registration parameters
 0.981809 -0.015836 -0.002459 0.270013
 0.009701 0.930506 -0.168149 -3.015697
 0.017777 0.177294 0.877184 5.718364
 0.000000 0.000000 0.000000 1.000000

I am assuming all is working now. Is there any thing else I should check before calling this a win?

from broccoli.

wanderine avatar wanderine commented on July 18, 2024

from broccoli.

dorianps avatar dorianps commented on July 18, 2024

The non-linear registration looks good, better than the affine. I am closing this issue for now and do more checks with AutoVOI.

Thank you for the help.

from broccoli.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.