zeiss / libczi Goto Github PK

The newly added/adapted code/functionality from PR #48 should be covered in a unit-test.

Describe the solution you'd like
Add a unit-test which executes the code/functionality.

Describe alternatives you've considered
N/A

Additional context
The code-coverage measurement should increase.

Is your feature request related to a problem? Please describe.
Currently, the permissions granted to GitHub Token are set to write-all on top-level. From a principle-of-least-privilege point of view this should be reduced to only the permissions needed. This was brought up by this run.

Describe the solution you'd like
Use permissions: read-all in top-level and extend on job-level if needed.

Describe alternatives you've considered
Disable the linter to report this issue - no functional change.
bridgecrewio/checkov#4127

Additional context

With Eigen already installed, CMake still insists to download and install it during buildtime.

Occurs due to line 7 in Src/CMakeLists.txt.

In contrast to #14, this cannot be turned off without hacking the script.

c.f. #73 - when doing a tile-composition within accessors, it is desirable to perform a decompression-operation concurrently. This should speed up the operation considerably (assuming the availability of multiple cores i.e.).

The implementation should be not compromise the versatility of libCZI. Best case, the libCZI-codebase should not be tied to a concurrency platform, and imperative threading should also be avoided.

Is your feature request related to a problem? Please describe.
c.f. here #12 - there should be a documented and defined sccheme for versioning.

Describe the solution you'd like
It should be documented

• which version is considered "stable" and is "recommended for which use"
• the "rules" for updating the version number here

  libczi/CMakeLists.txt

  Line 4 in afef9a2

  project(libCZI VERSION 0.46.0 )

Describe alternatives you've considered
N/A

Additional context
N/A

There is a new release of pugixml out (version 1.14), and the copy of pugixml used in libCZI should be updated.

Describe the bug
The default behavior of SingleChannelTileAccessor (after calling Clear()) is to sort all subblocks by the M-Index before accessing an ROI. Therefore, in regions with overlapping subblocks the subblock with the highest M-Index wins.
The other tile accessors (e.g. SingleChannelScalingTileAccessor) do not have this scaleByM option leading to different results in the described overlapping areas.

To Reproduce

1. Read an image with default options using the SingleChannelTileAccessor and save e.g. as PNG
2. Read an image with default oprions using the SingleChannelScalingTileAccessor (zoom=1.0) and save as PNG.
3. Observe that the diffece betwen the generated PNGs is not zero.

Expected behavior
The described results should always be the same independent from whether the SingleChannelTileAccessor or SingleChannelScalingTileAccessor with zoom=1.0 is used in their respective default configurations (Clear() method is called)

Desktop (please complete the following information):

• OS: Windows 10

Additional context
The Options struct in SingleChannelScalingTileAccessor (https://github.com/ZEISS/libczi/blob/main/Src/libCZI/libCZI_Compositor.h#L227) is missing the sortByM option that should be set to True in the respective Clear() function.
The same holds for all other tile accessors except the SingleChannelTileAccessor.

Describe the bug
The formatting, e.g. the usage of tabs-vs-spaces, is not consistent in the code-base.

To Reproduce
see #30 , and see cases like this all over the place

Expected behavior

• The formatting and the editor-settings should be consistent, and decision like tabs-vs-spaces should be documented.
• an EditorConfig file should be provided (c.f. here, here).

Is your feature request related to a problem? Please describe.
Job name test in workflow REUSE is misleading.

Describe the solution you'd like
Rename REUSE job from test to reuse and correspondingly adjust the branch protection rule to reference the renamed check.

Describe alternatives you've considered
None.

Additional context

In Src/libCZI/CMakeLists.txt, one line reads

add_library(libCZI SHARED ${LIBCZISRCFILES} ${LIBCZISRCEIGENFILES}  $<TARGET_OBJECTS:JxrDecodeStatic>)

However, LIBCZISRCEIGENFILES seems to not be set. Why would one need their source files, and just not their headers?

c.f. #73 - for the use-case where a plane is swept with a ROI (in case of using an accessor which performs tile-composition), the performance (in case of CZIs containing compressed data) is expected to greatly benefit from caching the decompressed bitmaps.

Esp. this scan-pattern is of interest here:

An implementation has to allow for controlling the memory-usage of the cache, and it must need to deal with cache eviction. The versatility of libCZI-implementation should not be compromised.

Is your feature request related to a problem? Please describe.
C. f. here -> #20 (comment)

Describe the solution you'd like
Update https://github.com/ZEISS/libczi/blob/afef9a21b56fef405ecd8117e64a8875994011b9/Src/Doxyfile so that it works with latest doxygen without warnings.

Describe alternatives you've considered
N/A

Additional context

Describe the bug
There is a significant difference in ROI reading speed, depending on the method of compression.

To Reproduce
The following Python script:

import timeit

from pylibCZIrw import czi


file_paths = ["uncompressed.czi", "jpeg-xr-compressed.czi", "zstd-compressed.czi"]

roi_x = -126748
roi_y = 46095
z = 0
t = 0
c = 0
scene = 0

num_read = 1

for file_path in file_paths:
    reader = czi.CziReader(file_path)
    roi = (roi_x, roi_y, 2000, 2000)
    read_time = timeit.timeit(
        lambda: reader.read(roi=roi, plane={"T": t, "Z": z, "C": c}, scene=scene),
        number=num_read,
        globals=globals(),
    )
    print(f"{file_path}: {read_time}s")

Outputs:

uncompressed.czi: 0.01224821200594306s
jpeg-xr-compressed.czi: 0.46801461999712046s
zstd-compressed.czi: 0.051155578999896534s

Expected behavior
Reading times of compressed and uncompressed documents don't differ, or differ less than 2x.

Desktop (please complete the following information):

• OS: Ubuntu 20.04.4 LTS
• Python Version: 3.9.11
• pyLibCZI Version: 3.5.1

Additional context
The original, JPEGXR-compressed document, was decompressed and ZSTD-compressed again using czicompress tool.

Test document will be delivered separately.

Is your feature request related to a problem? Please describe.
With the completion of #32 we still have some cpplint issues left to fix (or permanently accept) that are currently filtered out in CPPLINT.cfg.

Describe the solution you'd like
Apply some of the fixes primarily applied to Src/libCZI as part of #32 to other parts of the code like:

• Src/CZICmd
• Src/JxrDecode
• Src/libCZI_UnitTests

Src/libCZI largely passed cpplint by applying the following filter:

Any cpplint issues that are not going to be fixed (either because of feasibility or meaningfulness) should stay in the global CPPLINT.cfg. I strongly advise not to:

• introduce nested config files or
• exclude certain subdirectories from scanning

Additional context

Src/libCZI/CMakeLists.txt has a line that reads

target_link_libraries(libCZI PRIVATE libzstd)

However, this is incorrect as liblibzstd.so (or whatever filetype you'd like) won't exist, but libzstd will. Hence libzstd should be changed to zstd.

I was trying to use ISingleChannelPyramidLayerTileAccessor::Get(), which needs a PyramidLayerInfo struct. I didn't pay much attention to where this struct is actually defined. So I just took one instance obtained by CCZIReader::GetPyramidStatistics() and wanted to pass it to the accessor. To my surprise, it didn't compile, because the struct PyramidLayerInfo is defined in two places, leading to the error:

cannot convert from 'libCZI::PyramidStatistics::PyramidLayerInfo' to 'libCZI::ISingleChannelPyramidLayerTileAccessor::PyramidLayerInfo'

Ofc, I could just manually convert it. But I do wonder if there is a reason why there are two definitions of the very same PyramidLayerInfo struct? Would you consider removing one of the definitions?

c.f. #73 - an outcome of the analysis there is that when doing the tile-composition, tiles which are completely covered by other tiles are processed nevertheless.
The "test for visibility of a tile" has to take the z-ordering into account of course.

So - tiles for which it can be decided that they will not be visible in the tile-composition should not be loaded and processed.

With zstd already installed, the CMake script still fetches it and tries to install it.

Steps to reproduce the behavior:
Be on Arch Linux with package zstd already installed and type

mkdir build && cd build
cmake ..
cmake --build .

Describe the bug
CSingleChannelTileAccessor::Get (and probably also the other accessor implementations) returns uninitialized memory if called with pOptions=nullptr or if passing ISingleChannelTileAccessor::Options where backGroundColor has not been set to finite values explicitly. (Note that ISingleChannelTileAccessor::Options::Clear() sets the backGroundColor values to NaN.)

The reason is that CSingleChannelAccessorBase::Clear(), which is called by CSingleChannelTileAccessor::InternalGet() , only fills the bitmap if all values of backGroundColor are not NaN.

To Reproduce

auto reader = libCZI::CreateCZIReader();
reader->open(/* some stream */);
auto accessor = reader->CreateSingleChannelTileAccessor();
libCZI::CDimCoordinate coord;
// set some coordinates
auto bitmap = accessor->Get(/* some ROI which is not fully covered by tiles */, &coords, nullptr);
auto lock_info = bitmap->Lock();
// enjoy random values in regions not covered by tiles
bitmap->Unlock();

Expected behavior
Background pixels are initialized with a predictable value by default.

Desktop (please complete the following information):

• OS: Linux

In ZEISS' ZEN software, one is able to convert a czi-file to other formats, such as avi.

Is this possible with this library (or perhaps any other library released by ZEISS)?

If this is possible, it would be great to have an example in the documentation.

Is your feature request related to a problem? Please describe.

I want to link libczi with an app that uses the MSVC dll c runtime but that is in conflict with the line in CMakeLists.txt that chooses the static C runtime.

Describe the solution you'd like

Simply remove the CMAKE_MSVC_RUNTIME_LIBRARY line.

Describe alternatives you've considered
I am considering forking or applying a patch during my build process that removes this line before running CMake.
I'm not sure of any other way to override this. This same issue is true of the old libCZI also.

Additional context
My app has other dependencies which require the DLL C runtimes and I get crashes when linking against both.

Is your feature request related to a problem? Please describe.
codecov still showing unit flag that is however not used anymore. This applies to the codecov dashboard itself as well as PR comments.

Describe the solution you'd like
Do not show unit flag anymore in PR or codecov dashboard. It seems that we can simply disable carryforward as outlined here and then push to main. This should then only keep the flags as uploaded newly.
Since we anyway always test the whole test suite on every commit, I do not see a reason to enable carryforward again. As it says: "Carryforward Flags are designed for projects that do not upload total coverage for every commit (e.g., monorepos with multiple applications/languages, iterative/partial/delta testing setups, etc)."

Describe alternatives you've considered
None

Additional context

Is your feature request related to a problem? Please describe.
c.f. #20 (comment)

Describe the solution you'd like

• all commands should be described in the documenation
• in best case - they should come with "ready-to-run examples"
• one challenge here - how to provide/make available "sample data"

Describe alternatives you've considered
N/A

Additional context
N/A

Please refer issue created on old git repo
zeiss-microscopy/libCZI#63 (comment)

Describe the bug
Hi there, I am a developer at Indica Labs. We collab with Zeiss at times. We had a meeting last week where it was suggested I submit my inquiry to this repo.

Our HALO software (image analysis for digital pathology) uses libCZI to read sub block info on czi images. No problems there, it works great for us. My question is about interpreting czi images in general.

We use libCZI to read each sub block to gather coordinate info on tiles. We read all sub block values straight from the libCZI interface: the bounding box, the zoom, computing the appropriate x and y for the tile's zoom, etc. While doing so, we build a spatial index where we track tile_a_channel_i has (x, y, pyramid_lvl, scene_idx) or if it is a mosaic image then (x, y, pyramid_lvl, mosaic_idx). We then later use that spatial index to place tiles on the screen at different zooms.

Occasionally we run into czi images where the x or y of tile_a_channel_i does not line up with the x or y of tile_a_channel_j unless we apply the PixelShiftX and PixelShiftY values read out of the czi's metadata xml to the x and y respectfully. That is, tiles that are supposed to only differ in the channel dimension end up also differing in the x and/or y dimension unless we apply PixelShift. These PixelShift values are found in the xml's Track block, with info around it indicating what channel the PixelShift value matches to. Below is a screen shot of some of the xml to demonstrate.

To Reproduce
I have 2 czi images that I cannot share in their entirety because doing so would reveal private personal medical information, so I am hoping to just garner what I can from a conversation with your team. My problem is as follows.

Image 1

This image's channels line up only if I apply the PixelShift values to tiles in the bottom of the pyramid. If I apply to all tiles on all levels, it creates unaligned tiles in the channel dimension for all pyramid levels but the bottom one.

Image 2

This image's channels only line up if I apply PixelShift to all the tiles on all levels of the pyramid. If I do not apply to all levels, or to only the bottom pyramid level like Image 1, we end up with unaligned tiles in the channel dimension.

This seems to be implying that PixelShift values are only applied when something somewhere else in the czi's data indicates to do so. I have combed through both of these images' metadata as reported from libCZI, as well as their xmls, but I cannot find something indicating when PixelShift values should be applied. Neither can I find info on PixelShift values in the current czi spec I am working with, which is the one indicated in a screen shot posted at the bottom of this bug submission. The only appearant difference between these images is one has ComponentBitCount of 16, and the other 14, which idk if is relevant, but we have recently been encountering more and more 14 bit czi images that behave different than what we've seen in the past, so perhaps.

Expected behavior
Both of these images open in Zen Blue 2.3 lite and I can zoom in and out on them without issue, so it is as if there is some flag somewhere in the image's data indicating to Zen Blue when PixelShift values should be applied. I realize this is a unconventional "bug submission." I am just hoping your team can shed some light on a czi image's PixelShift.

1. The czi image spec I am using is from 2012, but I cannot find a newer one. Is there a newer one you could point me to? I am hoping it will have info on PixelShift.
2. Is there some flag somewhere in a czi's data that indicates on what pyramid levels a channel's PixelShift values should be applied?
3. Is there something unusual about 14 bit czi images that I am not checking?

Really appreciate any help. Thank you.
Screenshots

Desktop (please complete the following information):

• OS: [e.g. iOS]
• Browser [e.g. chrome, safari]
• Version [e.g. 22]

Smartphone (please complete the following information):

• Device: [e.g. iPhone6]
• OS: [e.g. iOS8.1]
• Browser [e.g. stock browser, safari]
• Version [e.g. 22]

Additional context
Add any other context about the problem here.

Describe the bug
The implementation and the semantic wrt concurrency of the CZIReader should be checked and, if necessary, improved.

To Reproduce
I am especially worried about calls to "Close" in case of concurrent calls to other methods. Goal is to ensure defined behavior in such scenarios.

Expected behavior
Defined behavior in case of concurrent calls.

Is your feature request related to a problem? Please describe.
There should be tests for CZIcmd - c.f. #20 (review)

Describe the solution you'd like
Runs of CZIcmd should be part of the CI/CD, and the results are to be checked for correctness.
The test-data used for those runs have to be made available "somehow".

Describe alternatives you've considered
N/A

Additional context
N/A

The documentation instructs Linux users to first install zstd, but CMake still installs another zstd whether or not the user has installed it.

Describe the bug
Code Scanning through a GitHub Action revealed several issues (see screenshot below).

To Reproduce
Steps to reproduce the behavior:

1. Go to https://github.com/zeiss/libczi/security/code-scanning (only Members with at least Write Access)

Expected behavior
No Code Scanning Alerts.

Screenshots