team-charls / charls Goto Github PK

I'm getting an error when reading (or maybe writing) 16-bit JPEG-LS files. I'm using the 1.x-master branch if that makes a difference. I'm attaching a sample program that shows what I'm doing.

CharLS_bitdepth_test.cpp.zip

It seems that the util.h is not installed in the final make install step. Should it be installed? I'm trying to package gdcm with system's charls, and it complains that CharLS/util.h not found.

Since #36 got a little crowded and hard to follow, I'll report this as a separate issue. Feel free to tell me how you prefer to get these kind of reports (I guess that there will be more to come, sorry).

Anyway, the following program dies with an assertion failure: /tmp/charls/src/decoderstrategy.h:129: void DecoderStrategy::MakeValid(): Assertion 'static_cast<size_t>(_validBits) <=bufType_bit_count - 8' failed.

#include "src/charls.h"

const unsigned char input[] = {
  0xff, 0xd8, 0xff, 0xf7, 0x00, 0x11, 0x08, 0x00, 0x01, 0x00, 0xfd, 0x02,
  0x01, 0x11, 0x00, 0x02, 0x11, 0x00, 0x03, 0x11, 0x00, 0xff, 0xda, 0x00,
  0x08, 0x01, 0x01, 0x00, 0x00, 0x00, 0x00, 0x80, 0x10, 0xd9
};

int main()
{
	char result[1024 * 1024];
	JpegLsDecode(result, sizeof(result), input, sizeof(input), nullptr, nullptr);
	return 0;
}

The problem seems to be... I don't know. ReadHighbits() has Peek0Bits() returning -1 and so calls Skip(15). However, _validBits is less than 15 at this point, so becomes negative, which causes the above assertion failure when MakeValid() is called the next time (casting a negative value to unsigned produces something huge and... well). So the "real error" happened way earlier than the assertion failure.

I'm not quite sure who is to blame here. ReadHighbits() assumes that _validBits >= 15 (otherwise Skip(15) is malformed), but this assumption is supposedly guaranteed by Peek0Bits() calling MakeValid() if there are less than 16 bits available. However, MakeValid() doesn't seem to do anything since the end of the input was already reached (_position >= _endPosition). So, $SOMETHING has to check for EOF, but I wonder how general this check has to be (in ReadHighbits()? In Skip()?).

windres.exe in MinGW doesn't like utf-8 and fails to parse charls.rc. Anyway, maybe this file and .def should only be added if MSVC (in CMakeLists.txt).

After successfully compiling latest VCS checkouts of CharLS on Linux make target install fails, apparently as it's expecting one of the source files at a location where it doesn't exist

# make install 
[ 42%] Built target CharLS
[100%] Built target charlstest
Install the project...
-- Install configuration: ""
-- Installing: /usr/lib/libCharLS.so.1.0
[...]
-- Installing: /usr/include/CharLS/losslesstraits.h
-- Installing: /usr/include/CharLS/scan.h
CMake Error at cmake_install.cmake:68 (file):
  file INSTALL cannot find
  "/usr/local/peter/Tests/CharLS/Projekt/charls/src/config.h".


Makefile:83: recipe for target 'install' failed
make: *** [install] Error 1

I'm getting an error on this line of code when trying to compile with VS 14.1

https://dev.azure.com/conda-forge/feedstock-builds/_build/results?buildId=106870&view=logs&j=2cc45e14-23e3-52d7-b33a-8c2744410b97&t=21c44aa7-1ae3-5312-cacc-7f19fefc82f4&l=688

This build will likely get destroyed, so I'm copying the error message

[1/22] Building CXX object CMakeFiles\charls.dir\src\charls_jpegls_decoder.cpp.obj
FAILED: CMakeFiles/charls.dir/src/charls_jpegls_decoder.cpp.obj 
C:\PROGRA~2\MICROS~1.0\VC\bin\amd64\cl.exe  /nologo /TP -DCHARLS_LIBRARY_BUILD -Dcharls_EXPORTS -I..\include -MD -GL  /DWIN32 /D_WINDOWS /W3 /GR /EHsc /MD /O2 /Ob2 /DNDEBUG   -std:c++14 /showIncludes /FoCMakeFiles\charls.dir\src\charls_jpegls_decoder.cpp.obj /FdCMakeFiles\charls.dir\ /FS -c ..\src\charls_jpegls_decoder.cpp
..\include\charls/charls.h(488): warning C4800: 'int32_t': forcing value to bool 'true' or 'false' (performance warning)
d:\bld\charls_1577727546940\work\src\util.h(121): error C3250: 'x': declaration is not allowed in 'constexpr' function body
d:\bld\charls_1577727546940\work\src\util.h(121): error C3249: illegal statement or sub-expression for 'constexpr' function
d:\bld\charls_1577727546940\work\src\util.h(122): error C3249: illegal statement or sub-expression for 'constexpr' function
d:\bld\charls_1577727546940\work\src\util.h(124): error C3249: illegal statement or sub-expression for 'constexpr' function
d:\bld\charls_1577727546940\work\src\util.h(124): note: see usage of '++'
d:\bld\charls_1577727546940\work\src\util.h(126): error C3256: 'x': variable use does not produce a constant expression
[2/22] Building CXX object CMakeFiles\charls.dir\src\charls_jpegls_encoder.cpp.obj
FAILED: CMakeFiles/charls.dir/src/charls_jpegls_encoder.cpp.obj 
C:\PROGRA~2\MICROS~1.0\VC\bin\amd64\cl.exe  /nologo /TP -DCHARLS_LIBRARY_BUILD -Dcharls_EXPORTS -I..\include -MD -GL  /DWIN32 /D_WINDOWS /W3 /GR /EHsc /MD /O2 /Ob2 /DNDEBUG   -std:c++14 /showIncludes /FoCMakeFiles\charls.dir\src\charls_jpegls_encoder.cpp.obj /FdCMakeFiles\charls.dir\ /FS -c ..\src\charls_jpegls_encoder.cpp
..\include\charls/charls.h(488): warning C4800: 'int32_t': forcing value to bool 'true' or 'false' (performance warning)
d:\bld\charls_1577727546940\work\src\util.h(121): error C3250: 'x': declaration is not allowed in 'constexpr' function body
d:\bld\charls_1577727546940\work\src\util.h(121): error C3249: illegal statement or sub-expression for 'constexpr' function
d:\bld\charls_1577727546940\work\src\util.h(122): error C3249: illegal statement or sub-expression for 'constexpr' function
d:\bld\charls_1577727546940\work\src\util.h(124): error C3249: illegal statement or sub-expression for 'constexpr' function
d:\bld\charls_1577727546940\work\src\util.h(124): note: see usage of '++'
d:\bld\charls_1577727546940\work\src\util.h(126): error C3256: 'x': variable use does not produce a constant expression
[3/22] Building CXX object CMakeFiles\charls.dir\src\jpegls.cpp.obj
d:\bld\charls_1577727546940\work\src\scan.h(82): error C3249: illegal statement or sub-expression for 'constexpr' function
d:\bld\charls_1577727546940\work\src\scan.h(88): error C3250: 'mappedError': declaration is not allowed in 'constexpr' function body
d:\bld\charls_1577727546940\work\src\scan.h(88): error C3249: illegal statement or sub-expression for 'constexpr' function
ninja: build stopped: subcommand failed.

During the encoding process the caller needs to pass the bitPerSample parameter and the buffer with the samples to encode. CharLS will not check if the samples are in the correct range, example:
bitPerSample = 10 (max value = 1014) and passing a sample of 2000.
In the debug build a ASSERT deep down will trigger that a computation failed. As most users use the release build they silently generate bad encoded bit streams.

I am interested in how jpeg-ls works. how can I build this project on linux? Thanks.

In release 2.1 encoder API (JpegLsEncode) produces corrupted images. This comes of an inversion of the width and the height in https://github.com/team-charls/charls/blob/master/src/charls_jpegls_encoder.cpp#L455

I meet this problem on release 2.0.0
In the function ModuloRange from file src/default_traits.h there are two inconsistencies

FORCE_INLINE int32_t ModuloRange(int32_t errorValue) const noexcept
    {
        ASSERT(std::abs(errorValue) <= RANGE);

        if (errorValue < 0)
        {
            errorValue += RANGE;
        }
        if (errorValue >= (RANGE + 1) / 2)
        {
            errorValue -= RANGE;
        }

        ASSERT(-RANGE / 2 <= errorValue && errorValue <= static_cast<int32_t>(ceil(static_cast<double>(RANGE) / 2)) - 1);
        return errorValue;
    }

why here is

if (errorValue < 0)
        {
            errorValue += RANGE;
        }

and not

if (errorValue < -RANGE / 2)
        {
            errorValue += RANGE;
        }

And rounding rules creates problem with eg. RANGE=25 and errorValue=12 The rounding rule do not apply, but end assertion aplly, so code ends with faliture. My suggestion for fixes is:

FORCE_INLINE int32_t ModuloRange(int32_t errorValue) const noexcept
    {
        ASSERT(std::abs(errorValue) <= RANGE);

        if (errorValue < -RANGE / 2)
        {
            errorValue += RANGE;
        }
        if (errorValue >= (RANGE + 1) / 2)
        {
            errorValue -= RANGE;
        }

        ASSERT(-RANGE / 2 <= errorValue && errorValue <= RANGE - (RANGE) / 2 - 1);
        return errorValue;
    }

But I`m not sure how it should work, so I open issue, not pull request. I can open pull request if you ask for it.

Looks like CharLS 1.1 trigger the following warning with clang:

/Users/builder/external/GDCM/Utilities/gdcmcharls/header.cpp:69:77: warning: instantiation of function 'JlsCodecFactory<EncoderStrategy>::GetCodec' required here, but no definition is available [-Wundefined-func-template]
        std::auto_ptr<EncoderStrategy> qcodec = JlsCodecFactory<EncoderStrategy>().GetCodec(info, _info.custom);
                                                                                   ^
/Users/builder/external/GDCM/Utilities/gdcmcharls/header.h:60:26: note: forward declaration of template entity is here
        std::auto_ptr<STRATEGY> GetCodec(const JlsParameters& info, const JlsCustomParameters&);
                                ^
/Users/builder/external/GDCM/Utilities/gdcmcharls/header.cpp:69:77: note: add an explicit instantiation declaration to suppress this warning if 'JlsCodecFactory<EncoderStrategy>::GetCodec' is explicitly instantiated in another translation unit
        std::auto_ptr<EncoderStrategy> qcodec = JlsCodecFactory<EncoderStrategy>().GetCodec(info, _info.custom);
                                                                                   ^
/Users/builder/external/GDCM/Utilities/gdcmcharls/header.cpp:136:78: warning: instantiation of function 'JlsCodecFactory<DecoderStrategy>::GetCodec' required here, but no definition is available [-Wundefined-func-template]
                std::auto_ptr<DecoderStrategy> qcodec = JlsCodecFactory<DecoderStrategy>().GetCodec(_info, _info.custom);       
                                                                                           ^
/Users/builder/external/GDCM/Utilities/gdcmcharls/header.h:60:26: note: forward declaration of template entity is here
        std::auto_ptr<STRATEGY> GetCodec(const JlsParameters& info, const JlsCustomParameters&);
                                ^
/Users/builder/external/GDCM/Utilities/gdcmcharls/header.cpp:136:78: note: add an explicit instantiation declaration to suppress this warning if 'JlsCodecFactory<DecoderStrategy>::GetCodec' is explicitly instantiated in another translation unit
                std::auto_ptr<DecoderStrategy> qcodec = JlsCodecFactory<DecoderStrategy>().GetCodec(_info, _info.custom);       

In performance.cpp, TestLargeImagePerformance and DecodePerformanceTests functions, some data files are referenced that are not included in the repository. From where can these images/data files be obtained?

Hello. Thank you for developing the CharLS library! I use it from Python via Cython.

I cannot seem to find a way to check the CharLS library version at compile or runtime.

Would it be possible to add some defines and/or functions to the API? For example

#define CHARLS_VERSION_MAJOR    2
#define CHARLS_VERSION_MINOR    1
#define CHARLS_VERSION_RELEASE  0
#define CHARLS_VERSION_NUMBER  (CHARLS_VERSION_MAJOR * 100*100 + CHARLS_VERSION_MINOR * 100 + CHARLS_VERSION_RELEASE)

CHARLS_API_IMPORT_EXPORT const char* CHARLS_API_CALLING_CONVENTION charls_get_version_string(void);
CHARLS_API_IMPORT_EXPORT const uint32_t CHARLS_API_CALLING_CONVENTION charls_get_version_number(void);

I have a raw image that I'm trying to compress using the library. My input data size is 57658368 and after calling JpegLsEncode, the bytesWritten output is 59951222. Any idea why the compressed length is larger than raw size?

JlsParameters params = JlsParameters();
params.width = 6576;
params.height = 4384;
params.bitsPerSample = 16;
params.stride = 0;
params.components = 1;
params.interleaveMode = InterleaveMode::None;
params.colorTransformation = ColorTransformation::None;

This is unfortunate but C++11 deprecated std::auto_ptr. CharLS codebase should be updated in favor of std:unique_ptr. It may be non-trivial to have a portable solution, see: http://programmers.stackexchange.com/questions/291141/how-to-handle-design-changes-for-auto-ptr-deprecation-in-c11

In version 2.1.0rc1 on gcc-5 the configuration of cmake pass, but compilation fail.

Cmake configuration

Build charls
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Building as master project, CMake version: 3.12.4
-- Configuring done
-- Generating done

Error info

[ 10%] Building CXX object CMakeFiles/charls.dir/src/charls_jpegls_encoder.cpp.o
/home/vsts/work/1/s/build_utils/libs_src/charls/src/charls_jpegls_encoder.cpp: In member function ‘void charls_jpegls_encoder::encode(const void*, size_t, uint32_t)’:
/home/vsts/work/1/s/build_utils/libs_src/charls/src/charls_jpegls_encoder.cpp:148:37: error: ‘interleave_mode’ is not a class, namespace, or enumeration
             if (interleave_mode_ != interleave_mode::none)
                                     ^
/home/vsts/work/1/s/build_utils/libs_src/charls/src/charls_jpegls_encoder.cpp:165:38: error: ‘color_transformation’ is not a class, namespace, or enumeration
         if (color_transformation_ != color_transformation::none)
                                      ^
/home/vsts/work/1/s/build_utils/libs_src/charls/src/charls_jpegls_encoder.cpp:181:33: error: ‘interleave_mode’ is not a class, namespace, or enumeration
         if (interleave_mode_ == interleave_mode::none)
                                 ^
CMakeFiles/charls.dir/build.make:75: recipe for target 'CMakeFiles/charls.dir/src/charls_jpegls_encoder.cpp.o' failed

As far as I understand the only requirement for c++11 is std::make_unique. Would you consider accepting a pull request which does:

$ cat src/util.h 
[...]
#if __cplusplus == 201103L
template<typename T, typename... Args>
std::unique_ptr<T> make_unique(Args&&... args)
{
    return std::unique_ptr<T>(new T(std::forward<Args>(args)...));
}
#endif

Hi,

I just started experimenting with american fuzzy lop and picked charls as my target. I wrote the following code to feed data to CharLS:

#include "src/charls.h"

#include <unistd.h>
#include <fcntl.h>
#include <string.h>

static char result[1024 * 1024];
static char input[1024 * 1024];

int main(int argc, char *argv[])
{
	int fd = 0;
	if (argc == 2)
	{
		fd = open(argv[1], O_RDONLY);
		if (fd < 0) {
			fprintf(stderr, "Failed to open '%s': %s\n",
					argv[1], strerror(errno));
			return 1;
		}
	}

	size_t input_length = read(fd, input, sizeof(input));
	std::cout << (int) JpegLsDecode(result, sizeof(result), input, input_length, nullptr, nullptr) << std::endl;
	return 0;
}

Within seconds, afl managed to crash this program.

Valgrind result for first file https://filebin.ca/3j4hDmAVjSvI:

==16586== Invalid read of size 8
==16586==    at 0x4E4793: JpegStreamReader::Read(ByteStreamInfo) (jpegstreamreader.cpp:124)
==16586==    by 0x41042D: JpegLsDecodeStream (interface.cpp:163)
==16586==    by 0x41042D: JpegLsDecode (interface.cpp:216)
==16586==    by 0x40DC91: main (charlsfuzz.cpp:24)
==16586==  Address 0x0 is not stack'd, malloc'd or (recently) free'd

The above seems to be JlsCodecFactory::CreateCodec returning a nullptr, which is something that JpegStreamReader::Read does not check for before dereferencing qcodec in line 124.

Plus, there are lots (currently 9) of files which cause the following assertion failure, one of them is https://filebin.ca/3j4hfYzIdJMm:

charlsfuzz: /tmp/charls/src/decoderstrategy.h:129: void DecoderStrategy::MakeValid(): Assertion `static_cast<size_t>(_validBits) <=bufType_bit_count - 8' failed.

Edit: Here is another failed assertion:
https://filebin.ca/3j4yTkNxVuQb

charlsfuzz: /tmp/charls/src/context.h:60: void JlsContext::UpdateVariables(int32_t, int32_t, int32_t): Assertion `std::abs(b) < 65536 * 256' failed.

Edit: https://filebin.ca/3j57Vrm3jPkZ

charlsfuzz: /tmp/charls/src/scan.h:239: SAMPLE JlsCodec<LosslessTraits<uint16_t, 16>, DecoderStrategy>::DoRegular(int32_t, int32_t, int32_t, DecoderStrategy *) [Traits = LosslessTraits<uint16_t, 16>, Strategy = DecoderStrategy]: Assertion `std::abs(ErrVal) < 65535' failed.

Edit: The above as a check-list since I see that some of these were already fixed (by the way, assuming I find more such "evil inputs", should I open one issue per file or continue grouping them like this? Should I open a new issue or keep updating this one?):

• https://filebin.ca/3j4hDmAVjSvI
• https://filebin.ca/3j4hfYzIdJMm
• https://filebin.ca/3j4yTkNxVuQb
• https://filebin.ca/3j57Vrm3jPkZ

The JPEG-LS standard ISO/IEC 14495-1, paragraph C.2.4.1.1.1 defines how to compute the default threshold values. 2 set of formulas are defined. One needs to switch between the two sets based on the MAXVAL >= 128 or not, MAXVAL = 2 ^ bits_per_sample.

CharLS only uses the set of formulas defined for MAXVAL >= 128. This may cause incorrect encoding and decoding of images with 2,4,5 or 6 bits per sample when the default threshold values are used. The typical case is to use the default threshold values.

The names 'jpeg-lossless' and 'JPEG-LOSSLESS' are not included in the names array in JPEGProviderInfo. These names are used by JAI to identify jpeg lossless readers and writers. Some packages explicitly look for these names. Please add these for JAI compatibility.

This is a minor issue but it would be nice to get rid of this.

scan.h:84:31: runtime error: left shift of 4031 by 63 places cannot be represented in type 'long int'

where:

$ tail -n +84 scan.h | head -n 1
    LONG sign = LONG(mappedError << (LONG_BITCOUNT-1)) >> (LONG_BITCOUNT-1);

How does JPEG-LS (and CharLS) handle 16 bit grayscale data signedness? Does it support both 16 bit signed and unsigned? If so, how does a caller determine whether the pixel data is signed or unsigned? If not, is 16 bit grayscale returned as signed or unsigned? Thanks

Note: this is an issue originally reported on codeplex: http://charls.codeplex.com/workitem/10742

Hi,

we received a bug report for the copy of CharLS that is part of the DCMTK:
http://forum.dcmtk.org/viewtopic.php?f=1&t=3412

I took a look at this file and I think that this is indeed a bug in CharLS.

The attached patch creates an EncoderStrategy instance and writes bits to it. This triggers the same assertion as the original test image. (Ignore my changes to encoderstrategy.h, that's just the debug output that I looked at for figuring this out):

encoderstrategy.h:95: void EncoderStrategy::AppendToBitStream(LONG, LONG): Assertion `bitpos >=0' failed.

My analsysis of what is going on is part of the patch, so be sure to read the comments. :-)

I tried to come up with a patch that fixes this. This is the following part of the diff:

if (bitpos >= 0)
if (bitpos > 0)

The idea here is that when 32 bits of input were read, we can Flush().

This should even fix a similar issue, where we have bitpos == 1 and add 31 input bits, so one bit gets added to valcurrent. If Flush() only writes out 30 bits, all the 30 bits of input left can go into the buffer.

Cheers,
Uli

I am using charLS to encode and decode DICOM medical images. During testing I came across a bad data file (a dcm file stored as an image in a dcm file). The storage was set the compress the file as JpegLS. During the compression of the data, charLS crashes with a EXC_BAD_ACCESS for pointer *_position accessing outside the output buffer in EncodeStratagy::flush(). _compressedLength goes negitive (bad for a size_t) Even though the data is not an image, the library should have returned a buffer overflow. I wrote a small program to read the file and compress it to prove it was not a bug in the main program. It is here:
encodeRaw.zip
This is the input data that caused the crash:
jpegls_crash_data.raw.zip
Try to compress it as a 944 x 944 x 8bits monochrome lossless image.

I am getting a couple of warning when compiling charls v2.0.0 with clang:

In file included from /home/mathieu/Perso/gdcm/Utilities/gdcmcharls/jpegls.cpp:27:
/home/mathieu/Perso/gdcm/Utilities/gdcmcharls/scan.h:202:10: warning: 'DecodeScan' overrides a member function but is not marked 'override' [-Winconsistent-missing-override]
    void DecodeScan(std::unique_ptr<ProcessLine> rawData, const JlsRect& size, ByteStreamInfo& compressedData, bool bCompare);
         ^
/home/mathieu/Perso/gdcm/Utilities/gdcmcharls/jpegls.cpp:113:50: note: in instantiation of template class 'JlsCodec<DefaultTraitsT<unsigned char, unsigned char>, DecoderStrategy>' requested here
        strategy = std::unique_ptr<STRATEGY>(new JlsCodec<DefaultTraitsT<uint8_t, uint8_t>, STRATEGY>(traits, params));
                                                 ^
/home/mathieu/Perso/gdcm/Utilities/gdcmcharls/jpegls.cpp:186:16: note: in instantiation of member function 'JlsCodecFactory<DecoderStrategy>::GetCodec' requested here
template class JlsCodecFactory<DecoderStrategy>;
               ^
/home/mathieu/Perso/gdcm/Utilities/gdcmcharls/decoderstrategy.h:35:18: note: overridden virtual function is here
    virtual void DecodeScan(std::unique_ptr<ProcessLine> outputData, const JlsRect& size, ByteStreamInfo& compressedData, bool bCheck) = 0;
                 ^
In file included from /home/mathieu/Perso/gdcm/Utilities/gdcmcharls/jpegls.cpp:27:

Would be super nice to fix in next release

Line 37 in util.h is #elif, should be #else

Here is one patch that we maintain within Debian:

https://salsa.debian.org/med-team/charls/blob/master/debian/patches/add_visibility.patch

Would be nice to handle upstream directly. Thanks

Clang <= 5 and AppleClang 9.3 fail to compile jpegls_error.cpp if using libc++ (and not libstdc++) because string (and system_error) are not included.

[ 44%] Building CXX object source_subfolder/CMakeFiles/charls.dir/src/jpegls_error.cpp.o
/home/conan/.conan/data/charls/2.1.0/bincrafters/testing/build/2b9f4a7d83ec1ec57c27b92740c5dea74386cdf8/source_subfolder/src/jpegls_error.cpp:19:12: error: implicit instantiation of undefined template 'std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >'
    string message(int error_value) const override
           ^
/usr/include/c++/v1/iosfwd:193:33: note: template is declared here
    class _LIBCPP_TYPE_VIS_ONLY basic_string;
                                ^
/home/conan/.conan/data/charls/2.1.0/bincrafters/testing/build/2b9f4a7d83ec1ec57c27b92740c5dea74386cdf8/source_subfolder/src/jpegls_error.cpp:21:16: error: implicit instantiation of undefined template 'std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >'
        return charls_get_error_message(static_cast<jpegls_errc>(error_value));
               ^
/usr/include/c++/v1/iosfwd:193:33: note: template is declared here
    class _LIBCPP_TYPE_VIS_ONLY basic_string;

Hi,

When I try to to compress this image:

t1.zip

with all pixel values in the 12 bits range, charls (version feb 2016) throws an assert; or in NDEBUG mode is not error free. This is an extremely rare example. Of course when the pixel values are outside the specified range, there is an ASSERT as well, but you have a bug report for that problem,

Regards,

Marcel (conquest DICOM server author)

Hey, I notice that BUILD_TESTING:BOOL=ON by default. But, it seems that there is no rule to make target 'test', so how could I run test after compiling charls? I'm trying to create a package for ArchLinux.

The current CI pipelines use Visual Studio 2019, GCC 7.4.0 and clang 7.0.0.

By adding an Azure Pipeline (free for open source projects) the coverage can be extended with:

Visual Studio 2017
Visual Studio 2015 (mostly C++14 compatible), see also issue #60
Xcode 11.3
Xcode 10.1

When browsing the code, I found the following questionable (?) assignment in colortransform.h, line 120

hp3.v1 = SAMPLE(G + ((hp3.v2 + hp3.v3)>>2)) - RANGE / 4;

I might be mistaken, but shouldn't the last right parenthesis be placed at the end of the assignment, i.e.

hp3.v1 = SAMPLE(G + ((hp3.v2 + hp3.v3)>>2) - RANGE / 4);

gcc5 fails to compile charls_jpegls_decoder.cpp (similar issue than #55)

[ 22%] Building CXX object source_subfolder/CMakeFiles/charls.dir/src/charls_jpegls_encoder.cpp.o
/home/conan/.conan/data/charls/2.1.0/bincrafters/testing/build/2de3d908e1d9b246c9b2b6ea5c083ff1297f138b/source_subfolder/src/charls_jpegls_decoder.cpp: In member function ‘size_t charls_jpegls_decoder::destination_size(uint32_t) const’:
/home/conan/.conan/data/charls/2.1.0/bincrafters/testing/build/2de3d908e1d9b246c9b2b6ea5c083ff1297f138b/source_subfolder/src/charls_jpegls_decoder.cpp:106:14: error: ‘interleave_mode’ is not a class, namespace, or enumeration
         case interleave_mode::none:
              ^
/home/conan/.conan/data/charls/2.1.0/bincrafters/testing/build/2de3d908e1d9b246c9b2b6ea5c083ff1297f138b/source_subfolder/src/charls_jpegls_decoder.cpp:109:14: error: ‘interleave_mode’ is not a class, namespace, or enumeration
         case interleave_mode::line:
              ^
/home/conan/.conan/data/charls/2.1.0/bincrafters/testing/build/2de3d908e1d9b246c9b2b6ea5c083ff1297f138b/source_subfolder/src/charls_jpegls_decoder.cpp:110:14: error: ‘interleave_mode’ is not a class, namespace, or enumeration
         case interleave_mode::sample:
              ^
source_subfolder/CMakeFiles/charls.dir/build.make:62: recipe for target 'source_subfolder/CMakeFiles/charls.dir/src/charls_jpegls_decoder.cpp.o' failed

Here is a patch (for 2.1.0) which fix this issue:

--- a/src/charls_jpegls_decoder.cpp
+++ b/src/charls_jpegls_decoder.cpp
@@ -103,11 +103,11 @@ struct charls_jpegls_decoder final

         switch (interleave_mode())
         {
-        case interleave_mode::none:
+        case charls::interleave_mode::none:
             return static_cast<size_t>(stride) * info.height * info.component_count;

-        case interleave_mode::line:
-        case interleave_mode::sample:
+        case charls::interleave_mode::line:
+        case charls::interleave_mode::sample:
             return static_cast<size_t>(stride) * info.height;
         }

Tasklist for the 2.1.0 release

• Create 2.1.0-rc1 tag and publish it to indicate that 2.1.0 is almost done
• Resolve important SonarCube issues
• Add unit test for planar RGB images
• Update README.md CMake chapter with description for Linux and macOS
• Improve Wiki: how to build CharLS
• Improve Wiki: how to convert from Legacy API to 2.1 API
• Verify and update the CHANGELOG.md
• Update the version numbers in CMakeFile.txt, C# projects, .rc file to 2.1.0
• Create Release tag and publish it
• Update and close all 2.1.0 issues
• Communicate release on Twitter
• Publish release on vcpkg

Right now c++14 is being used for both implementation detail and API in CharLS. It would be nice to make it optional so as to compile with newer/older standard for advanced users. See patch in Debian:

• https://salsa.debian.org/med-team/charls/blob/master/debian/patches/noexplicitstd.patch

Looks like the code only supports lossless compression, or at least that's what I understand from going over convert.c. Any chance we can add lossy compression, or add example code that calls a lossy compression?

Could you drop this in the src dir. It could help people build the library on Macs.
charLS.xcodeproj.zip

I would like to find a javascript version of a JPEG-LS decoder. Yesterday I used emscripten to compile CharLS into JS and it works great for small images but runs out of memory on larger images. Assuming I can fix the memory issue, this should meet my needs. I wanted to see if the CharLS project would be interested in a PR that adds a Emscripten JS build target. If not, I can fork and add my build step to that and keep it in sync. Thanks

PS - The codec would be used by the cornerstone open source medical imaging sdk here:
https://github.com/chafey/cornerstone

Hi,

I am trying to use 1.x-master. It lacks several files and therefore does not compile. I miss:

jpegstreamreader.h
jpegmarkercode.h
jpegstreamreader.cpp
jlscodecfactory.h

The versions from the main branch are not compatible.

Thanks,

Marcel

Here is another patch from Debian that we maintain: hiding of STL stuff being automagically exported. Here is the hack that I wrote long ago:

https://salsa.debian.org/med-team/charls/blob/master/debian/patches/stl_symbols_hack.patch

Would be nice to handle gracefully using some portable cmake magic.

I can reproduce a failure to decompress using CharLS 1.1 (issue is present in CharLS 1.0), this is originally a bug tracked in Debian. With the reference jls, I can get JpegLsDecodeStream to return TooMuchCompressedData:

(gdb) 
297     ByteStreamInfo outputInfo = FromByteArray(outputBuffer.data(), outputBuffer.size());
(gdb) 
298     err = JpegLsDecodeStream(outputInfo, compressedByteStream, &info);
(gdb) 
299     if (err != OK)
(gdb) p err
$1 = TooMuchCompressedData

Hi All,

First of all I am new to this code, so please pardon me if I am asking some silly query here.

I am getting "_validBits" as 0 in the MakeValid() method under decoderstrategy.h file and this is made as 0 in the "Skip(LONG length)" method, and this is happening only for few images. I would like to understand the reasons behind this behaviour.

Kindly help me get some idea about this and to proceed further on my investigation.

Regards,
Ramana Mohan

the code:
https://github.com/team-charls/charls/blob/master/src/encoderstrategy.h#L77

[...]
if (bitCount < 32)
{
int mask = (1 << (bitCount)) - 1;
[...]

does not look quite right for bitCount == 31 ... Maybe use unsigned integer (1u << bitCount)

The JPEG-LS standard provides the option to write custom coding parameters to a JPEG-LS bit stream. This can be done with a LSE marker segment. This marker segment may occur multiple times and at different locations in the bit stream.

In practice custom coding parameters are never used, except for 1 case.
The Java JAI library (not supported anymore by Oracle) can encode/decode JPEG-LS stream. This JAI library has a bug that has never been fixed, it computes invalid coding parameters for bit per sample > 12. As a workaround CharLS writes an explicit LSE marker before the first scan. If there are multiple components it will write however this marker multiple times. The output generated is a valid JPEG-LS bit stream.
Note: CharLS will only write this LSE Marker N times when:

• bitsPerSample > 12
• InterleaveMode == InterleaveMode::None
• ComponentCount > 1
  Monochrome images and standard color images (256 bit RGB) are not affected as CharLS will write no or only 1 LSE marker segment.

During the decoding process, CharLS will fail when it encounters a LSE Marker after the first SOS segment.
For example encoding and then decoding with:

• bitsPerSample = 16
• InterleaveMode == InterleaveMode::None
• ComponentCount = 3
  doesn't work.

Support for this JAI improved compatibly was introduced in commit 232a4c2

The initial V1.0 release of CharLS had its own unit test framework (in charlstest.exe)

Modern development environments come with support for C++ unit tests. CharLS start with Visual Studio C++ unit test framework as this was the only unit test framework with integrated unit test support. Recent versions of Visual Studio come with adapters to use all kind of unit tests, like google test, boost test, catch2, etc.
These unit test frameworks are all cross-platform, while VS unit test framework is limited to Windows.
By moving to a cross-platform, the unit test can be all moved to a single executable that integrates well and is usable on multiple platforms.

Note: which unit test framework to use is not yet decided.

The JPEG-LS standard allows encoding images with a custom RESET value in the JPEG-LS preset coding parameters.
A defect in the implementation would always cause a DefaultTraitsT<uint8_t, uint8_t> being used instead of DefaultTraitsT<uint16_t, uint16_t> when the bits per sample is greater than 8.

This defect has been reported by @renyu310, pull request #33

When using the HP3 color transform the encoding to decoding with lossless doesn't yield the same file bit to bit. Reproducing steps:

1- Use CharLSTest
2- Add this parameter to JlsParameters to function EncodePnm params.colorTransformation = charls::ColorTransformation::HP3;
3- encodepnm with the attached ppm file
4- decodetopnm the compressed file
5- comparepnm fails.

Land10.zip

Hello,

It seems like there is a problem in checking validity of data. The function stack is as follows:

-- ReadHighBits()
-- -- ReadBit()
-- -- -- MakeValid()

The matter is that in function MakeValid() the assert has a hit here -
ASSERT(static_cast<size_t>(validBits_) <=bufType_bit_count - 8);
It occures when validBits_ is negative - it happened in ReadHighBits() when Skip(15) is called.

Am I right that validBits_ is not supposed to be negative? Then the check if (validBits_ <= 0) in ReadBit() should be fixed in a way of throwing an exception when validBits_ < 0.

I could provide a simple PR but first I wanted to check why the test uses inverted convention. Eg:

$ dd if=/dev/zero of=/tmp/zero.jls count=512
512+0 records in
512+0 records out
262144 bytes (262 kB, 256 KiB) copied, 0.00496284 s, 52.8 MB/s
$ ./charlstest -decodetopnm /tmp/zero.jls tt.pgm
$ echo $?
0

On UNIX you need to return a non-zero value to indicate an error.