team-charls / charls Goto Github PK
View Code? Open in Web Editor NEWCharLS, a C++ JPEG-LS library implementation
License: BSD 3-Clause "New" or "Revised" License
CharLS, a C++ JPEG-LS library implementation
License: BSD 3-Clause "New" or "Revised" License
I'm getting an error when reading (or maybe writing) 16-bit JPEG-LS files. I'm using the 1.x-master branch if that makes a difference. I'm attaching a sample program that shows what I'm doing.
It seems that the util.h
is not installed in the final make install
step. Should it be installed? I'm trying to package gdcm with system's charls, and it complains that CharLS/util.h
not found.
Since #36 got a little crowded and hard to follow, I'll report this as a separate issue. Feel free to tell me how you prefer to get these kind of reports (I guess that there will be more to come, sorry).
Anyway, the following program dies with an assertion failure: /tmp/charls/src/decoderstrategy.h:129: void DecoderStrategy::MakeValid(): Assertion 'static_cast<size_t>(_validBits) <=bufType_bit_count - 8' failed.
#include "src/charls.h"
const unsigned char input[] = {
0xff, 0xd8, 0xff, 0xf7, 0x00, 0x11, 0x08, 0x00, 0x01, 0x00, 0xfd, 0x02,
0x01, 0x11, 0x00, 0x02, 0x11, 0x00, 0x03, 0x11, 0x00, 0xff, 0xda, 0x00,
0x08, 0x01, 0x01, 0x00, 0x00, 0x00, 0x00, 0x80, 0x10, 0xd9
};
int main()
{
char result[1024 * 1024];
JpegLsDecode(result, sizeof(result), input, sizeof(input), nullptr, nullptr);
return 0;
}
The problem seems to be... I don't know. ReadHighbits()
has Peek0Bits()
returning -1
and so calls Skip(15)
. However, _validBits
is less than 15 at this point, so becomes negative, which causes the above assertion failure when MakeValid()
is called the next time (casting a negative value to unsigned produces something huge and... well). So the "real error" happened way earlier than the assertion failure.
I'm not quite sure who is to blame here. ReadHighbits()
assumes that _validBits >= 15
(otherwise Skip(15)
is malformed), but this assumption is supposedly guaranteed by Peek0Bits()
calling MakeValid()
if there are less than 16 bits available. However, MakeValid()
doesn't seem to do anything since the end of the input was already reached (_position >= _endPosition
). So, $SOMETHING has to check for EOF, but I wonder how general this check has to be (in ReadHighbits()
? In Skip()
?).
windres.exe in MinGW doesn't like utf-8 and fails to parse charls.rc. Anyway, maybe this file and .def should only be added if MSVC (in CMakeLists.txt).
After successfully compiling latest VCS checkouts of CharLS on Linux make target install
fails, apparently as it's expecting one of the source files at a location where it doesn't exist
# make install
[ 42%] Built target CharLS
[100%] Built target charlstest
Install the project...
-- Install configuration: ""
-- Installing: /usr/lib/libCharLS.so.1.0
[...]
-- Installing: /usr/include/CharLS/losslesstraits.h
-- Installing: /usr/include/CharLS/scan.h
CMake Error at cmake_install.cmake:68 (file):
file INSTALL cannot find
"/usr/local/peter/Tests/CharLS/Projekt/charls/src/config.h".
Makefile:83: recipe for target 'install' failed
make: *** [install] Error 1
Line 121 in 20ed9e9
I'm getting an error on this line of code when trying to compile with VS 14.1
This build will likely get destroyed, so I'm copying the error message
[1/22] Building CXX object CMakeFiles\charls.dir\src\charls_jpegls_decoder.cpp.obj
FAILED: CMakeFiles/charls.dir/src/charls_jpegls_decoder.cpp.obj
C:\PROGRA~2\MICROS~1.0\VC\bin\amd64\cl.exe /nologo /TP -DCHARLS_LIBRARY_BUILD -Dcharls_EXPORTS -I..\include -MD -GL /DWIN32 /D_WINDOWS /W3 /GR /EHsc /MD /O2 /Ob2 /DNDEBUG -std:c++14 /showIncludes /FoCMakeFiles\charls.dir\src\charls_jpegls_decoder.cpp.obj /FdCMakeFiles\charls.dir\ /FS -c ..\src\charls_jpegls_decoder.cpp
..\include\charls/charls.h(488): warning C4800: 'int32_t': forcing value to bool 'true' or 'false' (performance warning)
d:\bld\charls_1577727546940\work\src\util.h(121): error C3250: 'x': declaration is not allowed in 'constexpr' function body
d:\bld\charls_1577727546940\work\src\util.h(121): error C3249: illegal statement or sub-expression for 'constexpr' function
d:\bld\charls_1577727546940\work\src\util.h(122): error C3249: illegal statement or sub-expression for 'constexpr' function
d:\bld\charls_1577727546940\work\src\util.h(124): error C3249: illegal statement or sub-expression for 'constexpr' function
d:\bld\charls_1577727546940\work\src\util.h(124): note: see usage of '++'
d:\bld\charls_1577727546940\work\src\util.h(126): error C3256: 'x': variable use does not produce a constant expression
[2/22] Building CXX object CMakeFiles\charls.dir\src\charls_jpegls_encoder.cpp.obj
FAILED: CMakeFiles/charls.dir/src/charls_jpegls_encoder.cpp.obj
C:\PROGRA~2\MICROS~1.0\VC\bin\amd64\cl.exe /nologo /TP -DCHARLS_LIBRARY_BUILD -Dcharls_EXPORTS -I..\include -MD -GL /DWIN32 /D_WINDOWS /W3 /GR /EHsc /MD /O2 /Ob2 /DNDEBUG -std:c++14 /showIncludes /FoCMakeFiles\charls.dir\src\charls_jpegls_encoder.cpp.obj /FdCMakeFiles\charls.dir\ /FS -c ..\src\charls_jpegls_encoder.cpp
..\include\charls/charls.h(488): warning C4800: 'int32_t': forcing value to bool 'true' or 'false' (performance warning)
d:\bld\charls_1577727546940\work\src\util.h(121): error C3250: 'x': declaration is not allowed in 'constexpr' function body
d:\bld\charls_1577727546940\work\src\util.h(121): error C3249: illegal statement or sub-expression for 'constexpr' function
d:\bld\charls_1577727546940\work\src\util.h(122): error C3249: illegal statement or sub-expression for 'constexpr' function
d:\bld\charls_1577727546940\work\src\util.h(124): error C3249: illegal statement or sub-expression for 'constexpr' function
d:\bld\charls_1577727546940\work\src\util.h(124): note: see usage of '++'
d:\bld\charls_1577727546940\work\src\util.h(126): error C3256: 'x': variable use does not produce a constant expression
[3/22] Building CXX object CMakeFiles\charls.dir\src\jpegls.cpp.obj
d:\bld\charls_1577727546940\work\src\scan.h(82): error C3249: illegal statement or sub-expression for 'constexpr' function
d:\bld\charls_1577727546940\work\src\scan.h(88): error C3250: 'mappedError': declaration is not allowed in 'constexpr' function body
d:\bld\charls_1577727546940\work\src\scan.h(88): error C3249: illegal statement or sub-expression for 'constexpr' function
ninja: build stopped: subcommand failed.
During the encoding process the caller needs to pass the bitPerSample parameter and the buffer with the samples to encode. CharLS will not check if the samples are in the correct range, example:
bitPerSample = 10 (max value = 1014) and passing a sample of 2000.
In the debug build a ASSERT deep down will trigger that a computation failed. As most users use the release build they silently generate bad encoded bit streams.
I am interested in how jpeg-ls works. how can I build this project on linux? Thanks.
In release 2.1 encoder API (JpegLsEncode) produces corrupted images. This comes of an inversion of the width and the height in https://github.com/team-charls/charls/blob/master/src/charls_jpegls_encoder.cpp#L455
I meet this problem on release 2.0.0
In the function ModuloRange
from file src/default_traits.h
there are two inconsistencies
FORCE_INLINE int32_t ModuloRange(int32_t errorValue) const noexcept
{
ASSERT(std::abs(errorValue) <= RANGE);
if (errorValue < 0)
{
errorValue += RANGE;
}
if (errorValue >= (RANGE + 1) / 2)
{
errorValue -= RANGE;
}
ASSERT(-RANGE / 2 <= errorValue && errorValue <= static_cast<int32_t>(ceil(static_cast<double>(RANGE) / 2)) - 1);
return errorValue;
}
why here is
if (errorValue < 0)
{
errorValue += RANGE;
}
and not
if (errorValue < -RANGE / 2)
{
errorValue += RANGE;
}
And rounding rules creates problem with eg. RANGE=25
and errorValue=12
The rounding rule do not apply, but end assertion aplly, so code ends with faliture. My suggestion for fixes is:
FORCE_INLINE int32_t ModuloRange(int32_t errorValue) const noexcept
{
ASSERT(std::abs(errorValue) <= RANGE);
if (errorValue < -RANGE / 2)
{
errorValue += RANGE;
}
if (errorValue >= (RANGE + 1) / 2)
{
errorValue -= RANGE;
}
ASSERT(-RANGE / 2 <= errorValue && errorValue <= RANGE - (RANGE) / 2 - 1);
return errorValue;
}
But I`m not sure how it should work, so I open issue, not pull request. I can open pull request if you ask for it.
Looks like CharLS 1.1 trigger the following warning with clang:
/Users/builder/external/GDCM/Utilities/gdcmcharls/header.cpp:69:77: warning: instantiation of function 'JlsCodecFactory<EncoderStrategy>::GetCodec' required here, but no definition is available [-Wundefined-func-template]
std::auto_ptr<EncoderStrategy> qcodec = JlsCodecFactory<EncoderStrategy>().GetCodec(info, _info.custom);
^
/Users/builder/external/GDCM/Utilities/gdcmcharls/header.h:60:26: note: forward declaration of template entity is here
std::auto_ptr<STRATEGY> GetCodec(const JlsParameters& info, const JlsCustomParameters&);
^
/Users/builder/external/GDCM/Utilities/gdcmcharls/header.cpp:69:77: note: add an explicit instantiation declaration to suppress this warning if 'JlsCodecFactory<EncoderStrategy>::GetCodec' is explicitly instantiated in another translation unit
std::auto_ptr<EncoderStrategy> qcodec = JlsCodecFactory<EncoderStrategy>().GetCodec(info, _info.custom);
^
/Users/builder/external/GDCM/Utilities/gdcmcharls/header.cpp:136:78: warning: instantiation of function 'JlsCodecFactory<DecoderStrategy>::GetCodec' required here, but no definition is available [-Wundefined-func-template]
std::auto_ptr<DecoderStrategy> qcodec = JlsCodecFactory<DecoderStrategy>().GetCodec(_info, _info.custom);
^
/Users/builder/external/GDCM/Utilities/gdcmcharls/header.h:60:26: note: forward declaration of template entity is here
std::auto_ptr<STRATEGY> GetCodec(const JlsParameters& info, const JlsCustomParameters&);
^
/Users/builder/external/GDCM/Utilities/gdcmcharls/header.cpp:136:78: note: add an explicit instantiation declaration to suppress this warning if 'JlsCodecFactory<DecoderStrategy>::GetCodec' is explicitly instantiated in another translation unit
std::auto_ptr<DecoderStrategy> qcodec = JlsCodecFactory<DecoderStrategy>().GetCodec(_info, _info.custom);
In performance.cpp, TestLargeImagePerformance
and DecodePerformanceTests
functions, some data files are referenced that are not included in the repository. From where can these images/data files be obtained?
Hello. Thank you for developing the CharLS library! I use it from Python via Cython.
I cannot seem to find a way to check the CharLS library version at compile or runtime.
Would it be possible to add some defines and/or functions to the API? For example
#define CHARLS_VERSION_MAJOR 2
#define CHARLS_VERSION_MINOR 1
#define CHARLS_VERSION_RELEASE 0
#define CHARLS_VERSION_NUMBER (CHARLS_VERSION_MAJOR * 100*100 + CHARLS_VERSION_MINOR * 100 + CHARLS_VERSION_RELEASE)
CHARLS_API_IMPORT_EXPORT const char* CHARLS_API_CALLING_CONVENTION charls_get_version_string(void);
CHARLS_API_IMPORT_EXPORT const uint32_t CHARLS_API_CALLING_CONVENTION charls_get_version_number(void);
I have a raw image that I'm trying to compress using the library. My input data size is 57658368 and after calling JpegLsEncode, the bytesWritten output is 59951222. Any idea why the compressed length is larger than raw size?
JlsParameters params = JlsParameters();
params.width = 6576;
params.height = 4384;
params.bitsPerSample = 16;
params.stride = 0;
params.components = 1;
params.interleaveMode = InterleaveMode::None;
params.colorTransformation = ColorTransformation::None;
This is unfortunate but C++11 deprecated std::auto_ptr
. CharLS codebase should be updated in favor of std:unique_ptr
. It may be non-trivial to have a portable solution, see: http://programmers.stackexchange.com/questions/291141/how-to-handle-design-changes-for-auto-ptr-deprecation-in-c11
In version 2.1.0rc1 on gcc-5 the configuration of cmake pass, but compilation fail.
Cmake configuration
Build charls
-- The C compiler identification is GNU 5.4.0
-- The CXX compiler identification is GNU 5.4.0
-- Check for working C compiler: /usr/bin/cc
-- Check for working C compiler: /usr/bin/cc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Check for working CXX compiler: /usr/bin/c++
-- Check for working CXX compiler: /usr/bin/c++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Building as master project, CMake version: 3.12.4
-- Configuring done
-- Generating done
Error info
[ 10%] Building CXX object CMakeFiles/charls.dir/src/charls_jpegls_encoder.cpp.o
/home/vsts/work/1/s/build_utils/libs_src/charls/src/charls_jpegls_encoder.cpp: In member function ‘void charls_jpegls_encoder::encode(const void*, size_t, uint32_t)’:
/home/vsts/work/1/s/build_utils/libs_src/charls/src/charls_jpegls_encoder.cpp:148:37: error: ‘interleave_mode’ is not a class, namespace, or enumeration
if (interleave_mode_ != interleave_mode::none)
^
/home/vsts/work/1/s/build_utils/libs_src/charls/src/charls_jpegls_encoder.cpp:165:38: error: ‘color_transformation’ is not a class, namespace, or enumeration
if (color_transformation_ != color_transformation::none)
^
/home/vsts/work/1/s/build_utils/libs_src/charls/src/charls_jpegls_encoder.cpp:181:33: error: ‘interleave_mode’ is not a class, namespace, or enumeration
if (interleave_mode_ == interleave_mode::none)
^
CMakeFiles/charls.dir/build.make:75: recipe for target 'CMakeFiles/charls.dir/src/charls_jpegls_encoder.cpp.o' failed
As far as I understand the only requirement for c++11 is std::make_unique
. Would you consider accepting a pull request which does:
$ cat src/util.h
[...]
#if __cplusplus == 201103L
template<typename T, typename... Args>
std::unique_ptr<T> make_unique(Args&&... args)
{
return std::unique_ptr<T>(new T(std::forward<Args>(args)...));
}
#endif
Hi,
I just started experimenting with american fuzzy lop and picked charls as my target. I wrote the following code to feed data to CharLS:
#include "src/charls.h"
#include <unistd.h>
#include <fcntl.h>
#include <string.h>
static char result[1024 * 1024];
static char input[1024 * 1024];
int main(int argc, char *argv[])
{
int fd = 0;
if (argc == 2)
{
fd = open(argv[1], O_RDONLY);
if (fd < 0) {
fprintf(stderr, "Failed to open '%s': %s\n",
argv[1], strerror(errno));
return 1;
}
}
size_t input_length = read(fd, input, sizeof(input));
std::cout << (int) JpegLsDecode(result, sizeof(result), input, input_length, nullptr, nullptr) << std::endl;
return 0;
}
Within seconds, afl managed to crash this program.
Valgrind result for first file https://filebin.ca/3j4hDmAVjSvI:
==16586== Invalid read of size 8
==16586== at 0x4E4793: JpegStreamReader::Read(ByteStreamInfo) (jpegstreamreader.cpp:124)
==16586== by 0x41042D: JpegLsDecodeStream (interface.cpp:163)
==16586== by 0x41042D: JpegLsDecode (interface.cpp:216)
==16586== by 0x40DC91: main (charlsfuzz.cpp:24)
==16586== Address 0x0 is not stack'd, malloc'd or (recently) free'd
The above seems to be JlsCodecFactory::CreateCodec
returning a nullptr, which is something that JpegStreamReader::Read
does not check for before dereferencing qcodec
in line 124.
Plus, there are lots (currently 9) of files which cause the following assertion failure, one of them is https://filebin.ca/3j4hfYzIdJMm:
charlsfuzz: /tmp/charls/src/decoderstrategy.h:129: void DecoderStrategy::MakeValid(): Assertion `static_cast<size_t>(_validBits) <=bufType_bit_count - 8' failed.
Edit: Here is another failed assertion:
https://filebin.ca/3j4yTkNxVuQb
charlsfuzz: /tmp/charls/src/context.h:60: void JlsContext::UpdateVariables(int32_t, int32_t, int32_t): Assertion `std::abs(b) < 65536 * 256' failed.
Edit: https://filebin.ca/3j57Vrm3jPkZ
charlsfuzz: /tmp/charls/src/scan.h:239: SAMPLE JlsCodec<LosslessTraits<uint16_t, 16>, DecoderStrategy>::DoRegular(int32_t, int32_t, int32_t, DecoderStrategy *) [Traits = LosslessTraits<uint16_t, 16>, Strategy = DecoderStrategy]: Assertion `std::abs(ErrVal) < 65535' failed.
Edit: The above as a check-list since I see that some of these were already fixed (by the way, assuming I find more such "evil inputs", should I open one issue per file or continue grouping them like this? Should I open a new issue or keep updating this one?):
The JPEG-LS standard ISO/IEC 14495-1, paragraph C.2.4.1.1.1 defines how to compute the default threshold values. 2 set of formulas are defined. One needs to switch between the two sets based on the MAXVAL >= 128 or not, MAXVAL = 2 ^ bits_per_sample.
CharLS only uses the set of formulas defined for MAXVAL >= 128. This may cause incorrect encoding and decoding of images with 2,4,5 or 6 bits per sample when the default threshold values are used. The typical case is to use the default threshold values.
The names 'jpeg-lossless' and 'JPEG-LOSSLESS' are not included in the names array in JPEGProviderInfo. These names are used by JAI to identify jpeg lossless readers and writers. Some packages explicitly look for these names. Please add these for JAI compatibility.
This is a minor issue but it would be nice to get rid of this.
scan.h:84:31: runtime error: left shift of 4031 by 63 places cannot be represented in type 'long int'
where:
$ tail -n +84 scan.h | head -n 1
LONG sign = LONG(mappedError << (LONG_BITCOUNT-1)) >> (LONG_BITCOUNT-1);
How does JPEG-LS (and CharLS) handle 16 bit grayscale data signedness? Does it support both 16 bit signed and unsigned? If so, how does a caller determine whether the pixel data is signed or unsigned? If not, is 16 bit grayscale returned as signed or unsigned? Thanks
Note: this is an issue originally reported on codeplex: http://charls.codeplex.com/workitem/10742
Hi,
we received a bug report for the copy of CharLS that is part of the DCMTK:
http://forum.dcmtk.org/viewtopic.php?f=1&t=3412
I took a look at this file and I think that this is indeed a bug in CharLS.
The attached patch creates an EncoderStrategy instance and writes bits to it. This triggers the same assertion as the original test image. (Ignore my changes to encoderstrategy.h, that's just the debug output that I looked at for figuring this out):
encoderstrategy.h:95: void EncoderStrategy::AppendToBitStream(LONG, LONG): Assertion `bitpos >=0' failed.
My analsysis of what is going on is part of the patch, so be sure to read the comments. :-)
I tried to come up with a patch that fixes this. This is the following part of the diff:
if (bitpos >= 0)
if (bitpos > 0)
The idea here is that when 32 bits of input were read, we can Flush().
This should even fix a similar issue, where we have bitpos == 1 and add 31 input bits, so one bit gets added to valcurrent. If Flush() only writes out 30 bits, all the 30 bits of input left can go into the buffer.
Cheers,
Uli
I am using charLS to encode and decode DICOM medical images. During testing I came across a bad data file (a dcm file stored as an image in a dcm file). The storage was set the compress the file as JpegLS. During the compression of the data, charLS crashes with a EXC_BAD_ACCESS for pointer *_position accessing outside the output buffer in EncodeStratagy::flush(). _compressedLength goes negitive (bad for a size_t) Even though the data is not an image, the library should have returned a buffer overflow. I wrote a small program to read the file and compress it to prove it was not a bug in the main program. It is here:
encodeRaw.zip
This is the input data that caused the crash:
jpegls_crash_data.raw.zip
Try to compress it as a 944 x 944 x 8bits monochrome lossless image.
I am getting a couple of warning when compiling charls v2.0.0 with clang:
In file included from /home/mathieu/Perso/gdcm/Utilities/gdcmcharls/jpegls.cpp:27:
/home/mathieu/Perso/gdcm/Utilities/gdcmcharls/scan.h:202:10: warning: 'DecodeScan' overrides a member function but is not marked 'override' [-Winconsistent-missing-override]
void DecodeScan(std::unique_ptr<ProcessLine> rawData, const JlsRect& size, ByteStreamInfo& compressedData, bool bCompare);
^
/home/mathieu/Perso/gdcm/Utilities/gdcmcharls/jpegls.cpp:113:50: note: in instantiation of template class 'JlsCodec<DefaultTraitsT<unsigned char, unsigned char>, DecoderStrategy>' requested here
strategy = std::unique_ptr<STRATEGY>(new JlsCodec<DefaultTraitsT<uint8_t, uint8_t>, STRATEGY>(traits, params));
^
/home/mathieu/Perso/gdcm/Utilities/gdcmcharls/jpegls.cpp:186:16: note: in instantiation of member function 'JlsCodecFactory<DecoderStrategy>::GetCodec' requested here
template class JlsCodecFactory<DecoderStrategy>;
^
/home/mathieu/Perso/gdcm/Utilities/gdcmcharls/decoderstrategy.h:35:18: note: overridden virtual function is here
virtual void DecodeScan(std::unique_ptr<ProcessLine> outputData, const JlsRect& size, ByteStreamInfo& compressedData, bool bCheck) = 0;
^
In file included from /home/mathieu/Perso/gdcm/Utilities/gdcmcharls/jpegls.cpp:27:
Would be super nice to fix in next release
Line 37 in util.h is #elif, should be #else
Here is one patch that we maintain within Debian:
https://salsa.debian.org/med-team/charls/blob/master/debian/patches/add_visibility.patch
Would be nice to handle upstream directly. Thanks
Clang <= 5 and AppleClang 9.3 fail to compile jpegls_error.cpp if using libc++ (and not libstdc++) because string (and system_error) are not included.
[ 44%] Building CXX object source_subfolder/CMakeFiles/charls.dir/src/jpegls_error.cpp.o
/home/conan/.conan/data/charls/2.1.0/bincrafters/testing/build/2b9f4a7d83ec1ec57c27b92740c5dea74386cdf8/source_subfolder/src/jpegls_error.cpp:19:12: error: implicit instantiation of undefined template 'std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >'
string message(int error_value) const override
^
/usr/include/c++/v1/iosfwd:193:33: note: template is declared here
class _LIBCPP_TYPE_VIS_ONLY basic_string;
^
/home/conan/.conan/data/charls/2.1.0/bincrafters/testing/build/2b9f4a7d83ec1ec57c27b92740c5dea74386cdf8/source_subfolder/src/jpegls_error.cpp:21:16: error: implicit instantiation of undefined template 'std::__1::basic_string<char, std::__1::char_traits<char>, std::__1::allocator<char> >'
return charls_get_error_message(static_cast<jpegls_errc>(error_value));
^
/usr/include/c++/v1/iosfwd:193:33: note: template is declared here
class _LIBCPP_TYPE_VIS_ONLY basic_string;
Hi,
When I try to to compress this image:
with all pixel values in the 12 bits range, charls (version feb 2016) throws an assert; or in NDEBUG mode is not error free. This is an extremely rare example. Of course when the pixel values are outside the specified range, there is an ASSERT as well, but you have a bug report for that problem,
Regards,
Marcel (conquest DICOM server author)
Hey, I notice that BUILD_TESTING:BOOL=ON
by default. But, it seems that there is no rule to make target 'test', so how could I run test after compiling charls? I'm trying to create a package for ArchLinux.
The current CI pipelines use Visual Studio 2019, GCC 7.4.0 and clang 7.0.0.
By adding an Azure Pipeline (free for open source projects) the coverage can be extended with:
Visual Studio 2017
Visual Studio 2015 (mostly C++14 compatible), see also issue #60
Xcode 11.3
Xcode 10.1
When browsing the code, I found the following questionable (?) assignment in colortransform.h, line 120
hp3.v1 = SAMPLE(G + ((hp3.v2 + hp3.v3)>>2)) - RANGE / 4;
I might be mistaken, but shouldn't the last right parenthesis be placed at the end of the assignment, i.e.
hp3.v1 = SAMPLE(G + ((hp3.v2 + hp3.v3)>>2) - RANGE / 4);
gcc5 fails to compile charls_jpegls_decoder.cpp (similar issue than #55)
[ 22%] Building CXX object source_subfolder/CMakeFiles/charls.dir/src/charls_jpegls_encoder.cpp.o
/home/conan/.conan/data/charls/2.1.0/bincrafters/testing/build/2de3d908e1d9b246c9b2b6ea5c083ff1297f138b/source_subfolder/src/charls_jpegls_decoder.cpp: In member function ‘size_t charls_jpegls_decoder::destination_size(uint32_t) const’:
/home/conan/.conan/data/charls/2.1.0/bincrafters/testing/build/2de3d908e1d9b246c9b2b6ea5c083ff1297f138b/source_subfolder/src/charls_jpegls_decoder.cpp:106:14: error: ‘interleave_mode’ is not a class, namespace, or enumeration
case interleave_mode::none:
^
/home/conan/.conan/data/charls/2.1.0/bincrafters/testing/build/2de3d908e1d9b246c9b2b6ea5c083ff1297f138b/source_subfolder/src/charls_jpegls_decoder.cpp:109:14: error: ‘interleave_mode’ is not a class, namespace, or enumeration
case interleave_mode::line:
^
/home/conan/.conan/data/charls/2.1.0/bincrafters/testing/build/2de3d908e1d9b246c9b2b6ea5c083ff1297f138b/source_subfolder/src/charls_jpegls_decoder.cpp:110:14: error: ‘interleave_mode’ is not a class, namespace, or enumeration
case interleave_mode::sample:
^
source_subfolder/CMakeFiles/charls.dir/build.make:62: recipe for target 'source_subfolder/CMakeFiles/charls.dir/src/charls_jpegls_decoder.cpp.o' failed
Here is a patch (for 2.1.0) which fix this issue:
--- a/src/charls_jpegls_decoder.cpp
+++ b/src/charls_jpegls_decoder.cpp
@@ -103,11 +103,11 @@ struct charls_jpegls_decoder final
switch (interleave_mode())
{
- case interleave_mode::none:
+ case charls::interleave_mode::none:
return static_cast<size_t>(stride) * info.height * info.component_count;
- case interleave_mode::line:
- case interleave_mode::sample:
+ case charls::interleave_mode::line:
+ case charls::interleave_mode::sample:
return static_cast<size_t>(stride) * info.height;
}
Tasklist for the 2.1.0 release
Right now c++14 is being used for both implementation detail and API in CharLS. It would be nice to make it optional so as to compile with newer/older standard for advanced users. See patch in Debian:
Looks like the code only supports lossless compression, or at least that's what I understand from going over convert.c
. Any chance we can add lossy compression, or add example code that calls a lossy compression?
Could you drop this in the src dir. It could help people build the library on Macs.
charLS.xcodeproj.zip
I would like to find a javascript version of a JPEG-LS decoder. Yesterday I used emscripten to compile CharLS into JS and it works great for small images but runs out of memory on larger images. Assuming I can fix the memory issue, this should meet my needs. I wanted to see if the CharLS project would be interested in a PR that adds a Emscripten JS build target. If not, I can fork and add my build step to that and keep it in sync. Thanks
PS - The codec would be used by the cornerstone open source medical imaging sdk here:
https://github.com/chafey/cornerstone
Hi,
I am trying to use 1.x-master. It lacks several files and therefore does not compile. I miss:
jpegstreamreader.h
jpegmarkercode.h
jpegstreamreader.cpp
jlscodecfactory.h
The versions from the main branch are not compatible.
Thanks,
Marcel
Here is another patch from Debian that we maintain: hiding of STL stuff being automagically exported. Here is the hack that I wrote long ago:
https://salsa.debian.org/med-team/charls/blob/master/debian/patches/stl_symbols_hack.patch
Would be nice to handle gracefully using some portable cmake magic.
I can reproduce a failure to decompress using CharLS 1.1 (issue is present in CharLS 1.0), this is originally a bug tracked in Debian. With the reference jls, I can get JpegLsDecodeStream
to return TooMuchCompressedData
:
(gdb)
297 ByteStreamInfo outputInfo = FromByteArray(outputBuffer.data(), outputBuffer.size());
(gdb)
298 err = JpegLsDecodeStream(outputInfo, compressedByteStream, &info);
(gdb)
299 if (err != OK)
(gdb) p err
$1 = TooMuchCompressedData
Hi All,
First of all I am new to this code, so please pardon me if I am asking some silly query here.
I am getting "_validBits" as 0 in the MakeValid() method under decoderstrategy.h file and this is made as 0 in the "Skip(LONG length)" method, and this is happening only for few images. I would like to understand the reasons behind this behaviour.
Kindly help me get some idea about this and to proceed further on my investigation.
Regards,
Ramana Mohan
the code:
https://github.com/team-charls/charls/blob/master/src/encoderstrategy.h#L77
[...]
if (bitCount < 32)
{
int mask = (1 << (bitCount)) - 1;
[...]
does not look quite right for bitCount == 31 ... Maybe use unsigned integer (1u << bitCount)
The JPEG-LS standard provides the option to write custom coding parameters to a JPEG-LS bit stream. This can be done with a LSE marker segment. This marker segment may occur multiple times and at different locations in the bit stream.
In practice custom coding parameters are never used, except for 1 case.
The Java JAI library (not supported anymore by Oracle) can encode/decode JPEG-LS stream. This JAI library has a bug that has never been fixed, it computes invalid coding parameters for bit per sample > 12. As a workaround CharLS writes an explicit LSE marker before the first scan. If there are multiple components it will write however this marker multiple times. The output generated is a valid JPEG-LS bit stream.
Note: CharLS will only write this LSE Marker N times when:
During the decoding process, CharLS will fail when it encounters a LSE Marker after the first SOS segment.
For example encoding and then decoding with:
Support for this JAI improved compatibly was introduced in commit 232a4c2
The initial V1.0 release of CharLS had its own unit test framework (in charlstest.exe)
Modern development environments come with support for C++ unit tests. CharLS start with Visual Studio C++ unit test framework as this was the only unit test framework with integrated unit test support. Recent versions of Visual Studio come with adapters to use all kind of unit tests, like google test, boost test, catch2, etc.
These unit test frameworks are all cross-platform, while VS unit test framework is limited to Windows.
By moving to a cross-platform, the unit test can be all moved to a single executable that integrates well and is usable on multiple platforms.
Note: which unit test framework to use is not yet decided.
The JPEG-LS standard allows encoding images with a custom RESET value in the JPEG-LS preset coding parameters.
A defect in the implementation would always cause a DefaultTraitsT<uint8_t, uint8_t> being used instead of DefaultTraitsT<uint16_t, uint16_t> when the bits per sample is greater than 8.
This defect has been reported by @renyu310, pull request #33
When using the HP3 color transform the encoding to decoding with lossless doesn't yield the same file bit to bit. Reproducing steps:
1- Use CharLSTest
2- Add this parameter to JlsParameters to function EncodePnm params.colorTransformation = charls::ColorTransformation::HP3;
3- encodepnm with the attached ppm file
4- decodetopnm the compressed file
5- comparepnm fails.
Hello,
It seems like there is a problem in checking validity of data. The function stack is as follows:
-- ReadHighBits()
-- -- ReadBit()
-- -- -- MakeValid()
The matter is that in function MakeValid() the assert has a hit here -
ASSERT(static_cast<size_t>(validBits_) <=bufType_bit_count - 8);
It occures when validBits_
is negative - it happened in ReadHighBits()
when Skip(15)
is called.
Am I right that validBits_
is not supposed to be negative? Then the check if (validBits_ <= 0)
in ReadBit() should be fixed in a way of throwing an exception when validBits_ < 0
.
I could provide a simple PR but first I wanted to check why the test uses inverted convention. Eg:
$ dd if=/dev/zero of=/tmp/zero.jls count=512
512+0 records in
512+0 records out
262144 bytes (262 kB, 256 KiB) copied, 0.00496284 s, 52.8 MB/s
$ ./charlstest -decodetopnm /tmp/zero.jls tt.pgm
$ echo $?
0
On UNIX you need to return a non-zero value to indicate an error.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.