renderkit / oidn Goto Github PK

When trying to build on macOS 10.13.6 (High Sierra), the make command returned the following error:

fsiddi@mbp  (master) $ make
Scanning dependencies of target OpenImageDenoise_sse42
[  1%] Building CXX object CMakeFiles/OpenImageDenoise_sse42.dir/core/network.cpp.o
In file included from /Users/fsiddi/Developer/oidn/core/network.cpp:17:
In file included from /Users/fsiddi/Developer/oidn/core/upsample.h:19:
In file included from /Users/fsiddi/Developer/oidn/core/node.h:19:
In file included from /Users/fsiddi/Developer/oidn/core/common.h:19:
/Users/fsiddi/Developer/oidn/common/platform.h:28:10: fatal error: 'cstdint' file not found
#include <cstdint>
         ^~~~~~~~~
1 error generated.
make[2]: *** [CMakeFiles/OpenImageDenoise_sse42.dir/core/network.cpp.o] Error 1
make[1]: *** [CMakeFiles/OpenImageDenoise_sse42.dir/all] Error 2
make: *** [all] Error 2

By adding -stdlib=libc++ to CMAKE_CXX_FLAGS I could build without errors.

Hey I just noticed that OIDN seems only to use the physical cores of my Threadripper and not all of the logical cores the system has to offer.
Any chance to utilize the full power of such systems?

seanvk@clr-eugene-svkelley ~/dev/oidn/build $ gcc -v
Using built-in specs.
COLLECT_GCC=/usr/bin/gcc
COLLECT_LTO_WRAPPER=/usr/lib64/gcc/x86_64-generic-linux/8.2.1/lto-wrapper
Target: x86_64-generic-linux
Configured with: ../gcc-8.2.0/configure --prefix=/usr --with-pkgversion='Clear Linux OS for Intel Architecture' --libdir=/usr/lib64 --enable-libstdcxx-pch --libexecdir=/usr/lib64 --with-system-zlib --enable-shared --enable-gnu-indirect-function --disable-vtable-verify --enable-threads=posix --enable-__cxa_atexit --enable-plugin --enable-ld=default --enable-clocale=gnu --disable-multiarch --enable-multilib --enable-lto --enable-linker-build-id --build=x86_64-generic-linux --target=x86_64-generic-linux --enable-languages=c,c++,fortran,go --enable-bootstrap --with-ppl=yes --with-isl --includedir=/usr/include --with-gxx-include-dir=/usr/include/c++/ --exec-prefix=/usr --with-glibc-version=2.19 --disable-libunwind-exceptions --with-gnu-ld --with-tune=haswell --with-arch=westmere --disable-libmpx
Thread model: posix
gcc version 8.2.1 (Clear Linux OS for Intel Architecture)

seanvk@clr-eugene-svkelley ~/dev/oidn/build $ cmake ..
-- The CXX compiler identification is GNU 8.2.1
-- Check for working CXX compiler: /usr/lib64/ccache/bin/g++
-- Check for working CXX compiler: /usr/lib64/ccache/bin/g++ -- works
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- The C compiler identification is GNU 8.2.1
-- Check for working C compiler: /usr/lib64/ccache/bin/gcc
-- Check for working C compiler: /usr/lib64/ccache/bin/gcc -- works
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Detecting C compile features
-- Detecting C compile features - done
-- Found TBB: /usr/include
-- VTune profiling environment is unset
-- Found PythonInterp: /usr/bin/python (found version "3.7.2")
-- Configuring done
-- Generating done
-- Build files have been written to: /home/seanvk/dev/oidn/build

error.log

I have included the following code in a VC++/MFC project:

#include "OpenImageDenoise/oidn.hpp"
#include "image_io.h"

#pragma comment(lib, "Ref/oidn-1.1.0.x64.vc14.windows/lib/OpenImageDenoise.lib")

	ImageBuffer color;
	oidn::loadImage(_T("D:\\Tempx\\memorial.pfm"));
	const int width = color.getWidth();
	const int height = color.getHeight();
	// Initialize the output image
	ImageBuffer output(width, height, 3);
	oidn::DeviceRef device = oidn::newDevice();

and when I compile this code, I get the following error:

error LNK2019: unresolved external symbol __imp__oidnNewDevice referenced in function "class oidn::DeviceRef __cdecl oidn::newDevice(enum oidn::DeviceType)" (?newDevice@oidn@@YA?AVDeviceRef@1@W4DeviceType@1@@Z)
error LNK2019: unresolved external symbol __imp__oidnReleaseDevice referenced in function "public: __thiscall oidn::DeviceRef::~DeviceRef(void)" (??1DeviceRef@oidn@@QAE@XZ)

What I should do more to can compile a simple sample for this library ?

There seems to be a crash somewhere in library initialization after we improperly fail to detect SSE4.1 ISA. This was noticed with the OIDN 0.7 packaged in ParaView 5.7.0. Is this a known issue that has been fixed in more recent releases by chance?

I am keeping track of this in the ParaView issue tracker here: https://gitlab.kitware.com/paraview/paraview/issues/19424
I have not yet done any serious investigation or reproduced it myself.

thanks

Would it be possible to include an option to build the library as a static library? Just switching the SHARED to STATIC basically works (at least on Ubuntu 18.04) though there are a couple errors to fix in the install module.

Since version 1.0 the denoiser doesn't work well with "high frequency" textures and details look more blurry in general. You can see the problem when looking at the carpet. Close to the window the carpet looses any structure:

Version 0.9 did a better job at this point:

You may find the input images (color, albedo, normal) here:
input_images.zip

Color image is SRGB.

Hi.

Are you considering denoising directly from samples? It’s unfortunate that so much information is thrown away, e.g. when rendering with 16spp I would expect much better results from 16 samples than from one pixel value.

I have tried the following code:

	ImageBuffer color = loadImage(_T("D:\\Temp\\memorial.pfm"));
	const int width = color.getWidth();
	const int height = color.getHeight();
	// Initialize the output image
	ImageBuffer output(width, height, 3);
	oidn::DeviceRef device = oidn::newDevice();
	oidn::FilterRef filter = device.newFilter("RT");
	filter.commit();
	filter.setImage("color", color.getData(), oidn::Format::Float3, width, height);
	filter.setImage("output", output.getData(), oidn::Format::Float3, width, height);
	filter.set("hdr", true);
	filter.commit();
	filter.execute();
	saveImage(_T("D:\\Temp\\memorial_output.pfm"), output);

but the output image are completely black ... the code seem simple and clear ... I have done something wrong ?

I attach here the input image and also the ouput result ... can you take a look ? Might be something simple, but I can't see what ...
memorial.zip
memorial_output.zip

So i'm interested in denoising high iso images from dslr. I've got noisy jpg and wanna have clean output. What should i write in cmd to get this? Thx :)

I currently submit oidn 0.8.2 package for Fedora under review and the reviewer reported this issue with that snippet:

shared-lib-calls-exit /usr/lib64/libOpenImageDenoise.so.0.8.2 exit@GLIBC_2.2.5
shared-lib-calls-exit:
This library package calls exit() or _exit(), probably in a non-fork()
context. Doing so from a library is strongly discouraged - when a library
function calls exit(), it prevents the calling program from handling the
error, reporting it to the user, closing files properly, and cleaning up any
state that the program has. It is preferred for the library to return an
actual error code and let the calling program decide how to handle the
situation.

Hi,
We're trying to build oidn 0.9.0 on Mac, and are hitting this error:

Building CXX object mkl-dnn/src/CMakeFiles/mkldnn.dir/common/batch_normalization.cpp.o
clang: error: unknown argument: '-fopenmp-simd'

clang -v gives

Apple LLVM version 9.1.0 (clang-902.0.29.2)

The mac is running macOS High Sierra Version 10.13.6.

It looks like Oidn doesn't work with Bidirectional path tracing renderings (with Metropolis sampler and spectral rendering) like this one:

or this one:

The output buffer it is just the same of the input, apparently untouched, even with a simple rendering like Cornell University box.

I'm not sure if this is a known limitation.

We use OIDN with a stream of inputs originally in half-float format (IEEE 754 half-float format, RGB/RGBA). Currently we convert the input images to 32-bit floats with SIMD intrinsics prior to denoising which works okay but does use extra memory and costs time; are there any plans to extend the RT filter to natively support half-float formats?

Thanks

Hi there!
I wanted to comment that I'm having kind of the same problem described in the "Loosing texture details since V1.0".

I've used "Blackman-Harris" pixel filter set up at 1.00 px.
As you can see after the denoise process the image gets really blurry. The "A" image is when I first noticed it and I thought that it was because the object was quite far from the camera, and there are not too much information/pixels to determine how it should look.

However, at close-ups, it also blurs the image in certain areas while in others don't.

I don't know if it is something that I might not be set up correctly in my scene or maybe something that the denoiser can't handle yet.

I've been using blender 2.81 and Luxcore for these renders.

Regards, saludos
Pablo

Hello,

context:
I'm not entirely sure this is a legitimate issue, since I'm not doing what is explicitly said on your README.md, I'm trying to use your library with cmake using add_subdirectory, I'm not sure if this way of linking the project is intended to be supported. Also, I'm on windows using the commandline( not sure if this is relevant but I'm not trying to use cmake-gui)

error:
when including the project with add_subdirectory, I get the following error:

CMake Error at external/oidn/CMakeLists.txt:55 (include): include could not find load file: package

temporary fix?
I can solve these errors by replacing all the includes on your CMakeLists.txt, for example:

include(${CMAKE_CURRENT_LIST_DIR}/cmake/install.cmake)

and then all problems seem to be solved. I have the gut feeling that my lack of knowledge about cmake might be the problem here, is there something special to do before using add_subdirectory on your project?
I hope this is not way too unrelated to your project, feel free to dismiss me if you feel that's the case, thanks for your time.

On windows, static builds are very small compared to the dll and don't work with Blender at least.
On Linux, when OIDN is compiled as a dynamic library it works. But when build as a static library, building Blender fails at 100%: the blender executable segfault while being called with the -h option to create the manpages.
To reproduce the bug:

• get Blender with OIDN from here https://github.com/skwerner/blender/tree/openimagedenoise
• modify build_files/build_environment/cmake/openimagedenoise.cmake to add -DOIDN_STATIC_LIB=ON
• modify build_files/build_environment/cmake/versions.cmake to download a newer version of oidn with the static option and the submodules (will require a new release on your side)
• modify build_files/cmake/modules/findopenimagedenoise.cmake to find openimagedenoise, mkldnn and common
• modify build_files/cmake/platform/platform_unix.cmake at "if(WITH_OPENIMAGEDENOISE)" with find_package_wrapper instead of find_package
• call "make deps"
• call "make full"

When I tried to generate the solution project, with CMake 3.13.4 (64bit), and select VS2017, I got these CMake errors (message):

Selecting Windows SDK version 10.0.17134.0 to target Windows 10.0.17763.
CMake Error at CMakeLists.txt:88 (add_subdirectory):

  The source directory

    D:/Project/oidn/mkl-dnn

  does not contain a CMakeLists.txt file.


CMake Error at CMakeLists.txt:91 (include):
  include could not find load file:

    mkl-dnn/cmake/Threading.cmake


CMake Error at CMakeLists.txt:92 (include):
  include could not find load file:

    mkl-dnn/cmake/TBB.cmake


CMake Error at CMakeLists.txt:93 (include):
  include could not find load file:

    mkl-dnn/cmake/platform.cmake


CMake Error at CMakeLists.txt:94 (include):
  include could not find load file:

    mkl-dnn/cmake/SDL.cmake


CMake Error at CMakeLists.txt:97 (append):
  Unknown CMake command "append".


Configuring incomplete, errors occurred!
See also "D:/Project/oidn/bin/CMakeFiles/CMakeOutput.log".

What else should I do in order to generate a solution for this library ?

We have received some report of Oidn failing to denoise image area with very light shadow. It is like if there is a threshold under which noise is not removed.

This is an example of the problem, the normal rendering:

and the is the Oidn output:

The very end of the monkey shadow is not denoised, there is a quite sharp cutoff.

This is a .zip with the .png/.exr rendering images, albedo and normal AOVs to replicate the problem:

img.zip

This is the thread in our forum where the problem is discussed: https://forums.luxcorerender.org/viewtopic.php?f=4&t=1122

From the above forum thread, another example of the problem in a real world scene can be seen here: https://forums.luxcorerender.org/viewtopic.php?f=4&t=1122&start=10#p13302

Tonemapped ldr images are working very nicely. My first attempt at denoising hdr is failing.
No errors reported but resulting data contains invalid floats (inf, ind, NaN, etc.) Pfm file below appears to fail in my code and in the denoise sample app. Units are cd/m^2.

win64 (Windows 10). Using prebuilt OpenImageDenoise.dll.

Example pfm

Related question about the denoise sample. Can you provide some sample data files and documentation? I tried downloading https://openimagedenoise.github.io/images/sponza_16spp_input.jpg, using ImageMagick to convert it to PFM, but was very successful when running denoise (came back with an Error: option expected). I figure my input isn't correct.
here is a response: #1 (comment) that is helpful, but I'm actually just looking for actual data sets.

Imagemagicks's convert automatically converts my EXR albedo images to grayscale PFM files (with an Pf header for grayscale instead of PF for color image) and denoise is refusing this type of image. Please add support for that.

Add “noise flicker removal feature"
As far as I know it first appeared in Disney’s own denoiser. I don’t quite know how it works, but it’s something like denoiser is analyzing next frames, and trying to avoid noise jitter, this is very useful for animations. This function is in standart blender cycles denoiser, but it is not in the interface, it seems to be used through python or smth, but I don’t know how, if you make this, I think you can use it as a source.
https://developer.blender.org/D3889

Hi there! How i can use it? Also would be awesome if someone can be create image for Colab notebook like in https://github.com/jantic/DeOldify
How i can denoise single jpg? Please write me steps what i need to do, i'm very need to denoise some of my family images, my father is dying by cancer and i wanna surprise him with this. Thanks!

I tried to join the mailing list (as per 'Support and Contact' section in the README), but Google says I don't have permission to access the content. I tried emailing [email protected], but it bounced. Is it actually open to public?

Is it possible to run on regular aliased images?

The idea is to provide an advantage on regular morphological AA and in ray tracing situations.

[Edited] regular morphological aa has the most problems with foilage.

[Edited] I wanted to investigate using this for Godot Engine as a dual use AA for both Raytracing and regular AA uses. Godot Engine currently doesn't have a raytracer engine, so there's no use using this unless it can also work for regular jagged images.

I am trying to denoise a rendered image from VRay with the OIDN, but the noise is removed only on some regions on the image, as show on the image bellow (click to enlarge).

The original image (a .png) is converted to .pfm with magick using this command:
magick convert -endian LSB PATH_IN PATH_OUT

And i am running the denoise with the following command:
denoise.exe -ldr PATH -alb PATH -nrm PATH -o PATH

Do someone have some tip for why this is happening? Can it be some incompatibility from my type of image with what OIDN can work it?

If someone want to test it, here is the original files.

The denoise example binary does not work on macOS.

This is seemingly due to it loading dylibs via @rpath/libOpenImageDenoise.0.dylib, but rpath itself is specified as /opt/intel//compilers_and_libraries_2019.1.144/mac/compiler/lib in the executable -- which very likely is a folder that does not exist on user's machine.

Probably needs to be built by setting rpath as needed for linker, or the executable patched after link via install_name_tool

Are there plans for Intel GPU support? MKL-DNN seems to be progressing in that direction, so I guess OIDN will also pick on that?

As I said in the title, is it possible to denoise an image with single-channel input?
Thanks!

Hi,
is there a documentation somewhere on how to retrain the network to support depth for example?

Oidn throws a segmentation fault when I provide a normal buffer without an albedo one. It is a small detail but I assume it should report an error and to not crash.

Hello ; i test version 0.8.1 and the new version 1.0.0
The image output from the old version looks better in the details unlike the new version more blurry
V 0.8.1

V 1.0.0

Most software componet is 32bit , it is impossible to change all dll to 64bit .

My question is that how to adatpor OpenImageDenoise with 32bit software

Hi,

I give the sample denoise app a pfm file in [0,1] range and it seems oidn returns greater than 1.0 values in the result buffer, is this an expected behavior?

My cmd args: -ldr rtow.pfm -o ouput.pfm

Thanks!

I tried using denoise.exe with raytraced images that I generated with Blender. I tried it with images generated using 64, 32, 16, 8, 4, 2, and 1 samples per pixel. There was no noticeable improvement of the image's quality. Moreover, the source and output images look identical (they both have "salt and pepper" look). However, when I use ImageMagick's compare utility, it shows some subtle differences. I generated 24-bpp (8 bits per channel) sample images (I saved them in BMP format) from Classroom sample scene available on Blender Web site. I then converted BMP files to PFM using my own program, then used them as input files with -ldr option, and then converted output images to BMP using my other program. All images have 3840x2160 dimensions.

Am I doing something wrong, or maybe something is wrong with the program?

I am on 64-bit Windows 10. I can share the code of my program that converts BMP to PFM if necessary.

I hit this when trying to build a debug version of OIDN on Windows: debug versions of TBB use the _debug suffix, which seemingly confuses the TBB.cmake script in your fork of MKL-DNN.

When can we expect a new release with Static Libraries problem solved and some new features?

Cheers!

I have 6 core / 12 threads intel processor and get this warning a bit randomly. Maybe 3-4 times. This is on Windows 7. What can be the cause?

With HDR images greater than 2K (4K, 8K, etc ..) there are SERIOUS memory problems, the request for images from 4K up is excessive in the order of 15-69 GB, how can this problem get around ???

1. I'm testing denoise.exe v0.8.2 on a windows platform and want to know what the parameter

-ref reference_output.pfm

in the command line stands for. Please can someone explain?

2. Are there any additional parameters for denoise.exe in which you can setup some values for the noise reduction filter settings like strength etc...?

This is not a bug as such, but a suggestion to improve the documentation regarding support on macOS. We just spent a bit of time figuring out why the denoiser would not run on mac OS 10.15 after being installed, but worked fine when building from source. The culprit was the new requirements for code-signing needed for notarization in macOS.

It turns out special entitlements are needed for the denoiser since the MKL DNN uses some jit-compilation that are not allowed with default settings for code signing. In particular, the entitlement com.apple.security.cs.allow-unsigned-executable-memory is needed (I could not get the more restrictive com.apple.security.cs.allow-jit to work).

It can be hard to figure out, because the code-signing itself does not fail, but the denoiser fails silently at runtime (no errors were reported). The only indication of a problem is a message from the kernel in the Console app with the message CODE SIGNING: 2396[....] vm_map_protect can’t have both write and exec at the same time. No error of any kind is displayed to the user or written to stdout.

Some documentation on macOS entitlements:
https://developer.apple.com/documentation/bundleresources/entitlements

This may be relevant for the MKL DNN project itself as well

I'm not sure if this can be defined a bug (see below) but I'm testing Oidn for animations and I'm observing a strange color bleeding.

This is my first simple test animation: https://drive.google.com/file/d/1fFW4UuKNqIG9ZryZTFTtYBxAuvA9s8TK/view?usp=sharing

This is a frame (raw image upper left, Oidn image upper right, albedo AOV lower leftm normals AOV lower right):

Do you see the slight red color bleeding on the out side of the left wall ? It is caused by Oidn.

I'm not sure if this is a bug because the problem goes away with more samples but it is till there with 256 samples/pixel (a quite high number for Oidn standards).

I assume the problem is caused by camera motion blur and the related noise on the albedo pass. There is a quite large number of features like motion blur, depth of field, SSS, volume rendering (like "God's ray") where the Albedo pass is going to be noisy and it is unclear how Oidn requires to handle the situation.

The albedo pass seems more detrimental than helpful in the above cases for Oidn.

Does we need some kind of Albedo denoiser to than feed Oidn with the result ?

Or there will be?

OID works really nice but makes some flickerings on animation so my question is this will be temporal coherent in the future?

Regards.

Row 56 in the file OpenImageDenoiseConfig.cmake of the latest release (0.8) has this line:

INTERFACE_LINK_LIBRARIES "C:/<build machine path>/lib/intel64/vc14/tbb.lib;C:/<build machine path>/lib/intel64/vc14/tbbmalloc.lib"

Where <build machine path> seems to be local to the release machine. Naturally, this path does not exist on my machine :) Setting it to:

INTERFACE_LINK_LIBRARIES "${_IMPORT_PREFIX}/lib/tbb.lib;${_IMPORT_PREFIX}/lib/tbbmalloc.lib"

fixes compilation issues.

This doesn't seem to be specified anywhere, but should the contents of the normal buffer be literal normals with potentially negative values, or should they be converted to positive values using N * 0.5 + 0.5? The bit about arbitrary lengths being allowed suggests the former and it does seem to work fine, but I just wanted to have hard confirmation. It's not completely obvious to me since the other buffer contents are always strictly nonnegative so maybe it's worth a mention in the readme either way?

I already have mkl-dnn installed. There should be no need to rebuild it again.

Not a bug, just a curiosity. Would it be possible to go into a bit more detail about how training data has been created for training OIDN? Something like the following which could be posted on the main website?

• Scenes
  • OIDN 1.2: We used 110 different scenes, native to 3 different Path Tracing engines, to train our model: 40 from OSPRay, 50 from Arnold, 20 from Cycles
  • OIDN 1.0: We used 80 different scenes, native to 2 different Path Tracing engines, to train our model: 40 from OSPRay, 40 from Arnold
• Ground Truth setup
  • We use 10000 samples as the ground-truth measurement for each scene
• Test images
  • We use a wide range of samples per pixel (spp) values, from 1 to almost fully converged, to generate the test corpus
  • To be exact, the following 15 spp values are used for each scene to generate training data:
    1, 2, 4, 10, 25, 50, 100, 200, 400, 500, 1000, 2000, 4000, 6000, 8000

To be clear, I'm not asking that the actual assets or images be posted etc. Just a more in depth, broad outline, of methodology with a version history perhaps?