GithubHelp home page GithubHelp logo

hanatos / vkdt Goto Github PK

View Code? Open in Web Editor NEW
344.0 31.0 32.0 5.4 MB

raw photography workflow that sucks less

Home Page: https://vkdt.org

License: BSD 2-Clause "Simplified" License

Makefile 1.23% C 62.89% C++ 31.10% GLSL 3.50% Jupyter Notebook 0.47% Python 0.28% Shell 0.21% Rust 0.32%
gpu photography vulkan workflow

vkdt's Introduction

raw image processing workflow which sucks less

vkdt is a workflow toolbox for raw photography and video. vkdt is designed with high performance in mind. it features a flexible processing node graph at its core, enabling real-time support for animations, timelapses, raw video, and heavy lifting algorithms like image alignment and better highlight inpainting. this is made possible by faster processing, allowing more complex operations.

the processing pipeline is a generic node graph (DAG) which supports multiple inputs and multiple outputs. all processing is done in glsl shaders/vulkan. this facilitates potentially heavy duty computational photography tasks, for instance aligning multiple raw files and merging them before further processing, as well as outputting intermediate results for debugging. the gui profits from this scheme as well and can display textures while they are still on GPU and output the data to multiple targets, such as the main view and histograms.

features

documentation

packages

there are up-to-date packages (deb/rpm/pkgbuild) in the

opensuse build system

also there are nixos packages vkdt and vkdt-wayland which you can use to try out/run on any linux distro, for instance:

  nix-shell -p vkdt

build instructions

you should have checked out this repo recursively. if not, do

  git submodule init
  git submodule update

to grab the dependencies in the ext/ folder. you should then be able to simply run 'make' from the bin/ folder. for debug builds (which enable the vulkan validation layers, so you need to have them installed), try

  cd bin/
  make debug -j12

simply run make without the debug for a release build. note that the debug build includes some extra memory overhead and performs expensive checks and can thus be much slower. make sanitize is supported to switch on the address sanitizer. changes to the compile time configuration as well as the compiler toolchain can be set in config.mk. if you don't have that file yet, you can copy it from config.mk.defaults.

running

the binaries are put into the bin/ directory. if you want to run vkdt from anywhere, create a symlink such as /usr/local/bin/vkdt -> ~/vc/vkdt/bin/vkdt.

  cd bin/
  ./vkdt -d all /path/to/your/rawfile.raw

raw files will be assigned the bin/default-darkroom.i-raw processing graph. if you run the command line interface vkdt-cli, it will replace all display nodes by export nodes. there are also a few example config files in bin/examples/. note that you have to edit the filename in the example cfg to point to a file that actually exists on your system.

licence

our code is licenced under the 2-clause bsd licence (if not clearly marked otherwise in the respective source files). there are parts from other libraries that are licenced differently. in particular:

rawspeed: LGPLv2 imgui: MIT

and we may link to some others, too.

dependencies

  • vulkan, glslangValidator (libvulkan-dev, glslang-tools, or use the sdk)
  • glfw (libglfw3-dev and libglfw3, only use libglfw3-wayland if you're on wayland)
  • submodule imgui
  • for raw input support:
    • either rawspeed (depends on pugixml, stdc++, zlib, jpeg, libomp, build-depends on cmake, libomp-dev, optionally libexiv2)
    • or rawler (depends on rust toolchain which will manage their own dependencies)
  • libjpeg
  • video input: libavformat libavcodec (minimum version 6)
  • sound: libasound2
  • video output: ffmpeg binary
  • build: make, pkg-config, clang, rsync, sed

the full list of packages used to build the nightly appimages can be found in the workflow yaml file.

optional (configure in bin/config.mk):

  • freetype (libfreetype-dev libpng16-16) for nicer font rendering
  • exiv2 (libexiv2-dev) for raw metadata loading to assign noise profiles, only needed for rawspeed builds
  • asound (libasound2) for audio support in mlv raw video
  • ffmpeg (ffmpeg, libavformat-dev libavcodec-dev) for the video input module i-vid and the output module o-ffmpeg

you can also build without rawspeed or rawler if that is useful for you.

faq

  • can i load canon cr3 files?
    yes. the rawspeed submodule in this repository is now by default already using a branch that supports cr3 files. this may go away in the future though. the rawler backend supports cr3.

  • does it work with wayland?
    vkdt has been confirmed to run on wayland, using amd and nvidia hardware.

  • can i run my super long running kernel without timeout?
    if you're using your only gpu in the system, you'll need to run without xorg, straight from a tty console. this means you'll only be able to use the command line interface vkdt-cli. we force a timeout, too, but it's something like 16 minutes. let us know if you run into this..

  • can i build without display server?
    there is a cli target, i.e. you can try to run make cli to only generate the command line interface tools that do not depend on xorg or wayland or glfw.

  • i have multiple GPUs and vkdt picks the wrong one by default. what do i do?
    make sure the GPU you want to run has the HDMI/dp cable attached (or else you can only run vkdt-cli) on it. then run vkdt -d qvk and find a line such as

[qvk] dev 0: NVIDIA GeForce RTX 2070

and then place this printed name exactly as written there in your
~/.config/vkdt/config.rc in a line such as, in this example:

strqvk/device_name:NVIDIA GeForce RTX 2070

if you have several identical models in your system, you can use the device number vkdt assigns (an index starting at zero, again see the log output), and instead use

intqvk/device_id:0
  • can i use my pentablet to draw masks in vkdt?
    yes, but you need a specific version of glfw to support it. you can for instance clone https://github.com/hanatos/glfw, for instance to /home/you/vc/glfw, and then put the following in your custom bin/config.mk:
VKDT_GLFW_CFLAGS=-I/home/you/vc/glfw/include/
VKDT_GLFW_LDFLAGS=/home/you/vc/glfw/build/src/libglfw3.a
VKDT_USE_PENTABLET=1
export VKDT_USE_PENTABLET
  • are there system recommendations?
    vkdt needs a vulkan capable GPU. it will work better with floating point atomics, in particular shaderImageFloat32AtomicAdd. you can check this property for certain devices on this website. also, vkdt requires 4GB video ram (it may run with less, but this seems to be a number that is fun to work with). a fast ssd is desirable since disk io is often times a limiting factor, especially during thumbnail creation.

  • can i speed up rendering on my 2012 on-board GPU?
    you can set the level of detail (LOD) parameter in your ~/.config/vkdt/config.rc file: intgui/lod:1. set it to 2 to only render exactly at the resolution of your screen (will slow down when you zoom in), or to 3 and more to brute force downsample.

  • can i limit the frame rate to save power?
    there is the frame_limiter option in ~/.config/vkdt/config.rc for this. set intgui/frame_limiter:30 to have at most one redraw every 30 milliseconds. leave it at 0 to redraw as quickly as possible.

  • where can i ask for support?
    try #vkdt on oftc.net or ask on pixls.us.

vkdt's People

Contributors

aferrero2707 avatar afoht avatar blitzgneisserin avatar butcherg avatar cryptomilk avatar dydyamotya avatar hanatos avatar houz avatar paolodepetrillo avatar phweyland avatar valpackett avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

vkdt's Issues

Packaging

As explained on pixls.us, I already did a package of vkdt for fedora, and I'm now trying to create a flatpak one.
I open this issue to centralise a bit all issues I found along the way. Here is a simple list:

  • have a reverse DNS name, used in .desktop file and or flatpak, ... useful to have unique app id
  • have .desktop file with probably an icon
  • have an appdata file that market the app purpose, where to find doc, report issues, ...

I'm currently trying to get a clang compiler working with LLVM to b able to compile the project.

Canvas no longer updated when timelapse stops playing after last frame

Steps to reproduce:

  • open a timelapse in the darkroom
  • start playing it by clicking on "play" in the "animation" module in the "esoteric" tab
  • wait for it to stop playing after reaching the last frame
  • move the slider of a development module like "exposure"

Expected behavior:

  • the last frame is continued to be displayed and updated as the slider is moved

Observed behavior:

  • the last frame is continued to be displayed but not updated as the slider is moved

please push changes to public repo - edgecase

A couple months ago you gave me a custom .zip with some features for waveform, CIE diag, and color picker (smaller font so all 6 rows of cc24 would fit on UI), I don't really remember but I just got it working with an X-rite CC24 again and I'm afraid the fixes will get lost.

Just making a ticket here to get the ball rolling.

Tagging

vkdt is more than promising on the development side but I miss some tagging features on lighttable.

My workflow is the following:

  1. Import. Today I use memory card import with renaming but I would afford to do that externally and import files where they are.
  2. Culling, sorting on lighttable
  3. Development
  4. Tagging on lighttable
    At this stage I don't care of any of the "standardized" metadata. I only care about information I want to associate to my images:
    • author, description
    • place (country, region, city, location, ...)
    • people (private tags)
    • taxonomic classification if applicable (mainly plants and animals). I don't use the full hierarchy but I still could simplify a bit the structure. I use synonyms to keep local and scientific names.
    • usage. For example, I've a piwigo site and I keep here the piwigo destination of images. Later on, I can rework that collection and refresh the piwigo images/metadata.
  5. Export. I always export on disk and hardly bear the other built-in exporters. Part of that step is the choice of the metadata I want to associate with the images, depending on their destination:
    • exif or not.
    • geotagged or not
    • here I associate my personal metadata (tags) to any of the standardized ones (note here that tags set as category are never exported):
      iptc: that's the info that piwigo understands for example:
      - country, region, ...
      - description, ...
      - keywords (synonyms or not)
      xmp:
      - rights
      - hierarchical tags (if I want to share the full metadata).
  6. Outside the end to end workflow, I can easily find my images based on any of those tags to reuse or rework them.

Here is an example of metadata export settings:
$(CATEGORYn(tag)) is the way to pick up the right tag.
image

[design] alpha channel handling

So far, modules/nodes are assumed to have opaque alpha channel (== 1 for each pixel) but the pixel data is RGBalpha nonetheless. Alpha blending can happen using a blending node.

The limitations of this design are:

  • we should add a blending node for each masked node, which might create a cluttered node graph if a simple opacity is used for blending,
  • we always assume that the alpha channel will be defined outside of a node, but for a drawing node, alpha is part of the brushes and will be tempered with inside of the node.

In the spirit of a full-fledged Porter-Duff alpha compositing with minimal pipeline assumptions and general use case, here is what I propose:

  1. Each node will have as input :
    • RGBalpha, coming from the previous node, and premultiplied. Let's call that alpha "alpha". It should be forced to 1 (full opacity) in nodes performing raw processing filters and color adaptation.
    • an extra alpha input, taking an user-defined mask (parametric or drawn) for that node. Let's call that "mask alpha".
  2. Each node will process either :
    • the alpha-premultiplied RGB internally ? (as agreed on IRC, just force alpha = 1.0 if opacity needed)
    • the alpha-demultiplied RGB (assuming full opacity), then re-apply the alpha ?
    • probably DSP-style and usual raw processing filters will use the latter, but drawing and creative nodes might prefer the former. Maybe this could be user parametric.
    • painting nodes will add new RGBalpha layers that will be merged and flattened with input RGBalpha before the next step.
  3. Each node will output :
    • premultiplied RGBapha using both alpha AND mask alpha : { R * a * ma, G * a * ma, B * a * ma, a*ma }, meaning alpha <- alpha * mask alpha.
  4. Each node output gets blended, but then I'm lost between possibilities :
    • current darktable has a linear pipeline which always assumes that a module input is an opaque background and blends the module output as a transparent foreground.
    • proposed vkdt has a non-linear pipeline that can use branches, and each branch (if using current proposal) will inherit the transparency of its previous nodes until the branch is explicitly composed (and it has to) against some opaque background at some point, using a blending node.
    • what if "simple" users want the simple blending transparent node output OVER opaque node input ? Do we still force them to use a blending node ? Do we duplicate a simple blending in-node for easiness ?
    • use a generic blending.h API and expose it full in-node. The blending node will be made of only this, other nodes will inherit it too but with background forced to node input and foreground force to node output. (as agreed on IRC)
  5. Blending node will have :
    • 2 RGBalpha inputs:
      • background
      • foreground
    • 3 unary controls by input, as defined in Porter-Duff paper [1], used to post-process the background and foreground RGBa in the blending context :
      • darken/brighten d, such that RGBa = { r * d, g * d, b * d, a } (useful to use multiply blending mode in place of overlay, to overlay a texture on a base image. The foreground texture could be adjusted such that "grey" hits 1, then it would act as a bumping map under the multiplication).
      • dissolve e, such that RGBa = { r * e, g * e, d * e, a * e } (dissolve(F, 0.5) PLUS dissolve(B, 0.5) = physically-accurate double exposure).
      • opacity o such that RGBa = { r, g, b, MAX(a * o, 1) } (useful to force an opaque background)
      • the background and foreground unary factors could have a linked mode that ensures their sum{d, e, o}_(back, front} == {1 , 1, 1 }.
    • a blending mode:
      • Porter-Duff classics (OVER == NORMAL, PLUS, IN, OUT, ATOP, XOR),
      • algebraic (MULTIPLY, DIVIDE, SUBTRACT, ADD == PLUS),
      • statistics (GEOMETRIC MEAN, HARMONIC MEAN),
      • and perhaps display-referred bullshit (SOFT/LINEAR/HARD-LIGHT and such).
    • 1 RGBalpha output, perhaps with an opacity override control ?
    • an option to choose between correlated, and uncorellated geometry to properly switch between alpha blending models (or try to implement [2] which appears to provide a more general approach). See [3] to get a sense of the implications.

Tell me what you think.

[1] http://graphics.pixar.com/library/Compositing/paper.pdf
[2] http://graphics.pixar.com/library/DeepCompositing/paper.pdf
[3] https://blender.stackexchange.com/a/67371

Pressing Esc to return to lighttable only works when canvas has focus

Steps to reproduce:

  • open an image in darkroom
  • click somewhere in the right area with histogram and slider, press Esc without clicking anywhere else before
  • nothing happens
  • click somewhere in the image canvas, press Esc without clicking anywhere else before
  • UI returns to lighttable

Bug: thumbnail selection is lighttable makes a weird flashing effect

Hi,

since a few commit, in lighttable, when selecting a picture (via simple click), the highlight done around the thumbnails double flashes in a weird fashion ... as if the thumbnail was very quickly selected+unselected+selected again.

Not sure this is expected

Thanks
GLLM

Inpaint module produces black smudges when used at the image border

steps to reproduce:

  • open an image in the darkroom
  • apply the "inpaint.pst" preset in the "esoteric" tab
  • uncollapse the new "draw inpaint" module in the "tweak all" tab and click "draw start"
  • draw a stroke that touches the image edge (preferreably in a bright area like the sky)

observed behaviour:

  • pixels close to the image edge are fading to black

expected behaviour:

  • pixels close to the image edge get colours of surrounding pixels (e.g. the sky)

Core dump on qvk_create_swapchain()

I'm just checking out what vkdt does, tried to run it on my archlinux install after building it from source. It crashes with this error:

[gui] monitor [0] LGD 0x0540 at 0 0
[gui] monitor [1] DEL DELL U2515H at 1920 0
[gui] vk extension required by GLFW:
[gui]   VK_KHR_surface
[gui]   VK_KHR_wayland_surface
MESA-INTEL: warning: Performance support disabled, consider sysctl dev.i915.perf_stream_paranoid=0

vkdt: qvk/qvk.c:227: VkResult qvk_create_swapchain(): Assertion `qvk.num_swap_chain_images < QVK_MAX_SWAPCHAIN_IMAGES' failed.
[1]    144589 abort (core dumped)  ./vkdt

The GPU is Intel HD graphics 630, this is the output from vulkaninfo: https://paste.sr.ht/~martijnbraam/69129017397a1c30779823a01a60a6bb41763a05

[features] local edits / nodes / multi-instanciation / pipe

Even if I know it can be a burden, I don't think we can avoid local edits and masking. If only for one reason, it would be because most softs don't blend in linear space as they should and don't make correct use of associated alpha blending.

Use cases involving local edits can be :

  • outdoors HDR scenes : separating ground and sky, to perform selective luminance/exposure and WB corrections,
  • indoors HDR scenes : separating windows/outdoors parts and indoors one, to do the same
  • portrait/fashion/macro : separating subject from background for selective colour, sharpness, luminance adjustments

The way current dt is doing it is problematic though, because drawn masks are blended from the input/output of the current module, and the parametric blending depends on what is done earlier in the pipe. Set up a luminance parametric mask for color edits, change the exposure earlier, and your mask is lost. Also, masks need to be defined for each filter, possibly duplicated (even if the last raster mask option is nice).

Some filters make sense at a general level (colour input/output transforms, camera exposure compensation, denoising, LUTs, reconstructive deblurring), some make sense at a local level (healing/cloning/in-painting), some make sense at both (lumiance/colour adjustments, blurring/local contrast). For this reason, having reorderable and multi-instanciable filters is a necessity.

So, I think it should be better to have a first round of global edits on the raw file, for operations that make sense at the scene level (relighting, spectrum modification), at the camera level (white balance, exposure compensation, demosaic, colour profile, lens profile), and the signal processing/reconstruction (healing, inpainting). At this step, I think the use of masking/blending equals tempering bad filters that break, so it should not happen ideally.

Then, a mask manager would split the global image into zones, defined by drawn and/or parametric masks or by auto/assisted image segmentation. Each zone would be loaded with the required filters, upon request, and possibly processed in parallel.

After that, the zones are blended with a proper associated alpha, and go to the final step of global edits (HDR/filmic curves, gamut mapping, output colour profile, film emulation LUTs, possibly brushed dodging & burning, etc.).

dt-pipe

In this architecture, parallel zones are loaded with filters stacked in serie, in parallel, so we limit the dependency of masking parameters to the initial step of global edits, and we don't need to blend after every filter. I don't think parametric blending will be necessary at all for these filters (inside zones), if algos behave properly (just a regular opacity, maybe some blendings like multiply or average).

I don't believe in several pipes (preview/full-size) because some operations are not scale-invariant and the branching it involves makes a lot of trouble to maintain, regarding scaling of masks coordinates and such. I would just use 2 intermediate full-size raster caches, to store the image state before and after the zones splitting, and maybe a third right after the demosaicing and denoising step, because these can be heavy. Or maybe just abuse the RAM and cache every filter's output as long as there is still space (8 GB = 20 copies of a 24 Mpx image in RGBa single-precision float).

FLT_MAX undeclared identifier (Arch)

Building master on ArchLinux fails with error:

gui/gui.c:181:25: error: use of undeclared identifier 'FLT_MAX'
  vkdt.view_look_at_x = FLT_MAX;
                        ^
gui/gui.c:182:25: error: use of undeclared identifier 'FLT_MAX'
  vkdt.view_look_at_y = FLT_MAX;
                        ^

Adding #include <float.h> in src/gui/gui.c allowed build to succeed.

vulkan validation error with vulkan 1.2.198 on AMD vulkan-pro driver

Using vkdt to process a RAW file on current master (e758f1b) results in the following vulkan validation error:

[qvk] validation layer: Validation Error: [ VUID-VkComputePipelineCreateInfo-layout-00703 ] Object 0: handle = 0x22ff760000000f24, type = VK_OBJECT_TYPE_SHADER_MODULE; Object 1: handle = 0x4e1bb10000000f23, type = VK_OBJECT_TYPE_PIPELINE_LAYOUT; | MessageID = 0xe63c2d8b | Push constant is used in VK_SHADER_STAGE_COMPUTE_BIT of VkShaderModule 0x22ff760000000f24[]. But VkPipelineLayout 0x4e1bb10000000f23[] doesn't set VK_SHADER_STAGE_COMPUTE_BIT. The Vulkan spec states: layout must be consistent with the layout of the compute shader specified in stage (https://vulkan.lunarg.com/doc/view/1.2.198.0/linux/1.2-extensions/vkspec.html#VUID-VkComputePipelineCreateInfo-layout-00703)
vkdt: qvk/qvk.c:96: VkBool32 vk_debug_callback(VkDebugUtilsMessageSeverityFlagBitsEXT, VkDebugUtilsMessageTypeFlagsEXT, const VkDebugUtilsMessengerCallbackDataEXT *, void *): Assertion `0' failed.

processing JPGs does not show the same behaviour, so it must be an RAW-only operation which causes the issue.

associated gdb backtrace: vkdt-bt-236671.txt

vulkaninfo output: vulkaninfo.txt

I'm running ubuntu 20.04.3 with amdgpu 21.40.1.40501 (installed via amdgpu-install --opencl=rocr --vulkan=pro).

Do not apply llap & filmcurv when opening JPEG files

Hello @hanatos,

I open this issue to keep track of my proposition, that I understand you do acknowledge to not apply RAW-dedicated modules: llap & filmcurv when opening JPEG files ... so that it is set out-of-the-box rather than needing the user modify the default-darkroom.i-jpg and default.i-jpg files.

The target is to get:

  • expo: 1.0
  • light: 1.0
  • contrast: 1.0
  • clarity: 0.0
  • shadows: 1.0
  • highlight: 1.0
  • when opening JPEG files, in both lighttable and darkroom.

Your proposed solution:
image

See: https://discuss.pixls.us/t/vkdt-dev-diary-pt2/28540/119?u=gllm

Thanks a lot,

GLLM

[implementation] Painting for Dodging & Burning

This is an differed answer on the IRC question

Goals of D&B

There are 3 main goals to dodging and burning:

  1. enhancing global contrast, usually to have better subject/background separation
  2. selectively adding more local contrast, usually on face features and body shapes, to increase depth and give a 3D feel, (this falls back to a make-up technic known as "contouring")
  3. selectively removing local contrast, usually in fashion/beauty shots, for examples eyes bags, ill-placed shadows, stretch marks, but keeping the fine skin texture untouched (example : use case, result).

Notice that the first goal is more easily covered by using an exposure compensation on areas defined by a guided filter mask (drawn or parametric), so this won't be covered here. Let's focus on the 2 others, that need actual painting.

D&B : Technic one, the bad

Create a grey layer, blend it in softlight or overlay mode, paint in white over areas to dodge, and black over areas to burn. Dodging and burning tools in PS/Gimp work around the same idea, just abstracting the initial grey layer.

https://youtu.be/4qsLJArkAe4?t=65

Why it's good:

  • it's intuitive
  • its fun

Why it's bad :

  • it's not non-destructive, meaning that if you need to increase the effect, you need to paint again on your mask, but if you need to decrease the effect, you might need to start over from the beginning. TL; DR : it's not very efficient.
  • mixing white light on pixels creates desaturation and hue shift because of the Abney effect. Even if it is physically accurate, it is not intuitive from an editing perspective (where you want to separate hue/lightness/saturation edits).
  • overlay and softlight blending mode expect display-referred encodings because they dodge and burn around 0.5 (assumed to be middle grey), which is quite an assumption for HDR scene-linear imagery.

D&B Technic : the ugly

Create "lightness up" and "lightness down" (gamma-like) adjustment (tone) curves, and paint over their opacity masks to uncover the desired areas.

https://youtu.be/ZeEXY2kIpVo?t=686

Why it's good :

  • now it's non-destructive : you can control the opacity of dodging and burning masks separately, but also edit the slope of the curves later

Why it's bad :

  • you need 2 steps (dodging, then burning), so it's probably less intuitive (is it though ?)
  • tone curves affect saturation in non-intuitive ways
  • it still works in display-referred encoding, where strokes blending fails and edges effects arise.

D&B Technic : the good

Same logic as before, but instead of tone curves, create an "exposure up" and "exposure down" layers, simply using a linear scaling of the scene-linear RGB code values. Then, paint over their opacity masks to uncover desired areas.

Why it's good :

  • since we use light transport models, with a simple exposure compensation, we don't get any weird colour shift,
  • we don't make any assumption on the mid-grey value and can work in a fully unbounded setup, between ]0 ; + infinity[
  • it's non-destructive, because the exposure compensations can be changed after painting,
  • strokes blending/merging will be fast and physically-accurate in scene-linear spaces.

Tools of the trade

You mainly need round soft brushes painting pure white and pure black with some opacity.

But… Photoshop has another really nice feature to its brushes : the flow.

https://youtu.be/-4qeu-TZLp8?t=200

The flow is basically simulating how wet ink would add up in a continuous stroke, but also helps blending seamlessly different strokes together, especially using hard brushes. See how it affects dodging and burning:

https://youtu.be/-4qeu-TZLp8?t=562

For implementations of this, maybe look at https://github.com/briend/libmypaint (Brien has been working on spectral color mixing for digital painting, I believe he has serious physically-accurate brushes in there).

Maybe this is close to what we want here:

https://github.com/briend/libmypaint/blob/d52e6bcd158fcb19cd62343289b7b1ba716be633/brushmodes.c#L291

Preview drifts to the right on keyboard/mouse events

Whenever I move my mouse or press a key, the preview moves to the right. It eventually drifts completely off the window and can only be recovered when the view randomly resets. Increasing the LOD reduces the distance that the preview moves per input event. Bisecting the commits reveals the issue started with ddc2379.

Zoom resets on mouse move

When I open an image, I can zoom in and out with the mouse wheel. But once I move the mouse even one pixel, zoom resets back to the initial size.

Ubuntu 21.04, Xorg session, GCC 10.3.0, GLFW 3.3.2-1

Crash (attempting to open non-existent folder?)

I just had a crash. Here is the console output:

[gui] monitor [0] XWAYLAND0 at 0 0
[gui] vk extension required by GLFW:
[gui]   VK_KHR_surface
[gui]   VK_KHR_xcb_surface
[gui] no joysticks found
[gui] no display profile file display.XWAYLAND0, using sRGB!
[gui] no display profile file display.XWAYLAND0, using sRGB!
vkdt: Datei oder Verzeichnis nicht gefunden.
ptrace: Die Operation ist nicht erlaubt.
/opt/vkdt/6378: Datei oder Verzeichnis nicht gefunden.
Warning: 'set logging off', an alias for the command 'set logging enabled', is deprecated.
Use 'set logging enabled off'.

Warning: 'set logging on', an alias for the command 'set logging enabled', is deprecated.
Use 'set logging enabled on'.

vkdt: Datei oder Verzeichnis nicht gefunden.
[New LWP 6379]
[New LWP 6380]
[New LWP 6381]
[New LWP 6382]
[New LWP 6383]
[New LWP 6384]
[New LWP 6385]
[New LWP 6386]
[New LWP 6387]
[New LWP 6388]
[New LWP 6389]
[New LWP 6390]
[New LWP 6397]
[New LWP 6402]
[New LWP 6403]
[New LWP 6404]
[New LWP 6405]
[New LWP 6406]
[New LWP 6407]
[New LWP 6408]
[New LWP 6409]
[New LWP 6410]
[New LWP 6411]
[New LWP 6412]
[New LWP 6413]
[New LWP 6414]
[New LWP 6415]
[New LWP 6416]
[New LWP 6417]
[New LWP 6418]
[New LWP 6419]
[New LWP 6420]
[New LWP 6421]
[New LWP 6422]
[New LWP 6423]
[New LWP 6424]

This GDB supports auto-downloading debuginfo from the following URLs:
https://debuginfod.archlinux.org
Enable debuginfod for this session? (y or [n]) [answered N; input not from terminal]
Debuginfod has been disabled.
To make this setting permanent, add 'set debuginfod enabled off' to .gdbinit.
[Thread debugging using libthread_db enabled]
Using host libthread_db library "/usr/lib/libthread_db.so.1".
0x00007ff1ae1020bf in poll () from /usr/lib/libc.so.6
Warning: 'set logging off', an alias for the command 'set logging enabled', is deprecated.
Use 'set logging enabled off'.

Warning: 'set logging on', an alias for the command 'set logging enabled', is deprecated.
Use 'set logging enabled on'.

backtrace written to /tmp/vkdt-bt-6378.txt
recovery data written to /tmp/vkdt-crash-recovery.*

It might have been that the last opened folder (that is loaded on opening) does not exist anymore? (I'm on 6b4da4d, latest master as of now).

vkdt-bt-6378.txt

Sliders effect applied too strongly

Hello @hanatos,

When using any slider to edit pictures, i always get the feeling that the effect (exposure, shadows, ... all of them) is applied very strongly even with a very tiny bit of wheeling/scrolling.

Is that on purpose ?

I understand effects may actually be applied linearly but I always overshoot PP and must everytime resort to play a second time with the mouse at pixel level to get the expected amount of effect.

Could effects be applied less linearly but more exponentially, so that the user may apply subtle amount of effect without struggle.

Note: I use vkdt on JPEG ;-) if this is of value

Thanks a lot,
GLLM

[features] colour control nodes worth considering

This is a list of thoughts about robust, all-purpose and usefull way to control color, worth considering adding:

Note : [xxx] denote vectors processed element-wise

1. Fulcrum-based contrast

  • By-the-book : [RGB] = fulcrum * ( [RGB] / fulcrum )^ contrast
  • How it works : the fulcrum is usually 18 % grey, but not necessarily. It does not assume bounds at 100% and pushes luminance in a perceptually uniform way around the fulcrum.
  • Problem : it adds saturation in low-lights, sometimes in an excessive fashion.
  • Work-around : DaVinci Resolve seems to do a weighted average between the by-the-book way in RGB, and Y = fulcrum * ( Y / fulcrum )^contrast in xyY (converted back to RGB). The weight parameter is user-defined.

2. Vibrance/saturation

  • By-the-book[RGB] = Y + saturation * ( [RGB] - Y ), Y from XYZ space
  • How it works : pushes away colors channels from the luminance.
  • Problem : sometimes you want to specifically resaturate low-saturation areas and not mess-up the properly saturated ones.
  • Work-around : [RGB]= Y + saturation * ( [RGB] - Y )^vibrance

3. ASC CDL

  • By-the-book : [RGB] = ( [slope] * [RGB]} + [offset] )^{power}, slope, offset and power being set for each channel
  • How it works : defines 3 1D transfer functions allowing to set RGB channels separately, to remove parasite colour casts or define a look (colour-grading)
  • Problem : mostly UI : 
    • does not compensate for luminance changes, so luminance and saturation are affected at once,
    • the power parameters are inversed (increasing them in one colour actually boosts the complimentary colour),
    • you probably want an HSV/HSL/HSY interface (derived from xyY polar coordinates), but how do you translate an HSY vector to RGB gains ? (we are not converting from pixel to pixel, but from gain to gain)
  • Work around : (see darktable colour balance)
    • (nasty) pretend the HSY input params are vectors, convert them to RGB, then normalise them with {RGB} - Y and pretend these are RGB params (but this assumes the RGB params are bounded, which they are not)
    • ???

4. Colour zones

Not yet defined.

Something to allow sective hue/saturation/lightness shift over a specific hue/saturation/lightness range. The problem is the inputs need to be in a perceptually even space (IPT, JzAzBz, Lab), otherwise hue shifts will be steeper in blues and softer in reds. But these spaces will irremediably fuck evey attempt of blending/masking/feathering between the corrected range and the rest of the spectrum : color noise and halos.

One possible way would be to apply the transfer functions (for hue/saturation shifting) in IPT/JzAzBz/Lab, then convert the result to RGB, and blend the output with the input using a gaussian/guided/whatever filter centered on the range to edit, but in linearly scaled RGB.

Another option could be to upsample to spectral domain, for example 8 primaries, drive a factor (gain) for each primary, and see what happens.

Bug: 'delete image(s)' does not reset when changing selection

Test case:

  • open a folder
  • select a picture
  • click on delete image(s) --> the confirmation button appears on its right
  • change your mind, do not proceed with deletion
  • switch to another file and select it

--> the deletion confirmation message is still there, as if it was on this 2nd file that I performed the click on 'delete images(s).

Expectation: the confirmation message disappears if the selection changes

Video of the issue

Video_2022-12-17_17-02-23.mp4

Thanks
GLLM

Feature request : zoom % indicator in darkroom

Hi Hanatos,

when zooming on my photos in DR, I always wonder how much I'm zoomed in : am I at 100% (which I usually want) or more (sometimes OK, sometimes not) ?

Do you think adding a zoom text indicator (xyz%) somewhere in DR could be feasible ?

And if this is too much UX/UI for the moment, dont hesitate to say so :-)

Many thanks !
GLLM

vkdt hangs on trying to open a non existing (deleted) image from tag collection

to reproduce:

  • tag some images (with or without *cfg) as "x"
  • delete one of these images
  • go to "x" collection.
  • try to open the deleted image

In darkroom_enter(), changing char graph_cfg[PATH_MAX+100]; to char graph_cfg[PATH_MAX+100] = {0}; avoids hanging but the issue may be deeper.
dt_db_load_directory() should check if the image exists or/and when deleting the image, the corresponding symlinks should be removed.

Related somehow, deleting an image in tag collection doesn't delete the image (only the symlink).

EDIT: That's a reason to be able to get the tags per image...

Make it easier for end users to use fast temporal reprojection antialiasing without motion vectors

Hi @hanatos :)

Your paper fast temporal reprojection without motion vectors is very interesting and solve a real problem, DLSS, XeSS and FSR 2.0 all require motion vectors which most games don't provide.
The only mainstream upscaler without motion vectors is AMD FSR 1.0, which is not temporal. Yet FSR 1.0 has become a huge sucess and is e.g. supported by most video games emulators (Yuzu (switch), RPCS3 (PS3), etc). This enables us to improve the visual quality of low polygon, old games (AKA most games that mankind has produced) and even improve the quality of recent games (e.g. Zelda Breath of the wild).
As such AMD FSR has been revolutionnary (in practice, not necessarily for scholars).
Your algorithm is the only motion vector free competitor to FSR 1.0 to my knowledge and, being temporal based, has a theoretical advantage.
Hence your algorithm, if contextually superior to FSR 1.0 would be revolutionnary for gamers arround the world !
If you think it could be competitive then the next step enabling it to become quickly mainstream in use, would be to either:
Implement it via Reshade/sweetfx https://github.com/crosire/reshade, hence enabling it for any game out of the box
or by integrating it in a console emulator such as Yuzu or RPCS3. It would BTW enable easy comparison with FSR 1.0. It might be BTW (no idea) that FSR 1.0 would be complementary and could be combined to your algorithm ? (since FSR is computationally very cheap and spatially based).
If you do this and it works, it will become instantaneously a huge success :)

work package list towards production use

the following lists and prioritises a few key features that need work to make vkdt useful for production use.

this is currently work in progress and certainly not in any good order yet or complete. shout if you see changes needed.

DAM/lighttable

  • ✔️ history support for edits (.cfg)
    • potential addition: automatic graph diff/insertion of ab module to compare two history states
  • history support for lighttable (vkdt.db)
  • ✔️ duplicate images in lt mode
  • filtering and sorting ui needs a lot of ui love
    • maybe add sort by last edit/edited at all yet
  • different presentation modes if useful?
    • more compact image only list
    • compare/select
  • tags need some work (can assign and collect but not view the list of tags of an image)
  • ✔️ "files" view with basic copy-from-cf/sd-card functionality (so it'd be "files" "lighttable" "darkroom" "node editor" from generic to special)

image processing/darkroom

  • ✔️ quake to replace ancient space invaders game
  • module versioning (for legacy_params as in dt or for versioned params and main.comp)
  • related to that last point: backward compatibility guarantee (starting at version 1.0, say)
  • ✔️ full blown 2D node editor
  • port essential processing modules
    • ✔️ colour balance: see grade module
    • ✔️ retouch see wavelet module
    • ✔️ tone equaliser zones module
    • diffuse/sharpen?
    • grid/guides overlay for crop/rotate/perspective (can use dt_draw postscript interpreter)
    • ✔️ postcard frame
    • ✔️ automatic chromatic aberration correction/defringing ca module
    • ✔️ contrast equaliser (eq wavelet with manual edge weight, threshold, and scale curves)
  • super resolution/aligned hdr bracket
  • ✔️ rate/label in darkroom mode too

other/general

  • scripting. lua or pybind11?
  • metadata (via scripting language?) is currently pretty much not supported. minimal metadata we get from rawspeed.
  • ui work
  • ✔️ export custom keybindings to make them persistent
  • ✔️ context sensitive ui help for keyboard + gamepad + mouse operations
  • ✔️ unify where is stuff (display profiles, default pipelines, fav ui, presets, etc): in ${HOME}/.config/vkdt or global bin/
  • ✔️ job scheduler for export in background (as well as copying in files view)

assert fail QVK_MAX_SWAPCHAIN_IMAGES on Wayland

vkGetSwapchainImagesKHR(qvk.device, qvk.swap_chain, &qvk.num_swap_chain_images, NULL);
assert(qvk.num_swap_chain_images < QVK_MAX_SWAPCHAIN_IMAGES);
vkGetSwapchainImagesKHR(qvk.device, qvk.swap_chain, &qvk.num_swap_chain_images, qvk.swap_chain_images);

I was able to get it working changing < to <=. my actual values were 4 <= 4

simple pipelines don't work

for instance

module:i-pfm:main
module:display:main
connect:i-pfm:main:output:display:main:input

has problems initialising. inserting a simple thing (exposure for instance) solves the issue.

cmake error when building on NixOS: The source dir /build/source doesn't have CMakeLists.txt

I'm trying to package vkdt for NixOS, which has nothing in the standard linux path.

When building, I get the following:

cmake flags: 
 -DCMAKE_FIND_USE_SYSTEM_PACKAGE_REGISTRY=OFF 
 -DCMAKE_FIND_USE_PACKAGE_REGISTRY=OFF 
 -DCMAKE_EXPORT_NO_PACKAGE_REGISTRY=ON 
 -DCMAKE_BUILD_TYPE=Release -DBUILD_TESTING=OFF 
 -DCMAKE_INSTALL_LOCALEDIR=/nix/store/cq585gb6wn8b2khl3jm3lbw0iinqpjk1-vkdt-8d03c97b128c2a848f9baf219269dee9eb010748/share/locale 
 -DCMAKE_INSTALL_LIBEXECDIR=/nix/store/cq585gb6wn8b2khl3jm3lbw0iinqpjk1-vkdt-8d03c97b128c2a848f9baf219269dee9eb010748/libexec 
 -DCMAKE_INSTALL_LIBDIR=/nix/store/cq585gb6wn8b2khl3jm3lbw0iinqpjk1-vkdt-8d03c97b128c2a848f9baf219269dee9eb010748/lib 
 -DCMAKE_INSTALL_DOCDIR=/nix/store/cq585gb6wn8b2khl3jm3lbw0iinqpjk1-vkdt-8d03c97b128c2a848f9baf219269dee9eb010748/share/doc/vkdt 
 -DCMAKE_INSTALL_INFODIR=/nix/store/cq585gb6wn8b2khl3jm3lbw0iinqpjk1-vkdt-8d03c97b128c2a848f9baf219269dee9eb010748/share/info 
 -DCMAKE_INSTALL_MANDIR=/nix/store/cq585gb6wn8b2khl3jm3lbw0iinqpjk1-vkdt-8d03c97b128c2a848f9baf219269dee9eb010748/share/man 
 -DCMAKE_INSTALL_OLDINCLUDEDIR=/nix/store/cq585gb6wn8b2khl3jm3lbw0iinqpjk1-vkdt-8d03c97b128c2a848f9baf219269dee9eb010748/include 
 -DCMAKE_INSTALL_INCLUDEDIR=/nix/store/cq585gb6wn8b2khl3jm3lbw0iinqpjk1-vkdt-8d03c97b128c2a848f9baf219269dee9eb010748/include 
 -DCMAKE_INSTALL_SBINDIR=/nix/store/cq585gb6wn8b2khl3jm3lbw0iinqpjk1-vkdt-8d03c97b128c2a848f9baf219269dee9eb010748/sbin 
 -DCMAKE_INSTALL_BINDIR=/nix/store/cq585gb6wn8b2khl3jm3lbw0iinqpjk1-vkdt-8d03c97b128c2a848f9baf219269dee9eb010748/bin 
 -DCMAKE_INSTALL_NAME_DIR=/nix/store/cq585gb6wn8b2khl3jm3lbw0iinqpjk1-vkdt-8d03c97b128c2a848f9baf219269dee9eb010748/lib 
 -DCMAKE_POLICY_DEFAULT_CMP0025=NEW 
 -DCMAKE_OSX_SYSROOT= 
 -DCMAKE_FIND_FRAMEWORK=LAST 
 -DCMAKE_STRIP=/nix/store/n95cd4q1dqzdvsiy1hmrkx9shwi3n4sh-gcc-wrapper-11.3.0/bin/strip 
 -DCMAKE_RANLIB=/nix/store/n95cd4q1dqzdvsiy1hmrkx9shwi3n4sh-gcc-wrapper-11.3.0/bin/ranlib 
 -DCMAKE_AR=/nix/store/n95cd4q1dqzdvsiy1hmrkx9shwi3n4sh-gcc-wrapper-11.3.0/bin/ar 
 -DCMAKE_C_COMPILER=gcc 
 -DCMAKE_CXX_COMPILER=g++ 
 -DCMAKE_INSTALL_PREFIX=/nix/store/cq585gb6wn8b2khl3jm3lbw0iinqpjk1-vkdt-8d03c97b128c2a848f9baf219269dee9eb010748
CMake Warning:
  Ignoring extra path from command line:

   ".."


CMake Error: The source directory "/build/source" does not appear to contain CMakeLists.txt.

Computational graph

Is the computational graph (graph and nodes) general enough that
could be repurposed for say deep learning?

Or would it be better to start from scratch?

Feature: in darkroom, add the ability to reset the history stack of the image being processed !

Hi,

yet another feature request ! :-o
In darkroom, would it be possible to have the ability to quickly reset the stack of the image being processed ?
I can do it in lighttable, but when, in DR, I messed things up in my pipeline, I need to go back in LT, make sure I'm on the right image and then use the selected images > reset history stack.

A one click action from within DR would be neat, whether in the 'Tweak all' or in the 'Pipeline config' tabs !

Thanks
GLLM

Thumbnails are not loading

Thumbnails are not loading. If I open a directory that I have not opened before, I just see a bunch of "fly" icons (or "bomb" icons in the latest version), but the thumbnails are not showing up. If I click on an image, it opens up fine in darkroom mode. After that, a thumbnail for this particular image will appear in lighttable mode too. If I open a directory that I opened in vkdt before, thumbnails show up fine.

I tried bisecting, and the first commit where the problem appears first seems to be 5a9ddaa

I tried running with -d all and in the console I see repeated messages like (this is with the latest commit 5739d25 )

[ERR] module dn2 has no connectors!
[gui] glfwGetVersionString() : 3.3.6 X11 GLX EGL OSMesa clock_gettime evdev shared
[gui] monitor [0] DisplayPort-0 at 0 0
[gui] vk extension required by GLFW:
[gui]   VK_KHR_surface
[gui]   VK_KHR_xcb_surface
[qvk] dev 0: vendorid 0x1002
[qvk] dev 0: AMD RADV RENOIR
[qvk] max number of allocations -1
[qvk] max image allocation size 16384 x 16384
[qvk] max uniform buffer range 4294967295
[qvk] dev 1: vendorid 0x10005
[qvk] dev 1: llvmpipe (LLVM 13.0.1, 256 bits)
[qvk] max number of allocations -1
[qvk] max image allocation size 16384 x 16384
[qvk] max uniform buffer range 65536
[ERR] device does not support requested feature shaderSampledImageArrayDynamicIndexing, trying anyways
[ERR] device does not support requested feature shaderStorageImageArrayDynamicIndexing, trying anyways
[ERR] device does not support requested feature inheritedQueries, trying anyways
[qvk] picked device 0 without ray tracing support
[qvk] num queue families: 2
[qvk] available surface formats:
[qvk] B8G8R8A8_SRGB
[qvk] B8G8R8A8_UNORM
[qvk] colour space: 0
[gui] no joysticks found
[gui] no display profile file display.DisplayPort-0, using sRGB!
[gui] no display profile file display.DisplayPort-0, using sRGB!
[qvk] available surface formats:
[qvk] B8G8R8A8_SRGB
[qvk] B8G8R8A8_UNORM
[qvk] colour space: 0
[db] allocating 1024.0 MB for thumbnails
[perf] upload source total:	   0.014 ms
[perf] upload for raytrace:	   0.229 ms
[mem] images : peak rss 0.00390625 MB vmsize 0.00390625 MB
[mem] staging: peak rss 0.000244141 MB vmsize 0.000244141 MB
[perf] record cmd buffer:	   0.329 ms
[perf] i-bc1    main    :	   0.004 ms
[perf] total time:	   0.004 ms
[perf] [thm] ran graph in   0ms
[perf] upload source total:	   0.011 ms
[perf] upload for raytrace:	   0.007 ms
[mem] images : peak rss 0.00390625 MB vmsize 0.00390625 MB
[mem] staging: peak rss 0.000244141 MB vmsize 0.000244141 MB
[perf] record cmd buffer:	   0.051 ms
[perf] i-bc1    main    :	   0.003 ms
[perf] total time:	   0.003 ms
[perf] [thm] ran graph in   0ms
[ERR] could not open directory '(null)'!
[ERR] [thm] no images in list!
[qvk] available surface formats:
[qvk] B8G8R8A8_SRGB
[qvk] B8G8R8A8_UNORM
[qvk] colour space: 0
[perf] upload source total:	   0.053 ms
[perf] upload for raytrace:	   0.024 ms
[mem] images : peak rss 0.00390625 MB vmsize 0.00390625 MB
[mem] staging: peak rss 0.000244141 MB vmsize 0.000244141 MB
[perf] record cmd buffer:	   0.180 ms
[perf] i-bc1    main    :	   0.015 ms
[perf] total time:	   0.015 ms
[perf] [thm] ran graph in   0ms
[perf] upload source total:	   0.046 ms
[perf] upload for raytrace:	   0.015 ms
[mem] images : peak rss 0.00390625 MB vmsize 0.00390625 MB
[mem] staging: peak rss 0.000244141 MB vmsize 0.000244141 MB
[perf] record cmd buffer:	   0.136 ms
[perf] i-bc1    main    :	   0.013 ms
[perf] total time:	   0.013 ms
[perf] [thm] ran graph in   0ms

and then repeating similar messages.

Git master doesn't load

Feel free to shoo me away for trying alpha software, but...

./vkdt /whatever/path/to/photos
[pipe] [global init] cannot open modules directory!
[gui] monitor [0] eDP-1 at 0 0
[gui] vk extension required by GLFW:
[gui]   VK_KHR_surface
[gui]   VK_KHR_xcb_surface
INTEL-MESA: warning: Performance support disabled, consider sysctl dev.i915.perf_stream_paranoid=0

[gui] no display profile file display.eDP-1, using sRGB!
[gui] no display profile file display.eDP-1, using sRGB!
vkdt: ../ext/imgui/imgui_draw.cpp:1909: ImFont* ImFontAtlas::AddFontFromFileTTF(const char*, float, const ImFontConfig*, const ImWchar*): Assertion `(0) && "Could not load font file!"' failed.
fish: './vkdt' terminated by signal SIGABRT (Abort)

I'm on commit ffd4654 here.

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.