photoshell / rawkit

ctypes based libraw bindings

rawkit

rawkit (pronounced rocket) is a ctypes-based LibRaw binding for Python inspired by the Wand API.

from rawkit.raw import Raw
from rawkit.options import WhiteBalance

with Raw(filename='some/raw/image.CR2') as raw:
  raw.options.white_balance = WhiteBalance(camera=False, auto=True)
  raw.save(filename='some/destination/image.ppm')

for more info, see the docs

rawkit's Issues

Make docstrings more readable

RTD supports napolean for parsing Google or Numpy docstring standards. These are much better than the restructred text garbage. Maybe we shoud switch to that?

Support python 2

ImportError: No module named bindings

Hi,

installed rawkit via pip on my Ubuntu 15.04 (stock Python 2.7.9) and libraw10. I get the following error:

>>> from rawkit.raw import Raw
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/usr/local/lib/python2.7/dist-packages/rawkit/raw.py", line 8, in <module>
    from libraw.bindings import LibRaw
ImportError: No module named bindings

Executing

libraw.bindings import LibRaw

works fine.

Same happens on a scientific linux (python 2.7.10).

Switch the docs theme to alabaster

It looks the same (it's a fork of the Flask theme), but it actually alignes codeblocks properly (and has a built in GitHub button, I think?):

https://github.com/bitprophet/alabaster

Add argtypes to LibRaw functions

If we're specifying return types, we might as well specify arg types too for LibRaw functions.

This would protect against segfaulting in the middle of raw development due to the wrong type being passed in and would throw a Python error (which users could gracefully recover from) instead.

Make tuple for supported export filetypes

We should make an enum-like tuple for our exported filetypes (tiff/ppm) like we do with options instead of just passing in an arbitrary string and then checking if it's in the list of supported types. This would feel cleaner, and more consistant.

Document LibRaw errors

Document LibRaw error work.

This comes in two stages:

• Document the Libraw error classes (dfe197c)
• Document existing methods that call LibRaw with what sort of errors they can return and when / why

Add option to re-unpack image or manually invalidate unpacked data if certain options are changed

Options like rotation can affect data unpacking. This means that if we do:

with Raw(filename='somefile.raw') as raw:
    raw.save(filename='somefile.ppm')
    raw.options.rotation = 180
    raw.save(filename='somefile_rotated.ppm')

It won't work for some filetypes / cameras (because data is already unpacked and cached after the first unpack call).

We should either invalidate the unpacked data if one of these options is used and unpack again, or have an option to force unpacking on the unpack method and any method that calls it (eg. save).

A discussion should happen about how this is done.

Support using Raw not as a context manager

Add make_buffer method

Add ability to get color data from Raw object

I've found myself wanting to get at color data for the raw file a lot lately (eg. the stuff in libraw_colordata_t) without having to go down into LibRaw land.

I'm not entirely sure where this should go; eg. in the metadata, in a separate named tuple, or just directly in the Raw class (eg. raw.darkness)?

Add deadpixels option and ability to autodetect dead pixels

Add an option for selecting a .badpixels file.

Use ICC profiles for color space selection

Add options to use ICC profile files for input and output colorspaces.

Blocking on #6

Use camera color matrix behavior

If you do not set the imgdata.params.use_camera_matrix option in LibRaw, it defaults to be the same as imgdata.params.use_camera_wb. Conditional defaults seem confusing to me.

In keeping with the mantra of "have reasonable defaults", should we just set this to "True" by default, or is it just me that thinks conditional defaults are confusing and following the camera wb is okay?

In all fairness, if you use the camera settings for one you probably also want to use it for the other, but I still don't like it.

Add ability to open Raw from file or blob

Possibly add the ability to open a file from a blob or a filehandle that is already open.

If we do want to do this, what does the API look like? Wand uses a somewhat un-Pythonic solution:

Raw(filename=None, file=None, blob=None)

and then does some aweful iteration to decide if any two arguments are set and throw a TypeError if so. This looks fairly terrible, but it does lead to a nice clean API.

The more Pythonic way to do this would probably be to just have a single method and just see what works (or even do a type check and attempt to open a file if a string is passed in, open a raw if bytes or a huge string is passed in, or read from the file if a file object is passed in). That would effectively keep the method signature the same:

Raw(data=None)

where data replaces file and can either be a string, blob, or file object.

Check the error code from the `make_mem` functions

libraw_dcraw_make_mem_image and libraw_dcraw_make_mem_thumb put a status code in a provided int pointer. We need to check and handle this.

Add per-channel darkness option

Add an option for setting the black level per channel (libraw: user_cblack).

I've made this a separate issue from the normal options issue because we need to experiment with it and decide on an API. My first thought was that if you set darkness to a 3 or 4 tuple it knows to do it per channel, otherwise do it globally. However, the options may stack (addative?) and if so we might want to support both (or we could just leave that up to the user and do it the original way... if they want extra black per channel they can just add some ammount per channel, it's not that hard).

Thoughts? /cc @campaul

TL;DR —

raw.options.darkness = (1, 2, 3, 4)
# or raw.options.darkness = 1, but we can't do both together

or

raw.options.darkness = 1
raw.options.channel_darkness = (1, 2, 3, 4) # Why didn't we just do (2, 3, 4, 5) (if that's actually how this works)?

Possibly blocking on #51 depending on what we decide to do.

Investigating running doctest against all source during CI

Investigate running the doctest module against all docstrings as part of CI to help us keep code in docstrings up to date.

Put docs somewhere

We should put the docs somewhere; probably just hosted on GitHub?

Setup coveralls

Write tests

We should really strive to keep this at 100% test coverage.

Expose raw buffer

Cleanup LibRaw error handling

Cleanup LibRaw error handling so that we don't have to wrap every single call out to libraw in the handler method.

Support libraw options

Setup pypi

Support LibRaw 0.18

Update the libraw structs to support LibRaw 0.18 (or whatever the current version is when the structs are updated).

Ensure that options are serializable

Have a simple way to serialize or export options (probably by making sure they can be easily turned into a dict which things like JSON and YAML libraries can consume). Having an easy way to turn this (string or dict) back into an Options object would be nice too.

Make a logo

Normalize the lightness/darkness options

The auto-brightness and darkness options should be normalized to behave in the same way. The darkness option should also be renamed to be less confusing.

For more information on how user_[c]black works see here.

How do we want them to work?

LibRaw options

The LibRaw options we're talking about are:

• user_cblack (will also need to look at the value of libraw_colordata_t.cblack)
• user_black (will also need to look at the value of libraw_colordata_t.black)
• no_auto_bright
• auto_bright_thr

Write detection function

recursively list all files that rawkit knows how to open

lightness option is broken

Not sure how I did this, but the lightness option doesn't work the way I thought it did. It's a simple True or False and should probably be retitled auto_brightness (as that's all it's doing). The thing I thought I was setting is a different option which should be added (I'm not actually sure what it is; bright or something).

Alternatively, make these a single option where lightness: None is the default (set it automatically based on the histogram), and any value sets LibRaw's no_auto_bright to 1 and brightness to whatever the value is.

Consider using mkdocs instead of Sphinx for documentation?

Analyze mkdocs and see if it meets all the needs for documentation. if so, switch to it immediately and kill all the stupid restructured text garbage with fire.

Restructure options / libraw in docs.

The options list is getting rather long. Maybe we should split the libraw structs out into their own doc file, and start dividing the options up by section (eg. interpolation options, color options, etc.).

Support thumbnails

Support multiple versions of libraw

Add darkframe option and generation

Add functionality for generating a dark frame (eg. a Raw class with preset options, or an options constant which the user can develop with that matches the functionality of dcraw -D -4 -j -t 0).

In dcraw land -4 is the same as -6 -W -g 1 1 so we may be blocking on figuring out the lightness value that dcraw applies by default if we specify -W (I'm assuming it's the default of -b 1.0).

Also an option to actually select the darkframe.

Blocking on #6, #70, #72.

Setup travis

Add documentation for each interpolation algorithm

I have had a hard time finding information about the various interpolation algorithms supported by LibRaw and the demosaic packs. We should have a page in the docs that describes each algorithm and its pros and cons as best we can. We should be able to find info about the common ones easily enough, though I don't know where we'd find information about some of the less standard ones in the demosaic pack.

libraw_dcraw_make_mem_image takes 2 arguments, one given

My first attempt to use rawkit failed immediately (I'm good at breaking things). Fresh install with pip, then tried to use raw.to_buffer():

...
    data = raw.to_buffer()
  File "/usr/lib/python2.7/site-packages/rawkit/raw.py", line 139, in to_buffer
    processed_image = self.libraw.libraw_dcraw_make_mem_image(self.data)
TypeError: this function takes at least 2 arguments (1 given)

Handle libraw return values

Right now we just assume all the libraw calls work.

Write documentation

Write better documentation and setup sphinx or someting.

• Add docstrings to everything
• Setup sphinx or something

Add constants for the demosaic options in the demosaic packs

Add constants for the following:

• 5 —Modified AHD interpolation by Paul Lee
• 6 — AFD interpolation (5-pass)
• 7 — VCD interpolation
• 8 — Mixed VCD/Modified AHD interpolation
• 9 — LMMSE interpolation
• 10 — AMaZE intepolation

and a note that if they're not available AHD will be used.

They were left out originally because we were worried that it would be a violation of the GPLv2 or GPLv3 licenses used by the demosaic packs, but I'm reasonably sure we're okay... we're just including constants, and our library is open source. Based on this StackExchange question, I think we're okay to add it.

Add a way to set the "none" value for options

Right now if an option is set to "none" it's just not written. This is fine normally (the defaults get used), but if you set the option, then set it back to none, it doesn't get written and the option isn't reverted to the default.

Adding an extra argument (none_value?) to the option constructor would do the trick (also a separate method like param_writer when we're doing things manually).

This is needed for #46 in particular.

Add option for Fuji camera rotation / no-resize on non-square pixel cameras

Add an option similar to dcraw -j or LibRaw's use_fuji_rotate.

What should we call it?

From the dcraw docs:

-j For Fuji Super CCD cameras, show the image tilted 45 degrees. For cameras with non-square pixels, do not stretch the image to its correct aspect ratio. In any case, this option guarantees that each output pixel corresponds to one raw pixel.

Python 3 string encoding error

I left off the filename, we should probably be checking for that since the error is kind of confusing as is:

$ sudo pip install rawkit
$ python3 test.py
Traceback (most recent call last):
  File "test.py", line 4, in <module>
    raw.save()
  File "/usr/lib/python3.4/site-packages/rawkit/raw.py", line 89, in save
    self.data, filename.encode('ascii'))

where test.py is:

from rawkit.raw import Raw

with Raw(filename="TEST.CR2") as raw:
    raw.save()

ReadTheDocs not showing namedtuples

The documentation on RTD isn't showing the constants (named tuples) in the rawkit.options docs. Building locally works fine.

Might need to figure out what exactly RTD is doing to build its docs and what option is different to see if we can fix it.

MathJAX broken on RTD

The MathJAX template doesn't work on ReadTheDocs (works fine when building locally) so math shows up as raw text.

Add more options

Add the following as options:

• interpolation_quality — See libraw's user_qual. This should also have a constant with:
  • 0 — linear interpolation
  • 1 — VNG interpolation
  • 2 — PPG interpolation
  • 3 — AHD interpolation
  • 4 — DCB interpolation
  • 5 —Modified AHD interpolation by Paul Lee
  • 6 — AFD interpolation (5-pass)
  • 7 — VCD interpolation
  • 8 — Mixed VCD/Modified AHD interpolation
  • 9 — LMMSE interpolation
  • 10 — AMaZE intepolation
  • For 5–9 see #44
• med_passes — The number of median filter passes (see libraw med_passes)
• auto_brightness — Allow turning off auto brightness or setting the threshold (lightness)
• adjust_maximum_thr — Allow setting the channel maximum value threshold in case of channel overflows?
• green_matching — Do secondary processing to fix the green channel imbalance.
• dcb_iterations — Number of passes when performing DCB interpolation.
• dcb_enhance_fl — Enhance colors when performing DCB interpolation.
• fbdd_denoise — FBDD noise reduction.
• es_med_passes — Number of edge-sensitive median filter passes after VCD+AHD demosaic.
• ca_correc — Turn on / off CA correction. Doesn't this already work? Why are there more values for it?
• cared,cablue — See above... why are these different from the dcraw ones?

... TODO: Add more? That's a lot of stuff

Other:

These either have their own issue, or I'm not sure if we care.

• darkness tuple — Allow setting the darkness value per-channel (see #46)
• use_fuji_rotate — Ensures a 1:1 mapping of raw pixels to output pixels (see #70)
• rotation — Allow rotating or flipping the image (see #72)
• sony_arw2_hack — Investigate this libraw value?
• eeci_refine — No idea what this does.

Separate rawkit and libraw into two "virtual" packages

This gives us division between the high level stuff and the low level stuff. You still get both by pip install rawkit-ing, but they're separate top level packages.

Eg. instead of:

from rawkit.libraw import ph1_t
from rawkit.errors import LibRawDataError
from rawkit.raw import Raw

we'd have:

from libraw.structs import ph1_t
from libraw.errors import DataError

from rawkit.raw import Raw

This is a breaking API change (which is okay since we're not at 1.0 yet).

Add rotation option

Add an option to flip or rotate the image (or disable all rotation, or use the camera's builtin rotation)

This should either take degrees, or a set of bitwise-or-able flags as input.