loris-imageserver / loris Goto Github PK

Performance with thumbnails and smaller full region images (not tiles or ROI) will be better if we can use the next largest level to generate the image.

Hi Jon,
I just finished to update my Loris setup guide for dummies:
http://doc.biblissima-condorcet.fr/loris-setup-guide-ubuntu-debian
I first made it for my own needs, as a reminder, but I thought it could be a complement to your readme file, primarily intended for web admin beginners like me.
Feel free to criticize or disseminate if you think it's worth.
R.

catch whatever is raised when nothing in patokah.url_map is matched and return a 400.

Tests are passing, but, e.g.:

http://libimages.princeton.edu/loris/new_jp2s%2Fpudl0052%2F6131707%2F00000001.jp2/info.json

Why only ['native', 'bitonal']? Could be a problem with the image.

Logging is a mess at the moment; it's tough to configure and only app.py actually logs anything.

I'm having some trouble deploying loris on RHEL6 using mod_wsgi (I don't think this is your fault, but I'm hoping you or someone else here might have some guidance, as I'm not really a python person).

I have everything installed correctly, as far as I can tell. When I try to serve an image, however, the call to '/usr/bin/mkfifo' fails with a permission denied error (from line 279 of transforms.py). Changing the call to os.mkfifo() doesn't help, so it's not like I have the wrong path to mkfifo.

I can't for the life of me figure this out, and I'm hoping someone can help, or might have had the same issue. I've posted a similar question on StackOverflow:

http://stackoverflow.com/questions/21260404/mod-wsgi-cant-run-mkfifo

Thanks!

Somehow this was missed and there's no test coverage. Regions can only be requested from jp2s!

Daemon Mode >> Adjust

• under [handler_err] and [handler_out] set args to reflect the correct log dir, .e.g. something like args=('/var/log/loris/loris.err', 'midnight', 1, 14)

Is that something that should be there, or was it taken out?

Hi Jon,

When I request "http://interne.biblissima-condorcet.fr/loris/T0000009.jp2", it makes a 303 redirect to this URI "http://interne.biblissima-condorcet.fr/T0000009.jp2/info.json", which throws a 404 error (I have "redirect_base_uri=1" in loris.conf). The "/loris" prefix is removed. Is it a normal behaviour ?

Thx
Regis

While attempting to integrate loris in the image browser, noticed a lot has changed since I last used it. Someone with basic skills should be able to get this to roll over in under an hour woking with basic test images. Convention over configuration. For example, found this in the loris/resolver.py file:

SRC_IMG_ROOT='/home/jstroop/workspace/loris/test_img'

... shouldn't this be in a conf file somewhere? And I would argue for a single conf file... even though it may get long, convention should be that most people don't have to change but a few values.

As an image repository, I'm interested in giving donors a sense of control of their images once posted on the website. For downloading or printing any image above a certain resolution, I'd like to add a watermark that provides attribution and/or discourages unlicensed exploitation of the image.

We're discussing using loris at UNC-Chapel Hill Libraries and were wondering if there was an Ubuntu or Redhat/Centos package for loris? Our sys. admin was hoping not to have to create a local package if possible.

When you build it are you all using Ubuntu LTS or a more unstable version?

Also does loris require python 2.7? Redhat, unfortunately is only up to python 2.6 in their releases.

thanks!

See this: http://stackoverflow.com/questions/5166129/how-do-i-stream-a-file-using-werkzeug

It might be useful for larger files.

The following should work:
http://example.org/image-service/abcd1234/pct:10,10,50,50/full/0/native.jpg

However, Loris returns the following error:

local variable 'fifo_path' referenced before assignment

Furthermore, the following returns a tile that is 50x50.

http://example.org/image-service/abcd1234/0,0,100,100/pct:50/0/native.jpg

After reading through the spec, it looks like the scaling of the tile and not the original image is intended. However, it's difficult to create a zoom viewer without being able to manipulate the size of the image and maintain a consistent tile size.

Finally, trying to go around the problem, I noticed the following should return a 100x100 tile from the full sized image, but for some reason returns a 7920x7920 tile (problem persists for ",3600" size, etc.) :

http://example.org/image-service/abcd1234/0,0,100,100/,7200/0/native.jpg

Creating a tile-based viewer is pretty much impossible without being able to have "non-full" parameters in both Region and Size slots. The spec is very light on those examples and could probably be more clear. I personally do not think that the region specified should be scaled based on the image, but perhaps there's some reason for that.

Before this becomes public, we should add license info to the README. GPL?

Check out http://www.w3.org/TR/media-frags/

E.g. color version of a greyscale image? 4xx?

Again, only sometimes, "Check size with !width,height" fails, but when it does it looks like this:

url: http://libimages.princeton.edu/loris/67352ccc-d1b0-11e1-89ae-279075081939.jp2/full/%21441%2C587/0/native
expected: 441,441
got: 587,587
type: size

My algorithm (which I believe 4.2 permits) choosing which dimension to respect/retain is as follows:
preferred_dimension=height

if:
    both dimensions are in bounds of the requested region in or both are out of
    bounds of the requested region
        use the preferred_dimension
else if:
    the requested width is wider than the requested region
        use height
else
    use width

see: https://github.com/pulibrary/loris/blob/development/loris/parameters.py#L321

Again, I think this is legit...no? Maybe the validator should be expecting one of two values?

Hrm. 4.2 on !w.h says:

The image content is scaled for the best fit such that the resulting width and height are less than or equal to the requested width and height. The exact scaling MAY be determined by the service provider, based on characteristics including image quality and system performance. The dimensions of the returned image content are calculated to maintain the aspect ratio of the extracted region.

Which I think is supposed to be interpreted as:
outW <= inW and outH <= inH and outAspect == imgAspect
Which wouldn't be true as 587 is not < 441 for width.
But I can see your interpretation too -- (outW <= inW or outH <= inH) and outAspect == imgAspect
Can you bring it up on the list?

BAH. Subtle, but I think you're correct. Both w and h have to be <= the arguments in the request; somehow I missed that bit. What would you like me to bring up on the list? The interpretation (mine is wrong), or a change for the next version?

Just when I thought Loris was out of my life for a while....

When working with networked storage of src image files, it may make sense to copy the source image to the local disk cache. This could all be implemented in the resolver, by adding a second param, the root of the local disk cache, to the resolver implementation's constructor.

If the value of the Origin header is in this list, set Access-Control-Allow-Origin to its value. Otherwise set Access-Control-Allow-Origin to the name of the server.

Unit tests are all in one modle at the moment. Refactor into separate modules that match the modules they're testing.

(Separated from from #15 ; issue #13 is also related.)

/0,0,100,100/,7200/0/native.jpg

returns a 7920x7920 tile (problem persists for ",3600" size, etc.).

Start here: http://code.google.com/p/modwsgi/wiki/QuickConfigurationGuide

Just as with Kakadu so that a different version of ImageMagick can be used from the one on the system PATH.

Also, build the environment variable dictionary for shell-outs once globally, rather than on every call.

(i.e. with %2F) See http://www-sul.stanford.edu/iiif/image-api/#url_encoding, 5th example.

More of a comment, but loris seems to work just fine with Pillow as well. Do you all have any plans to make Pillow the default imaging library? Thanks for such a cool project.

As with images, if use_415 is set to False, there should be a default format for the info service (probably json)

currently not covered

e.g. JPEG, TIFF and so support different libraries for format conversion. Maybe this should be abstracted out of the Loris class, so that any function that implements a common signature can be called.

Bill's proposal is here.

It can be done with kakadu 7.2+. Instead of creating a bmp as the intermediate file, we need to make a tif, which will then carry the color profile along. From there, PIL, with the help of Little CMS can read the restricted profile out of the tiff and convert it, e.g. SRGB.

Proof:
The jp2, supplied by Bill: https://www.dropbox.com/s/5ugtmjyjx7hgbpg/47102787.jp2

The JPEG produced by the code that follows:

Loris would currently produce something quite different: http://libimages.princeton.edu/loris/47102787.jp2/full/full/0/native.jpg

This starts with a TIFF that would normally be in a named pipe in Loris as a result of a kdu_expand subprocess.

from PIL import Image
from ImageCms import profileToProfile
import cStringIO

image_path = "sample.tif"
SRGB = 'sRGB_v4_ICC_preference.icc'

original_image = Image.open(image_path)
if 'icc_profile' in original_image.info:
    src_profile = cStringIO.StringIO(original_image.info['icc_profile'])
    converted_image = profileToProfile(original_image, src_profile, SRGB)
    converted_image.save('converted.jpg')

Two issues:

1. This is very likely to slow tile creation down...not a problem once they're cached, but the first pass will be slower.
2. It's a massive deployment headache because PIL needs to be uninstalled, and rebuilt with Little CMS support. Here's the approximate procedure for Ubuntu 12.04 (64 bit):

sudo apt-get install libjpeg-turbo8 libjpeg-turbo8-dev libfreetype6 libfreetype6-dev zlib1g-dev
sudo apt-get install liblcms liblcms-dev liblcms-utils
sudo apt-get build-dep python-imaging

sudo ln -s /usr/lib/`uname -i`-linux-gnu/libfreetype.so /usr/lib/
sudo ln -s /usr/lib/`uname -i`-linux-gnu/libjpeg.so /usr/lib/
sudo ln -s /usr/lib/`uname -i`-linux-gnu/libz.so /usr/lib/
sudo ln -s /usr/lib/`uname -i`-linux-gnu/liblcms.so /usr/lib/ # check this

# Build PIL from source
cd /tmp
wget http://effbot.org/downloads/Imaging-1.1.7.tar.gz
cd Imaging-1.1.7
python setup.py build_ext -i

# Check we have all the deps

python selftest.py

# Output MUST include these lines:
# ...
# --- PIL CORE support ok
# --- JPEG support ok
# --- ZLIB (PNG/ZIP) support ok
# --- FREETYPE2 support ok
# --- LITTLECMS support ok
# ...

# See the README for what to do if there's trouble, otherwise, do:

sudo python setup.py install

For these reasons, we can make this something that is configurable in Loris, with the caveat that you're going to have extra work to do to enable it.

See explanation here:

#43 (comment)

See http://www-sul.stanford.edu/iiif/image-api/#implementation #1
and http://tools.ietf.org/html/rfc6266#section-4.1

Per Jay, something like:

log_to=[file|console]
log_level=[CRITICAL|ERROR|WARNING|INFO|DEBUG]
log_dir=/var/log/loris

(and the last one wouldn't apply if log_to=console).

See:

http://img.princeton.edu/loris/pudl0001/4609321/s42/00000523/full/pct:25/0/grey.jpg

http://img.princeton.edu/loris/pudl0001/4609321/s42/00000523/full/pct:25/0/bitonal.jpg

Which points out that there's no test for this (e.g. is the resultant img grey, bitonal. color, etc.)

This is not broken on other test instances, which suggests a dependency version issue, probably of ImageMagick.

3 TODOs:

• Fix the bug on img.princeton.edu
• Write a test that checks, probably by bit-depth, the resultant images 'quality'
• Add info about versions of kdu_expand and convert to docs, and maybe see if these can be checked in the tests

Hi Jon,

I need to run some more tests with another instance of Loris but I ran into errors while deploying the latest build:

(Python version is 2.7.3)

"python test.py" returns:

Hello Jon,

I have been struggling with Loris for a few days and I think it would be interesting to give you some feedback.
I started with the loris-master version and tried to test it with a sample jp2 file. Finally I obtained a "502 Proxy Error. Reason: Error reading from remote server" (maybe from the reverse proxy ? no idea about that...).
So I decided to test Simeon's fork of Loris (I thought it would be easier to manipulate static images as source), but I did not manage to install it.
Then I saw the latest developments you made this week and tried again. I encountered some problems with setup.py (which is not yet updated to take into account changes in etc/loris.conf), so I had to install it manually. Now I think the setup is done but I am facing a bug.

Here is the final error from Apache logs. I posted another issue that give you all the configuration and environment details (it could be useful to other python newbies like me in the future).

[Fri Jun 28 12:46:04 2013] [error] [client 10.14.1.38] mod_wsgi (pid=10518): Exception occurred processing WSGI script '/var/www/loris/loris.wsgi'.
[Fri Jun 28 12:46:04 2013] [error] [client 10.14.1.38] Traceback (most recent call last):
[Fri Jun 28 12:46:04 2013] [error] [client 10.14.1.38]   File "/usr/local/loris/lib/python2.7/site-packages/Loris-0.1.0dev-py2.7.egg/loris/webapp.py", line 215, in __call__
[Fri Jun 28 12:46:04 2013] [error] [client 10.14.1.38]     return self.wsgi_app(environ, start_response)
[Fri Jun 28 12:46:04 2013] [error] [client 10.14.1.38]   File "/usr/local/loris/lib/python2.7/site-packages/Loris-0.1.0dev-py2.7.egg/loris/webapp.py", line 206, in wsgi_app
[Fri Jun 28 12:46:04 2013] [error] [client 10.14.1.38]     response = self.dispatch_request(request)
[Fri Jun 28 12:46:04 2013] [error] [client 10.14.1.38]   File "/usr/local/loris/lib/python2.7/site-packages/Loris-0.1.0dev-py2.7.egg/loris/webapp.py", line 212, in dispatch_request
[Fri Jun 28 12:46:04 2013] [error] [client 10.14.1.38]     return getattr(self, 'get_'+endpoint)(request, **values)
[Fri Jun 28 12:46:04 2013] [error] [client 10.14.1.38]   File "/usr/local/loris/lib/python2.7/site-packages/Loris-0.1.0dev-py2.7.egg/loris/webapp.py", line 371, in get_img
[Fri Jun 28 12:46:04 2013] [error] [client 10.14.1.38]     headers['Link']['profile'] = constants.COMPLIANCE
[Fri Jun 28 12:46:04 2013] [error] [client 10.14.1.38] TypeError: 'unicode' object does not support item assignment

Looking at some scrolls, at levels that should start to tile I'm seeing multiple links to identical images.

The symlinks:

.
├── 10
│   ├── 0_0.jpg -> /var/cache/loris/pudl0052/6131707/00000001/full/pct:1.5625/0/native.jpg
│   ├── 1_0.jpg -> /var/cache/loris/pudl0052/6131707/00000001/full/pct:1.5625/0/native.jpg
│   ├── 2_0.jpg -> /var/cache/loris/pudl0052/6131707/00000001/full/pct:1.5625/0/native.jpg
│   └── 3_0.jpg -> /var/cache/loris/pudl0052/6131707/00000001/full/pct:1.5625/0/native.jpg
├── 11
│   ├── 1_0.jpg -> /var/cache/loris/pudl0052/6131707/00000001/full/pct:3.125/0/native.jpg
│   ├── 2_0.jpg -> /var/cache/loris/pudl0052/6131707/00000001/full/pct:3.125/0/native.jpg
│   ├── 3_0.jpg -> /var/cache/loris/pudl0052/6131707/00000001/full/pct:3.125/0/native.jpg
│   └── 4_0.jpg -> /var/cache/loris/pudl0052/6131707/00000001/full/pct:3.125/0/native.jpg
├── 6
│   └── 0_0.jpg -> /var/cache/loris/pudl0052/6131707/00000001/full/pct:0.09765625/0/native.jpg
├── 7
│   └── 0_0.jpg -> /var/cache/loris/pudl0052/6131707/00000001/full/pct:0.1953125/0/native.jpg
├── 8
│   └── 0_0.jpg -> /var/cache/loris/pudl0052/6131707/00000001/full/pct:0.390625/0/native.jpg
└── 9
    ├── 0_0.jpg -> /var/cache/loris/pudl0052/6131707/00000001/full/pct:0.78125/0/native.jpg
    └── 1_0.jpg -> /var/cache/loris/pudl0052/6131707/00000001/full/pct:0.78125/0/native.jpg

The targets:

├── [       4096]  pct:0.09765625
│   └── [       4096]  0
│       └── [        727]  native.jpg
├── [       4096]  pct:0.1953125
│   └── [       4096]  0
│       └── [       1226]  native.jpg
├── [       4096]  pct:0.390625
│   └── [       4096]  0
│       └── [       3903]  native.jpg
├── [       4096]  pct:0.78125
│   └── [       4096]  0
│       └── [      12595]  native.jpg
├── [       4096]  pct:10
│   └── [       4096]  0
│       └── [     933782]  native.jpg
├── [       4096]  pct:1.5625
│   └── [       4096]  0
│       └── [      39114]  native.jpg
└── [       4096]  pct:3.125
    └── [       4096]  0
        └── [     130138]  native.jpg

">" might be taking care of if, but it seems like it might be a good idea to pass the dimensions of the image (if full) or otherwise region to SizeParameter#to_convert_arg(). This might make it possible to allow upsampling as a configurable option as well.

cors_whitelist should be able to take a '*' as its value, indicating that the info service should be completely open.

Confirm that they're getting through, else:
if native quality is color and request is native or color

convert_call += '-profile '
convert_call += os.path.join(self.patoka_data_dir, 'sRGB.icc') + ' '

if native quality is grey and request is native or grey

convert_call += '-profile '
convert_call += os.path.join(self.patoka_data_dir, 'gray22.icc ')

The "Full Featured" Werkzeug Response mixes in a bunch of stuff I don't need, and, meanwhile, these four lines:

r = Response()
link_header = RelatedLinksHeader()
link_header['profile'] = constants.COMPLIANCE
r.headers['Link'] = link_header.to_header()

are repeated over and over. Could I make a custom LorisResponse that extends BaseResponse. to always include the compliance link header?

See src for BaseResponse.__init__(). Just super __init__ and then add the link header.

See, e.g. http://mypythonnotes.wordpress.com/2008/09/04/__slots__/

there were path issues that came up with tests.py when they weren't run through Sublime.