hooram / ownphotos

Self hosted alternative to Google Photos

Ownphotos development has moved to LibrePhotos

Ownphotos development has moved to LibrePhotos Use it here: https://github.com/LibrePhotos/librephotos

OLD project for archive only, use LibrePhotos

Ownphotos

Screenshots

Live demo

Live demo avilable here. User is demo, password is demo1234.

Discord Server

https://discord.gg/dPCdTBN

What is it?

• Self hosted wannabe Google Photos clone, with a slight focus on cool graphs
• Django backend & React frontend.
• In development. Contributions are welcome!

Currently the project is in very early stages, so run it only for the sake of checking it out.

Features

- Currently implemented:

• Label some faces manually, and train a face classifier to label the rest.
• View photos by people in them.
• Automatically generate "event" albums with nice titles, like "Thursday in Berlin"
• See photos on the map
• Backend caching
• View photos grouped by date
• "Optimized" frontend (react virtualized... I tried my best.)
• Detect objects in photos, and make them searchable by objects
• Search photos by the location
• Authentication (using JWT)
• Create custom albums
• Docker ready

- Upcoming

• Short term:

  • Share photos/albums
  • Stability

• Longer term, i.e. haven't thought much about them

  • Basic photo editing, like rotation
  • Tag undetected face
  • Add cool graphs

What does it use?

• Face detection: face_recognition
• Face classification/clusterization: scikit-learn
• Object detection: densecap, places365
• Reverse geocoding: Mapbox: You need to have an API key. First 50,000 geocode lookups are free every month.

How do I run it?

Docker

Ownphotos comes with separate backend and frontend servers. The backend serves the restful API, and the frontend serves, well, the frontend. They are connected via a proxy. The easiest way to do it is using Docker.

If you want the backend server to be reachable by ownphotos-api.example.com and the frontend by ownphotos.example.com from outside. You must account for the corsaCross-Origin Resource Sharing (CORS) in your proxy.

Docker-compose method (Recommended)

wget https://raw.githubusercontent.com/hooram/ownphotos/dev/docker-compose.yml.template
cp docker-compose.yml.template docker-compose.yml

Open docker-compose.yml in your favorite text editor and make changes in the lines with # CHANGE ME

docker-compose up -d

You should have ownphotos accessible after a few seconds of bootup on: localhost:3000 User is admin, password is admin and its important you change it on a public server via the docker-compose.yml file.

First steps after setting up

You need to log in as the admin user, and set up the directory for the users. To do this, click the top right button, and go to "Admin Area". On this page, it will show a list of users, and manually set the "Scan Directory" for the desired user. Only an admin can do this. And then you can go to Dashboard - Library and click the Green "Scan photos (file system)" button. If you have a Nextcloud instance, you can also input this in the Dashboard-Library page. Once logged in (the little circle next to "Nextcloud Scan Directory will be green), you can choose a top level directory in your logged in Nextcloud account. Once this works, you can click the blue "Scan photos (Nextcloud)". The backend system will essentially copy the contents of the Nextcloud directory you specified.

The basic idea is this:

• For scanning photos that reside in the local file system
  • Only the admin user can change the "scan directory" of the users, including the admin itself.
  • Normal users cannot change his/her own "scan directory"
  • Only the admin can find the page to control this under the "user icon (top right) - admin area"
• For scannign photos that reside in external Nextcloud instances
  • Any user can change his/her own Nextcloud endpoint, and choose a top level directory in the Nextcloud account.

ownphotos's Issues

Feature idea: intelligently parse and import Google Takeout from Google Photos

For example, dealing with how Google Takeout annoyingly gives two copies of every photo that's in an album and in your main library, and not ending up with duplicates.

Google Takeout also provides .jpg.json files that could be parsed for additional information.

View progress of scan?

Hi

I am currently running a scan of a large directory and was hoping that there could be some way to see how many files have been processed or, even better, have some sort of progress bar.

Thanks! :)

Cannot read property 'status' of undefined

When attempting to login with latest build, receive the following:

Chrome: "Cannot read property 'status' of undefined"
Firefox: "e.response is undefined"
IE: "Unable to get property 'status' of undefined or null reference"

Login does not complete

Generate albums from directory structrure

Bug: docker build - Illegal instruction

I am trying to build the backend server whith the provided dockerfile.
Build is ok but i get the following error when trying to run the container :

./entrypoint.sh: line 5:    10 Illegal instruction     (core dumped) python manage.py migrate
./entrypoint.sh: line 6:    15 Illegal instruction     (core dumped) python manage.py migrate --run-syncdb
./entrypoint.sh: line 12:    20 Illegal instruction     (core dumped) python manage.py shell  <<EOF
from django.contrib.auth.models import User
User.objects.filter(email='$ADMIN_EMAIL').delete()
User.objects.create_superuser('$ADMIN_USERNAME', '$ADMIN_EMAIL', '$ADMIN_PASSWORD')
EOF

Running server...
./entrypoint.sh: line 16:    25 Illegal instruction     (core dumped) python manage.py runserver 0.0.0.0:8000

Any idea how i shall fix it?

Add New Photo Automatically

Is it possible to have new photos automatically when they are added to the folder?

Internal Server Error when trying to view phjotos

The current branch on demo server gives when viewing the images:

Internal Server Error: /media/photos/8a2c3c09e57a61ec0b94cb131cd27fdb6.jpg
Traceback (most recent call last):
  File "/miniconda/lib/python3.7/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/miniconda/lib/python3.7/site-packages/django/core/handlers/base.py", line 126, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/miniconda/lib/python3.7/site-packages/django/core/handlers/base.py", line 124, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/miniconda/lib/python3.7/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/miniconda/lib/python3.7/site-packages/django/views/generic/base.py", line 68, in view
    return self.dispatch(request, *args, **kwargs)
  File "/miniconda/lib/python3.7/site-packages/rest_framework/views.py", line 483, in dispatch
    response = self.handle_exception(exc)
  File "/miniconda/lib/python3.7/site-packages/rest_framework/views.py", line 443, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/miniconda/lib/python3.7/site-packages/rest_framework/views.py", line 480, in dispatch
    response = handler(request, *args, **kwargs)
  File "/code/api/views.py", line 1842, in get
    if photo.public:
AttributeError: 'NoneType' object has no attribute 'public'
Internal Server Error: /media/thumbnails_small/8a2c3c09e57a61ec

After that just keep getting:

Not Found: /media/photos/45ff89100220ee80c7e87eca9b78cab01.jpg
Not Found: /media/thumbnails_small/45ff89100220ee80c7e87eca9b78cab01.jpg
Not Found: /media/photos/821214c0b8658165136276ca7d9f6b411.jpg
Not Found: /media/thumbnails_small/821214c0b8658165136276ca7d9f6b411.jpg

EXIF GPS date/time should be used when available

It seems like Ownphotos currently thinks that imported date from EXIF is always in UTC but it seems like it is not the case. I have older images from Android Nexus 6P where "Create Date" is in local time.

Here are some date related attributes for that image as displayed by exiftool:

File Name                       : IMG_20160619_025324.jpg
Create Date                     : 2016:06:19 02:53:26
Modify Date                     : 2016:06:19 02:53:26
GPS Date Stamp                  : 2016:06:18
GPS Time Stamp                  : 23:53:24
Date/Time Original              : 2016:06:19 02:53:26
GPS Date/Time                   : 2016:06:18 23:53:24Z

As you can see the image was shot in Eastern European Summer Time (EEST) so it is off by 3 hours.

In cases where GPS date/time is available, it should be used to determine correct time the image was taken.

Small face thumbnail missing on "Label Faces" modal

When on the "Label Faces" modal, the small face thumbnail on the top left is missing. When inspecting it looks like this:

<img auth="[object Object]" height="40" width="40" src="http://backend/media/faces/c5291c1156b701f0b5d4ff2cc10545a74_0.jpg" class="ui image">

Note http://backend

I'm running this by checking out the dev branch and using docker-compose up.
I've tweaked the docker-compose.yml file only by changing all 3000 to 3007 and updating BACKEND_HOST to machine.local:3007. I'm wondering if the proxy is handling this correctly with 3000 being hard-coded into the nginx config file.

Note 2 If I replace "http://backend/media/faces/c5291c1156b701f0b5d4ff2cc10545a74_0.jpg" with "http://machine.local:3007/media/faces/c5291c1156b701f0b5d4ff2cc10545a74_0.jpg", the file does exist correctly. I'm not seeing this issue on any of the other images in the system so far.

Ownphotos fails to start, complains about COCO import

Looks like something in the container breaks /code/api/im2txt/build_vocab.py

There is an import there from pycocotools.coco import COCO

Which now gives in the commandline:

>>> from pycocotools.coco import COCO
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/venv/lib/python3.5/site-packages/pycocotools/coco.py", line 55, in <module>
    from . import mask as maskUtils
  File "/venv/lib/python3.5/site-packages/pycocotools/mask.py", line 3, in <module>
    import pycocotools._mask as _mask
  File "__init__.pxd", line 918, in init pycocotools._mask
ValueError: numpy.ufunc size changed, may indicate binary incompatibility. Expected 216 from C header, got 192 from PyObject
>>> 

Run without AVX instruction

The backend should be able to run without the AVX instructions

Abort a photo scan

The photo scan seems to take a long time during which some other things aren't possible (e.g. no face scanning).

Is there a way to abort a scan?

TypeError: Cannot read property 'im2txt' of null

On opening one of the scanned photos, I got the following error (the photo had a broken thumbnail).

TypeError: Cannot read property 'im2txt' of null
LightBox.render
src/app/src/components/lightBox.js:494
  491 | <Form.TextArea
  492 |   disabled={this.props.isPublic}
  493 |   fluid
> 494 |   placeholder={
  495 |     this.props.photoDetails[
  496 |       this.props.idx2hash[
  497 |         this.props.lightboxImageIndex
View compiled
▼ 29 stack frames were expanded.
finishClassComponent
src/app/node_modules/react-dom/cjs/react-dom.development.js:14301
updateClassComponent
src/app/node_modules/react-dom/cjs/react-dom.development.js:14264
beginWork
src/app/node_modules/react-dom/cjs/react-dom.development.js:15082
performUnitOfWork
src/app/node_modules/react-dom/cjs/react-dom.development.js:17820
workLoop
src/app/node_modules/react-dom/cjs/react-dom.development.js:17860
HTMLUnknownElement.callCallback
src/app/node_modules/react-dom/cjs/react-dom.development.js:149
invokeGuardedCallbackDev
src/app/node_modules/react-dom/cjs/react-dom.development.js:199
invokeGuardedCallback
src/app/node_modules/react-dom/cjs/react-dom.development.js:256
replayUnitOfWork
src/app/node_modules/react-dom/cjs/react-dom.development.js:17107
renderRoot
src/app/node_modules/react-dom/cjs/react-dom.development.js:17979
performWorkOnRoot
src/app/node_modules/react-dom/cjs/react-dom.development.js:18837
performWork
src/app/node_modules/react-dom/cjs/react-dom.development.js:18749
performSyncWork
src/app/node_modules/react-dom/cjs/react-dom.development.js:18723
requestWork
src/app/node_modules/react-dom/cjs/react-dom.development.js:18592
scheduleWork
src/app/node_modules/react-dom/cjs/react-dom.development.js:18401
enqueueSetState
src/app/node_modules/react-dom/cjs/react-dom.development.js:12313
Connect../node_modules/react/cjs/react.development.js.Component.setState
src/app/node_modules/react/cjs/react.development.js:372
Connect.onStateChange
src/app/node_modules/react-redux/es/components/connectAdvanced.js:205
notify
src/app/node_modules/react-redux/es/utils/Subscription.js:23
Subscription.notifyNestedSubs
src/app/node_modules/react-redux/es/utils/Subscription.js:62
Connect.onStateChange
src/app/node_modules/react-redux/es/components/connectAdvanced.js:202
notify
src/app/node_modules/react-redux/es/utils/Subscription.js:23
Subscription.notifyNestedSubs
src/app/node_modules/react-redux/es/utils/Subscription.js:62
Connect.onStateChange
src/app/node_modules/react-redux/es/components/connectAdvanced.js:202
dispatch
src/app/node_modules/redux/es/redux.js:214
(anonymous function)
src/app/node_modules/react-router-redux/es/middleware.js:13
(anonymous function)
src/app/node_modules/redux-logger/dist/redux-logger.js:1
(anonymous function)
src/app/node_modules/redux-thunk/es/index.js:11
dispatch
src/app/node_modules/redux/es/redux.js:611
▲ 29 stack frames were expanded.
(anonymous function)
src/app/src/actions/photosActions.js:393
  390 | dispatch({ type: "FETCH_PHOTO_DETAIL", payload: image_hash });
  391 | Server.get(`photos/${image_hash}/`, { timeout: 100000 })
  392 |   .then(response => {
> 393 |     dispatch({
  394 |       type: "FETCH_PHOTO_DETAIL_FULFILLED",
  395 |       payload: response.data
  396 |     });
View compiled
This screen is visible only in development. It will not appear if the app crashes in production.
Open your browser’s developer console to further inspect this error.

Show relevant photos on the map view

7e519e1
2c1f91f
hooram/ownphotos-frontend@4d3c6c0
hooram/ownphotos-frontend@3f9f206

ownphotos-backend is unavailable during `docker-compose up`

Symptom

Unable to complete docker-compose up:

Pulling backend (ownphotos-backend:)...
ERROR: The image for the service you're trying to recreate has been removed. If you continue, volume data could be lost. Consider backing up your data before continuing.

Continue with the new image? [yN]y
Pulling backend (ownphotos-backend:)...
ERROR: pull access denied for ownphotos-backend, repository does not exist or may require 'docker login'

Steps to reproduce:

git clone [email protected]:hooram/ownphotos.git
cd ownphotos
docker-compose up

More info

• Related line in docker-compose.yml:

  ownphotos/docker-compose.yml

  Line 11 in e1285a7

  image: ownphotos-backend

photo scan is very slow

This is not a bug, but I wonder if it is possible to let users to decide what is being done when the photos are imported. Currently it runs very slow on my NAS. I have a large amount of photos that I don't need face detection and auto-tagging. If I can import photos first and do face detection and object detection later, it could be more efficient. On my NAS, ownphotos processed 3000+ photos in about 8 hours. And I have 20000+ photos in total.

Request failed with status code 400

I've used a docker-compose.yaml to set it up that I found posted here. I get "request failed with status code 400"

version: '3'
services:
    ownphotos:
        image: hooram/ownphotos:master
        links:
        - ownphotos-db
        - ownphotos-redis
        volumes:
        - ownphotos-data:/data
        - ownphotos-metadata:/code/media
        environment:
        - SECRET_KEY=SECRET_KEY
        - [email protected]
        - ADMIN_USERNAME=USER
        - ADMIN_PASSWORD=PASSWORD
        - DEBUG=false
        - DB_BACKEND=postgresql
        - DB_NAME=ownphotos
        - DB_USER=ownphotos
        - DB_PASS=PASSWORD
        - DB_HOST=ownphotos-db
        - DB_PORT=5432
        - REDIS_HOST=ownphotos-redis
        - REDIS_PORT=6379
        - MAPBOX_API_KEY=MAPBOX_API_KEY
        - BACKEND_HOST=192.168.2.24:8000 <THIS IS THE IP OF MY SERVER>
        ports:
        - 8000:80
        - 3000:3000
        - 5000:5000
    ownphotos-db:
        image: postgres:alpine
        container_name: postgres
        restart: always
        volumes:
         - ownphotos-db:/var/lib/postgresql/data
       environment:
        - POSTGRES_DB=ownphotos
        - POSTGRES_USER=ownphotos
        - POSTGRES_PASSWORD=PASSWORD
    ownphotos-redis:
        image: redis:alpine
        container_name: redis
        volumes:
        - ownphotos-redis:/data
    volumes:
        ownphotos-data:
        ownphotos-metadata:
        ownphotos-db:
        ownphotos-redis:

and here is the error message in chrome:

Problems with first login

Hi,

I deployed ownphotos with the following docker-compose.yml, trying to replicate the docker settings from the readme. But I cannot login with the credentials from the environment, I get the error "Invalid token specified". The labels are for traefik instead of nginx, but I don't think thats a problem, I can access both backend and frontend without any problems.

version: '2'

services:
  ownphotos:
    image: ownphotos
    volumes:
      - /srv/ownphotos/data:/data
      - /srv/ownphotos/media:/code/media
    labels:
      - "com.centurylinklabs.watchtower.enable=true"
      - "traefik.enable=true"
      - "traefik.front.frontend.rule=Host:pics.example.com"
      - "traefik.front.port=5000"
      - "traefik.back.frontend.rule=Host:backend.pics.example.com"
      - "traefik.back.port=80"
    environment:
      - SECRET_KEY=asdf
      - [email protected]
      - ADMIN_USERNAME=myuser
      - ADMIN_PASSWORD=mypass
      - DEBUG=false
      - DB_BACKEND=postgresql
      - DB_NAME=ownphotos
      - DB_USER=ownphotos
      - DB_PASS=dbpass
      - DB_HOST=db
      - DB_PORT=5432
      - REDIS_HOST=redis
      - REDIS_PORT=6379
      - MAPBOX_API_KEY=CHANGE_ME
      - BACKEND_HOST=backend.pics.example.com
    links:
      - "db"
      - "redis"

  db:
    image: postgres
    environment:
      POSTGRES_DB: ownphotos
      POSTGRES_USER: ownphotos
      POSTGRES_PASSWORD: dbpass
    volumes:
      - /srv/ownphotos/db:/var/lib/postgresql/data

  redis:
    image: redis

I cannot find anything in the log, even with DEBUG=true. Any idea where to look further?

Thanks,
Jakob

Is this still being developed?

This looks like a pretty cool project.

is there any plans to continue the work on this?

Make a page with screenshots and doc

Hi,
I installed ownphotos to see how it looked, and it's really amazing.
I've been looking into face recognition lately and was about to try to build something like ownphotos and it turns out you built almost exactly what I had in mind.

However, it's not obvious for people not used to github to find https://github.com/hooram/ownphotos/tree/master/screenshots

I think it would be great to have a page showing what ownphotos can do.

A public (limited) demo would be even better, but that requires an host, while a static page with screenshots could be hosted on github pages.

Use a job queue system for long running tasks, instead of returning http responses eons later

• scanning photos
• generating event albums
• generating event album titles
• training a face classifier

The admin username & email are switched when the user is being created

In entrypoint.sh the username and the email are reversed

docker-compose fails

When cloning the git repo and issuing a docker-compose up it starts building with the supplied dockerfile. It stops with this error:

npm WARN optional SKIPPING OPTIONAL DEPENDENCY: [email protected] (node_modules/react-scripts/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm WARN optional SKIPPING OPTIONAL DEPENDENCY: fsevents@^1.0.0 (node_modules/chokidar/node_modules/fsevents):
npm WARN notsup SKIPPING OPTIONAL DEPENDENCY: Unsupported platform for [email protected]: wanted {"os":"darwin","arch":"any"} (current: {"os":"linux","arch":"x64"})
npm ERR! Linux 4.13.9-1-ARCH
npm ERR! argv "/usr/bin/node" "/usr/bin/npm" "install"
npm ERR! node v6.11.5
npm ERR! npm  v3.10.10
npm ERR! code ELIFECYCLE

npm ERR! [email protected] install: `gulp install`
npm ERR! Exit status 130
npm ERR! 
npm ERR! Failed at the [email protected] install script 'gulp install'.
npm ERR! Make sure you have the latest version of node.js and npm installed.
npm ERR! If you do, this is most likely a problem with the semantic-ui package,
npm ERR! not with npm itself.
npm ERR! Tell the author that this fails on your system:
npm ERR!     gulp install
npm ERR! You can get information on how to open an issue for this project with:
npm ERR!     npm bugs semantic-ui
npm ERR! Or if that isn't available, you can get their info via:
npm ERR!     npm owner ls semantic-ui
npm ERR! There is likely additional logging output above.

npm ERR! Please include the following file with any support request:
npm ERR!     /usr/src/app/npm-debug.log
ERROR: Service 'frontend' failed to build: The command '/bin/sh -c npm install && npm cache clean --force' returned a non-zero code: 1

When docker container upgraded, ownphotos doesn't use existing admin user

Instead, it marks the existing admin user as 'deleted' and creates a new admin user, who does not inherit the photos previously uploaded. and i cannot access the photos that were part of the previous 'admin' user (not 'deleted')

Handling of Broken Scan Photos Jobs

Hey, I'm really enjoying playing with OwnPhotos, but I ran into an issue that I can't figure out how to fix without digging into the database. Essentially, I accidentally added a photo with a non ascii character (∞) and ran scan photos. The scan photos job failed with the below error, but OwnPhotos still thinks that the job is running. I stopped the docker container and deleted the file, and restarted it. The web UI still thinks that scan photos is running, but this is not reflected in the logs.

I was able to fix the issue by removing the most recent entry in the api_longrunningjob table of my postgres database. I used the following command:

DELETE FROM api_longrunningjob WHERE id in (SELECT id from api_longrunningjob ORDER BY id desc LIMIT 1);

00:11:01 UnicodeEncodeError: 'utf-8' codec can't encode character '\udce2' in position 49: surrogates not allowed Traceback (most recent call last): File "/venv/lib/python3.5/site-packages/rq/worker.py", line 793, in perform_job rv = job.perform() File "/venv/lib/python3.5/site-packages/rq/job.py", line 599, in perform self._result = self._execute() File "/venv/lib/python3.5/site-packages/rq/job.py", line 605, in _execute return self.func(*self.args, **self.kwargs) File "/code/api/directory_watcher.py", line 100, in scan_photos if not Photo.objects.filter(image_path=image_path).exists(): File "/venv/lib/python3.5/site-packages/django/db/models/query.py", line 715, in exists return self.query.has_results(using=self.db) File "/venv/lib/python3.5/site-packages/django/db/models/sql/query.py", line 509, in has_results return compiler.has_results() File "/venv/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 1036, in has_results return bool(self.execute_sql(SINGLE)) File "/venv/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 1067, in execute_sql cursor.execute(sql, params) File "/venv/lib/python3.5/site-packages/django/db/backends/utils.py", line 100, in execute return super().execute(sql, params) File "/venv/lib/python3.5/site-packages/django/db/backends/utils.py", line 68, in execute return self._execute_with_wrappers(sql, params, many=False, executor=self._execute) File "/venv/lib/python3.5/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers return executor(sql, params, many, context) File "/venv/lib/python3.5/site-packages/django/db/backends/utils.py", line 85, in _execute return self.cursor.execute(sql, params)

Thank you for your work on this project!

Request - Package as a Nextcloud / Owncloud app

Please consider packaging and releasing in the Nextcloud and Owncloud app store for easy deployment. Thanks for all your good work!

Processes inside docker container running as root

Please fix the docker container to not run ownphotos processes as root user. If ownphotos is exposed to outside world and someone finds a bug that allows remote code execution, it can make it easy for attacker to escape from the container to the host system and take it over.

Publishing photos on Pixelfed/other federated image publishing platforms

While sharing photos via a direct link is ultimately gonna be a feature of ownphotos, having the option to publish photos on some platforms would help them reach a wider audience. Then better be on platforms built to empower users, like Pixelfed.

There are multiple ways to see publishing on federated platforms. The simplest way is to use one implementation's API. Another one is to publish via ActivityPub (the underlying federation protocol).

Unauthorized: /api/rqavailable/

Backend continuously giving "unauthorized" messages like this (every few sec)

The bigger problem is that whenever i give the worker something to do (e.g., scan photos) it never completes the job. (after adding the photos)

Unauthorized: /api/jobs/
Unauthorized: /api/sitesettings/
Unauthorized: /api/jobs/
Unauthorized: /api/user/
Unauthorized: /api/dirtree/
Unauthorized: /api/jobs/
Unauthorized: /api/sitesettings/
Unauthorized: /api/jobs/
Unauthorized: /api/user/
Unauthorized: /api/dirtree/
Unauthorized: /api/rqavailable/
Unauthorized: /api/rqavailable/
Unauthorized: /api/stats/
Unauthorized: /api/sitesettings/
Unauthorized: /api/user/1/
Unauthorized: /api/nextcloud/listdir/
Unauthorized: /api/stats/
Unauthorized: /api/jobs/
Unauthorized: /api/stats/
Unauthorized: /api/sitesettings/
Unauthorized: /api/user/1/
Unauthorized: /api/nextcloud/listdir/
Unauthorized: /api/stats/
Unauthorized: /api/jobs/

ValueError: No group keys passed!

Many faces process well, but I do get quite a lot of these errors (all of the same type). I don't know if this is the reason why quite a few faces remain untagged. In any case, I'll further explore your brilliant work 👍

serializing photos of faces took 0.01 seconds
Unauthorized: /api/rqavailable/
Internal Server Error: /api/locationsunburst/
Traceback (most recent call last):
  File "/venv/lib/python3.5/site-packages/django/core/handlers/exception.py", line 34, in inner
    response = get_response(request)
  File "/venv/lib/python3.5/site-packages/django/core/handlers/base.py", line 126, in _get_response
    response = self.process_exception_by_middleware(e, request)
  File "/venv/lib/python3.5/site-packages/django/core/handlers/base.py", line 124, in _get_response
    response = wrapped_callback(request, *callback_args, **callback_kwargs)
  File "/venv/lib/python3.5/site-packages/django/views/decorators/csrf.py", line 54, in wrapped_view
    return view_func(*args, **kwargs)
  File "/venv/lib/python3.5/site-packages/django/views/generic/base.py", line 68, in view
    return self.dispatch(request, *args, **kwargs)
  File "/venv/lib/python3.5/site-packages/rest_framework/views.py", line 483, in dispatch
    response = self.handle_exception(exc)
  File "/venv/lib/python3.5/site-packages/rest_framework/views.py", line 443, in handle_exception
    self.raise_uncaught_exception(exc)
  File "/venv/lib/python3.5/site-packages/rest_framework/views.py", line 480, in dispatch
    response = handler(request, *args, **kwargs)
  File "/code/api/views.py", line 1727, in get
    res = get_location_sunburst()
  File "/code/api/api_util.py", line 330, in get_location_sunburst
    df.columns.tolist()).size().reset_index().rename(columns={4: 'count'})
  File "/venv/lib/python3.5/site-packages/pandas/core/generic.py", line 6665, in groupby
    observed=observed, **kwargs)
  File "/venv/lib/python3.5/site-packages/pandas/core/groupby/groupby.py", line 2152, in groupby
    return klass(obj, by, **kwds)
  File "/venv/lib/python3.5/site-packages/pandas/core/groupby/groupby.py", line 599, in __init__
    mutated=self.mutated)
  File "/venv/lib/python3.5/site-packages/pandas/core/groupby/groupby.py", line 3320, in _get_grouper
    raise ValueError('No group keys passed!')
ValueError: No group keys passed!
serializing photos of faces took 0.01 seconds
/venv/lib/python3.5/site-packages/rest_framework/pagination.py:198: UnorderedObjectListWarning: Pagination may yield inc
onsistent results with an unordered object_list: <class 'api.models.Person'> QuerySet.
  paginator = self.django_paginator_class(queryset, page_size)

ownphotos-backend invalid instruction

Backend services are not starting (getting a 502 bad gateway error).

I notice in my backend logs I have this.

...done.
./entrypoint.sh: line 18: 40 Illegal instruction (core dumped) python manage.py shell <<EOF
from api.models import User
User.objects.filter(email='$ADMIN_EMAIL').delete()
User.objects.create_superuser('$ADMIN_USERNAME', '$ADMIN_EMAIL', '$ADMIN_PASSWORD')
EOF
Running backend server...
[2019-01-10 21:24:31 +0000] [43] [INFO] Starting gunicorn 19.8.1
[2019-01-10 21:24:31 +0000] [43] [INFO] Listening at: http://0.0.0.0:8001 (43)
[2019-01-10 21:24:31 +0000] [43] [INFO] Using worker: sync
[2019-01-10 21:24:31 +0000] [47] [INFO] Booting worker with pid: 47
[2019-01-10 21:24:32 +0000] [49] [INFO] Booting worker with pid: 49
[2019-01-10 21:24:33 +0000] [51] [INFO] Booting worker with pid: 51
[2019-01-10 21:24:34 +0000] [53] [INFO] Booting worker with pid: 53

Obviously the big problem is the core dump in the entrypoint.sh from the following lines.

python manage.py shell <<EOF
from api.models import User
User.objects.filter(email='$ADMIN_EMAIL').delete()
User.objects.create_superuser('$ADMIN_USERNAME', '$ADMIN_EMAIL', '$ADMIN_PASSWORD')
EOF

Also seems odd the worker keeps restarting like it can't connect either, probably related.

I'm using the latest images and have tried resetting everything and pruning the volumes many times.. Also the other odd thing is on my Macbook Pro the same docker-compose.yml works just fine but not on my Nextcloud server.

As noted that you were thinking it was odd architecture and it's not it's a Core2 Duo with Intel CPU.

• Architecture: x86_64
• CPU op-mode(s): 32-bit, 64-bit
• Byte Order: Little Endian
• CPU(s): 2
• On-line CPU(s) list: 0,1
• Thread(s) per core: 1
• Core(s) per socket: 2
• Socket(s): 1
• NUMA node(s): 1
• Vendor ID: GenuineIntel
• CPU family: 6
• Model: 23
• Model name: Intel(R) Core(TM)2 Duo CPU T9300 @ 2.50GHz
• Stepping: 6
• CPU MHz: 800.000
• CPU max MHz: 2500.0000
• CPU min MHz: 800.0000
• BogoMIPS: 4999.55
• Virtualization: VT-x
• L1d cache: 32K
• L1i cache: 32K
• L2 cache: 6144K
• NUMA node0 CPU(s): 0,1
• Flags: fpu vme de pse tsc msr pae mce cx8 apic sep mtrr pge mca cmov pat pse36 clflush dts acpi mmx fxsr sse sse2 ss ht tm pbe syscall nx lm constant_tsc arch_perfmon pebs bts rep_good nopl aperfmperf pni dtes64 monitor ds_cpl vmx est tm2 ssse3 cx16 xtpr pdcm sse4_1 lahf_lm kaiser tpr_shadow vnmi flexpriority dtherm

Creating a library for the ownphotos api

Hey all,
More of a feature in progress.
I started writing a python package that lets you interact with ownphotos backend.

You can see it here:
https://pypi.org/project/ownphotos-client/

I hope this will let us have better documentation for all the backend functions.

Help would be appreciated.

At the moment it can log in, and get a list of photos, hope to add paging options as I progress.

If backend is stoped in middle of LongRunningJob it will get stuck in the db forever

Steps to reproduce:

1. start ownphotos
2. run a directory scan with a large directory (few hundred images)
3. stop containers/process before its finished.
4. start service

Expected:
All stuck db jobs should be cleaned .

Actual:
Job hangs finished_at field has NULL.
The issue can be seen using Adminer, looks like this:

Return proper response (not 500) for some face apis

URLs

• /api/facetolabel/
• /api/trainfaces/

should return 200 when there is no labeled faces.

https://github.com/hooram/ownphotos-backend/blob/232f43eec933e591fab48dbe4df37bd181dba04b/api/face_classify.py#L22

https://github.com/hooram/ownphotos-backend/blob/232f43eec933e591fab48dbe4df37bd181dba04b/api/face_classify.py#L52

Feature Idea - Sync the person DB with CarDAV server for easy bootstrapping

I think that syncing the persons database to a remote contact server (CarDAV, Google & others) would be great.

It would:

• Remove manual creation of persons, at least for most of them (all of my closest friends are in my contact list for example)
• Persons could have their face matched automatically, this would allow to perform a face match without any user interaction, yay!
• Names would be updated automatically over time
• Contact pictures will be automatically updated over time (will strengthen face id)

For the 2nd point, of course if a face has been wrongly matched (several persons in one picture) it'll be possible to undo that manually.

I'll be proposing a PR soon to get things started.

docker image for arm distributions

Hi there!

I'm so interesting on experimenting with ownphotos in my modest NAS server, an Odroid HC2. Unfortunately, I'm not able to run it through docker so I presume it is build for x86/x86_64 system and the Odroid uses ARMv7-A processors.

Would it be possible somehow to port the docker image to this architecture? Otherwise, could you point the files I should modify so as to try myself the port? I'm not very familiar with docker configurations more than running or tuning images, but I could give it a try.

Many thanks!!

Add left-right navigation on full page modal photo view

View photo button doesn't work on face dashboard

[Feature] Allow Multiple Users,...

I would really like to use OwnPhotos for more than one user.

We would need different user-logins and datafolders.
Best would be if we put this in the database somewhere.

This way, we could also easily integrate it with nextcloud, if we can use nextcloud as a login provider.

#19

Backend won't boot

Hello, I'm trying to install using docker-compose

wget https://raw.githubusercontent.com/hooram/ownphotos/dev/docker-compose.yml
docker-compose up -d

The backend cannot boot with the logs:

backend.log

Isn't it related to an old numpy version?

Pictures having not EXIF metadata should try to determine date based on filename

Most of the time it's possible to determine the date the picture was taken by parsing the filename.

Docker images for front-end and back-end should be built separately

The current Dockerfile (and the one I updated with #40) build an image that contains both the front-end and the back-end.

It might be a better practice to build them separately, and using Docker Compose to orchestrate all the containers and run everything automatically.

Also, I noticed that the entry point builds JS stuff on startup. I think it might be better if it were built at image build time, as unless changes are placed in a volume, they're lost when the container is stopped/restarted.

How do you add photos to OwnPhotos

I finally got OwnPhotos up and running. Now I want to try it out, but I cannot figure out how to add pictures. Can anyone help?

No connection to backend server

Have used the docker compose method:

wget https://raw.githubusercontent.com/hooram/ownphotos/dev/docker-compose.yml
docker-compose up -d

Once it's finished, I go to localhost:3000, login with admin as user and admin as password and the following red box appears underneath.

I've check the ownphotos-backend container and it seems to be running fine. Any ideas what I'm doing wrong?

Picture not having exif timestamp should also be displayed in the `Browse` page

Picture not having exif timestamp should also be displayed in the Browse page.

It is currently not the case as the API filter photo not having a timestamp.

Two solutions :

1. Return an Unknown album from the API containing those pictures
2. Set a default timestamp

What do you think of it? @hooram

I'll probably implement it in a few days, but I'd like to have your opinion so my work won't go straight to the trash.

I hope PR are welcomed :)

Share photos/albums

• allow multiple users
• signup page
  • admin settings to allow signups
• set owners to photos and albums
• user can only train / label faces in photos that belong to him
• set permissions to allow users to see only their photos / albums
• admin user management settings to set a specific subdirectory that belong to a specific user
  • If you point the nextcloud data root directory on the host to container's /data when running a docker container, the admin can specify a given user's nextcloud data folder. When this user runs 'scan photos', it will import photos from the specified directory.
• public sharing
  • photos (no plans for albums)
• sharing between users
  • sharing photos between users (view only)
  • sharing manually created albums between users (view only)

Not possible to finish the setup

Hello,

i use the Docker-compose.yml file from this github repo.

But when i try to login i get the Message "no connection to the backend server"
when i look at http://domain.tld:3000/api i see this error:

`

Request Method:	GET
http://backend/api/auth/token/obtain
2.1.2
OperationalError
FATAL: password authentication failed for user "postgres"
/venv/lib/python3.5/site-packages/psycopg2/init.py in connect, line 130
/venv/bin/python3
3.5.2
['', '/code', '/venv/bin', '/venv/lib/python35.zip', '/venv/lib/python3.5', '/venv/lib/python3.5/plat-x86_64-linux-gnu', '/venv/lib/python3.5/lib-dynload', '/usr/lib/python3.5', '/usr/lib/python3.5/plat-x86_64-linux-gnu', '/venv/lib/python3.5/site-packages', '/venv/lib/python3.5/site-packages/dlib-19.16.99-py3.5-linux-x86_64.egg', '/venv/lib/python3.5/site-packages/IPython/extensions', '/root/.ipython']
Wed, 16 Jan 2019 17:58:31 +0000

`

The Settings are correct in the docker-compose.yml file

` - POSTGRES_PASSWORD=AaAa1234

• POSTGRES_DB=ownphotos

[...]
- DB_BACKEND=postgresql
- DB_NAME=ownphotos
- DB_USER=postgres
# This db password is internal, you can change it if you want, but also change it in ownphotos-db container
- DB_PASS=AaAa1234

• DB_HOST=ownphotos-db

`

Docker Log:

backend log:

2019-01-16 17:58:31.218 UTC [58] FATAL: password authentication failed for user "postgres" 2019-01-16 17:58:31.218 UTC [58] DETAIL: Password does not match for user "postgres". Connection matched pg_hba.conf line 95: "host all all all md5"

The passwords are the same...

Could not find a version that satisfies the requirement en-core-web-sm==2.0.0 mkl-random==1.0.1

Getting

Could not find a version that satisfies the requirement en-core-web-sm==2.0.0

And after removing that I get:

Could not find a version that satisfies the requirement mkl-random==1.0.1

When running pip install -r requirements.txt from this repo.

Handle deleted/moved files in source directory

Errors during photo scan

I used the trick mentioned in #51 to view the progress of the scan for new photos. Here I noticed lots of error messages concerning the date times. It also seems that this affects the results as many photos have missing timestamps.

It must be said that long after the counter reached 100%, the "Scan Photos" is still active.

12it [00:04,  2.65it/s]1
1
4832548215a188a3bfd991c2071300501
64ee96024ed9f9e43bee7615212a9f4b1
/venv/lib/python3.5/site-packages/django/db/models/fields/__init__.py:1421: RuntimeWarning: DateTimeField Photo.exif_tim
estamp received a naive datetime (2017-10-28 12:55:41) while time zone support is active.
  RuntimeWarning)
photo not added to album place
Error in directory scan, got exception:
duplicate key value violates unique constraint "api_photo_pkey"
DETAIL:  Key (image_hash)=(64ee96024ed9f9e43bee7615212a9f4b1) already exists.

Traceback (most recent call last):
  File "/venv/lib/python3.5/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
psycopg2.IntegrityError: duplicate key value violates unique constraint "api_photo_pkey"
DETAIL:  Key (image_hash)=(64ee96024ed9f9e43bee7615212a9f4b1) already exists.


The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/code/api/directory_watcher.py", line 117, in handle_new_image
    photo._add_to_album_thing()
  File "/code/api/models.py", line 511, in _add_to_album_thing
    album_thing.cover_photos.add(self)
  File "/venv/lib/python3.5/site-packages/django/db/models/fields/related_descriptors.py", line 926, in add
    self._add_items(self.source_field_name, self.target_field_name, *objs)
  File "/venv/lib/python3.5/site-packages/django/db/models/fields/related_descriptors.py", line 1075, in _add_items
    new_ids.difference_update(vals)
  File "/venv/lib/python3.5/site-packages/django/db/models/query.py", line 268, in __iter__
    self._fetch_all()
  File "/venv/lib/python3.5/site-packages/django/db/models/query.py", line 1186, in _fetch_all
    self._result_cache = list(self._iterable_class(self))
  File "/venv/lib/python3.5/site-packages/django/db/models/query.py", line 176, in __iter__
    for row in compiler.results_iter(chunked_fetch=self.chunked_fetch, chunk_size=self.chunk_size):
  File "/venv/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 1017, in results_iter
    results = self.execute_sql(MULTI, chunked_fetch=chunked_fetch, chunk_size=chunk_size)
  File "/venv/lib/python3.5/site-packages/django/db/models/sql/compiler.py", line 1065, in execute_sql
    cursor.execute(sql, params)
  File "/venv/lib/python3.5/site-packages/django/db/backends/utils.py", line 100, in execute
    return super().execute(sql, params)
  File "/venv/lib/python3.5/site-packages/django/db/backends/utils.py", line 68, in execute
    return self._execute_with_wrappers(sql, params, many=False, executor=self._execute)
  File "/venv/lib/python3.5/site-packages/django/db/backends/utils.py", line 77, in _execute_with_wrappers
    return executor(sql, params, many, context)
  File "/venv/lib/python3.5/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
  File "/venv/lib/python3.5/site-packages/django/db/utils.py", line 89, in __exit__
    raise dj_exc_value.with_traceback(traceback) from exc_value
  File "/venv/lib/python3.5/site-packages/django/db/backends/utils.py", line 85, in _execute
    return self.cursor.execute(sql, params)
django.db.utils.IntegrityError: duplicate key value violates unique constraint "api_photo_pkey"
DETAIL:  Key (image_hash)=(64ee96024ed9f9e43bee7615212a9f4b1) already exists.

The error also mentions duplicate key, while the scan details seem to indicate that an md5 check should avoid this.