hadryan / recognize

👁 👂 Smart media tagging for Nextcloud: recognizes faces, objects, landscapes, music genres

Recognize for Nextcloud

Smart media tagging for Nextcloud

This app goes through your media collection and adds fitting tags, automatically categorizing your photos and music.

• 📷 👪 Recognizes faces from contact photos
• 📷 🏔 Recognizes animals, landscapes, food, vehicles, buildings and other objects
• 📷 🗼 Recognizes landmarks and monuments
• 👂 🎵 Recognizes music genres
• ⚡ Tagging works via Nextcloud's Collaborative Tags, allowing access by any of your apps
  • 👂 listen to your tagged music with the audioplayer app
  • 📷 view your tagged photos with the photos app

Note: This project is quite young and thus still a bit rough around the edges.

Examples

(Screenshot by @_DigitalWriter_)

Privacy

This app does not send any sensitive data to cloud providers or similar services. All image processing is done on your nextcloud machine, using Tensorflow.js running in Node.js, which comes bundled with this app.

Categories

This is the list of recognized things and which categories they are currently mapped to. I'm happy to accept pull requests for this file to fine tune predictions.

Behind the scenes

Recognize uses

• a pre-trained EfficientNet v2 model for ImageNet object detection.
• a pre-trained model trained on the Landmarks v1 dataset for landmark recognition.
• face-api.js to extract and compare face features.
• a Musicnn neural network architecture to classify audio files into music genres. Also see the original musicnn repository.

Install

Requirements

• php 7.3 and above
• App "collaborative tags" enabled
• For native speed:
  • Processor: x86 64bit (with support for AVX instructions)
  • System with glibc (usually the norm on Linux; Alpine linux and FreeBSD are not such systems)
• For sub-native speed (using JavaScript mode)
  • Processor: x86 64bit, arm64, armv7l (no AVX needed)
  • System with glibc (Required by the bundled node.js binary; alternatively you can set the path to custom node.js binary)
• ~4GB of free RAM (if you're cutting it close, make sure you have some swap available)

One click

Go to "Apps" in your nextcloud, search for "recognize" and click install (currently only works on Nextcloud v22+. If you're below that, you'll need to install manually).

Configuration

Any configuration is done in Settings/Recognize of your Nextcloud instance.

You can also ignore directories (and their children) by adding a .noimage, .nomusic or .nomedia file in them.

Manual install

Dependencies

• git
• Node.js and npm
• php
• composer

Setup

cd /path/to/nextcloud/apps/
git clone https://github.com/marcelklehr/recognize.git
cd recognize
make

Maintainers

• Marcel Klehr

Donate

If you'd like to support the creation and maintenance of this software, consider donating.

Contribute

We always welcome contributions. Have an issue or an idea for a feature? Let us know. Additionally, we happily accept pull requests.

In order to make the process run more smoothly, you can make sure of the following things:

• Announce that you're working on a feature/bugfix in the relevant issue
• Make sure the tests are passing
• If you have any questions you can let the maintainers above know privately via email, or simply open an issue on github

Please read the Code of Conduct. This document offers some guidance to ensure Nextcloud participants can cooperate effectively in a positive and inspiring atmosphere, and to explain how together we can strengthen and support each other.

More information on how to contribute: https://nextcloud.com/contribute/

Happy hacking ❤️

License

This software is licensed under the terms of the AGPL written by the Free Software Foundation and available at COPYING.

The recognize logo Smart tag by Xinh Studio from the Noun Project is licensed under a Creative Commons Attribution license.