mytardis / mytardis-app-atom

MyTardis app for Atom consumption.

MyTardis Atom App

This app can be used to ingest datasets via Atom. Please see tests/atom_test for format examples.

New metadata is ingested first, with data files being copied asynchronously afterwards.

Installation

Symlink this app into a MyTardis tardis/apps directory. The preferred name for the app is atom.

Configuration

Celery is used to schedule periodic file ingestion.

The atom_ingest.walk_feeds task takes a variable number of feeds and updates them. Here's an example for settings.py that checks two Picassa feeds every 30 seconds:

CELERYBEAT_SCHEDULE = dict(CELERYBEAT_SCHEDULE.items() + {
  "update-feeds": {
    "task": "atom_ingest.walk_feeds",
    "schedule": timedelta(seconds=30),
    "args": ('http://example.org/feed.atom',
             'http://example.test/feed.atom')
  },
}.items())

You must run celerybeat and celeryd for the scheduled updates to be performed. MyTardis provides a Procfile for this purpose, but you can run both adhoc with:

bin/django celeryd --beat

HTTP Basic password protection is available via settings.py in MyTardis:

REMOTE_SERVER_CREDENTIALS = [
  ('http://localhost:4272/', 'username', 'password')
]

In a production environment, you should combine HTTP Basic password protection with SSL for security.

mytardis-app-atom's Issues

Experiment creation permission doesn't inherit from groups

(As discussed with Steve A and Grischa)

1. Create user U with no permissions
2. Create group G with all the permissions
3. Assign U to G
4. U can't create experiments.

Follow 'next' page links

ATOM allows the following of 'next page' links to serve more entries. The nodejs-based atom-dataset-provider supports this, for example.

The harvester needs to know how to follow 'next' links to harvest as many updated entries as needed.

Merge Security Features from steveandroulakis fork

2 features:

• basicauth security options
• serving via SSL

In the CVL branch on https://github.com/steveandroulakis/atom-dataset-provider/branches should be merged.

Celery Issues

Hey Tim,

Here's a screenie (for coloured syntax!) of the error I'm encountering trying to run django celery --beat. Looks like something isn't being wired up.

http://i.imgur.com/ZvLP7.png

(I think imgur hates OS X screenies.. renders large)

I'm running a fresh buildout install, all tests pass, preliminary clicking around of the site has worked, and the celery tables etc have been created fine.

Cheers,
Steve