GithubHelp home page GithubHelp logo

hotosm / osm-tasking-manager2 Goto Github PK

View Code? Open in Web Editor NEW
425.0 57.0 156.0 3.56 MB

Designed and built for Humanitarian OpenStreetMap Team collaborative emergency/disaster mapping, the OSM Tasking Manager 2.0 divides an area into individual squares that can be rapidly mapped by thousands of volunteers.

Home Page: http://tasks.hotosm.org

License: Other

Python 27.98% Mako 14.28% Shell 0.17% CSS 27.51% HTML 0.68% JavaScript 29.38%

osm-tasking-manager2's Introduction

OpenStreetMap Tasking Manager v2

Build Status Coverage Status

About

OpenStreetMap Tasking Manager enables collaborative work on specific areas in OpenStreetMap by defining clear workflows to be achieved and by breaking tasks down into pieces.

This is version 2.0 of the Tasking Manager. Most development work is now taking place on version 3.0

V2 Tasking Manager still powers many Tasking Manager installations. It is written in Python using the Pyramid framework.

Installation

First clone the git repository:

git clone --recursive git://github.com/hotosm/osm-tasking-manager2.git

Installing OSMTM in a Virtual Python environment is recommended.

To create a virtual Python environment:

cd osm-tasking-manager2
sudo easy_install virtualenv
virtualenv --no-site-packages env
./env/bin/pip install -r requirements.txt

Tip: if you encounter problems installing psycopg2 especially on Mac, it is recommended to follow advice proposed here.

Database

OSMTM requires a PostgreSQL/PostGIS database. Version 2.3 or higher of PostGIS is required.

First create a database user/role named www-data:

sudo -u postgres createuser -SDRP www-data

Then create a database named osmtm:

sudo -u postgres createdb -T template0 osmtm -E UTF8 -O www-data
sudo -u postgres psql -d osmtm -c "CREATE EXTENSION postgis;"

Local settings

You certainly will need some local specific settings, like the db user or password. For this, you can create a local.ini file in the project root, where you can then override every needed setting. For example:

[app:main]
sqlalchemy.url = postgresql://www-data:PASSWORD@localhost/osmtm
default_comment_prefix = #yourinstancename-project
check_expiration_interval = 60

Note: you can also put your local settings file anywhere else on your file system, and then create a LOCAL_SETTINGS_PATH environment variable to make the project aware of this.

Currently, these are the settings you can over-ride:

  • sqlalchemy.url: Postgres URL to use for database connection
  • default_comment_prefix: Default prefix to use for changeset comments, defaults to #hotosm-project
  • check_expiration_interval: The interval at which the database should be checked for expired tasks, in seconds. Defaults to 5 seconds.

Populate the database

You're now ready to do the initial population of the database. An initialize_osmtm_db script is available in the virtual env for that:

./env/bin/initialize_osmtm_db

Launch the application

./env/bin/pserve --reload development.ini

You will see messages, hopefully including a line like serving on http://0.0.0.0:6543. Visit that address in your web browser - you should see your local Tasking Manager!

Running the application behind proxy server

You need to make the following changes to the osmtm/views/osmauth.py file.

# Add the below lines in the starting
import httplib2
httplib2.debuglevel = 4
PROXY = httplib2.ProxyInfo(httplib2.socks.PROXY_TYPE_HTTP_NO_TUNNEL, 'PROXY-SERVER', PROXY-PORT)

NOTE: Replace the PROXY-SERVER with your proxy server address and PROXY-PORT with the port number on which your proxy is established.

# then add "proxy_info=PROXY" for every line in oauth.Client.
client = oauth.Client(consumer, proxy_info=PROXY)

client = oauth.Client(consumer, token, proxy_info=PROXY)

Replace the host address in the development.ini file with your IP address of the system.

host='SYSTEM-IP-ADDRESS'

Styles

The CSS stylesheet are compiled using less. Launch the following command as soon as you change the css:

lessc -ru osmtm/static/css/main.less > osmtm/static/css/main.css

Launch the application

env/bin/pserve --reload development.ini

Tests

The tests use a separate database. Create that database first:

sudo -u postgres createdb -O www-data osmtm_tests
sudo -u postgres psql -d osmtm_tests -c "CREATE EXTENSION postgis;"

Create a local.test.inifile in the project root, where you will add the settings for the database connection. For example:

[app:main]
sqlalchemy.url = postgresql://www-data:www-data@localhost/osmtm_tests

To run the tests, use the following command:

./env/bin/nosetests

Application deployment

  1. pull latest updates from the repository: git pull origin
  2. update the submodules: git submodule update --init
  3. update/install python modules: ./env/bin/pip install -r requirements.txt
  4. create database dump: pg_dump -Fc -f osmtm2_latest.dmp database_name
  5. run database migrations: ./env/bin/alembic upgrade head
  6. compile messages: ./env/bin/python setup.py compile_catalog
  7. restart application server

Installation as a mod_wsgi Application

an example Apache configuration file

WSGIDaemonProcess OSMTM_process user=ubuntu group=ubuntu processes=1 \
        threads=4 \
        python-path=/home/ubuntu/osm-tasking-manager2:/home/ubuntu/osm-tasking-manager2/env/lib/python2.7/site-packages
WSGIRestrictStdin Off

<VirtualHost *:80>
        ServerName example.org
        # Use only 1 Python sub-interpreter.  Multiple sub-interpreters
        # play badly with C extensions.
        WSGIPassAuthorization On
        WSGIScriptAlias /osmtm /home/ubuntu/osm-tasking-manager2/env/OSMTM.wsgi

        <Location />
            WSGIProcessGroup OSMTM_process
            WSGIApplicationGroup %{GLOBAL}
        </Location>

        <Directory /home/ubuntu/osm-tasking-manager2/env>
            <Files OSMTM.wsgi>
                Require all granted
            </Files>
            Order allow,deny
            Allow from all
        </Directory>

        LogLevel warn

        CustomLog /var/log/apache2/osmtm-access.log combined
        ErrorLog /var/log/apache2/osmtm-error.log
</VirtualHost>

Customization

In case you install your own instance you may want to customize its look and feel. You can do so by modifying the following files: osmtm/templates/custom.mako, osmtm/static/css/custom.less & osmtm/static/img/favicon.ico

Localization

OSMTM is localized on Transifex service.

It's possible to create translations for two resources: current and master. Current resource represents currently deployed instance of the OSMTM http://tasks.hotosm.org. Master resource represents actively developed code that will become current once it gets deployed.

Initializing translation files

In general managing translation files involves:

  • generate pot file: ./env/bin/python setup.py extract_messages
  • initialize a message catalogue file (english): ./env/bin/python setup.py init_catalog -l en
    • if the catalogue is already created use: ./env/bin/python setup.py update_catalog
  • eventually compile messages: ./env/bin/python setup.py compile_catalog
  • append new language to the available_languages configuration variable in production.ini file, for example available_languages = en fr

Using Transifex service

  • in the project top level directory, initialize transifex service (after installing transifex-client): tx init
    • the init process will ask for service URL and username/password, which will be saved to ~/.transifexrc file
  • if the project has already been initialized, but you are missing ~/.transifexrc, create the file and modify it's access privileges chmod 600 ~/.transifexrc

Example .transifexrc file:

[https://www.transifex.com]
hostname = https://www.transifex.com
password = my_super_password
token =
username = my_transifex_username
  • after creating the project on the Transifex service: osm-tasking-manager2, generate the pot file, and add it as a master resource on the project, full resource name, in this case, is osm-tasking-manager2.master

Setting up resources

  • add initial source file, in this case English:
    • tx set --source -r osm-tasking-manager2.master -l en osmtm/locale/en/LC_MESSAGES/osmtm.po
  • add existing source files, in this case French:
    • tx set -r osm-tasking-manager2.master -l fr osmtm/locale/fr/LC_MESSAGES/osmtm.po
  • push resources on the transifex service (this will overwrite any existing resources on the service)
    • tx push -s -t
    • -s - pushes source files (English)
    • -t - pushes translation files (French)

Pulling changes

  • to pull latest changes from Transifex service execute: tx pull
    • this will pull every available language from the Transifex service, even the languages that are not yet mapped
    • if the language is not mapped, translated language file will be saved to the local .tx directory, which is not what we want so we need to define the mapping
    • for example, if we want to correctly map Croaian language you need to execute: tx set -r osm-tasking-manager2.master -l hr osmtm/locale/hr/LC_MESSAGES/osmtm.po
      • there is no need to create the actual po file or the directory structure, Transifex client will manage that for us
  • there is also a possibility to pull specific languages: tx pull -l hr
  • or pull only languages that have a certain completeness percentage: tx pull --minimum-perc=90

Transifex workflow

Updating source files, locally and on the service

  • update pot and source po file

    • ./env/bin/python setup.py extract_messages
    • ./env/bin/python setup.py init_catalog -l en
  • push the source file to Transifex service

    • tx push -s

Pull latest changes from the service

  • when adding a new language:
    • we need to configure local mapping: tx set -r osm-tasking-manager2.master -l hr osmtm/locale/hr/LC_MESSAGES/osmtm.po
    • append the new language to the available_languages configuration variable in production.ini file: available_languages = en fr hr
  • after there are some translation updates, pull latest changes for mapped resources
    • tx pull -l fr,hr
  • compile language files
    • ./env/bin/python setup.py compile_catalog

API

The tasking manager exposes some of its functionality via a RESTful API. It is documented on the following page: https://github.com/hotosm/osm-tasking-manager2/wiki/API.

osm-tasking-manager2's People

Contributors

alejandroscf avatar audkuan avatar aurimasf avatar baladkb avatar bgirardot avatar clockwerx avatar cquest avatar danstowell avatar david-hotosm avatar derickr avatar dkogan avatar dodobas avatar elemoine avatar ethan-nelson avatar filbertkm avatar geobrando avatar greenie10 avatar gustry avatar harry-wood avatar jarmokivekas avatar kaibot3000 avatar mikelmaron avatar milovanderlinden avatar oeon avatar pgiraud avatar piskvor avatar ravilacoya avatar skalca avatar stefanb avatar tonio avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

osm-tasking-manager2's Issues

required mapping experience

Some projects are appropriate for beginners whereas some are for experienced mappers.
Let's have a field in the database for that.

Mark different levels as done

Instead of one simple "Mark as done" for an entire square, it would be great to be able to mark different levels as done. Those levels could possibly be created by the project admin and wouldn't necessarily have to be hierarchical. We could then start with low detail and come back to a square for more detail.

For example, being able to check these separately as done: highway=unclassified and up, highway=track, highway=path, rivers, streams, places, buildings, or whatever the project admin chooses.

There could also be a "Mark all as done" option.

Force admins to set an imagery link of some kind to be used for each job.

On the /edit page for the jobs, it would be good to have a dropdown menu box listing what kind of imagery to use. The box should have three entries:

  • The first (and default selected entry) should be something like "Choose background imagery", but trying to save the page with this setting will result in an error.
  • Then Bing imagery which, if selected, results in some kind of notice on the task instructions page where the imagery link would be indicating to just use the default Bing imagery as a background
  • The last option should be a "Custom URL", which should enable the current imagery URL box that is there and then add the imagery URL links like it does now.

opening iD should ask for license agreement

To be checked in V2.

one feature that needs fairly immediate attention for both v1 and v2 relates
to the license acknowledgment and its behavior in the application buttons.

For instance, currently if someone has not acknowledged the NextView license
(only located on the bottom of the Instructions tab), and they select a tile
and open directly in iD from the Task tab, then iD will not throw an error
and will simply load bing imagery as the default. This causes massive
confusion for new users and will only continue to grow as more donated (and
separately licensed) imagery grows.

I would recommend that the Task Manager intercept the open in iD call if the
user does not have the right license creditials and take them directly to
the acknowledgement screen.

ERROR: role "www-data" does not exist

Hi - when running the installation instructions for osmtm2 (on ubuntu), I get as far as the postgres install script. When I run it the output is:

 dropdb: database removal failed: ERROR:  database "osmtm" does not exist
 createdb: database creation failed: ERROR:  role "www-data" does not exist
 dropdb: database removal failed: ERROR:  database "osmtm_tests" does not exist
 createdb: database creation failed: ERROR:  role "www-data" does not exist

I think the "dropdb" errors are safe to ignore. But the "createdb" ones are not - it fails to create the db, and the TM can't work. Is there a step missing from the install instructions, or from the postgres script?

Ubuntu installation incomplete: "install_postgis_mac.sh" equivalent?

Thanks for updating the instructions (from my previous issue). I notice that there's no Ubuntu equivalent for "install_postgis_mac.sh". So when I run the server it simply says the database doesn't exist (of course).

By looking at "install_postgis_mac.sh" and https://gist.github.com/1081907 I've done the following:

sudo su postgres
dropdb postgis_template
createdb postgis_template
createlang plpgsql postgis_template

# Import Postgis Data
psql -d postgis_template -f /usr/share/postgresql/9.1/contrib/postgis-1.5/postgis.sql
psql -d postgis_template -f /usr/share/postgresql/9.1/contrib/postgis-1.5/spatial_ref_sys.sql
# psql -d postgis_template -f /usr/share/postgresql/9.1/contrib/postgis-1.5/rtpostgis.sql             ############# not found
# psql -d postgis_template -f /usr/share/postgresql/9.1/contrib/postgis-1.5/legacy_minimal.sql        ############# not found 
# psql -d postgis_template -f /usr/share/postgresql/9.1/contrib/postgis-1.5/topology.sql              ############# not found

#Test if works
psql -d postgis_template -c "SELECT postgis_full_version();"

#3. Set template permissions to gisgroup
createuser -R -S -L -D -I gisgroup

psql -d postgis_template -c "ALTER DATABASE postgis_template OWNER TO gisgroup;"
psql -d postgis_template -c "ALTER TABLE geometry_columns OWNER TO gisgroup;"
psql -d postgis_template -c "ALTER TABLE spatial_ref_sys OWNER TO gisgroup;"
psql -d postgis_template -c "CREATE SCHEMA gis_schema AUTHORIZATION gisgroup;"

#4. Adds your app's user
createuser -i -l -S -R -d www-data 

psql -d postgres -c 'GRANT gisgroup TO "www-data";'

This all seems to go fine EXCEPT note that I've commented out three postgis SQL files that simply do not exist even though I have installed postgis-1.5.

When I run env/bin/pserve development.ini and go to the homepage I get

ProgrammingError: (ProgrammingError) relation "users" does not exist
LINE 3: FROM users) AS anon_1
             ^
 'SELECT count(*) AS count_1 \nFROM (SELECT users.username AS users_username \nFROM users) AS anon_1' {}

Add a remote link for the export.osm file on tasks

As reported by @AndrewBuck on v1:

Currently there is a .osm file associated with each job in a task that is the bounding box of that task. The process of doawnloading and then opening that .osm file for each tile you take is an extra step that consumes time. The josm remote plugin supports loading a .osm file from a url as documented at: http://wiki.openstreetmap.org/wiki/JOSM/Plugins/RemoteControl#import_command

The docs there say it adds the .osm data to the current data set, so it may need an additional parameter added to the remote control to make it add the dataset in a new layer, or this may already exist and the documentaion may just be lacking. If this can be made to work, however, it would be nice to have a remote url similar to the one that is already there for loading the area. This allows someone to download a larger area and then easily use the remote link on new tasks to just draw in the bbox which works a lot better when you are working in an area with lots of roads/streams/etc that go outside the bbox.

Don't compute percent done on-the-fly

Compute done/validate percent each time a task state is changed. This should improve performances a bit.
But this requires new fields in the database.

Page layout not adapted to small screens

TM Page layout should take into account the small screen devices.

For smaller windows (width of 640 pixels or less), the Text and Map columns are truncated since they do not fit into the window.

The CSS container, spanxx and offsetxx selectors define the width attribute in pixels.

This could be modified easily to adapt to smaller screens, either by defining the attribute in percentage of with a specific rule for the smaller screens.

t

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    ๐Ÿ–– Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. ๐Ÿ“Š๐Ÿ“ˆ๐ŸŽ‰

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google โค๏ธ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.