nansencenter / geo-spaas-vagrant Goto Github PK

View Code? Open in Web Editor NEW

0.0 0.0 1.0 496 KB

Vagrant configuration for Geo-Scientific Platform as a Service

License: GNU General Public License v3.0

Python 100.00%

geo-spaas-vagrant's People

Contributors

Watchers

Forkers

korvinos

geo-spaas-vagrant's Issues

GEOSException: Could not parse version info string

Shared folders

At the moment we define shared folders for each virtual machine, e.g., shared/develop_vm. Is this necessary? Why not have just one folder shared/code or something like that?

Provisioning fails due to gdal error

Suddenly, I can't provision my vm's. They fail at nansat installation because gdal isn't installed. Has anyone changed anything in conda?

noaa_ndbc is not a part of geospaas

Ansible playbook for geospaas settings.py should be fixed according to the changes in geospaas core applications. Since noaa_ndbc is a separate module now, the app must be set separately

Dynamic generation of static_root, media_root, etc

So far generation of directory for static files is hardcoded in django_deploy.
Instead ti should be dynamic. Four _root variables are needed:
static_root
media_root
product_root
download_root

django_deploy shall make sure these folders exists and add them to setting.py

Tasks and defaults in roles/geospaas_catalog need better documentation

Please explain to make it easier to update...

Create documentation

We need to decide about documentation - e.g., what tools and where to store it. Also we need to summarize content...

Let's list things that need to be covered in the docs in comments under this issue as a start.. This can be done in relation to the NSC pilot project.

Unable to load the SpatiaLite library extension "mod_spatialite" because: mod_spatialite.so: undefined symbol: libiconv

I am trying to deploy VM on my mac and get an error connected with spatialite. The error is related to an issue in mod_spatialite: mod_spatialite.so: undefined symbol: libiconv. I have tried to find a solution but it has been unsuccessful yet. Installation of libiconv from the source did not help

Error report

vagrant@ubuntu-xenial:~/geospaas_doppler$ python manage.py makemigrations
/home/vagrant/miniconda/lib/python2.7/site-packages/nansat-1.0.20-py2.7-linux-x86_64.egg/nansat/nansatmap.py:47: NansatFutureWarning: Nansatmap() will be disabled in Nansat 1.1. and moved to a separate packageIt is not covered by unittests intentionally.
  warnings.warn(NANSATMAP_WARNING, NansatFutureWarning)
Traceback (most recent call last):
  File "manage.py", line 22, in <module>
    execute_from_command_line(sys.argv)
  File "/home/vagrant/miniconda/lib/python2.7/site-packages/django/core/management/__init__.py", line 364, in execute_from_command_line
    utility.execute()
  File "/home/vagrant/miniconda/lib/python2.7/site-packages/django/core/management/__init__.py", line 356, in execute
    self.fetch_command(subcommand).run_from_argv(self.argv)
  File "/home/vagrant/miniconda/lib/python2.7/site-packages/django/core/management/base.py", line 283, in run_from_argv
    self.execute(*args, **cmd_options)
  File "/home/vagrant/miniconda/lib/python2.7/site-packages/django/core/management/base.py", line 330, in execute
    output = self.handle(*args, **options)
  File "/home/vagrant/miniconda/lib/python2.7/site-packages/django/core/management/commands/makemigrations.py", line 110, in handle
    loader.check_consistent_history(connection)
  File "/home/vagrant/miniconda/lib/python2.7/site-packages/django/db/migrations/loader.py", line 282, in check_consistent_history
    applied = recorder.applied_migrations()
  File "/home/vagrant/miniconda/lib/python2.7/site-packages/django/db/migrations/recorder.py", line 65, in applied_migrations
    self.ensure_schema()
  File "/home/vagrant/miniconda/lib/python2.7/site-packages/django/db/migrations/recorder.py", line 52, in ensure_schema
    if self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor()):
  File "/home/vagrant/miniconda/lib/python2.7/site-packages/django/db/backends/base/base.py", line 254, in cursor
    return self._cursor()
  File "/home/vagrant/miniconda/lib/python2.7/site-packages/django/db/backends/base/base.py", line 229, in _cursor
    self.ensure_connection()
  File "/home/vagrant/miniconda/lib/python2.7/site-packages/django/db/backends/base/base.py", line 213, in ensure_connection
    self.connect()
  File "/home/vagrant/miniconda/lib/python2.7/site-packages/django/db/backends/base/base.py", line 189, in connect
    self.connection = self.get_new_connection(conn_params)
  File "/home/vagrant/miniconda/lib/python2.7/site-packages/django/contrib/gis/db/backends/spatialite/base.py", line 65, in get_new_connection
    six.reraise(ImproperlyConfigured, ImproperlyConfigured(new_msg), sys.exc_info()[2])
  File "/home/vagrant/miniconda/lib/python2.7/site-packages/django/contrib/gis/db/backends/spatialite/base.py", line 60, in get_new_connection
    cur.execute("SELECT load_extension(%s)", (self.spatialite_lib,))
  File "/home/vagrant/miniconda/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py", line 328, in execute
    return Database.Cursor.execute(self, query, params)
django.core.exceptions.ImproperlyConfigured: Unable to load the SpatiaLite library extension "mod_spatialite" because: /home/vagrant/miniconda/lib/././mod_spatialite.so: undefined symbol: libiconv

Spatialite error with course vm. Linux

TASK [geospaas_catalog : Setup DB | Run makemigrations on django-geo-spaas project] ***
task path: /vagrant/provisioning/roles/geospaas_catalog/tasks/setup_db.yml:2
Using module file /usr/local/lib/python2.7/dist-packages/ansible/modules/web_infrastructure/django_manage.py
<192.168.33.10> ESTABLISH LOCAL CONNECTION FOR USER: vagrant
<192.168.33.10> EXEC /bin/sh -c 'echo ~ && sleep 0'
<192.168.33.10> EXEC /bin/sh -c '( umask 77 && mkdir -p "` echo /home/vagrant/.ansible/tmp/ansible-tmp-1513717508.43-226667677386440 `" && echo ansible-tmp-1513717508.43-226667677386440="` echo /home/vagrant/.ansible/tmp/ansible-tmp-1513717508.43-226667677386440 `" ) && sleep 0'
<192.168.33.10> PUT /tmp/tmpRtKDaL TO /home/vagrant/.ansible/tmp/ansible-tmp-1513717508.43-226667677386440/django_manage.py
<192.168.33.10> EXEC /bin/sh -c 'chmod u+x /home/vagrant/.ansible/tmp/ansible-tmp-1513717508.43-226667677386440/ /home/vagrant/.ansible/tmp/ansible-tmp-1513717508.43-226667677386440/django_manage.py && sleep 0'
<192.168.33.10> EXEC /bin/sh -c 'GDAL_DATA=/home/vagrant/miniconda/share/gdal PATH=/home/vagrant/miniconda/bin:/usr/sbin:/usr/bin:/sbin:/bin LC_ALL=en_US.UTF-8 /usr/bin/python /home/vagrant/.ansible/tmp/ansible-tmp-1513717508.43-226667677386440/django_manage.py; rm -rf "/home/vagrant/.ansible/tmp/ansible-tmp-1513717508.43-226667677386440/" > /dev/null 2>&1 && sleep 0'
fatal: [192.168.33.10]: FAILED! => {
    "changed": false, 
    "cmd": "./manage.py makemigrations --settings=geospaas_project.settings --pythonpath=/vagrant/shared/course_vm/geospaas_project", 
    "invocation": {
        "module_args": {
            "app_path": "/vagrant/shared/course_vm/geospaas_project", 
            "apps": null, 
            "cache_table": null, 
            "clear": null, 
            "command": "makemigrations", 
            "database": null, 
            "failfast": false, 
            "fixtures": null, 
            "link": null, 
            "liveserver": null, 
            "merge": null, 
            "pythonpath": "/vagrant/shared/course_vm/geospaas_project", 
            "settings": "geospaas_project.settings", 
            "skip": null, 
            "testrunner": null, 
            "virtualenv": null
        }
    }, 
    "msg": "\n:stderr: Traceback (most recent call last):\n  File \"./manage.py\", line 22, in <module>\n    execute_from_command_line(sys.argv)\n  File \"/home/vagrant/miniconda/lib/python2.7/site-packages/django/core/management/__init__.py\", line 364, in execute_from_command_line\n    utility.execute()\n  File \"/home/vagrant/miniconda/lib/python2.7/site-packages/django/core/management/__init__.py\", line 356, in execute\n    self.fetch_command(subcommand).run_from_argv(self.argv)\n  File \"/home/vagrant/miniconda/lib/python2.7/site-packages/django/core/management/base.py\", line 283, in run_from_argv\n    self.execute(*args, **cmd_options)\n  File \"/home/vagrant/miniconda/lib/python2.7/site-packages/django/core/management/base.py\", line 330, in execute\n    output = self.handle(*args, **options)\n  File \"/home/vagrant/miniconda/lib/python2.7/site-packages/django/core/management/commands/makemigrations.py\", line 110, in handle\n    loader.check_consistent_history(connection)\n  File \"/home/vagrant/miniconda/lib/python2.7/site-packages/django/db/migrations/loader.py\", line 282, in check_consistent_history\n    applied = recorder.applied_migrations()\n  File \"/home/vagrant/miniconda/lib/python2.7/site-packages/django/db/migrations/recorder.py\", line 65, in applied_migrations\n    self.ensure_schema()\n  File \"/home/vagrant/miniconda/lib/python2.7/site-packages/django/db/migrations/recorder.py\", line 52, in ensure_schema\n    if self.Migration._meta.db_table in self.connection.introspection.table_names(self.connection.cursor()):\n  File \"/home/vagrant/miniconda/lib/python2.7/site-packages/django/db/backends/base/base.py\", line 254, in cursor\n    return self._cursor()\n  File \"/home/vagrant/miniconda/lib/python2.7/site-packages/django/db/backends/base/base.py\", line 229, in _cursor\n    self.ensure_connection()\n  File \"/home/vagrant/miniconda/lib/python2.7/site-packages/django/db/backends/base/base.py\", line 213, in ensure_connection\n    self.connect()\n  File \"/home/vagrant/miniconda/lib/python2.7/site-packages/django/db/backends/base/base.py\", line 189, in connect\n    self.connection = self.get_new_connection(conn_params)\n  File \"/home/vagrant/miniconda/lib/python2.7/site-packages/django/contrib/gis/db/backends/spatialite/base.py\", line 65, in get_new_connection\n    six.reraise(ImproperlyConfigured, ImproperlyConfigured(new_msg), sys.exc_info()[2])\n  File \"/home/vagrant/miniconda/lib/python2.7/site-packages/django/contrib/gis/db/backends/spatialite/base.py\", line 60, in get_new_connection\n    cur.execute(\"SELECT load_extension(%s)\", (self.spatialite_lib,))\n  File \"/home/vagrant/miniconda/lib/python2.7/site-packages/django/db/backends/sqlite3/base.py\", line 328, in execute\n    return Database.Cursor.execute(self, query, params)\ndjango.core.exceptions.ImproperlyConfigured: Unable to load the SpatiaLite library extension \"/home/vagrant/miniconda/lib/libspatialite.so.7\" because: /home/vagrant/miniconda/lib/./libspatialite.so.7: undefined symbol: sqlite3_spatialite_init\n", 
    "path": "/home/vagrant/miniconda/bin:/usr/sbin:/usr/bin:/sbin:/bin", 
    "state": "absent", 
    "syspath": [
        "/tmp/ansible_iPIMPH", 
        "/tmp/ansible_iPIMPH/ansible_modlib.zip", 
        "/tmp/ansible_iPIMPH/ansible_modlib.zip", 
        "/usr/lib/python2.7", 
        "/usr/lib/python2.7/plat-x86_64-linux-gnu", 
        "/usr/lib/python2.7/lib-tk", 
        "/usr/lib/python2.7/lib-old", 
        "/usr/lib/python2.7/lib-dynload", 
        "/usr/local/lib/python2.7/dist-packages", 
        "/usr/lib/python2.7/dist-packages"
    ]
}

Install conda in the shared dir

Problem

Now we have an awkward procedure for copying packages from conda_pkgs back and forth to somewhat speed up deployment of a new machine.

Solution

Install conda in /vagrant/shared/miniconda
Create conda environment for each machine

Caveats

May not work in windows since conda makes symbolic links. Can probably be overriden with allow_softlinks: False