heremaps / xyz-qgis-plugin

Plugin for QGIS to connect to the HERE Platform IML

HERE Maps for QGIS Plugin

HERE Maps for QGIS is a QGIS plugin which can be used to visualize, edit and analyze data from Interactive Map Layers (IML) on HERE Platform and from xyz-hub spaces directly on a map.

Installation

Prerequisite

• QGIS version 3.0 or later

Install via plugin repository in QGIS

1. In QGIS, navigate to menu Plugins > Manage and Install Plugins... > All
2. Search for HERE Maps for QGIS > Install plugin
3. Switch to tab Installed, make sure the plugin HERE Maps for QGIS is enabled.

Install manually from zip file in QGIS

1. Download the latest release zip file
2. In QGIS, navigate to menu Plugins > Manage and Install Plugins... > Install from ZIP, then select the downloaded zip file.
3. Switch to tab Installed, make sure the plugin HERE Maps for QGIS is enabled.

Usage

Once user have installed and enabled the plugin HERE Maps for QGIS, it can be accessed via the toolbar, or the menu Web. To use the plugin, user first need to create an account in HERE Platform. Once done, user can try to login in the plugin either with the same HERE Platform user credential or with the app credential (in case the first option does not work).

Login

Add User Credential

1. Ensure that HERE Platform is selected in the Server drop down.
2. If the User credential HERE Platform User Login exist in the Credential drop down, proceed directly to step 7
3. Click on Add to add your credential.
4. Tick the box User login
5. Enter "HERE Platform User Login" in the Name, user email in the User email and the realm from the Account confirmation email.
6. Click on OK to save the credential
7. Select the User credential and click Connect to login for the first time. The Platform login webpage will pop up.
8. Follow the steps in the Platform login webpage. Once login succesfully, the popup windows will be closed automatically and the list of available IML layers in your Platform account shall show up.

Add App Credential (optional)

1. Ensure that HERE Platform is selected in the Server drop down.
2. Click on Add to add your credential.
3. Enter "My App credential" as the name.
4. Select a credentials.properties containing your HERE Platform app credential (typically location in {$HOME}/.here/credentials.properties)
5. If user don't already have an app or a credentials.properties file then user can create one, following the instructions here.
6. Grant access on the data to the HERE platform app so that it can access your project, following the instructions here.
7. Click on OK to save the credential, and close the connection window.
8. Click Connect to show the list of available IML layers.

Connect and list catalogs and layers

1. Select either the User credential or the App credential and click Connect
2. The list of available IML layers in your Platform account shall show up in the tabular form
3. User use the search bar to find the catalogs and layers of interest

Features

The main dialog contains 5 tabs:

• Load : load data from a HERE Platform IML into QGIS, given a valid credential.

  • Mode: there are 3 modes of loading

    • Live loading: interactively refresh features in tiles within the current canvas. Useful for visualization of dynamic dataset with full editing capabilities
    • Incremental loading: interactively refresh and cached features in tiles within the current canvas (no features delete). Useful for visualization and exploration of large dataset
    • Static loading: load and cache all features in space in background. Useful for importing and analysis of static dataset

  • Chunk size indicates the number of features per tile or iteration. Default: 100. For large layer of point, lines features, value of 10000 is recommended. For large layer of polygon features, value of 1000 is recommended.

  • Layering controls the organization of data in one layer into different QGIS layers based on fields similarity, with 3 levels:

    • single: merge data into 1 layer per geometry type
    • maximal: do not merge data, as many layers per geometry type
    • balanced: merge only similar data into 1 layer, balanced number of layers per geometry type

• Upload : upload local QGIS Vector Layer to the server. For this to work, make sure that your token also has write-level permissions.

• Manage : create new space, edit or delete existing space.

• Map Tile : create HERE Map Tile layer given valid app_id and app_code. Alternatively, api_key can also be used.

• Settings :

  • Clear cache : empty the temporary cache folder. Active layers will be invalid after clearing cache.
  • Clear cookies : clear the login cookies. This could be helpful when login or loading show Authorization error

When the user make some edits to the loaded layer, the changes can be pushed to the server via the button Push changes in the toolbar.

Note: User can also add background map tiles via the "Map Tile" tab. User will need to use your API key following the instructions here.

Testing

Test with unittest

./runTest.sh -m unittest test

Test with pytest-xdist

./runTest.sh -m pytest -n 2 test

Development

1. Clone this repository.
2. Build the plugin by running sh makeBuild.sh <VERSION NUMBER>, e.g. sh makeBuild.sh 1.5.5.

Changelog

CHANGELOG

License

Copyright (C) 2019-2024 HERE Europe B.V.

This project is licensed under the MIT license - see the LICENSE file in the root of this project for license details.

xyz-qgis-plugin's Issues

Switching to App Key option to load HERE Tiles

Due to a recent change, the switch to the new authentication type is needed to load HERE Map Tiles:
https://developer.here.com/blog/announcing-two-new-authentication-types

Examples:
https://developer.here.com/documentation/map-tile/dev_guide/topics/quick-start-map-tile.html

qgis sometime crash right after press upload

Crash ID: a7a5ab4eb043484af16cf1c12b3890c3d82b3388

Stack Trace

QRasterPaintEngine::ensurePen :
QRasterPaintEngine::drawTextItem :
QPainter::drawTextItem :
QTextLine::draw :
qt_draw_helper :
QPainter::drawText :
QStyle::drawItemText :
qt_plugin_query_metadata :
QWindowsStylePrivate::devicePixelRatio :
QDockWidget::paintEvent :
QgsPresetSchemeColorRamp::clone :
QWidget::event :
QDockWidget::event :
QgsPresetSchemeColorRamp::clone :
QApplicationPrivate::notify_helper :
QApplication::notify :
QgsApplication::notify :
QCoreApplication::notifyInternal2 :
QWidgetPrivate::drawWidget :
QWidgetPrivate::paintSiblingsRecursive :
QWidgetPrivate::drawWidget :
QApplication::windowIcon :
QWidget::event :
QgisApp::event :
QApplicationPrivate::notify_helper :
QApplication::notify :
QgsApplication::notify :
QCoreApplication::notifyInternal2 :
QWidgetPrivate::scrollRect :
QWidgetPrivate::invalidateBuffer_resizeHelper :
QWidget::repaint :
QWidget::repaint :
PyMethodDef_RawFastCallKeywords :
PyMethodDef_RawFastCallKeywords :
PyEval_EvalFrameDefault :
PyFunction_FastCallDict :
PyMethodDef_RawFastCallDict :
PyObject_FastCallDict :
PyObject_CallFunctionObjArgs :
PyFile_WriteObject :
PyUnicode_GetLength :
PyMethodDef_RawFastCallKeywords :
PyMethodDef_RawFastCallKeywords :
PyEval_EvalFrameDefault :
PyMethodDef_RawFastCallKeywords :
PyEval_EvalFrameDefault :
PyEval_EvalCodeWithName :
PyMethodDef_RawFastCallKeywords :
PyEval_EvalFrameDefault :
PyMethodDef_RawFastCallKeywords :
PyEval_EvalFrameDefault :
PyFunction_FastCallDict :
PyException_SetTraceback :
PyEval_EvalFrameDefault :
PyFunction_FastCallDict :
PyMethodDef_RawFastCallDict :
PyObject_Call :
PyInit_sip :
QThreadPoolPrivate::reset :
QThread::start :
BaseThreadInitThunk :
RtlUserThreadStart :

QGIS Info
QGIS Version: 3.4.4-Madeira
QGIS code revision: f6ddc62fdb
Compiled against Qt: 5.11.2
Running against Qt: 5.11.2
Compiled against GDAL: 2.4.0
Running against GDAL: 2.4.0

System Info
CPU Type: x86_64
Kernel Type: winnt
Kernel Version: 10.0.17134

can't create new space

I tried to create a new space with an "all-in" token. Unfortunately it shows me some WRITE issues:
Request:
https://xyz.api.here.com/hub/spaces

Response:
{"type":"ErrorResponse","streamId":"8717d1df-7e1c-11e9-beed-dbc5dfb205cb","error":"Exception","errorMessage":"Insufficient rights. Token access: {"manageSpaces":[{"owner":"HERE-0217530d-346e-41cc-abad-252821e8b2de","space":"Gxt7o9l9"}],"createFeatures":[{"owner":"HERE-0217530d-346e-41cc-abad-252821e8b2de","space":"Gxt7o9l9"}],"updateFeatures":[{"owner":"HERE-0217530d-346e-41cc-abad-252821e8b2de","space":"Gxt7o9l9"}],"deleteFeatures":[{"owner":"HERE-0217530d-346e-41cc-abad-252821e8b2de","space":"Gxt7o9l9"}],"readFeatures":[{"owner":"HERE-0217530d-346e-41cc-abad-252821e8b2de","space":"Gxt7o9l9"}]}\nRequest access: {"manageSpaces":[{"owner":"HERE-0217530d-346e-41cc-abad-252821e8b2de","space":"99cmJ0rh","packages":[]}]}"}

the layer is a point layer with 200 points QGIS 3.4.4, Python 3.5.2.

Wrong tile data loaded from virtual space in here schema. Use different tile schema instead

Data from virtual space have wrong coordinates.

As here schema is deprecated for tile endpoint, other schema should be used instead, e.g. Web Mercator, TMS or Quadkey. This change can potentially fix the wrong coordinates issues in virtual space

Support new XYZ Hub API capabilities

• Support filter and selection query in request to XYZ Hub

Automatically apply SLD/QML styles for specified spaces

Allow to choose a QGIS supported style to be applied automatically to a specific space or a specific geometry type

Plugin does not work properly after internet interruption

Externally deleted features are not synced in QGIS

When a feature is deleted externally (not via QGIS), after refreshing the canvas, the feature is not deleted in QGIS.

Expected: The feature should be deleted in QGIS

Support custom XYZ Hub API server

Required features:

• Update UI for custom server
• Server url belongs to the Connection model (with token, space id, etc.)
• Save server url into project
• Update token persistent file (backward compatibility required)

installation shows error

I used the plugin installer to install the plugin in QGIS 3.4.4... Unfortunately it "fails" with this message:

  File "/home/ricckli/.local/share/QGIS/QGIS3/profiles/default/python/plugins/XYZHubConnector/modules/layer/__init__.py", line 11, in 
    from .layer import XYZLayer
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 672, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/ricckli/.local/share/QGIS/QGIS3/profiles/default/python/plugins/XYZHubConnector/modules/layer/layer.py", line 141
    sql_constraint = f'"{parser.QGS_XYZ_ID}" TEXT UNIQUE ON CONFLICT REPLACE' # replace older duplicate

My setup:

Python version: 3.5.2 (default, Nov 12 2018, 13:43:14) [GCC 5.4.0 20160609] 
QGIS version: 3.4.4-Madeira Madeira, f6ddc62 

On Ubuntu/Elementary OS

Manage token by name

• When add a new token, required to input name + token
• Display only the given name in the main dialog
• Has option to edit, delete name/token
• Persist tokens (including names) across versions

Allow XYZ Layer group to be nested in Layers Panel

Currently, the data is continously loaded in live or incremental mode when the corresponding Qgis layer group is located at root level. This limits the capability to organize layers

Proposal: the layer group should be able to continuously load data even when located in group or sub group in Layers Panel

Doesnt connect

No longer connects, wrong url settings etc, they have moved

Select multiple spaces to load at once

Extending the functionality within the space loading dialogue to load a set of spaces at once with a common loading setup.
Function would be mark per "shift" and click -> load all marked spaces at once.

Handle property conflicting with reserved QgsField name

When the data contains a property conflicting with the reserved QgsField name, e.g. 'FID' conflicts with 'fid' used internally in gpkg file. Since the column names are case-insensitive in Geopackage file, it may cause problems with the loaded data, for instance:

• "balanced" layering yield too many layers
• trying to merge data from conflicting fields

Is the custom repo necessary?

bug: progress bar keeps showing when all task finished or failed

can't load layer (sqlite3.OperationalError: cannot commit - no transaction is active)

was trying to load a space with 204 items. had some issues in downloading:
An error has occurred while executing Python code:

sqlite3.OperationalError: cannot commit - no transaction is active
Traceback (most recent call last):
File "/home/ricckli/.local/share/QGIS/QGIS3/profiles/default/python/plugins/XYZHubConnector/plugin.py", line 398, in start_load_layer
con_load.start_args(args)
File "/home/ricckli/.local/share/QGIS/QGIS3/profiles/default/python/plugins/XYZHubConnector/modules/loader/layer_loader.py", line 64, in start_args
self.start( *a, **kw)
File "/home/ricckli/.local/share/QGIS/QGIS3/profiles/default/python/plugins/XYZHubConnector/modules/loader/layer_loader.py", line 53, in start
self.layer = XYZLayer(conn_info, meta, tags=tags)
File "/home/ricckli/.local/share/QGIS/QGIS3/profiles/default/python/plugins/XYZHubConnector/modules/layer/layer.py", line 50, in init
self._init_ext_layer(geom, crs)
File "/home/ricckli/.local/share/QGIS/QGIS3/profiles/default/python/plugins/XYZHubConnector/modules/layer/layer.py", line 143, in _init_ext_layer
self._init_constraint(fname, sql_constraint, db_layer_name)
File "/home/ricckli/.local/share/QGIS/QGIS3/profiles/default/python/plugins/XYZHubConnector/modules/layer/layer.py", line 195, in _init_constraint
cur.execute(s)
sqlite3.OperationalError: cannot commit - no transaction is active

Python version: 3.5.2 (default, Nov 12 2018, 13:43:14) [GCC 5.4.0 20160609]
QGIS version: 3.4.4-Madeira Madeira, f6ddc62

Python Path:
/usr/share/qgis/python
/home/ricckli/.local/share/QGIS/QGIS3/profiles/default/python
/home/ricckli/.local/share/QGIS/QGIS3/profiles/default/python/plugins
/usr/share/qgis/python/plugins
/usr/lib/python35.zip
/usr/lib/python3.5
/usr/lib/python3.5/plat-x86_64-linux-gnu
/usr/lib/python3.5/lib-dynload
/home/ricckli/.local/lib/python3.5/site-packages
/usr/local/lib/python3.5/dist-packages
/usr/lib/python3/dist-packages
/home/ricckli/.local/share/QGIS/QGIS3/profiles/default/python
/home/ricckli/.local/lib/python3.5/site-packages/IPython/extensions

Problem continuing loading data in layer with Virtual Field

Steps to reproduce:

1. Load a DataHub space into QGIS Layer
2. Create a Virtual field in the QGIS layer
3. Save and close the project
4. Reopen the saved project

Expected:
The plugin shall continue loading data into the existing layer,

Actual:
The plugin failed to load data, as there is mismatch between list of fields (properties) of QGIS Layer and DataHub space

Feature has too many attributes (expecting 7, received 10)

Installation on QGIS 3.22 / Ubuntu 20.04 throws `AttributeError: type object 'Qgis' has no attribute 'None' `

Couldn't load plugin 'XYZHubConnector' due to an error when calling its classFactory() method

AttributeError: type object 'Qgis' has no attribute 'None' 

Traceback (most recent call last):
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 423, in _startPlugin
    plugins[packageName] = package.classFactory(iface)
  File "/home/pi/.local/share/QGIS/QGIS3/profiles/default/python/plugins/XYZHubConnector/__init__.py", line 22, in classFactory
    from .plugin import XYZHubConnector
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 888, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/pi/.local/share/QGIS/QGIS3/profiles/default/python/plugins/XYZHubConnector/plugin.py", line 24, in 
    from .gui.space_dialog import MainDialog
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 888, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/pi/.local/share/QGIS/QGIS3/profiles/default/python/plugins/XYZHubConnector/gui/space_dialog.py", line 15, in 
    from ..xyz_qgis.controller import make_qt_args
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 888, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/pi/.local/share/QGIS/QGIS3/profiles/default/python/plugins/XYZHubConnector/xyz_qgis/controller/__init__.py", line 13, in 
    from ..common.signal import (BasicSignal, make_qt_args, output_to_qt_args,
  File "/usr/lib/python3/dist-packages/qgis/utils.py", line 888, in _import
    mod = _builtin_import(name, globals, locals, fromlist, level)
  File "/home/pi/.local/share/QGIS/QGIS3/profiles/default/python/plugins/XYZHubConnector/xyz_qgis/common/signal.py", line 47, in 
    qgis_level = dict(zip(map(lambda k: getattr(Qgis,k), level), level))
  File "/home/pi/.local/share/QGIS/QGIS3/profiles/default/python/plugins/XYZHubConnector/xyz_qgis/common/signal.py", line 47, in 
    qgis_level = dict(zip(map(lambda k: getattr(Qgis,k), level), level))
AttributeError: type object 'Qgis' has no attribute 'None'

Python version: 3.8.10 (default, Sep 28 2021, 16:10:42) [GCC 9.3.0] 
QGIS version: 3.22.0-Białowieża Białowieża, d9022691f1 

Tested with version 1.8.3 from hub.qgis.org and 1.9.0 from Github releases.

Add multi language support

Support switching to different language in the plugin UI

loaded space with unicode char in properties does not shown properly (in QGIS Attribute Table)

enhance support tags / adding property search

XYZ/DataHub supports filtering via property search and via tags.
Adding this functionality into the plugin data loading dialogue to enable to user making use of this feature.

Authentication using SSO

Authentication using SSO in addition to token authentication

can't edit space description

I've tried to edit a space in QGIS. Unfortunately it throws a warning:
XYZ Hub : ChainInterrupt(TypeError("QNetworkAccessManager.sendCustomRequest(QNetworkRequest, QByteArray, QIODevice data=None): argument 3 has unexpected type 'bytes'",), 0, '1/2 in chain', )
The edits were not pushed to XYZ.

Add option for automatic flatner of any json attribute in xyz-layer loaded.

As an analyst i would like to jump straight away into labeling and symbology in Qgis using flat attributes and avoiding the need to dive into the exposed string json structures received when loading xyz layers.

Please see below an image example of a flaten attribute.

bug: when main connect dialog is opened for a while, clicking buttons doing nothing

The network requests (count) might happen before the UI is rendered, which caused UI to hang

The controllers corresponding to the button might be deleted by Qt after a while.

• Need a way to keep them persistent,
• or recreate them on button clicked