3liz / py-qgis-wps

An implementation of the Web Processing Service standard from the Open Geospatial Consortium based on the QGIS processing API

Py-QGIS-WPS

New in 1.8: OGC api processes support

Py-QGIS-WPS is an implementation of the Web Processing Service standard from the Open Geospatial Consortium based on the QGIS Processing API.

Since 1.8 Py-QGIS-WPS supports OGC API REST processes api

This implementation allows you to expose and run on a server:

• QGIS Processing algorithms available on Desktop (side note, exposing a QGIS model or Processing script are recommended)
• QGIS Processing models and scripts
• QGIS plugins having a Processing provider according to their metadata.txtfile

It is written in Python and was originally a fork of PyWPS.

Requirements and limitations :

• Python 3.7 minimum
• Windows is not officially supported
• Redis server

Documentation

Latest documentation is available on docs.3liz.org.

Why Py-QGIS-WPS ?

Py-QGIS-WPS differs from PyWPS in the following:

• QGIS centric
• Handle all requests in an asynchronous way: all jobs run in a non-blocking way, even when synchronous operation is requested.
• Use multiprocessing Pool to handle task queue instead of instantiating a new process each time.
• Uniform Logging with the 'logging' module
• Implements OGC processes api.
• Use Redis for asynchronous status storage.
• Support streamed/chunked requests for stored data
• Add extensions to WPS: TIMEOUT and EXPIRE
• No Windows support

All these changes were not easy to implement without some drastic changes of the original code, and we think that it deviates too much from the PyWPS original intentions.

That is, we have decided to fork the original project and go along with it.

So, we are really grateful to the original authors of PyWPS for the nice piece of software that helped us very much to start quickly this project.

Why moving to Tornado instead WSGI

• We need to support asyncio: asyncio requires a blocking running loop. This cannot be achieved simply in a WSGI architecture.
• Tornado is fully integrated with native python asyncio library and provide a great framework for developing an HTTP server.

Extensions to WPS

TIMEOUT extension

Specify the timeout for a process: if the process takes more than TIMEOUT seconds to run, the worker is then killed, and an error status is returned.

Set the TIMEOUT=<seconds> in GET requests.

In POST requests, set the timeout=<seconds> attribut in the <ResponseDocument> tag.

The server may configure the maximum timeout value.

EXPIRE extension

Specify the expiration time for stored results: after EXPIRE seconds after the end of the wps process, all results will be flushed from disks and local cache. Trying to request the results again will return a 404 HTTP error.

Set the EXPIRE=<seconds> in GET requests.

In POST requests, set the expire=<seconds> attribut in the <ResponseDocument> tag.

The server may configure maximum expiration value.

Status API

Now implemented with the processes api: The status REST api will return the list of the stored status for all running and terminated wps processes.

Example for returning all stored statuses:

http://localhost:8080/jobs

Example for returning status for one given job from its id:

http://localhost:8080/jobs/<job_id>

Extensions to processes api:

Files

http://localhost:8080/jobs/<job_id>/files

Running QGIS processing

WPS input/output layer mapping

With QGIS desktop, QGIS processing algorithms usually apply on a QGIS source project and computed layers are displayed in the same context as the source project.

Py-qgis-wps works the same way: a qgis project will be used as a source of input layers. The difference is that, when an algorithm runs, it creates a QGIS project file associated with the current task and register computed layers to it.

The created project may be used as an OWS source with QGIS Server. Output layers are returned as complex objects holding a reference to a WMS/WFS uri that can be used directly with QGIS server. The URI template is configurable using the server/wms_response_uri configuration setting.

Contextualized input parameters

Tasks parameters are contextualized using the MAP query param. If a MAP parameters is given when doing a DescripProcess requests, allowed values for input layers will be taken from the qgis source project according the type of the input layers.

QGIS project (.qgs) files and project stored in Postgres databases are both supported.

The best practice is to always provide a MAP parameters and include the possible input layer in a qgs project. This way you may connect whatever data source supported by qgis and use them as input data in a safe way.

If you need to pass data to your algorithm from client-side, prefer inputs file parameter and small payloads.

Dependencies

See requirements.txt file.

Installation from python package

ADVICE: You should always install in a python virtualenv. If you want to use system packages, set up your environment with the --system-site-packages option.

See the official documentation for how to set up a python virtualenv: https://virtualenv.pypa.io/en/stable/.

From source

Install in development mode

make build
pip install -e .

From python package server

pip install py-qgis-wps

Running the server

The server from a command line interface:

The server does not run as a daemon by itself, there are several ways to run a command as a daemon.

For example:

• Use Supervisor http://supervisord.org/ will give you full control over logs and server status notifications.
• Use the daemon command.
• Use Docker

Running the server

Usage

usage: wpsserver [-h] [-d] [-c [PATH]]
                 [--version] [-p PORT] [-b IP] [-u SETUID]

WPS server

optional arguments:
  -h, --help            show this help message and exit
  -d, --debug           Set debug mode  
  -c [PATH], --config [PATH]
                        Configuration file
  --version             Return version number and exit
  -p PORT, --port PORT  http port
  -b IP, --bind IP      Interface to bind to
  -u SETUID, --setuid SETUID
                        uid to switch to

Requests to OWS services

The OWS requests use the following format: /ows/?<ows_query_params>

Example:

http://myserver:8080/ows/?SERVICE=WPS&VERSION=1.0.0&REQUEST=GetCapabilities

Configuration

From config ini file

By default, the wps server is not using any config file, but one can be used with the --config option. A config file is a simple ini file, a sample config file is given with the sources.

From environment variables

The server can be configured with environnement variables:

Configuration is done with environment variables:

• QGSWPS_SERVER_WORKDIR: set the current dir processes, all processes will be running in that directory.
• QGSWPS_SERVER_HOST_PROXY: When the service is behind a reverse proxy, set this to the proxy entrypoint.
• QGSWPS_SERVER_PARALLELPROCESSES: Number of parallel process workers
• QGSWPS_SERVER_RESPONSE_TIMEOUT: The max response time before killing a process.
• QGSWPS_SERVER_RESPONSE_EXPIRATION: The max time (in seconds) the response from a WPS process will be available.
• QGSWPS_SERVER_WMS_SERVICE_URL: The base url for WMS service. Default to /wms. Responses from processing will be retourned as WMS urls. This configuration variable sets the base url for accessing results.
• QGSWPS_SERVER_RESULTS_MAP_URI

Logging

• QGSWPS_LOGLEVEL: the log level, should be INFO in production mode, DEBUG for debug output.

REDIS storage configuration

• QGSWPS_REDIS_HOST: The redis host
• QGSWPS_REDIS_PORT: The redis port. Default to 6379
• QGSWPS_REDIS_DBNUM: The redis database number used. Default to 0

QGIS project Cache configuration

• QGSWPS_CACHE_ROOTDIR: Absolute path to the qgis projects root directory, projects referenced with the MAP parameter will be searched at this location

Processing configuration

• QGSWPS_PROCESSSING_PROVIDERS_MODULE_PATH: Path to look for processing algorithms provider to publish, algorithms from providers specified here will be runnable as WPS processes.

Exposing algorithms as WPS services

Note that since 1.1, the __algorithms__.py method for declaring providers is no longer supported.

Processing providers following the same rules as QGIS regular plugin with a special factory entrypoint: WPSClassFactory(iface) in the __init__.py file.

The metadata.txt file

As regular QGIS plugin, a metadata.txt file must be present with a special entry wps=True indicating that the plugin is available as a WPS service provider.

Registering providers

The iface parameter is a instance of WPSServerInterface which provide a registerProvider( provider: QgsAlgorithmProvider, expose: bool = True) -> Any method.

Exposed providers as WPS services must be registered using the registerProvider method

Example:

def WPSClassFactory(iface: WPSServerInterface) -> Any:

    from TestAlgorithmProvider1 import  AlgorithmProvider1
    from TestAlgorithmProvider2 import  AlgorithmProvider2

    iface.registerProvider( AlgorithmProvider1() )
    iface.registerProvider( AlgorithmProvider2() )

Controlling what is exposed:

Processing algorithm with the flag FlagHideFromToolbox set will not be exposed as WPS process.

Parameters with the flag FlagHidden set won't be exposed in a DescribeProcess request

References

• OGC standards
• OGC Api processes
• Introduction to WPS
• Py-qgis-server at FOSS4G 2019.

py-qgis-wps's Issues

VERSION file is missing

The setup.py file tries to open a VERSION file (get_version() function) but this file does not exist.

Need example for how to pass client-side data to an algorithm

What I have

I currently have an algorithm that performs certain actions on an input layer. That layer is already included in a qgs project and I only need to provide the name of the layer as an input parameter.

What I need

I would now like to change the algorithm so that I can provide the actual layer (in my specific case a shape file) as an input parameter, but so far I haven't been able to achieve this.

My problem

In the docs under Contextualized input parameters it only says:

If you need to pass data to your algorithm from client-side, prefer inputs file parameter and small payloads.

Unfortunately, that didn't help me much further. Have tried it with both Postman and curl, but wasn't able to get it working. Can you provide a more detailed explanation on how something like that could be achieved?

Idea/reason behind all this

The already in the qgs project included layer gives a baseline reference. A user can download a copy of the original layer, edit it, re-run the algorithm and compare the results.

Update docker image OS

Would it be possible to update the underlying debian to something more new?
Or provide an ubuntu image?

If i try to add other software like GDAL, i always run into trouble with oudated packages.

Add an ows:Metadata with processing:geometryTypes for QgsProcessingParameterGeometry

A QgsProcessingParameterGeometry has already an ows:Metadata with processing:type

<ows:Metadata xlink:title="processing:type" xlink:href="geometry" xlink:type="simple"/>
``

It will be useful to add an other `ows:Metadata` but with `processing:geometryTypes`. For example:
```xml
<ows:Metadata xlink:title="processing:type" xlink:href="geometry" xlink:type="simple"/>
<ows:Metadata xlink:title="processing:geometryTypes" xlink:href="point" xlink:type="simple"/>
...
<ows:Metadata xlink:title="processing:type" xlink:href="geometry" xlink:type="simple"/>
<ows:Metadata xlink:title="processing:geometryTypes" xlink:href="line" xlink:type="simple"/>
...
<ows:Metadata xlink:title="processing:type" xlink:href="geometry" xlink:type="simple"/>
<ows:Metadata xlink:title="processing:geometryTypes" xlink:href="polygone" xlink:type="simple"/>

Could not access processing script via WPS

Hi Team,

As I could manage to setup WPS service on Linux machine and able to run it from source code, however, I could send request with simplevalue processing script. When I access http://localhost:8080/store/pyqgiswps_Main/simplevalue?service=WPS, it shows below error message and console shows the python file is expected to be available in output folder. Not sure whether I did anything wrong or is it a bug in the code (as I suspect).

<ows:Exception exceptionCode="NoApplicableCode" locator="">
ows:ExceptionTextResource not found</ows:ExceptionText>
</ows:Exception>

2021-04-15 11:21:24,933 [1563] ERROR File '/home/adminuser/py-qgis-wps-master/outputdata_/pyqgiswps_Main/simplevalue' not found
2021-04-15 11:21:24,933 [1563] ERROR Exception: code: 404, description: Resource not found, locator:
2021-04-15 11:21:24,933 [1563] DEBUG Request failed wint message: Resource not found HTTP 404: Not Found
2021-04-15 11:21:24,933 [1563] DEBUG Serializing XML response

As I am pretty new to this py-qgis-wps, wondering whether we have,

1. any dashboard url to see the status of jobs in webpage, (I see the webui code and html but could not get the url to access)
2. any request response sample to send geometry geojson in request and input processing parameter for the same.

It would be really helpful to me if someone can help me in this regards.

-Prem

Load QGIS Processing Config to use Script provider

The QGIS procesing Script provider https://github.com/qgis/QGIS/tree/master/python/plugins/processing/script can be used to defined algorithms without providing a QGIS plugin.

The scripts for QGIS Processing has been loaded from directories. The default one is in the processing user folder:
https://github.com/qgis/QGIS/blob/master/python/plugins/processing/script/ScriptUtils.py#L49

def defaultScriptsFolder():
    folder = str(os.path.join(userFolder(), "scripts"))
    mkdir(folder)
    return os.path.abspath(folder)

Other folders can be defined through QGIS Processing config
https://github.com/qgis/QGIS/blob/master/python/plugins/processing/script/ScriptUtils.py#L55

def scriptsFolders():
    folder = ProcessingConfig.getSetting(SCRIPTS_FOLDERS)
    if folder is not None:
        return folder.split(";")
    else:
        return [ScriptUtils.defaultScriptsFolder()]

userFolder: https://github.com/qgis/QGIS/blob/master/python/plugins/processing/tools/system.py#L41
ProcessingConfig: https://github.com/qgis/QGIS/blob/master/python/plugins/processing/core/ProcessingConfig.py

Help string on a parameter in QGIS 3.16

The new help string has been merged in QGIS 3.16.

In QgsProcessingParameter, there is now a help() and setHelp().

Question: expiration time

The expire time is configurable in seconds, is it still possible (without huge side effects) to choose an expiration time of several days or a week?

use of default value in a request

As a test I have created this simple process with QGIS. It computes the buffer of a vector layer:

The "number_of_segments" has a default value equal to 5.

I can run this process with the following xml:

<?xml version="1.0" encoding="UTF-8"?>
<wps:Execute xmlns:wps="http://www.opengis.net/wps/1.0.0" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ows="http://www.opengis.net/ows/1.1" xmlns:gml="http://www.opengis.net/gml" service="WPS" version="1.0.0">
  <ows:Identifier>model:buffer_simple</ows:Identifier>
  <wps:DataInputs>
    <wps:Input>
      <ows:Identifier>input_layer</ows:Identifier>
      <wps:Data>
        <wps:LiteralData>/home/foo/data.shp</wps:LiteralData>
      </wps:Data>
    </wps:Input>
    <wps:Input>
      <ows:Identifier>number_of_segments</ows:Identifier>
      <wps:Data>
        <wps:LiteralData>6</wps:LiteralData>
      </wps:Data>
    </wps:Input>
    <wps:Input>
      <ows:Identifier>buffer_layer</ows:Identifier>
      <wps:Data>
        <wps:LiteralData>buffer_layer</wps:LiteralData>
      </wps:Data>
    </wps:Input>
  </wps:DataInputs>
</wps:Execute>

It works. However, "number_of_segments" has a default value. So, I would expect the default value to be used if the parameter is missing in the xml request:

<?xml version="1.0" encoding="UTF-8"?>
<wps:Execute xmlns:wps="http://www.opengis.net/wps/1.0.0" xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:ows="http://www.opengis.net/ows/1.1" xmlns:gml="http://www.opengis.net/gml" service="WPS" version="1.0.0">
  <ows:Identifier>model:buffer_simple</ows:Identifier>
  <wps:DataInputs>
    <wps:Input>
      <ows:Identifier>input_layer</ows:Identifier>
      <wps:Data>
        <wps:LiteralData>/home/foo/data.shp</wps:LiteralData>
      </wps:Data>
    </wps:Input>
    <wps:Input>
      <ows:Identifier>buffer_layer</ows:Identifier>
      <wps:Data>
        <wps:LiteralData>buffer_layer</wps:LiteralData>
      </wps:Data>
    </wps:Input>
  </wps:DataInputs>
</wps:Execute>

However, it fails with the following error because "number_of_segments" is missing.

<?xml version="1.0" encoding="UTF-8"?>
<!-- py-qgis-wps 1.8.7rc0 -->
<ows:ExceptionReport xmlns:ows="http://www.opengis.net/ows/1.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/ows/1.1 http://schemas.opengis.net/ows/1.1.0/owsExceptionReport.xsd" version="1.0.0">
  <ows:Exception exceptionCode="MissingParameterValue" locator="number_of_segments" >
      <ows:ExceptionText>number_of_segments</ows:ExceptionText>
  </ows:Exception>
</ows:ExceptionReport>

I would have expected the default value (5) to be used in that case. Is it the expected behavior?

Thanks.

Provide python packages

Like py-qgis-server 3liz/py-qgis-server#27 publishing Py-QGIS-WPS on pypi.org

WPS exit with code 139 randomly

Randomly when executing make run in tests directory i have those logs above.
Then in LWC I don't have algorithms displayed.

wps_1     | /usr/lib/python3/dist-packages/pyparsing.py:943: DeprecationWarning:
wps_1     | 
wps_1     | Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
wps_1     | 
wps_1     | /usr/lib/python3/dist-packages/pyparsing.py:3245: DeprecationWarning:
wps_1     | 
wps_1     | Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
wps_1     | 
wps_1     | /usr/share/qgis/python/plugins/processing/gui/wrappers.py:1757: DeprecationWarning:
wps_1     | 
wps_1     | invalid escape sequence \A
wps_1     | 
wps_1     | /usr/share/qgis/python/plugins/processing/gui/wrappers.py:1772: DeprecationWarning:
wps_1     | 
wps_1     | invalid escape sequence \A
wps_1     | 
wps_1     | 2020-06-03 07:53:47,147	[1]	ERROR	[qgis:1] Processing: Problem with SAGA installation: SAGA was not found or is not correctly installed
wps_1     | /usr/lib/python3/dist-packages/pyparsing.py:943: DeprecationWarning:
wps_1     | 
wps_1     | Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
wps_1     | 
wps_1     | /usr/lib/python3/dist-packages/pyparsing.py:3245: DeprecationWarning:
wps_1     | 
wps_1     | Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated, and in 3.8 it will stop working
wps_1     | 
wps_1     | /usr/share/qgis/python/plugins/processing/gui/wrappers.py:1757: DeprecationWarning:
wps_1     | 
wps_1     | invalid escape sequence \A
wps_1     | 
wps_1     | /usr/share/qgis/python/plugins/processing/gui/wrappers.py:1772: DeprecationWarning:
wps_1     | 
wps_1     | invalid escape sequence \A
wps_1     | 
wps_1     | 2020-06-03 07:53:47,147	[67]	ERROR	[qgis:67] Processing: Problem with SAGA installation: SAGA was not found or is not correctly installed
wps_1     | Traceback (most recent call last):
wps_1     |   File "/usr/share/qgis/python/plugins/processing/script/ScriptAlgorithmProvider.py", line 66, in load
wps_1     |     ScriptUtils.defaultScriptsFolder(),
wps_1     |   File "/usr/share/qgis/python/plugins/processing/script/ScriptUtils.py", line 51, in defaultScriptsFolder
wps_1     |     mkdir(folder)
wps_1     |   File "/usr/share/qgis/python/plugins/processing/tools/system.py", line 109, in mkdir
wps_1     |     os.mkdir(newdir)
wps_1     | FileExistsError: [Errno 17] File exists: '/home/qgis/.local/share/QGIS/QGIS3/profiles/default/processing/scripts'
wps_1     | 2020-06-03 07:53:47,177	[67]	INFO	[qgis:67] QGis processing initialized
wps_1     | 2020-06-03 07:53:47,178	[67]	DEBUG	= Registering provider 'script'
wps_1     | 2020-06-03 07:53:47,179	[67]	DEBUG	= Registering provider 'model'
wps_1     | 2020-06-03 07:53:47,185	[67]	INFO	Loaded plugin 'algorithms'
wps_1     | 2020-06-03 07:53:47,189	[67]	DEBUG	Starting ZMQ worker loop
tests_wps_1 exited with code 139

The output project has no CRS

In old QGIS, before 3.16, projects have a default CRS. With QGIS 3.16 and upper, projects can have no CRS. For QGIS Server, the project has to have a CRS, but the project build by WPS has no CRS.

To fix it, py-qgis-wps can set the project CRS with the project used to perform the processes.

QgsProcessingParameterVectorDestination TypeVector only WFS output

If the type of a QgsProcessingParameterVectorDestination is set to QgsProcessing.TypeVector the algorithm will return a table (with or without geometry). When this type used for a sink this indicates the sink has no geometry.

If the type QgsProcessing.TypeVector is defined the corresponding WPS output has not to be a WMS one.

It is the same for QgsProcessingOutputVectorLayer

Cannot set different port for docker instance

I'm trying to setup both the py-qgis-server and py-qgis-wps on one server using Docker (as described in your docs). Since both use by default the 8080 port, I tried to change the port for th py-qgis-wps instance like this:

docker run -it --rm -p 8081:8081 --net mynet \
       -v /path/to/processing/:/processing \
       -v /path/to/qgis/projects:/projects \
       -v /path/to/processing/output/dir:/srv/data \
       -e QGSWPS_SERVER_HTTP_PORT=8081 \
       ... other environment variables ...    
       3liz/qgis-wps:3.22

However, I'm unable to call http://my-server-ip:8081/ows/?SERVICE=WPS&VERSION=1.1.0&REQUEST=GetCapabilities

Running the docker instance with the default 8080 port works just fine. I'm also able to alter the port for the py-qgis-server using above command. So, am I missing something here or could this be a bug in py-qgis-wps?

py-qgis-wps with flask

I am interesting to use py-qgis-wps in flask framework. It seems that for rapid succession in this project It is necessary to show an example of application in flask framework.

Erreur 'Unknown process' à l'exécution d'une requête Execute

Bonjour,

Je rencontre des difficultés à faire fonctionner py-qgis-wps.
J’ai installé py-qgis-wps depuis une image Docker (qgis-wps:ltr-rc).

J'ai suivi les instructions présentes sur Docker-hub :
docker network create network_py_qgis_wps
docker run -d --rm --name redis --net network_py_qgis_wps redis:4

docker run -it --rm -p 127.0.0.1:8080:8080 --net network_py_qgis_wps -v '/home/administrateur/py-qgis-wps/processing':/processing -v '/home/administrateur/py-qgis-wps/projects':/projects -v '/home/administrateur/py-qgis-wps/outputs':/srv/data -e QGSWPS_SERVER_WORKDIR=/srv/data -e QGSWPS_SERVER_PARALLELPROCESSES=2 -e QGSWPS_SERVER_LOGSTORAGE=REDIS -e QGSWPS_PROCESSSING_PROVIDERS=provider1,provider2 -e QGSWPS_PROCESSSING_PROVIDERS_MODULE_PATH=/processing -e QGSWPS_CACHE_ROOTDIR=/projects -e QGSWPS_USER=1000:1000 3liz/qgis-wps:ltr-rc

Sur 172.18.0.3:8080/ j’obtiens un fichier JSON avec des informations sur py-qgis-wps. (la même chose s’affiche sur 127.0.0.1:8080/)
Lorsque je me rends sur http://172.18.0.3:8080/ows/?service=WPS&request=GetCapabilities, j’obtiens bien un xml des métadonnées.

Dans mon dossier /home/administrateur/py-qgis-wps/processing, j’ai un traitement appelé ‘ExampleProcessingAlgorithm’ (ExampleProcessingAlgorithm.py)

Dans init.py, j’ai inséré les lignes suivantes :

def WPSClassFactory(iface: WPSServerInterface) -> Any:

    from ExampleProcessingAlgorithm import ExampleProcessingAlgorithm

    iface.registerProvider( ExampleProcessingAlgorithm() )

Cependant, lorsque je lance une requête DescribeProcess ou Execute : http://172.18.0.3:8080/ows/?service=WPS&version=1.0.0&request=Execute&Identifier=ExampleProcessingAlgorithm, j’obtiens un xml contenant « Unknown process ''ExampleProcessingAlgorithm'' ».

127.0.0.1:8080/ui me renvoit une erreur 404.

Je n'arrive pas à trouver ce à côté de quoi je passe.

Merci d'avance pour l'aide que vous pourrez me fournir !

Value with select for source input is not valid

To run an algorithm on a part of a layer, it is possible to add a select param to the layer name:

• layersio.py#L244
• test_algorithms.py#L303

But if you do so, py-qgis-wps returns Exception: code: 400, description: Input data not valid using mode 1

<?xml version="1.0" encoding="UTF-8"?>
<!-- PyQgisWPS 1.4.3 -->
<ows:ExceptionReport xmlns:ows="http://www.opengis.net/ows/1.1" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/ows/1.1 http://schemas.opengis.net/ows/1.1.0/owsExceptionReport.xsd" version="1.0.0">
  <ows:Exception exceptionCode="InvalidParameterValue" locator="" >
      <ows:ExceptionText>Input data not valid using mode 1</ows:ExceptionText>
  </ows:Exception>
</ows:ExceptionReport>

pyyaml missing in requirements.txt ?

Hello,

I am trying your server (I am interested in re-using your WPS tornado handler).

Just as a feedback so far. When I installed as follow:
pip install https://github.com/3liz/py-qgis-wps/archive/1.4.1.tar.gz

then launched:
% wpsserver -h
Failed to read manifest !: [Errno 2] No such file or directory: '/Users/loubrieu/PycharmProjects/incubator-sdap-nexus/qgis_wps/lib/python3.7/site-packages/pyqgiswps/build.manifest'
Traceback (most recent call last):
File "/Users/loubrieu/PycharmProjects/incubator-sdap-nexus/qgis_wps/bin/wpsserver", line 11, in
load_entry_point('py-qgis-wps==1.4.1', 'console_scripts', 'wpsserver')()
File "/Users/loubrieu/PycharmProjects/incubator-sdap-nexus/qgis_wps/lib/python3.7/site-packages/pkg_resources/init.py", line 489, in load_entry_point
return get_distribution(dist).load_entry_point(group, name)
File "/Users/loubrieu/PycharmProjects/incubator-sdap-nexus/qgis_wps/lib/python3.7/site-packages/pkg_resources/init.py", line 2793, in load_entry_point
return ep.load()
File "/Users/loubrieu/PycharmProjects/incubator-sdap-nexus/qgis_wps/lib/python3.7/site-packages/pkg_resources/init.py", line 2411, in load
return self.resolve()
File "/Users/loubrieu/PycharmProjects/incubator-sdap-nexus/qgis_wps/lib/python3.7/site-packages/pkg_resources/init.py", line 2417, in resolve
module = import(self.module_name, fromlist=['name'], level=0)
File "/Users/loubrieu/PycharmProjects/incubator-sdap-nexus/qgis_wps/lib/python3.7/site-packages/pyqgiswps/wpsserver.py", line 17, in
from .runtime import run_server
File "/Users/loubrieu/PycharmProjects/incubator-sdap-nexus/qgis_wps/lib/python3.7/site-packages/pyqgiswps/runtime.py", line 27, in
from .handlers import (RootHandler, WPSHandler, StoreHandler, StatusHandler,
File "/Users/loubrieu/PycharmProjects/incubator-sdap-nexus/qgis_wps/lib/python3.7/site-packages/pyqgiswps/handlers/init.py", line 11, in
from .wpshandler import WPSHandler, StatusHandler
File "/Users/loubrieu/PycharmProjects/incubator-sdap-nexus/qgis_wps/lib/python3.7/site-packages/pyqgiswps/handlers/wpshandler.py", line 22, in
from ..accesspolicy import new_access_policy
File "/Users/loubrieu/PycharmProjects/incubator-sdap-nexus/qgis_wps/lib/python3.7/site-packages/pyqgiswps/accesspolicy.py", line 5, in
import yaml
ModuleNotFoundError: No module named 'yaml'

I've installed pyyaml and the error disappeared.

I guess if pyyaml is added in the requirements.txt then the package will include the dependency.

Thanks for doing this development, I will go on looking at it.

Wrong document format for Job Results endpoint

wpsserver 1.8.2rc0 (build 202209290824,commit 39d1857)

The /jobs/job-id/results endpoint is declared to return JSON mime-type, but returns a XML document.

/jobs/job-id/results GET headers Content-Type: application/json;charset=utf-8

/jobs/job-id/results GET response:

<wps:ExecuteResponse xmlns:xlink="http://www.w3.org/1999/xlink" xmlns:wps="http://www.opengis.net/wps/1.0.0" xmlns:ows="http://www.opengis.net/ows/1.1" xmlns:gml="http://www.opengis.net/gml" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" xsi:schemaLocation="http://www.opengis.net/wps/1.0.0 http://schemas.opengis.net/wps/1.0.0/wpsExecute_response.xsd" service="WPS" version="1.0.0" xml:lang="en-US" serviceInstance="https://localhost/?service=WPS&amp;request=GetCapabilities" statusLocation="https://localhost/ows/?service=WPS&amp;request=GetResults&amp;uuid=2ddc3650-3fdc-11ed-8850-0242ac160003">
    <wps:Process wps:processVersion="1.0generic">
        <ows:Identifier>script:gdal_rastercalculator</ows:Identifier>
        <ows:Title>GDAL Raster calculator on 2 Layers</ows:Title>
        <ows:Abstract>1 Performs calculation on 2 input layers (rasters or bands) and produces resulting raster</ows:Abstract>
    </wps:Process>
    <wps:Status creationTime="2022-09-29T09:50:42Z">
        <wps:ProcessSucceeded>Task finished</wps:ProcessSucceeded>
    </wps:Status>
    <wps:ProcessOutputs>
        <wps:Output>
            <ows:Identifier>OUTPUT</ows:Identifier>
            <ows:Title>Calculated Layer</ows:Title>
            <ows:Abstract>Calculated Layer</ows:Abstract>
            <wps:Reference href="https://localhost/?MAP=/etc/qgisserver/2ddc3650-3fdc-11ed-8850-0242ac160003/script_gdal_rastercalculator.qgs&amp;service=WMS&amp;request=GetCapabilities&amp;layers=result" mimeType="application/x-ogc-wms"/>
        </wps:Output>
    </wps:ProcessOutputs>

The alternate URL for a job (text/html) does not work (404 not found)

wpsserver 1.8.1 (build 202209270744,commit 0f4908d)

The alternate link in a job status page (jobs/<job_id>) leads to a 404 not found page.
Example of link: /jobs/9e6b0e36-3f42-11ed-9cbb-0242ac130002.html
(an extra / is added to this link in the target page)

/jobs.html also leads to the same error.

The relevant logs entry is:

wps-wps-1    | 2022-09-28 15:31:54,303  [1]     REQ     171.18.2.58     301     GET     /jobs.html      0       0       Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36   
wps-wps-1    | 2022-09-28 15:31:54,645  [1]     REQ     171.18.2.58     404     GET     /jobs.html/     0       69      Mozilla/5.0 (Macintosh; Intel Mac OS X 10_15_7) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/106.0.0.0 Safari/537.36   
wps-wps-1    | 2022-09-28 15:32:02,668  [1]     ERROR   Exception: code: 404, description: Invalid resource path., locator: 
wps-wps-1    | 2022-09-28 15:32:02,668  [1]     DEBUG   Request failed with message: Invalid resource path. HTTP 404: Not Found

config.sample.ini is not up to date

The section [cache] has been be updated to [projects.cache].

WKT_EXPR.match(wkt): cannot use a string pattern on a bytes-like object

ERROR	Worker Error: cannot use a string pattern on a bytes-like object
Traceback (most recent call last):
  File "/opt/local/pyqgiswps/lib/python3.8/site-packages/pyqgiswps/poolserver/worker.py", line 84, in worker_handler
    result = (True, func( *args, **kwargs ))
  File "/opt/local/pyqgiswps/lib/python3.8/site-packages/pyqgiswps/executors/processingexecutor.py", line 285, in _run_process
    handler(wps_request, wps_response)
  File "/opt/local/pyqgiswps/lib/python3.8/site-packages/pyqgiswps/executors/processingprocess.py", line 351, in _handler
    parameters = dict( input_to_processing(ident, inp, alg, context) for ident,inp in request.inputs.items() )
  File "/opt/local/pyqgiswps/lib/python3.8/site-packages/pyqgiswps/executors/processingprocess.py", line 351, in <genexpr>
    parameters = dict( input_to_processing(ident, inp, alg, context) for ident,inp in request.inputs.items() )
  File "/opt/local/pyqgiswps/lib/python3.8/site-packages/pyqgiswps/executors/processingio.py", line 298, in input_to_processing
    geometryio.get_processing_value(param, inp, context) or \
  File "/opt/local/pyqgiswps/lib/python3.8/site-packages/pyqgiswps/executors/io/geometryio.py", line 215, in get_processing_value
    value = input_to_point( inp[0] )
  File "/opt/local/pyqgiswps/lib/python3.8/site-packages/pyqgiswps/executors/io/geometryio.py", line 189, in input_to_point
    g = input_to_geometry( inp )
  File "/opt/local/pyqgiswps/lib/python3.8/site-packages/pyqgiswps/executors/io/geometryio.py", line 175, in input_to_geometry
    return wkt_to_geometry(inp.data)
  File "/opt/local/pyqgiswps/lib/python3.8/site-packages/pyqgiswps/executors/io/geometryio.py", line 117, in wkt_to_geometry
    m = WKT_EXPR.match(wkt)
TypeError: cannot use a string pattern on a bytes-like object

The wkt provided to wkt_to_geometry is not a string available regex match method.
The wkt comes from WPSInput.data.

The request was:

<wps:Execute xmlns:wps="http://www.opengis.net/wps/1.0.0" version="1.0.0" service="WPS" xsi:schemaLocation="http://www.opengis.net/wps/1.0.0 http://schemas.opengis.net/wps/1.0.0/wpsAll.xsd" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance">
  <ows:Identifier xmlns:ows="http://www.opengis.net/ows/1.1">pyqgiswps_test:testpoint</ows:Identifier>
  <wps:DataInputs>
    <wps:Input>
      <ows:Identifier xmlns:ows="http://www.opengis.net/ows/1.1">INPUT</ows:Identifier>
      <ows:Title xmlns:ows="http://www.opengis.net/ows/1.1">Point</ows:Title>
      <wps:Data>
        <wps:ComplexData mimeType="application/wkt" encoding="utf-8" schema=""><![CDATA[CRS=4326;POINT(-4 48)]]></wps:ComplexData>
      </wps:Data>
    </wps:Input>
  </wps:DataInputs>
  <wps:ResponseForm>
    <wps:ResponseDocument storeExecuteResponse="true" status="true">
      <wps:Output>
        <ows:Identifier xmlns:ows="http://www.opengis.net/ows/1.1">OUTPUT</ows:Identifier>
        <ows:Title xmlns:ows="http://www.opengis.net/ows/1.1"/>
        <ows:Abstract xmlns:ows="http://www.opengis.net/ows/1.1"/>
      </wps:Output>
    </wps:ResponseDocument>
  </wps:ResponseForm>
</wps:Execute>

The algorithm is:

""" Test getting point
"""

from qgis.core import (QgsProcessingParameterPoint,
                       QgsProcessingOutputString,
                       QgsProcessingAlgorithm)


class TestPoint(QgsProcessingAlgorithm):

    INPUT = 'INPUT'
    OUTPUT = 'OUTPUT'

    def __init__(self):
        super().__init__()

    def name(self):
        return 'testpoint'

    def displayName(self):
        return 'Test Point'

    def createInstance(self, config={}):
        """ Virtual override

            see https://qgis.org/api/classQgsProcessingAlgorithm.html
        """
        return self.__class__()

    def initAlgorithm(self, config=None):
        """ Virtual override

           see https://qgis.org/api/classQgsProcessingAlgorithm.html
        """
        self.addParameter(QgsProcessingParameterPoint(self.INPUT, 'Point'))
        self.addOutput(QgsProcessingOutputString(self.OUTPUT, "Output"))

    def processAlgorithm(self, parameters, context, feedback):
        """ Virtual override

            see https://qgis.org/api/classQgsProcessingAlgorithm.html
        """
        point = self.parameterAsPoint(parameters, self.INPUT, context)
        crs = self.parameterAsPointCrs(parameters, self.INPUT, context)

        return {self.OUTPUT: point.toString(8)+' ('+crs.authid()+')'}

With "native" provider, a Python error occured : pyqgiswps.exceptions.ProcessException: Failed to initialize Qgis processes

Maybe this is not an issue but I'm writing here just to get your help. When I start the server, I get this warning:

WARNING No published processes !

And then in lizmap, the wps process select is always empty. I also the wps:ProcessOfferings/ empty when calling /ows/?request=GetCapabilities&service=WPS.
Do I need to enable something in the Qgis server? (I'm using the official Qgis server and not the py-qgis-server, should I used it instead)

Below are the logs when I start py-qgis-wps:

2023-07-03 14:02:02,262 [963]   INFO    Proxy configuration enabled
2023-07-03 14:02:02,262 [963]   INFO    WPS Server ready
2023-07-03 14:02:02,263 [963]   INFO    Running WPS server on port 0.0.0.0:8081
2023-07-03 14:02:02,264 [963]   WARNING No WPS plugin found in /usr/share/qgis_wps/processing
2023-07-03 14:02:02,266 [963]   WARNING No WPS plugin found in .
2023-07-03 14:02:02,641 [966]   INFO    Starting Qgis application: 3.30.3-'s-Hertogenbosch
2023-07-03 14:02:02,642 [966]   INFO    Setting offscreen mode
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
2023-07-03 14:02:03,511 [966]   INFO    [qgis:966] Installing cleanup hook
2023-07-03 14:02:03,536 [966]   INFO    [qgis:966] Qgis application initialized......
2023-07-03 14:02:05,565 [966]   ERROR   [qgis:966] Processing: Problem with OTB installation: OTB folder is not set.
2023-07-03 14:02:05,582 [966]   INFO    [qgis:966] QGis processing initialized
2023-07-03 14:02:05,677 [966]   WARNING No published processes !
2023-07-03 14:02:05,682 [972]   INFO    Starting worker pool
2023-07-03 14:02:05,717 [963]   INFO    Starting supervisor
2023-07-03 14:02:05,718 [963]   INFO    WPS Server ready
2023-07-03 14:02:06,120 [973]   INFO    Starting Qgis application: 3.30.3-'s-Hertogenbosch
2023-07-03 14:02:06,121 [973]   INFO    Setting offscreen mode
QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-root'
2023-07-03 14:02:06,856 [973]   INFO    [qgis:973] Installing cleanup hook
2023-07-03 14:02:06,857 [973]   INFO    [qgis:973] Qgis application initialized......
2023-07-03 14:02:07,286 [973]   ERROR   [qgis:973] Processing: Problem with OTB installation: OTB folder is not set.
2023-07-03 14:02:07,291 [973]   INFO    [qgis:973] QGis processing initialized

I didn't published any scripts by my self. Shouldn't the default qgis geoprocessing functions like "intersects", "join", etc, appear by default?

Thanks.

Model output format

Hi,
thanks a lot for this interesting project.

We are testing some QGIS models. In QGIS I can choose as output whether to create a temporary layer, generate a gpkg or save with another format (e.g. csv).

With py-qgis-wps, I always get the result as gpkg. If I define the parameter of the output file such as file.csv, a gpkg with a layer called "file.csv" is created instead of a csv file.

Is it possible to specify somehow what the desired output format should be?

wps server hangs on Macos when Reloading Qgis providers

py-qgis-wps running on docker on Macos, with variable QGSWPS_SERVER_RESTARTMON set to /tmp/restart_workers.

When touching the file from command line connected to wps docker (touch $QGSWPS_SERVER_RESTARTMON), wps docker is reloading qgis providers as expected but hangs after that. Extract of log:

wps-wps-1         | 2022-09-26 13:18:12,473     [1]     DEBUG   running update hook for ['/tmp/restart_workers']
wps-wps-1         | 2022-09-26 13:18:12,476     [1]     INFO    Reloading Qgis providers
wps-wps-1         | Application state:
wps-wps-
...
wps-wps-1         | 2022-09-26 13:18:13,344     [52]    INFO    [qgis:52] Qgis application initialized......

Then I cannot access wps server anymore and must send several ctrl+c to stop container from docker compose up command

new release

Could it be possible to get a new release please?

1.8.6 was in April 2023 and there has been several changes since this version.

Thanks

how can i send geojson files as input to a process exist in the server (in PSOT and GET)

Hello,
how can i send geojson files to server in order to run it ?
how can i display layers of qgis projects in browser using MAP parameter ?
I am executing this request http://127.0.0.7:8040/?MAP=/home/ayuob19971/Bureau/PFE/v2/py-qgis-wps-1.6.0/tests/unittests/data/france_parts.qgs&service=WMS&request=GetCapabilities
The results is :

author | "3Liz"
author_url | "http://3liz.com"
config |  
DEFAULT | {}
logging |  
level | "DEBUG"
logstorage:redis |  
dbnum | "0"
host | "localhost"
port | "6379"
prefix | "pyqgiswps"
metadata:main |  
contact_address | "Mailing Address"
contact_email | "Email Address"
contact_name | "Lastname, Firstname"
contact_role | "pointOfContact"
contact_url | "Contact URL"
identification_abstract | "Py-Qgis-WPS is an implementation of the Web Processing Service standard from the Open Geospatial Consortium. Py-Qgis-WPS is written in Python."
identification_accessconstraints | "NONE"
identification_fees | "NONE"
identification_keywords | "Py-Qgis-WPS,WPS,OGC,QGIS,processing"
identification_keywords_type | "theme"
identification_title | "Py-Qgis-WPS Processing Service"
provider_name | "Organization Name"
provider_url | "https://github.com/3liz/py-qgis-wps"
processing |  
accesspolicy | "/accesspolicy.yml"
default_module_path | "/usr/share/qgis_wps/processing"
exposed_providers | "script,model"
providers_module_path | ""
projects.cache |  
rootdir | ""
size | "10"
strict_check | "yes"
projects.schemes | {}
qgis.projects |  
wmsurl | "${server:wms_service_url}"
qgis.settings | {}
qgis.settings.folders |  
Processing/Configuration/MODELS_FOLDER | "/models"
Processing/Configuration/SCRIPTS_FOLDERS | "/scripts"
server |  
CACHE_ROOTDIR | "/home/ayuob19971/Bureau/PFE/v2/py-qgis-wps-1.6.0/tests/unittests/data"
PROCESSING_EXPOSED_PROVIDERS | "TestCopyLayer,TestAlgorithmProvider"
SERVER_PARALLELPROCESSES | "3"
SERVER_WORKDIR | "/home/ayuob19971/Bureau/PFE/v2/py-qgis-wps-1.6.0"
cleanup_interval | "600"
download_ttl | "30"
enable_filters | "yes"
encoding | "utf-8"
host_proxy | ""
interfaces | "127.0.0.7"
language | "en-US"
maxbuffersize | "1m"
maxinputsize | "100m"
maxqueuesize | "100"
outputfile_as_reference | "yes"
parallelprocesses | "1"
port | "8040"
processlifecycle | "1"
providers_module_path | "/home/ayuob19971/Bureau/PFE/v2/py-qgis-wps-1.6.0/tests/unittests/algorithms"
response_expiration | "86400"
response_timeout | "1800"
status_url | "{host_url}ows/?service=WPS&request=GetResults&uuid={uuid}"
store_url | "{host_url}store/{uuid}/{file}?service=WPS"
sys_config_path | "/usr/share/qgis_wps"
url | "{host_url}"
wms_response_uri | "${wms_service_url}?MAP={map_url}&service=WMS&request=GetCapabilities"
wms_service_url | "${wps.request:host_url}"
workdir | "/tmp"
wps_result_map_uri | "wps-results:"
wps.request | {}
qgis_version | "31802 (Zürich)"
styles | {}
tornado_ver | "6.1"
version | "n/a"

Erreur lors du chargement d'un fond de carte tuile XYZ personnalisé

Le serveur QGIS plante et redémarre lors de chargement d'un fond de carte personnalisé et implémenté au format tuile XYZ.
Avec la même version de QGIS Server (3.10) installé directement sur la même machine, le fond de carte est bien rendu.
Rendu fonctionnel avec OSM par exemple ou IGN.

Les logs :
13:57:40 WARNING PostGIS[333]: NOTICE: row number 0 is out of range 0..-1 qt.svg: Cannot open file ':/images/composer/missing_image.svg', because: No such file or directory qt.svg: Cannot open file ':/images/composer/missing_image.svg', because: No such file or directory qt.svg: Cannot open file ':/images/composer/missing_image.svg', because: No such file or directory 2021-12-31 13:57:46,739 DEBUG [333] Cleaning config cache entry /projects/rep3/veloroutes.qgs 2021-12-31 13:57:47,286 WARNING [333] Qgis: : formatString is: image/jpeg 13:57:47 WARNING [333]: formatString is: image/jpeg 2021-12-31 13:57:47,287 INFO [333] Qgis: Server: BBOX:-1411824.84092,5156961.528564,1880470.84092,6624552.471436 13:57:47 INFO Server[333]: BBOX:-1411824.84092,5156961.528564,1880470.84092,6624552.471436 2021-12-31 13:57:47,287 INFO [333] Qgis: Server: CRS:EPSG:3857 13:57:47 INFO Server[333]: CRS:EPSG:3857 2021-12-31 13:57:47,287 INFO [333] Qgis: Server: DPI:96 13:57:47 INFO Server[333]: DPI:96 2021-12-31 13:57:47,287 INFO [333] Qgis: Server: EXCEPTIONS:application/vnd.ogc.se_inimage 13:57:47 INFO Server[333]: EXCEPTIONS:application/vnd.ogc.se_inimage 2021-12-31 13:57:47,288 INFO [333] Qgis: Server: FORMAT:image/jpeg 13:57:47 INFO Server[333]: FORMAT:image/jpeg 2021-12-31 13:57:47,288 INFO [333] Qgis: Server: HEIGHT:600 13:57:47 INFO Server[333]: HEIGHT:600 2021-12-31 13:57:47,288 INFO [333] Qgis: Server: LAYERS:AF3V 13:57:47 INFO Server[333]: LAYERS:AF3V 2021-12-31 13:57:47,288 INFO [333] Qgis: Server: MAP:rep3/veloroutes.qgs 13:57:47 INFO Server[333]: MAP:rep3/veloroutes.qgs 2021-12-31 13:57:47,288 INFO [333] Qgis: Server: REQUEST:GetMap 13:57:47 INFO Server[333]: REQUEST:GetMap 2021-12-31 13:57:47,288 INFO [333] Qgis: Server: SERVICE:WMS 13:57:47 INFO Server[333]: SERVICE:WMS 2021-12-31 13:57:47,288 INFO [333] Qgis: Server: STYLES:défaut 13:57:47 INFO Server[333]: STYLES:défaut 2021-12-31 13:57:47,288 INFO [333] Qgis: Server: VERSION:1.3.0 13:57:47 INFO Server[333]: VERSION:1.3.0 2021-12-31 13:57:47,289 INFO [333] Qgis: Server: WIDTH:1346 13:57:47 INFO Server[333]: WIDTH:1346 2021-12-31 13:57:47,290 INFO [333] Qgis: Server: WMS Request parameters: 13:57:47 INFO Server[333]: WMS Request parameters: 2021-12-31 13:57:47,291 INFO [333] Qgis: Server: - CRS : EPSG:3857 13:57:47 INFO Server[333]: - CRS : EPSG:3857 2021-12-31 13:57:47,291 INFO [333] Qgis: Server: - WIDTH : 1346 13:57:47 INFO Server[333]: - WIDTH : 1346 2021-12-31 13:57:47,291 INFO [333] Qgis: Server: - HEIGHT : 600 13:57:47 INFO Server[333]: - HEIGHT : 600 2021-12-31 13:57:47,291 INFO [333] Qgis: Server: - BBOX : -1411824.84092,5156961.528564,1880470.84092,6624552.471436 13:57:47 INFO Server[333]: - BBOX : -1411824.84092,5156961.528564,1880470.84092,6624552.471436 2021-12-31 13:57:47,291 INFO [333] Qgis: Server: - LAYERS : AF3V 13:57:47 INFO Server[333]: - LAYERS : AF3V 2021-12-31 13:57:47,291 INFO [333] Qgis: Server: - STYLES : défaut 13:57:47 INFO Server[333]: - STYLES : défaut 2021-12-31 13:57:47,292 INFO [333] Qgis: Server: - FORMAT : image/jpeg 13:57:47 INFO Server[333]: - FORMAT : image/jpeg 2021-12-31 13:57:47,292 INFO [333] Qgis: Server: - DPI : 96 13:57:47 INFO Server[333]: - DPI : 96 2021-12-31 13:57:47,292 INFO [333] Qgis: Server: - VERSION : 1.3.0 13:57:47 INFO Server[333]: - VERSION : 1.3.0 2021-12-31 13:57:47,380 INFO [333] Qgis: server: Parallel rendering deactivated 13:57:47 INFO server[333]: Parallel rendering deactivated 2021-12-31 13:57:59,653 ERROR [1] Request timeout error 2021-12-31 13:57:59,653 REQ [1] 192.168.10.11 504 POST /ows/ 20005 83 Lizmap 2021-12-31 13:57:59,653 CRITICAL [22] Killing stalled process 333 2021-12-31 13:57:59,701 WARNING [22] Qgis Worker exited with code -9 2021-12-31 13:57:59,726 DEBUG [494] Initializing qgis server 2021-12-31 13:57:59,728 INFO [494] Starting Qgis application: 3.10.14-A Coruña WARNING: QApplication was not created in the main() thread. QStandardPaths: XDG_RUNTIME_DIR not set, defaulting to '/tmp/runtime-' 2021-12-31 13:57:59,929 DEBUG [21] REQUEST b'OWS-SERVER-1' b'\xaa\x01\x91~jA\x11\xec\x85\xb0\x02B\xac\x11\x00\x02' 2021-12-31 13:57:59,930 DEBUG [21] SND client: b'OWS-SERVER-1' -> worker: b'\x9e\x05gLjA\x11\xec\x86B\x02B\xac\x11\x00\x02' : b'\xaa\x01\x91~jA\x11\xec\x85\xb0\x02B\xac\x11\x00\x02' 2021-12-31 13:57:59,931 DEBUG [343] RCV b'OWS-SERVER-1': b'\xaa\x01\x91~jA\x11\xec\x85\xb0\x02B\xac\x11\x00\x02' 2021-12-31 13:57:59,934 DEBUG [343] Handling request: b'\xaa\x01\x91~jA\x11\xec\x85\xb0\x02B\xac\x11\x00\x02' 2021-12-31 13:57:59,948 DEBUG [343] Resolving 'file' protocol 2021-12-31 13:57:59,949 DEBUG [343] Reading Qgis project /projects/rep3/veloroutes.qgs 2021-12-31 13:57:59,989 INFO [494] Qgis: Installing cleanup hook 2021-12-31 13:57:59,990 INFO [494] Qgis: Qgis application initialized...... 2021-12-31 13:57:59,991 INFO [494] Qgis: Server: Skipping GDAL ECW drivers in server.

WPS exit with code 139 randomly

https://github.com/3liz/lizmap-wps-web-client-module/issues/7#issue-629780449

Publish the documentation on docs.3liz.org

Guidance about use of MAP=postgres connection string

Hi, I'm having some troubles with using MAP and a connection string to a postgres source, from which we expect to load a QGIS project which at same time contains references to layers defined in the same database.

Important:
the decoded connection string "postgres://postgres:[email protected]:5432?sslmode=require&dbname=postgres&schema=myschema&project=my_qgis_project_indb" points to my_qgis_project_indb project. That project was saved from QGIS to the database and layers defined on it are postgis based layers. That is those "tables" are located in the same schema. Then you have the qgis project table + all the tables/layers in the same schema

A call like this:
https://localhost:port/wps/ows/?SERVICE=WPS&VERSION=1.0.0&REQUEST=DescribeProcess&identifier=model:my_model&MAP=postgres%3A%2F%2Fpostgres%3Apassword%40postgres.database.url.com%3A5432%3Fsslmode%3Drequire%26dbname%3Dpostgres%26schema%3Dmyschema%26project%3Dmy_qgis_project_indb

should be allowing to show the list of available layers in the ows:AllowedValues tag under the input (for the model, which is simply getting a vector layer as input), but list of layers is empty.

If instead of referencing the MAP to a postgres, I use the file equivalent (that is the qgs file with same layers defined) then I can get the list of allowed values (vectorial layers).

I'm not sure if I'm missing something in middle or if it's a kind of bug or expected behavior (as the QGIS project in this case is loaded dinamically - cause it's not a known "project", as you receive it by MAP parameter - then not sure if the projects cache is considering that. The weird thing is that it's not showing an error, it just simply returns an empty list of layers)

Any thoughts/ideas ?

Exposing Processing algorithms with py-qgis-wps and docker-compose

Hello,
I'm trying to expose the processing algorithms with py-qgis-wps to be used with Lizmap. I'm using the docker-compose variant because it is easy to install and use. Everything went well until I decided to expose the "native" algorithms.
In the docker-compose.yml file, I set the provider property in this way:
QGSWPS_PROCESSING_EXPOSED_PROVIDERS: native,qgis

When executing docker compose up I'm getting the error reported below:

This error occurs just when using the "native" provider; if I use just the qgis (QGSWPS_PROCESSING_EXPOSED_PROVIDERS: qgis) provider, everything works well but I can't see the "native" algorithms, e.g. native:buffer

PD: I'm using the docker image: 3liz/qgis-wps:3.30, but I also used 3liz/qgis-wps:release-rc and 3liz/qgis-wps:ltr-rc, but i got the same error.

Can you give me some hints on how to solve this problem?

Thanks!!

Acitvate WCS and WFS OGC protocols for the generated qgis projects

It would be nice if WCS and WFS could be activated on the qgis projected generated after a WPS Execute command.

processing.runAndLoadResults does not load results

I have recently started using QGIS WPS. I am using the docker image 3liz/qgis-wps:3.30. For testing I wrote a very simple algorithm with no INPUT or OUTPUT:

        feedback.pushDebugInfo("Step 1")
        res1 = processing.runAndLoadResults(
            "qgis:rastercalculator",
            {
                'EXPRESSION': '1',
                'LAYERS': None,
                'CELLSIZE': 10,
                'EXTENT': '67740.870400000,79959.890400000,341924.860400000,360502.590400000 [EPSG:31255]',
                'CRS': QgsCoordinateReferenceSystem('EPSG:31255'),
                'OUTPUT': '/tmp/test1.tiff'}, context=context, feedback=feedback)
        
        feedback.pushDebugInfo("Step 2")
        res2 = processing.run(
            "qgis:rastercalculator",
            {
                'EXPRESSION': '"test1@1"',
                'LAYERS': None,
                'CELLSIZE': 10,
                'EXTENT': '67740.870400000,79959.890400000,341924.860400000,360502.590400000 [EPSG:31255]',
                'CRS': QgsCoordinateReferenceSystem('EPSG:31255'),
                'OUTPUT': 'TEMPORARY_OUTPUT'}, context=context, feedback=feedback)

it simply creates a raster layer and uses it in the next step to create a second raster layer. It works absolutely fine when I run it in QGIS Desktop, but when I run it in through WPS I get the following log:

2023-04-25 07:11:57,357	[28]	INFO	Starting task 770e7536:script:test
2023-04-25 07:11:57,360	[28]	DEBUG	Resolving 'file' protocol
2023-04-25 07:11:57,362	[28]	DEBUG	Reading Qgis project /projects/klimalinz/KLIMALINZ_v1.4.qgz
2023-04-25 07:11:59,050	[28]	DEBUG	script:test:770e7536 Step 1
2023-04-25 07:11:59,082	[28]	INFO	script:test:770e7536 Results: {'OUTPUT': '/tmp/test1.tiff'}
2023-04-25 07:11:59,146	[28]	DEBUG	script:test:770e7536 Step 2
2023-04-25 07:11:59,184	[28]	CRITICAL	Traceback (most recent call last):
  File "/opt/local/pyqgiswps/lib/python3.9/site-packages/pyqgiswps/executors/processingprocess.py", line 242, in run_processing_algorithm
    results, ok = alg.run(parameters, context, feedback, configuration=create_context,
_core.QgsProcessingException: An error occurred while performing the calculation

2023-04-25 07:11:59,184	[28]	INFO	script:test:770e7536 memory: start=183.469Mb end=213.340Mb delta=29.871Mb

I investigated a little further and the problem is that runAndLoadResults does not load the output layer. When I instead use the following code for step 1 it works fine:

        res1 = processing.run(
            "qgis:rastercalculator",
            {
                'EXPRESSION': '1',
                'LAYERS': None,
                'CELLSIZE': 10,
                'EXTENT': '67740.870400000,79959.890400000,341924.860400000,360502.590400000 [EPSG:31255]',
                'CRS': QgsCoordinateReferenceSystem('EPSG:31255'),
                'OUTPUT': 'TEMPORARY_OUTPUT'}, context=context, feedback=feedback)
        
        outputlayer = QgsRasterLayer(res1['OUTPUT'], 'test1')
        QgsProject.instance().addMapLayer(outputlayer)

Since it works in QGIS Desktop but not in WPS I assume this might be a bug.

I use the following docker-compose.yml:

version: '3.8'
services:
  wps:
    image: 3liz/qgis-wps:3.30
    environment:
      QGSWPS_SERVER_PARALLELPROCESSES: '2'
      QGSWPS_SERVER_LOGSTORAGE: REDIS
      QGSWPS_REDIS_HOST: redis
      QGSWPS_PROCESSING_PROVIDERS_MODULE_PATH: /processing
      QGSWPS_CACHE_ROOTDIR: /projects
      QGSWPS_SERVER_WORKDIR: /srv/data
      QGSWPS_USER: 1000:1000
      QGSWPS_LOGLEVEL: DEBUG
    volumes:
      - ./processing:/processing
      - ./projects:/projects
      - ./data:/srv/data
      - ./qgis:/home/qgis
    ports:
      - 127.0.0.1:8888:8080
  redis:
      image: redis:5-alpine

Complete algorithm: https://pastebin.com/8EcRwdVi