north-road / qgis-redistricting-plugin Goto Github PK

2018-05-30T11:54:32 INFO : Preparing redistricting

2018-05-30T11:55:04 SUCCESS : Switched to “General (North Island)”

2018-05-30T12:20:48 SUCCESS : Created electorate “Just testing”

2018-05-30T12:20:59 SUCCESS : Redistricted selected meshblocks to Just testing

2018-05-30T12:20:59 WARNING Python error : An error has occurred while executing Python code: See message log (Python Error) for more details.

2018-05-30T12:20:59 WARNING Traceback (most recent call last):
File "C:/Users/gbyrom/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\redistrict\redistrict.py", line 570, in redistrict_selected
gui_handler.show_stats_for_district(dlg.selected_district)
File "C:/Users/gbyrom/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\redistrict\linz\linz_redistrict_gui_handler.py", line 49, in show_stats_for_district
'POPULATION'])
File "C:/Users/gbyrom/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\redistrict\linz\linz_district_registry.py", line 181, in get_variation_from_quota_percent
return round(100 * (population - quota) / quota)
TypeError: unsupported operand type(s) for -: 'QVariant' and 'int'

"After importing a database, there’s no layers loaded. Even if I import the same database as I was using to start with.

Begin Redistricting does nothing, and if I choose Toggle Editing I get a python error."

Need a class for interfacing with the Statistics NZ population API. Implement as a QgsTask subclass so that we can take advantage of QGIS 3.0 task manager.

On undo or rollback of unsaved changes, we also need to remove the staged log entries.

If the changes are then re-done, the original log entries will need to be restored in an atomic fashion (i.e. as though no undo actions were ever performed).

Also I just came across a sudden “undo” while doing interactive
redistricting… not sure what happened there but everything suddenly went
back to the last staged edit. I was nowhere near the button so didn’t hit it
accidentally.

This repo requires a Travis CI setup using the QGIS master docker container.

Python unit tests are kicked off via a call to make test

During previous email exchanges I proposed to use PKI authentication for authentication.

https://github.com/linz/qgis-redistricting-plugin/wiki/Requirements section F10 does not mention any of the PKI available methods but instead it sais:

Security

Application must be capable of working through a secure connection to transmit and receive data via the Statistics NZ API, transport integration will be secured to TLS1.2 standard, mutual > authentication will apply with IP whitelisting (if possible) Application works through a secure connection possibly with a VPN to a whitelisted IP address

This does not look like PKI, instead it suggests that the authentication is handled outside QGIS at the presentation level (VPN/TLS).

If that is correct, we don't need to do anything inside QGIS and we don't have to test anything because the whole auth would be delegated to the O.S.

"I think I’ve done this before, but got a different issue, which may have been fixed but now this one. This happened when I went to import the master database, but chose to import the same one I’m already working on. I typed I ACCEPT into the box, saved the current database as a new name, but then this happened when importing.

May not be something I should be attempting to do… but worth closing loops so that the user can’t do it at all might be a good idea.

Everything’s gone blank and before I can “begin redistricting” I need to re-load the project. However all the data layers from the database refuse to load now, with the “handle bad layers” box displaying the Electorates, Meshblocks, and all the layers from the Tables group.

When I do this on a different database copy (as I should always do), it seems to work fine."

During "selected meshblocks" redistricting, is it possible to display the (rounded) percentage of the quota that each whole selection (not each meshblock) will make, somewhere on the screen (preferably the map, but it could be in the redistricting window)? So as you add more meshblocks to the selection, a total percentage within them starts getting larger, up to the amount you need to add. This might be more difficult as it means totalling up the populations inside a selection, I'm not even sure if that's possible. But what would be good to know is how much difference a particular selection will make, so we know whether we need to add more meshblocks to it. Without this it's difficult to know whether you should select more meshblocks or not.

"I select import a database, I get the open file dialog to open a database. Select the file I want, I get the dialog to type in “I ACCEPT”. Then I get the open file dialog again and have to select the file a second time."

We need to block redistricting "stop" operations if there's an unsaved edits in the meshblock layer

…appears to be OSGEO4W/bin

Might be better if there was a default location for this so the database you’re using currently is presented to make it obvious where database are stored?

I’m not sure I should be able to select this from the “Redistrict selected meshblocks” dialog.

SO, I opened the Electorates attribute table, curious as to what this actually is, and did a “zoom to feature” on it. I’m not sure where it’s taken me but my screen shows nothing, and when I zoom out a few times, I get this:

This will probably fail on start

https://travis-ci.org/north-road/qgis-redistricting-plugin/jobs/375785564#L547

"Redistrict some meshblocks, then click on the Electorate Statistics button, and click in the area just redistricted. Electorate statistics for the electorate they were previously assigned to still appear."

During interactive redistricting, is it possible to display the percentage under/over quota on the labels as well as the changed populations?

On switching to a scenario which I had just copied from the same database… this happened:

Stack Trace

std::basic_string,std::allocator >::end :

std::_String_alloc > >::_Getal :

PyCFunction_FastCallDict :

PyObject_GenericGetAttr :

PyEval_EvalFrameDefault :

PyFunction_FastCallDict :

PyObject_CallFunctionObjArgs :

PyObject_Call :

PyInit_sip :

std::basic_string,std::allocator >::shrink_to_fit :

PyInit__core :

QgsTask::start :

QThreadPoolPrivate::reset :

QThread::start :

BaseThreadInitThunk :

RtlUserThreadStart :

> Basically I did a “Import scenario from database”, and imported an existing one that was already in the same database. It started the import and then this happened. Let me try again and see if I can do the same thing… I’ll wait until the current switch has finished first so it doesn’t confuse the issue ;-)

> This happens AFTER importing a new scenario from the database (I happened to sue the same database as I was already using, I haven’t yet tried on a new database… I’m pretty sure I had already done that though and it did work, but can’t recall) but doesn’t happen during the import itself, it happens when I go to switch to the imported scenario.

 
> Importing a scenario, in this instance, seems to take no time – it just says “scenario imported” almost immediately, I suspect it hasn’t actually imported anything – which might be why the crash L


> Switching to it, then comes up with the progress bar saying “Preparing switch…” and after a few seconds I get the crash.

 

When I import a database, having just exported one (and therefore closed down everything), nothing shows up – do I need to re-load the project as well? If I do, is it going to load the old database or the one I’ve selected?

The API is aync: a boundaryRequest job is submitted and the processing results are retrieved at a later time by different call.

How do we handle the case when a user submitted a request and before getting back the results she wanted to submit another boundaryChange request?

The options I see here are:

• lock at the plugin level (a boundary change calculation is in progress: do you want to submit a new one?)
• handle a queue of requests and response (keyed by requestId)

"I’ve redistricted some of Taranaki-King Country into Hamilton West to see if I could bring it within quota, and it’s now reading 0% (very slightly under quota according to the figures on the label), but when I then run the validation, it reports both electorates as still outside quota tolerance. See below."

Add a simple shortcut to deactivate the interactive redistricting map tool - currently requires switching to another map tool. Make the button togglable, where deactivating it switches automatically to the pan tool.

Needs a canvas refresh after fetching stats

When I validate electorates, then I click the button to look up statistics for an electorate while validation is in progress, the validation list never comes up. The Invalid Electorates map gets updated, but there’s no separate way to bring up that list.

It would be good if the list appeared as a separate window, make it a tab on the Invalid Electorates window would make sense (although I’m not sure how that would work in practice as tabs are just separate docked windows anyway).

Should fix:

3.   Can still switch scenarios again while the switching scenarios process is already in progress. This might be OK but not sure what the effect will be?
   

4.   While this is going on, only one switching message shows at a time (there’s a “1 more” message at the end of it, but if I click on that, I can’t see what else is processing in the background. I can click the X to stop the process though).
   

5.   While switching scenarios process is in progress, validating statistics throws an error. It still seems to work it though. Not sure which scenario I’m validating however – it doesn’t tell me whether it’s the new one or the old one (old one I presume?)
   

error comes up when it can’t connect to the API (“Forbidden”), which is good, the statistics in the window still show as “updating” however.

As soon as the “Begin redistricting…” process is started, the Redistrict Selected meshblocks button is now active (I think this is a change?) even though there’s no meshblocks selected

Create new electorate – works fine, but when I then go to assign selected meshblocks to it, I get a python error, and the meshblocks end up with no electorate assigned:

An error has occurred while executing Python code:

TypeError: '>' not supported between instances of 'NoneType' and 'int'

Traceback (most recent call last):
File "C:/Users/gbyrom/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\redistrict\redistrict.py", line 635, in redistrict_selected
gui_handler.show_stats_for_district(dlg.selected_district)
File "C:/Users/gbyrom/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\redistrict\linz\linz_redistrict_gui_handler.py", line 50, in show_stats_for_district
if contents['VARIATION'] > 0:
TypeError: '>' not supported between instances of 'NoneType' and 'int'

Python version: 3.6.0 (v3.6.0:41df79263a11, Dec 23 2016, 08:06:12) [MSC v.1900 64 bit (AMD64)]
QGIS version: 3.1.0-Master Master, 4d3ff9a5c7

Python Path:

· C:/OSGEO4~1/apps/qgis-dev/./python

· C:/Users/gbyrom/AppData/Roaming/QGIS/QGIS3\profiles\default/python

· C:/Users/gbyrom/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins

· C:/OSGEO4~1/apps/qgis-dev/./python/plugins

· C:\OSGeo4W64\bin\python36.zip

· C:\OSGEO4~1\apps\Python36\DLLs

· C:\OSGEO4~1\apps\Python36\lib

· C:\OSGeo4W64\bin

· C:\OSGEO4~1\apps\Python36

· C:\OSGEO4~1\apps\Python36\lib\site-packages

· C:/Users/gbyrom/AppData/Roaming/QGIS/QGIS3\profiles\default/python

C:/Users/gbyrom/Desktop/QGIS Redistricting project

This one occurred when switching from General (North Island) to General (South Island) before I had clicked “begin redistricting”.

2018-05-29T14:17:38 WARNING Traceback (most recent call last):
File "C:/Users/gbyrom/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\redistrict\redistrict.py", line 458, in set_task_and_show_progress
self.set_task(task)
File "C:/Users/gbyrom/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\redistrict\redistrict.py", line 432, in set_task
self.meshblock_scenario_bridge.task = task
AttributeError: 'NoneType' object has no attribute 'task'

Switching to General South Island, BEFORE Begin Redistricting selected…

An error has occurred while executing Python code:

AttributeError: 'NoneType' object has no attribute 'task'

Traceback (most recent call last):
File "C:/Users/gbyrom/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\redistrict\redistrict.py", line 481, in set_task_and_show_progress
self.set_task(task)
File "C:/Users/gbyrom/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins\redistrict\redistrict.py", line 455, in set_task
self.meshblock_scenario_bridge.task = task
AttributeError: 'NoneType' object has no attribute 'task'

Python version: 3.6.0 (v3.6.0:41df79263a11, Dec 23 2016, 08:06:12) [MSC v.1900 64 bit (AMD64)]
QGIS version: 3.1.0-Master Master, e7da103096

Python Path:

· C:/OSGEO4~1/apps/qgis-dev/./python

· C:/Users/gbyrom/AppData/Roaming/QGIS/QGIS3\profiles\default/python

· C:/Users/gbyrom/AppData/Roaming/QGIS/QGIS3\profiles\default/python/plugins

· C:/OSGEO4~1/apps/qgis-dev/./python/plugins

· C:\OSGeo4W64\bin\python36.zip

· C:\OSGEO4~1\apps\Python36\DLLs

· C:\OSGEO4~1\apps\Python36\lib

· C:\OSGeo4W64\bin

· C:\OSGEO4~1\apps\Python36

· C:\OSGEO4~1\apps\Python36\lib\site-packages

· C:/Users/gbyrom/AppData/Roaming/QGIS/QGIS3\profiles\default/python

C:/Users/gbyrom/Desktop/QGIS Redistricting project

" This is stupidly trivial, but I wonder if it’s possible to make QGIS sound a small “bong” or similar sound when the mouse moves over a new meshblock being redistricted during interactive redistricting?

I think this would help a bit for the user to work with the performance issue as it makes it a bit easier for the user to know that they’ve changed something, and it seems a bit blank not having a sound. The visual change of meshblock colour accompanied by a small sound just seems to make sense. It might slow things down though?"

(with option to disable!)

"One thing to note is that the rollback rolls back the meshblocks but doesn’t seem to change the black boundary line back to how it was before. See below."

"Rollback and then redo still results in the boundary not being moved.

•      The meshblock colour remains as the redistricted colour
  

•      Clicking in the redistricted meshblocks gives the stats for the original electorate, not the redistricted one"
  

It seems to work correctly after an Undo. Just not after a rollback.

"Scenario switching takes a LOOOOOONG time – can this be speeded up?"

When I export a database, the Electorate Statistics window stays up with the stats for the last selected electorate in it, even though nothing’s open

"Select the Meshblocks layer by default, so that I don’t have to manually select it in the list. If it’s not currently selected and, say, the electorates layer is, then when I click a meshblock to redistrict the whole electorate gets selected instead – and I have to cancel that and select the meshblocks layer and then select the meshblock again. This could be confusing for users other than myself."

"Switching scenarios only changes the boundary but doesn’t update the meshblocks if they’ve already been redistricted in the previous scenario."

When I choose this option to end the process, I still seem to be able to continue redistricting.

The Statistics window stays open, and I can still perform the process of redistricting meshblocks.

Switching to General South Island, AFTER Begin Redistricting selected, but before the progress bar had completed:

h2. User Feedback

h2. Report Details

Crash ID: 6d2737cd15e245e5695a9c3d7674f5ba50835d05

Stack Trace

std::basic_string,std::allocator >::find_first_not_of :

std::basic_string,std::allocator >::find_first_not_of :

std::basic_string,std::allocator >::_Check_offset_exclusive :

PyCFunction_FastCallDict :

PyObject_GenericGetAttr :

PyEval_EvalFrameDefault :

PyErr_Occurred :

PyFunction_FastCallDict :

PyObject_CallFunctionObjArgs :

PyObject_Call :

PyInit_sip :

std::basic_string,std::allocator >::_Check_offset_exclusive :

std::basic_string,std::allocator >::_Check_offset_exclusive :

QgsLegendStyle::setMargin :

QgsLegendStyle::setMargin :

QThreadPoolPrivate::reset :

QThread::start :

BaseThreadInitThunk :

RtlUserThreadStart :

QGIS Info

QGIS Version: 3.1.0-Master

QGIS code revision: commit:e7da103096

Compiled against Qt: 5.9.2

Running against Qt: 5.9.2

Compiled against GDAL: 2.2.4

Running against GDAL: 2.2.4

And show as grey or yellow in invalid districts map view

List needs clearing (and map redrawn) as soon as a new validation task is started.

Project needs updating as a result

E.g. north island codes must start with 'N'