davidhuser / dhis2-pk

Command-line tools for DHIS2

Hi there 👋

dhis2-pk's Issues

Virtualenv installation is broken

I tried to install it in a virtualenv.

Output:

$ virtualenv env
Using base prefix '/usr'
New python executable in /tmp/tmp.d9C10LWOLM/env/bin/python3
Also creating executable in /tmp/tmp.d9C10LWOLM/env/bin/python
Installing setuptools, pip, wheel...done.

$ pip install dhis2-pocket-knife
Collecting dhis2-pocket-knife
  Downloading dhis2-pocket-knife-0.0.1.tar.gz
Requirement already satisfied (use --upgrade to upgrade): requests in /usr/lib/python3.5/site-packages (from dhis2-pocket-knife)
Installing collected packages: dhis2-pocket-knife
  Running setup.py install for dhis2-pocket-knife ... error
    Complete output from command /usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-zvzrk2dr/dhis2-pocket-knife/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-nxuzib7n-record/install-record.txt --single-version-externally-managed --compile:
    running install
    running build
    running build_scripts
    creating build
    creating build/scripts-3.5
    copying and adjusting scripts/delete-objects.py -> build/scripts-3.5
    copying and adjusting scripts/share-objects.py -> build/scripts-3.5
    copying and adjusting scripts/user-orgunits.py -> build/scripts-3.5
    changing mode of build/scripts-3.5/delete-objects.py from 644 to 755
    changing mode of build/scripts-3.5/share-objects.py from 644 to 755
    changing mode of build/scripts-3.5/user-orgunits.py from 644 to 755
    running install_egg_info
    running egg_info
    writing dependency_links to dhis2_pocket_knife.egg-info/dependency_links.txt
    writing top-level names to dhis2_pocket_knife.egg-info/top_level.txt
    writing dhis2_pocket_knife.egg-info/PKG-INFO
    writing requirements to dhis2_pocket_knife.egg-info/requires.txt
    warning: manifest_maker: standard file '-c' not found

    reading manifest file 'dhis2_pocket_knife.egg-info/SOURCES.txt'
    writing manifest file 'dhis2_pocket_knife.egg-info/SOURCES.txt'
    Copying dhis2_pocket_knife.egg-info to /usr/lib/python3.5/site-packages/dhis2_pocket_knife-0.0.1-py3.5.egg-info
    error: [Errno 13] Permission denied: '/usr/lib/python3.5/site-packages/dhis2_pocket_knife-0.0.1-py3.5.egg-info'

    ----------------------------------------
Command "/usr/bin/python -u -c "import setuptools, tokenize;__file__='/tmp/pip-build-zvzrk2dr/dhis2-pocket-knife/setup.py';exec(compile(getattr(tokenize, 'open', open)(__file__).read().replace('\r\n', '\n'), __file__, 'exec'))" install --record /tmp/pip-nxuzib7n-record/install-record.txt --single-version-externally-managed --compile" failed with error code 1 in /tmp/pip-build-zvzrk2dr/dhis2-pocket-knife/

improve print message when metadata or user groups not found

current:

* WARNING  2020-12-03 17:14:04,003  No categoryOptions found - check your filter

does not print what exactly is not found.

Also the full traceback is printed when a userGroup was not found (exiting the full sharing command):

Traceback (most recent call last):
  File "/home/david/.local/bin/dhis2-pk", line 8, in <module>
    sys.exit(pocketknife_run())
  File "/home/david/.local/lib/python3.8/site-packages/src/main.py", line 82, in pocketknife_run
    share_main(args, password)
  File "/home/david/.local/lib/python3.8/site-packages/src/share.py", line 610, in main
    usergroups = UserGroupsCollection(api, args.groups)
  File "/home/david/.local/lib/python3.8/site-packages/src/share.py", line 423, in __init__
    usergroups = self.get_usergroup_uids(filter_list, root_junction)
  File "/home/david/.local/lib/python3.8/site-packages/src/share.py", line 452, in get_usergroup_uids
    raise PKClientException("No userGroup found with {}".format(filter_list))
src.common.exceptions.PKClientException: No userGroup found with ['name:eq:Admin_userGroup']

dhis2-pk-delete-objects: exit on error

I ran the following command:

$ dhis2-pk-delete-objects --server=play.dhis2.org/demo --uid_file=UIDS.txt --object_type=dataElements --username=admin --password=district
Deleting FHD3wiSM7Sn
[409] https://play.dhis2.org/demo/api/dataElements/FHD3wiSM7Sn
{"httpStatus":"Conflict","httpStatusCode":409,"status":"ERROR","message":"Could not delete due to association with another object: DataValue"}

Maybe you could catch the error and continue with the execution of the remaining items? It is bulk processing, isn't it?

Overwriting/extending existing sharing settings

Hi David.

Please see the following issue from one of our Lead Data Managers:

Farai

We were given the following script to use when changing sharing settings.

dhis2-pk-share -s https://staging.dhis.dhmis.org/fs_591606 -u ##### -p ##### -t dataelement -f name:ilike:antenatal -a readwrite -g "name:like:Facilitator" readwrite -g "name:like:Dashboard Managers" none

In the example above the script will change any data element with the word “antenatal to “Public Access” readwrite and it will change the selected groups “Facilitator” & “Dashboard Managers” to readwrite and none respectively. The problem we have is that if there where any other groups already in the sharing settings that we do not want to amend it will now be removed from the selection above as it is not part of the script. What we need the script to do is ignore current groups where no amendment is made and leave it in the sharing settings.

Hope it is clear

Thanks

Kind Regards

Hans Naude

Team Coordinator: Data Management

OS X El Capitan & six

If trouble with installing dhis2-pocket-knife / six:
Install like this: pypa/pip#3165 (comment)

include analytics period boundaries in the program indicator output

dhis2-pk indicator-definitions to include columns for Analytics Period Boundaries when using argument -t programIndicators

getpass support

Support for getpass instead of argparse for passwords

support for loading from file

e.g. similar to -m from https://github.com/olavpo/dhis2-sharing-from-file

user-agent is not accurate

Problem: User-agent header transmitted with every request is not accurate / identifiable to pocketknife.

Solution:
https://github.com/davidhuser/dhis2-pk/blob/master/src/common/utils.py#L16
dhis2-src/0.35.1 should become dhis2-pk/0.35.1

Failed to install dhis2-pk on a Windows 7 machine

Hi after doing several installations of dhis2-pk on other machines, I failed to install dhis2-pk on a Windows 7 machine. Tried using pip and also without pip.

add DeprecationWarning for Python 2.7 and Python <=3.5

as they all reached End of Life

Fetch the native data integrity results as well

https://docs.dhis2.org/en/develop/using-the-api/dhis-core-version-234/web-api.html#webapi_data_integrity

python 3 compability

in my virtualenv:

Python 3.5.2 (default, Jun 28 2016, 08:46:01) 
[GCC 6.1.1 20160602] on linux

I get the following errors:

$ share-objects.py 
  File "/tmp/tmp.d9C10LWOLM/env/bin/share-objects.py", line 28
    print "Getting " + usergroup_name + " UID..."

$ user-orgunits.py 
  File "/tmp/tmp.d9C10LWOLM/env/bin/user-orgunits.py", line 45
    print "Checking " + str(no_of_users) + " users against sub-orgunit assignments of root orgunit " + orgunit_root['name']

$ delete-objects.py 
  File "/tmp/tmp.d9C10LWOLM/env/bin/delete-objects.py", line 33
    print msg

I would either write in the README that you are only targeting python 2.7 or make the code work in both 2.7 and 3. IMHO because the codebase is not really large it would be okay to try to write code that runs in both versions.

Incorporate new user lookup API into userinfo

use userInfo endpoint from 2.35 release https://www.dhis2.org/235#api for userinfo script

Error: "command not found" on macOS

An upgrade to macOS Catalina seems to unrecognize the command.
Strangely pip show dhis2-pocket-knife command gives me an existing dhis2-pocket-knife v0.32.3 however in the location, /Users/username/Library/Python/2.7/lib/python/site-packages I notice it is pointing to version 2.7 instead of my previous 3.7 - could this be the problem?

No dhis2 object type for 'charts'

If you want to share charts with dhis2-pk share you will get an error:

src.common.exceptions.PKClientException: No DHIS2 object type for 'charts'

which is an bug in DHIS2 not dhis2-pk: https://jira.dhis2.org/browse/DHIS2-10591

Data integrity: check that program rule variables don't contain invalid keywords

Program Rule Variables cannot contain or / and / not (mind the spaces before and after the word) b/c it creates to issues for Android Capture app so that no Program Rule will work.

Reference: https://github.com/dhis2/dhis2-utils/blob/master/resources/sql/check_program_rule_variable_name.sql

Solution: implement GET requests assuring these are empty.

• https://play.dhis2.org/2.36.0/api/programRules.json?paging=false&fields=id,name,condition&filter=condition:like:%20or%20&filter=condition:like:%20and%20&filter=condition:like:%20not%20&rootJunction=OR
• https://play.dhis2.org/2.36.0/api/programRuleVariables.json?paging=false&fields=id,name&filter=name:like:%20or%20&filter=name:like:%20and%20&filter=name:like:%20not%20&rootJunction=OR

self-standing executables

no need to install python, pip, pipenv in order to run it and put it up into releases, e.g. via https://nuitka.net/doc/user-manual.html

Script not found: modifying PATH

If you install it with pip install dhis2-pocket-knife --user and then try to run any of the scripts, e.g. dhis2-pk-share --help, it may be that the script could not be found.

Fix:
You need to add it to your PATH in order to find it.

MacOS

Replace $USER with your macOS username, then paste it into a terminal.
echo 'export PATH=$PATH:/Users/$USER/Library/Python/2.7/bin' >> ~/.bash_profile && source ~/.bash_profile