aidbox / aidbox-python-sdk

aidbox-python-sdk

1. Create a python 3.8+ environment pyenv
2. Set env variables and activate virtual environment source activate_settings.sh
3. Install the required packages with pipenv install --dev
4. Make sure the app's settings are configured correctly (see activate_settings.sh and aidbox_python_sdk/settings.py). You can also use environment variables to define sensitive settings, eg. DB connection variables (see example .env-ptl)
5. You can then run example with python example.py.

Add APP_FAST_START_MODE=TRUE to env_tests for fast start mode.

Getting started

Minimal application

from aidbox_python_sdk.main import create_app as _create_app
from aidbox_python_sdk.settings import Settings
from aidbox_python_sdk.sdk import SDK

settings = Settings(**{})
sdk = SDK(settings, resources=resources, seeds=seeds)

async def create_app():
    return await _create_app(sdk)

Register handler for operation

import logging
from aiohttp import web

from yourappfolder import sdk 


@sdk.operation(
    methods=["POST", "PATCH"],
    path=["signup", "register", {"name": "date"}, {"name": "test"}],
    timeout=60000  ## Optional parameter to set a custom timeout for operation in milliseconds
)
def signup_register_op(operation, request):
    """
    POST /signup/register/21.02.19/testvalue
    PATCH /signup/register/22.02.19/patchtestvalue
    """
    logging.debug("`signup_register_op` operation handler")
    logging.debug("Operation data: %s", operation)
    logging.debug("Request: %s", request)
    return web.json_response({"success": "Ok", "request": request["route-params"]})

Validate request

schema = {
    "required": ["params", "resource"],
    "properties": {
        "params": {
            "type": "object",
            "required": ["abc", "location"],
            "properties": {"abc": {"type": "string"}, "location": {"type": "string"}},
            "additionalProperties": False,
        },
        "resource": {
            "type": "object",
            "required": ["organizationType", "employeesCount"],
            "properties": {
                "organizationType": {"type": "string", "enum": ["profit", "non-profit"]},
                "employeesCount": {"type": "number"},
            },
            "additionalProperties": False,
        },
    },
}


@sdk.operation(["POST"], ["Organization", {"name": "id"}, "$update"], request_schema=schema)
async def update_organization_handler(operation, request):
    location = request["params"]["location"]
    return web.json_response({"location": location})

Valid request example

POST /Organization/org-1/$update?abc=xyz&location=us

organizationType: non-profit
employeesCount: 10

aidbox-python-sdk's Issues

Implement fast start mode

Now we always do whole app initialization. Whan app starts it should check that local manifest matches app manifest. If they are the same we should start the app without initialization phase.

Make sdk.db.raw_sql more high-level

raw_sql should raise an error or return the result without wrappers from Aidbox

Decrease default read timeout to 30s

Improve escaping for json - can not insert/update resource with escaped double quotes

Example:

async def raw_update(resource: AsyncAidboxResource):
    cleaned_resource = funcy.omit(
        resource.serialize(), ['resourceType', 'id', 'meta'])
    if 'meta' in resource:
        cleaned_meta = funcy.omit(
            resource['meta'], ['versionId', 'lastUpdated'])
        if cleaned_meta:
            cleaned_resource['meta'] = cleaned_meta

    table = getattr(sdk.db, resource.resource_type).__table__
    stmt = table.update().where(table.c.id == resource.id).values(
        resource=cleaned_resource,
        status='updated',
        ts=sqlalchemy.func.now()).returning(table)
    await sdk.db.alchemy(stmt)

for resource = {"field": "content \" "} I always get an error.

Add default aidbox alchemy func definitions

Let's make them available via sqlalchemy.func by default (project independent):

• knife_extract_text
• knife_extract_max_numeric
• knife_extract_min_numeric
• knife_extract_min_timestamptz
• knife_extract_max_timestamptz
• knife_date_bound
• aidbox_text_search

@ir4y What do you think? They can be useful for custom endpoints with sqlalchemy.

Update to python 3.11

There is an issue with pytest. It is partially fixed now at this branch https://github.com/Aidbox/aidbox-python-sdk/tree/update-python-3-11
Unfortunately, aidbox initializes on each test. It should be fixed. Aidbox should be initialized only once per test suite run.

Add typehints for sdk.operation/sdk.subscription

Currently, type hints are not defined. I suggest adding typehints for them. Since some projects are using python3.9, let's write types using python3.9+

Seed resource with default value

Creating default resource via Manifest overrides resource value even if it was modified after
pervious deploy. It is useful to have an option to seed resource value just once.

Improve operations logic

Let's change the signature of the operation function (#4).

And review endpoints definitions via decorator. The problem is the same as in #22. Decorators make the code confusing.
I suggest following the django way - define endpoints via routing.

@ir4y @mkizesov Any thoughts about it?

Add ability to provide operation's docstring as Aidbox operation description

Check if we can add Operation.description in app manifest.
This feature should be optional (we can set flag for this in ENV file)

DEV: Catch exception in operation handler

Now it's difficult to debug issues if backend fails with exception (e.g. KeyError):

@sdk.operation(..)
def operation(_, request):
    key = request['resource']['userType']
    ...

If we don't pass userType - the code fails, and Aidbox returns 500 without details.

Let's add some useful information for DEV environment (stack trace + error message)

Add an example how to use app client/db outside operation/subscription

Sometimes we need to use client/db outside, e.g. in corn tasks.

We need to add an example to readme how to achieve it.

One of the possible solutions:

1. move aidbox_python_sdk.main.init_client to a separate file, give a proper name
2. move db initialization DBProxy(app["settings"]) into a separate file
3. make them accessible as context managers (async with)

And use in the cron as:

@crontab("* * * * *")
async def import_hospital_data_task_opoo():
    async with init_client(sdk_settings) as client:
        await client.resources('Patient').fetch()

The only thing that needs to be solved, is how to get rid of settings here, maybe always passing SDK object that already contains settings

Add row_to_resource helper

From https://github.com/fhirbase/fhirbase.py/blob/master/fhirbase/__init__.py#L54

Improve subscriptions logic

Let's make subscription callback more high-level:

def patient_send_email_on_create(action, resource):
    pass

where action is current event['action'] and resource is an instance of AidboxResource from event['resource']

And the current logic with the decorator is confusing. Let's initialize manifest somehow with subscription definitions:

subscriptions = {
    'Patient': [patient_send_email_on_create, sub2, ...]
}

sdk = SDK(
    sdk_settings,
    entities=entities,
    resources=meta_resources,
    seeds=seeds,
    migrations=migrations,
    subscriptions=subscriptions,
    on_ready=on_ready)

Using decorator allows having only one subscription and decorators can be used in any file so that makes impossible to understand the whole picture how the app works.

Remove per-request logs

Health checks on productions fill logs and make it total unreadable

Stop after fail connection to AidBox

If an app can't connect to the AidBox, it should fail and stop working.
Now app continue working, so it accepts hooks requests from AidBox and fails in runtime.

Implement healthcheck endpoint

async def healthcheck():
    return web.json_response({'is_ready': sdk.is_ready.result()})

Inconsistent request in operation and subscription handlers

• request arg in operation - it's request from aidbox (dict)
• request arg in subscription - it's an aiohttp request

Get rid of APP_OVERRIDE_AIDBOX_BASE_URL

It's redundant. Use APP_INIT_URL always

Add changelog

$sql endpoint support

• Add https://www.sqlalchemy.org/
• Load information about resources via /Entity?type=resource and bootstrap SQL Alchemy models at runtime
• Provide manifest.alchemy as entrypint for SQL Alchemy based queries
• Provide manifest.raw_sql as entrypint for raw SQL request as strings

create_app should resolve promise after successful config stage

The function create_app should wait for config Aidbox initializing message.

Add support for content type yaml for custom endpoints

Use FHIR subscription instead of app subscriptions

App subscriptions don't guarantee receiving

Aidbox app initialized failed. Response from Aidbox: 500 {"message":"No manifest provided"}

Hello! I am trying to use and SDK but get the following error during app initializaion - Response from Aidbox: 500 {"message":"No manifest provided"}

Here are my findings of possible reasons:

in aidbox_python_sdk/main.py the manifest looks the follwoing:

json = {
            'url': app['settings'].APP_URL,
            'app_id': app['settings'].APP_ID,
            'secret': app['settings'].APP_SECRET,
        }

However Aidbox documentation suggest more diverse object:

resourceType: App
id: myorg.myapp  # id of your application
apiVersion: 1    # App API version
type: app        # type of app - app | addon
endpoint:        # communication protocol between aidbox and app
   type: http-rpc         # type of protocol - http-rpc | ws-rpc
   host: app.mydomain.com # app service host for http protocols 
   scheme: https          # app service schema for http protocols - default = https
   port: 8080             # app service port - default = 80
   secret: <yoursercret>  # will be used to secure aidbox - app communication
entities: <Resource's Definitions, Profiles & Hooks>
operations: <Operation's Definitions>
subscriptions: <Subscriptions>

It this mismatch a source of error?

Can you please facilitate with fixing this. Thanks!

Provide a way to define FHIR operations overriding

Now we have a problem when we want to make additional business logic on some actions, e.g. Patient creation.
We used two methods:

• Subscriptions
• Custom operation (usually /Patient/$create or something similar)

Custom operation is good enough and it should be used for complex logic.
Subscription isn't good for business logic (especially mutating) because of non-atomic nature: we can't unroll resource creation/updating/deletion when something went wrong in the subscription.

Discussed with @ir4y we decided that we need something new - FHIR operation overriding.

Proposal:
POST /Patient

might be overridden with a custom operation where we can call super method.

It means we need something like POST /original/{resource-type} (URL need to be discussed).

Add policy argument to operation decorator

This argument accept JSON AccessPolicy schema and apply it for the operation

Change order of arguments for operation

Now we have operation and request args, but operation isn't useful for endpoints. I think request should be the first argument.

Handle unconnected state

1. When the app tries to connect to aidbox https://github.com/Aidbox/aidbox-python-sdk/blob/with-aidbox-py/aidbox_python_sdk/main.py#L52 it uses the default timeout.
   In a kubernetes cluster, it gets a timeout for the first request and it takes about 5 minutes.
   Let's restore timeout set to 3 seconds and fail if it reached 670361d

2. In the proves of new deploy, It is possible that application gets a request from the AidBox but, it will not be initialized yet. In this case, SDK should respond to AidBox with 503 code, instead of a passing request to the handler.

Add typehints for sdk.operation/sdk.subscription

New types should be added, somethling like:

class SDKOperationRequestApp(TypedDict):
    client: AsyncAidboxClient
    db: DBProxy
    sdk: SDK


class SDKOperationRequest(TypedDict):
    app: SDKOperationRequestApp

And sdk.operation/sdk.subscription should return function with SDKOperationRequest typehint

Usage example:

@sdk.operation(["POST"], ["$myop"])
async def myop_op(_operation, request: SDKOperationRequest):
    client = request["app"]["client"]
    return web.json_response({})

pytest_plugin imports create_app from main

https://github.com/Aidbox/aidbox-python-sdk/blob/master/aidbox_python_sdk/pytest_plugin.py#L11

Was it done explicitly or by mistake?

Before the upgrade, it wasn't a problem to run tests on python 3.9 and old aidbox python SDK