google / fhir-py Goto Github PK
View Code? Open in Web Editor NEWPython utilities for working with FHIR, including libraries to build simple, flat FHIR views in BigQuery.
License: Apache License 2.0
Python utilities for working with FHIR, including libraries to build simple, flat FHIR views in BigQuery.
License: Apache License 2.0
Would a new release to PyPI be possible?
Hi! Our team is very interested in using this package, but the lack of 4.X protobuf support is blocking us pulling it into our repo (todos in a few setup.py ex).
# TODO(b/276635321): Fix compatibility issue with protobuf 4.x
Curious if there's any chance this will be addressed soon / if the incompatibilities could be elaborated on, we'd be willing to contribute.
Thanks!
Answer field is not captured while creating dataframe from QuestionnaireResponse parquet files using fhir-views.
Code for creating dataframe
import pandas
import json
from sqlalchemy import dialects
from sqlalchemy import engine
from google.fhir.views import r4
from google.fhir.views import spark_runner
fhir_dataset = 'default'
analysis_dataset = 'statin_analysis_example'
dialects.registry.register('hive', 'pyhive.sqlalchemy_hive', 'HiveDialect')
query_engine = engine.create_engine('hive://localhost:10001/default')
runner = spark_runner.SparkRunner(
query_engine=query_engine,
fhir_dataset=fhir_dataset,
view_dataset=analysis_dataset,
snake_case_resource_tables=True,
)
views = r4.base_r4()
qrs = views.view_of('QuestionnaireResponse')
QuestionnaireResponse_df = runner.to_dataframe(
qrs.select(
{
"id": qrs.id,
"tag": qrs.meta.tag.code,
"patient_id":qrs.subject.reference,
"encounter_id": qrs.encounter.reference,
"questionnaire": qrs.questionnaire,
"item":qrs.item
}
)
)
when I do QuestionnaireResponse_df.iloc[0].item
dataframe_json.zip
when I query for the same data using sql
Please refer the attached Json file
qr.zip
Clearly the nested items and answer field are not captured in the dataframe. For example if I take the first object of the dataframe json and sql output
##Dataframe output
[
{
"id": null,
"linkId": "vitals",
"definition": null,
"text": null,
"answer": null,
"item": [
{
"id": null,
"linkId": "3.1",
"definition": null,
"text": "ANC service provided at *"
},
{
"id": null,
"linkId": "service-type-details",
"definition": null,
"text": null
},
{
"id": null,
"linkId": "machine-available",
"definition": null,
"text": "Is BP Machine Available? *"
},
{
"id": null,
"linkId": "3.2",
"definition": null,
"text": null
},
{
"id": null,
"linkId": "3.3",
"definition": null,
"text": null
},
{
"id": null,
"linkId": "3.4",
"definition": null,
"text": null
},
{
"id": null,
"linkId": "3.5",
"definition": null,
"text": null
}
]
##Sql output
[
{
"id": null,
"linkId": "vitals",
"definition": null,
"text": null,
"answer": null,
"item": [
{
"id": null,
"linkId": "3.1",
"definition": null,
"text": "ANC service provided at *",
"answer": [
{
"id": null,
"value": {
"boolean": null,
"decimal": null,
"integer": null,
"date": null,
"dateTime": null,
"time": null,
"string": null,
"uri": null,
"attachment": null,
"coding": {
"system": null,
"version": null,
"code": "home",
"display": "Home",
"userSelected": null
},
"quantity": null,
"reference": null
},
"item": null
}
],
"item": null
},
{
"id": null,
"linkId": "service-type-details",
"definition": null,
"text": null,
"answer": null,
"item": [
{
"id": null,
"linkId": "service-type",
"definition": null,
"text": null,
"answer": [
{
"id": null,
"value": {
"boolean": null,
"decimal": null,
"integer": null,
"date": null,
"dateTime": null,
"time": null,
"string": null,
"uri": null,
"attachment": null,
"coding": {
"system": "https://iprdgroup.com/custom-codes",
"version": null,
"code": "anc-visit",
"display": "ANC Visit",
"userSelected": null
},
"quantity": null,
"reference": null
},
"item": null
}
],
"item": null
}
]
},
{
"id": null,
"linkId": "machine-available",
"definition": null,
"text": "Is BP Machine Available? *",
"answer": [
{
"id": null,
"value": {
"boolean": false,
"decimal": null,
"integer": null,
"date": null,
"dateTime": null,
"time": null,
"string": null,
"uri": null,
"attachment": null,
"coding": null,
"quantity": null,
"reference": null
},
"item": null
}
],
"item": null
},
{
"id": null,
"linkId": "3.2",
"definition": null,
"text": null,
"answer": null,
"item": null
},
{
"id": null,
"linkId": "3.3",
"definition": null,
"text": null,
"answer": null,
"item": null
},
{
"id": null,
"linkId": "3.4",
"definition": null,
"text": null,
"answer": null,
"item": [
{
"id": null,
"linkId": "3.4.1",
"definition": null,
"text": null,
"answer": null,
"item": [
{
"id": null,
"linkId": "weight",
"definition": null,
"text": "Weight (in Kgs) *",
"answer": [
{
"id": null,
"value": {
"boolean": null,
"decimal": 79.0,
"integer": null,
"date": null,
"dateTime": null,
"time": null,
"string": null,
"uri": null,
"attachment": null,
"coding": null,
"quantity": null,
"reference": null
},
"item": null
}
],
"item": null
},
{
"id": null,
"linkId": "3.4.1.2",
"definition": null,
"text": null,
"answer": [
{
"id": null,
"value": {
"boolean": null,
"decimal": null,
"integer": null,
"date": null,
"dateTime": null,
"time": null,
"string": "Kgs",
"uri": null,
"attachment": null,
"coding": null,
"quantity": null,
"reference": null
},
"item": null
}
],
"item": null
}
]
},
{
"id": null,
"linkId": "3.4.2",
"definition": null,
"text": null,
"answer": null,
"item": [
{
"id": null,
"linkId": "3.4.2.1",
"definition": null,
"text": null,
"answer": [
{
"id": null,
"value": {
"boolean": null,
"decimal": null,
"integer": null,
"date": null,
"dateTime": null,
"time": null,
"string": null,
"uri": null,
"attachment": null,
"coding": {
"system": "http://fhir.org/guides/who/anc-cds/CodeSystem/anc-custom-codes",
"version": null,
"code": "ANC.B8.DE3",
"display": "Current weight",
"userSelected": null
},
"quantity": null,
"reference": null
},
"item": null
}
],
"item": null
},
{
"id": null,
"linkId": "3.4.2.2",
"definition": null,
"text": null,
"answer": [
{
"id": null,
"value": {
"boolean": null,
"decimal": null,
"integer": null,
"date": null,
"dateTime": null,
"time": null,
"string": null,
"uri": null,
"attachment": null,
"coding": {
"system": "http://hl7.org/fhir/observation-status",
"version": null,
"code": "preliminary",
"display": "Preliminary",
"userSelected": null
},
"quantity": null,
"reference": null
},
"item": null
}
],
"item": null
},
{
"id": null,
"linkId": "3.4.2.3",
"definition": null,
"text": null,
"answer": [
{
"id": null,
"value": {
"boolean": null,
"decimal": null,
"integer": null,
"date": null,
"dateTime": null,
"time": null,
"string": null,
"uri": null,
"attachment": null,
"coding": {
"system": "http://hl7.org/fhir/ValueSet/observation-category",
"version": null,
"code": "vital-signs",
"display": "Vital Signs",
"userSelected": null
},
"quantity": null,
"reference": null
},
"item": null
}
],
"item": null
}
]
}
]
},
{
"id": null,
"linkId": "3.5",
"definition": null,
"text": null,
"answer": null,
"item": [
{
"id": null,
"linkId": "3.5.1",
"definition": null,
"text": null,
"answer": null,
"item": [
{
"id": null,
"linkId": "3.5.1.1",
"definition": null,
"text": "Height (in cms) *",
"answer": [
{
"id": null,
"value": {
"boolean": null,
"decimal": 100.0,
"integer": null,
"date": null,
"dateTime": null,
"time": null,
"string": null,
"uri": null,
"attachment": null,
"coding": null,
"quantity": null,
"reference": null
},
"item": null
}
],
"item": null
},
{
"id": null,
"linkId": "3.5.1.2",
"definition": null,
"text": null,
"answer": [
{
"id": null,
"value": {
"boolean": null,
"decimal": null,
"integer": null,
"date": null,
"dateTime": null,
"time": null,
"string": "cms",
"uri": null,
"attachment": null,
"coding": null,
"quantity": null,
"reference": null
},
"item": null
}
],
"item": null
}
]
},
{
"id": null,
"linkId": "3.5.2",
"definition": null,
"text": null,
"answer": null,
"item": [
{
"id": null,
"linkId": "3.5.2.1",
"definition": null,
"text": null,
"answer": [
{
"id": null,
"value": {
"boolean": null,
"decimal": null,
"integer": null,
"date": null,
"dateTime": null,
"time": null,
"string": null,
"uri": null,
"attachment": null,
"coding": {
"system": "http://fhir.org/guides/who/anc-cds/CodeSystem/anc-custom-codes",
"version": null,
"code": "ANC.B8.DE1",
"display": "Height",
"userSelected": null
},
"quantity": null,
"reference": null
},
"item": null
}
],
"item": null
},
{
"id": null,
"linkId": "3.5.2.2",
"definition": null,
"text": null,
"answer": [
{
"id": null,
"value": {
"boolean": null,
"decimal": null,
"integer": null,
"date": null,
"dateTime": null,
"time": null,
"string": null,
"uri": null,
"attachment": null,
"coding": {
"system": "http://hl7.org/fhir/observation-status",
"version": null,
"code": "registered",
"display": "Registered",
"userSelected": null
},
"quantity": null,
"reference": null
},
"item": null
}
],
"item": null
},
{
"id": null,
"linkId": "3.5.2.3",
"definition": null,
"text": null,
"answer": [
{
"id": null,
"value": {
"boolean": null,
"decimal": null,
"integer": null,
"date": null,
"dateTime": null,
"time": null,
"string": null,
"uri": null,
"attachment": null,
"coding": {
"system": "http://hl7.org/fhir/ValueSet/observation-category",
"version": null,
"code": "vital-signs",
"display": "Vital Signs",
"userSelected": null
},
"quantity": null,
"reference": null
},
"item": null
}
],
"item": null
}
]
}
]
}
]
I'm working with a BigQuery FHIR dataset from a Google FHIR store export (analytics_v2 schema). I need to extract the partOf.id of an Organization, but the FHIR Views library's FHIRPath builder doesn't seem to account for the analytics_v2 schema's structure where partOf.id is partOf.organizationId.
Here is the code,
org = views.view_of('Organization')
child_orgs = org.select({
'id': org.id,
'name': org.name,
# partOf.id throws: BadRequest: 400 Field name id does not exist in STRUCT>, text STRING>, system STRING, ...>, reference STRING, ...> at [1:89]
'org_id': org.partOf.id,
# partOf.organizationId throws: AttributeError: ('No such field organizationId in partOf', "Expected something in ['display', 'extension', 'id', 'identifier', 'reference', 'type']")
'org_id2': org.partOf.organizationId,
})
Any guidance on how to handle this would be greatly appreciated. Thanks!
I am using https://colab.sandbox.google.com/ with these lines:
!git clone https://github.com/google/fhir-py.git
!cd fhir-py; ./install_all.sh
This is my output:
Processing ./google-fhir-core
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
Preparing metadata (setup.py) ... error
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Processing ./google-fhir-r4
error: subprocess-exited-with-error
× python setup.py egg_info did not run successfully.
│ exit code: 1
╰─> See above for output.
note: This error originates from a subprocess, and is likely not a problem with pip.
Preparing metadata (setup.py) ... error
error: metadata-generation-failed
× Encountered error while generating package metadata.
╰─> See above for output.
note: This is an issue with the package mentioned above, not pip.
hint: See above for details.
Looking in indexes: https://pypi.org/simple, https://us-python.pkg.dev/colab-wheels/public/simple/
Processing ./google-fhir-views
Preparing metadata (setup.py) ... done
Collecting absl-py~=0.10.0
Using cached absl_py-0.10.0-py3-none-any.whl (127 kB)
ERROR: Could not find a version that satisfies the requirement google-fhir-core~=0.8 (from google-fhir-views[bigquery,r4]) (from versions: none)
ERROR: No matching distribution found for google-fhir-core~=0.8
Hi. Thanks for the great work!
I was just wondering if support for the latest FHIR R5 specification is in the books or not.
Hello, I am using FHIR view and its bigquery runner to query fhir data from a bigquery table on GCP. Wondering how we can work with extensions in this case?
Specifically, the bq table is exported from a fhir-store, and bq automatically strips away extensions (it maps each extension onto its own column). For example the following extension in a resource will actually create a column named retired
in the bq table for the resource, and the bq table will NOT have a column with the name extension.
"extension": [
{
"url": "https://my-definition/StructureDefinition/retired",
"valueString": "N"
},
]
For context, due to our usecase, we are using the SQL on FHIR to specify the query, i.e
simple_pats_config = {
'resource': 'Patient',
'select': [
{
'alias': 'extensions',
'path': 'extension',
},
...
]
}
The problem with this is that the SQL generated will attempt to UNNEST(extension)
, which bigquery will error google.api_core.exceptions.BadRequest: 400 Unrecognized name: extension
since such column does not exist in the bq table. So the question is how do we work with the bq table in this case? Or alternatively what settings/config do we need when exporting from fhir-store to bq to preserve the extension
column so this works?
And a separate but related question is whether there will be support for extension.url
matching. Currently, if the fhir-path used is extension.where(url = 'some-url')
, views.from_view_definition(query)
will throw an error (ValueError: Identifier url not in dict_keys(['id', 'extension', 'value'])
). This is perhaps because (if i am correct) url is currently not part of child_defs
of extension
in FhirPathDataType
. Maybe I am mistaken and if this is somehow supported it would be great!
Thanks in advance.
Hi, we have a multi language project where we define APIs that depends on the fhir protos in google-fhir. So far we've been able to use the google-fhir python package for the generated protos.
Now it seems like the google-fhir python package is effectively deprecated and the fhir protos are provided in here (google-fhir-r4, google-fhir-core). However, it also seems like the fhir protos are copied in this repo with a different path, resulting a diffrent import path. And we have a problem when try to deprecate the use of google-fhir
python package in favor of google-fhir-r4
, I'll describe the problem below.
In our API proto definition, we have this (we clone the google-fhir repo's protos).
import "proto/google/fhir/proto/r4/core/resources/patient.proto";
Then, the generated python protobuf files will have
from proto.google.fhir.proto.r4.core.resource import patient_pb2
which works with the deprecated google-fhir python package.
However, if we want to use the google-fhir-r4/core
package, it is using this import path (due to it's own folder structure for the protos)
from google.fhir.r4.proto.core.resources import patient_pb2
Do you have any suggestions on how we can migrate to google-fhir-r4? We can stay with google-fhir
package for now but we also want to use the google-fhir-views package for our next project.
A declarative, efficient, and flexible JavaScript library for building user interfaces.
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
An Open Source Machine Learning Framework for Everyone
The Web framework for perfectionists with deadlines.
A PHP framework for web artisans
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
Some thing interesting about web. New door for the world.
A server is a program made to process requests and deliver data to clients.
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
Some thing interesting about visualization, use data art
Some thing interesting about game, make everyone happy.
We are working to build community through open source technology. NB: members must have two-factor auth.
Open source projects and samples from Microsoft.
Google ❤️ Open Source for everyone.
Alibaba Open Source for everyone
Data-Driven Documents codes.
China tencent open source team.