Data missing when submitting POST query about dkan

[dafeder]

@tedmoorman the "index" referred to in the API docs should probably be explained better. We're talking about the index of the distribution array. DKAN can have multiple distributions/resources associated with one dataset record, but on the Open Payments website there is only a single distribution for each dataset. So the "index" on datastore API queries will always be "0".

To paginate through the data use the "limit" and "offset" parameters in the request itself.

You can also use the "download" endpoint with "format" = "csv" to stream the full response as a CSV file. We can't currently stream data in JSON format but may add this capability in the future.

[tedmoorman]

I added '"format": "csv"' to the JSON portion of the request. In the API tool on the page, I see a download link: blob:https://openpaymentsdata.cms.gov/4aaa8ac9-9bcb-4f98-a99d-0e2eb04f9a51 . (See screen shot below).

I don't see this URL in any of the output (in my programming interface). How do I locate that URL? Or, is there a way I need to format the JSON portion of my request to "use the 'download' endpoint"?
Not sure how to do this. Thank you for your help.

[dafeder]

@tedmoorman the documentation on the specific datasets page is kind of minimal, there are additional endpoints described at https://openpaymentsdata.cms.gov/about/api

For instance for your request you could do a simple GET request to

https://openpaymentsdata.cms.gov/api/1/datastore/query/0380bbeb-aea1-58b6-b708-829f92a48202/0/download?format=csv

The POST version of this would be

POST https://openpaymentsdata.cms.gov/api/1/datastore/query/0380bbeb-aea1-58b6-b708-829f92a48202/0/download

{
  "format": "csv"
}

You can then add any filters or other query elements you wish. It's not really necessary to perform an API request if you just want a CSV file of the full dataset without any filters; simply download the original CSV file linked from the dataset.

And of course you can also use the distribution ID instead of the dataset ID without the "/0" on the end, but I would reccomend sticking to the [dataset ID]/[index] endpoint, as distribution IDs can change over time but dataset IDs should not. Again, we should explain this better in the dataset-specific API docs.

[tedmoorman]

@dafeder Well, that's definitely what I was missing!

Any idea how I create an "OR" condition in the JSON part of a POST request?

For example, how might I edit this?

{
  "name": "dan",
  "or",
  "name": "ted"
}

Thank you again!

[dafeder]

See the request schema in any of the datastore POST endpoints on that API docs page. I believe for your example it would look like:

{
   "conditions": [
    {
      "groupOperator": "or",
      "conditions": [
        {
          "property": "name",
          "value": "dan"
        },
        {
          "property": "name",
          "value": "tod"
        }
      ]
    }
  ]
}

for AND the condition group isn't necessary, you can just do:

{
  "conditions": [
    {
      "property": "name",
      "value": "dan"
    },
    {
      "property": "name",
      "value": "tod"
    }
  ]
}

[tedmoorman]

@dafeder thank you so much, Dan!

The example you gave is exactly what I was looking for. Works like a charm!