metaodi / tecdottir

API for weather stations of the Wasserschutzpolizei Zurich

tecdottir

This is a project to get measurements from tecson meteo stations.

The City of Zurich has two such stations:

• Tiefenbrunnen
• Mythenquai

The data is available as CSV from Open Data Zurich.

tecdottir is an API based on this data.

Showcases

Projects based on this API are shown.

• Zürichhorn: map of Zürichhorn with temperature and wind by Markus Steiger

(if you want on the list, create an issue or PR to add your project)

Development

We use swagger to document the API. Please install the CLI tool to edit the swagger file:

npm install -g swagger

(you might need root privileges to do so)

After installing the CLI tool, you can start editing:

swagger project edit

To start the server use:

swagger project start

Tests

To run the tests, use the following command:

npm test

Local database

For local developlment you might want to use a local database in a docker container

docker run --rm -P -p 127.0.0.1:5432:5432 -e POSTGRES_PASSWORD="1234" --name pg postgres:alpine

Or use the provided script:

start_local_database.sh # start in it's own shell

Then you can set DATABASE_URL in .env to postgresql://postgres:1234@localhost:5432/postgres

Once the db is there, import the data:

setup_local_database.sh

Create a new release

1. Update the version number in package.json and api/swagger/swagger.yaml
2. Update the CHANGELOG.md
3. Push code to GitHub
4. Create a new release on GitHub
5. Wait for the GitHub Action to deploy the app to heroku

tecdottir's Issues

water temperature for tiefenbrunnen is broken

it returns

"Wassertemp. (DEFEKT)": {
          "value": "14.8",
          "unit": "°C"
        },

no actual data

actual data is missing. last timestamp is 2022-11-17T16:30:00.000Z

'wind_force_avg_10min' reports value of 'wind_speed_avg_10min'

Just noticed that the value of 'wind_force_avg_10min' reports the value of 'wind_speed_avg_10min'. Apparently the transformation from m/s to bft is not done

Example:
https://tecdottir.herokuapp.com/measurements/mythenquai?startDate=2023-08-07 reports

Large requests (e.g. over one year) cause memory exhaustion on heroku

If a large request is made, a lot of memory is consumed and therefore the memory quote of heroku is exhausted. This leads to a 500 error code from heroku and a failed request.

Maybe it's possible to cache the data or split those requests up to not hit the memory limit.

Enable TLS 1.3

I'm trying to use the API from a system that doesn't support TLS 1.2. However, at the moment the API only supports TLS 1.2. Heroku says it supports TLS 1.2 and 1.3, so maybe there's a button somewhere that you could click to enable 1.3? Thanks!

API request error: Unauthorized

Recently, the API query no longer works. The following error message appears:

{"ok":false,"message":"Tecson returned an error: Error: Unauthorized"}

The following query was performed:
https://tecdottir.herokuapp.com/measurements/tiefenbrunnen?startDate=2022-01-06&endDate=2022-01-06

Do I have to log in or submit a token now?

It also no longer works on the API reference page:
https://tecdottir.herokuapp.com/docs/

Use CSV from Open Data Zurich for historical data

Migrate to new CSV API of Tecson

https://www.tecson-data.ch/zurich/mythenquai/download/
Username: wetter
PW: mythenquai

https://www.tecson-data.ch/zurich/tiefenbrunnen/download/
Username: wetter
PW: tiefenbrunnen

Add limit and offset parameter for pagination

Add default values to prevent memory problems (e.g. 1000 records)

Migrate to express-openapi

https://gist.github.com/camsjams/6c6ac145324786006972bedbf25ce1c4

Check if air quality could be integrated

There are (at least) 4 stations, that meassure the air quality, see https://data.stadt-zuerich.ch/dataset/luftqualitaet-historisierte-messungen

Either import the historical data or check with OSTLUFT if the API could be used (?)

Data Refresh

Hello, first of all, thanks for your work!
Secondly, I wanted to ask if it is possible to increase the number of updates during the day.
I'm trying to create an "Alexa" skill to retrieve the temperature but as far as I can see the data is rather old.
For example, I'm performing a query at 16:09 and the result is "value": "2023-02-02T13:30:00+01:00".
Let me already say thank you if you would have the chance to work on this suggestion!

Recent data is empty for the current month

Hey I have a question

I was running this command successfully since a very long time. But since approx a year, the current month is always empty:

curl -X 'GET' 'https://tecdottir.herokuapp.com/measurements/tiefenbrunnen?startDate=2023-03-13&endDate=2023-03-14&sort=timestamp_cet%20desc&offset=0' -H "accept: application/json"

{"ok":true,"total_count":0,"row_count":0,"result":[]}%

Any idea what I'm doing wrong?

allow to query both stations at once

maybe use the endpoint without a st.tion for this.

New Value "Windböen"

Its possible to add the new provided value "Windböen (max) 10 Min" ?

Add sort parameter (asc/desc)

Default query with no startDate is empty

Since a while it seems that the default queries return empty results. Only by specifying a startDate that is in the past (multiple days) the request returns data.
My integration is broken for a while but did not find the time to check.
Is there a change needed in order to get the current values?

empty results:

curl -X 'GET' \
  'https://tecdottir.herokuapp.com/measurements/tiefenbrunnen?sort=timestamp_cet%20desc&limit=500&offset=0' \
  -H 'accept: application/json'

compared to:

curl -X 'GET' \
  'https://tecdottir.herokuapp.com/measurements/tiefenbrunnen?startDate=2022-08-05&sort=timestamp_cet%20desc&limit=500&offset=0' \
  -H 'accept: application/json'

Use morph.io scraper

Consider to use morph io to scrape the data, to keep the data and tackle the caching problem (see #2)

Add test for new time period limit

I added a new limit on the time period (in days) you're allowed to query at once. There should be a test for this new behaviour.

No new data since 2022-08-05

Hi,
Unfortunately, the latest data in the csv from stadt-zuerich (and therefore also in your API) is from 2022-08-05T19:30:00+00:00 and no newer data is available. The https://www.tecson-data.ch/zurich/tiefenbrunnen/index.php is still up to date. I'm not sure if this is the same issue as #32 if so, sorry for creating a new one.
I'm guessing the .csv are no longer automatically updated which is no fault of the tecdottir API here, but it would mean getting the current data would require to now access the password protected part of the tecson-data page :(

Hacked keyname

The hardware to measuring the water temperature is at the moment defect. The dataowner solved this with a special(!) "keyname". Is there a simple solution to recognize this "hacked" keyname and give a valid response?

Only load new data

Keep the historical data in the database and only update the current year.

Provide CSV and or Parquet as file format

Add automatic check if data is missing

the results of the check could be made availablejas API endpoint.

Use parametrized queries

https://node-postgres.com/features/queries#parameterized-query