GithubHelp home page GithubHelp logo

fl550 / simple_dwd_weatherforecast Goto Github PK

View Code? Open in Web Editor NEW
27.0 4.0 26.0 1.32 MB

This is a python package for simple access to hourly forecast data for the next 10 days and reported weather where this data is provided.

License: MIT License

Python 100.00%
dwd weather-forecast forecast-data deutscherwetterdienst deutscher-wetterdienst forecast weather-data

simple_dwd_weatherforecast's Introduction

Simple DWD weather forecast

BuyMeCoffee

DISCLAIMER: This project is a private open source project and doesn't have any connection with Deutscher Wetterdienst.

Weather data

This is a python package for simple access to hourly forecast data for the next 10 days. The data is updated every six hours and updated when needed. Some stations also have actual reported weather, which you can also retrieve.

Available station-IDs can be found here or you can use the method dwdforecast.get_nearest_station_id(latitude, longitude) which tries to find it for you.

Forecasted weather conditions are evaluated using this table and then converted into these possible weather conditions:

  • cloudy
  • fog
  • lightning-rainy
  • partlycloudy
  • pouring
  • rainy
  • snowy
  • snowy-rainy
  • sunny

The reported weather is delayed (roughly one hour), so have a close look at the time within the presented data.

The weather report for the region which is available on the DWD homepage (see an example here) can also be retrieved via a method which maps the station to the relevant region.

Weather maps

You can also retrieve weather maps from the DWD GeoServer with this package.

Installation

python3 -m pip install simple_dwd_weatherforecast

Usage

Weather data

Usage example

from simple_dwd_weatherforecast import dwdforecast
from datetime import datetime, timedelta, timezone

#Find nearest Station-ID automatically
#id = dwdforecast.get_nearest_station_id(50.1109221, 8.6821267)

dwd_weather = dwdforecast.Weather("10385") # Station-ID For BERLIN-SCHOENEFELD
time_now = datetime.now(timezone.utc)
temperature_now = dwd_weather.get_forecast_data(dwdforecast.WeatherDataType.TEMPERATURE, time_now)
time_tomorrow = datetime.now(timezone.utc)+timedelta(days=1)
temperature_tomorrow = dwd_weather.get_forecast_data(dwdforecast.WeatherDataType.TEMPERATURE, time_tomorrow)

Available methods

All methods return their values as string. The datetime value has to be in UTC. If no data is available for this datetime, None will be returned. With the optional boolean shouldUpdate an automated check for new updates can be prevented by setting this parameter to False. Otherwise data is updated if new data is available with every function call.

There is also data available which is updated every hour by DWD. Be careful though, as this will download each time roughly 37MB of data. Furthermore, some elements are missing from this data:

  • PRECIPITATION_PROBABILITY
  • PRECIPITATION_DURATION

You can use this data by using the optional parameter force_hourly=True.

is_valid_station_id(station_id) #Checks if given station_id is valid

get_nearest_station_id(latitude, longitude) #Returns nearest Station-ID for the coordinates. latitude and longitude expect float values.

update(optional bool force_hourly) #Force fetch of new data from the DWD server. With this parameter set to True, there will be missing the precipitation forecast. See above.

class WeatherDataType(Enum):
    CONDITION = "condition"
    TEMPERATURE = "TTT" # Unit: K
    DEWPOINT = "Td" # Unit: K
    PRESSURE = "PPPP" # Unit: Pa
    WIND_SPEED = "FF" # Unit: m/s
    WIND_DIRECTION = "DD" # Unit: Degrees
    WIND_GUSTS = "FX1" # Unit: m/s
    PRECIPITATION = "RR1c" # Unit: kg/m2
    PRECIPITATION_PROBABILITY = "wwP" # Unit: % (0..100)
    PRECIPITATION_DURATION = "DRR1" # Unit: s
    CLOUD_COVERAGE = "N" # Unit: % (0..100)
    VISIBILITY = "VV" # Unit: m
    SUN_DURATION = "SunD1" # Unit: s
    SUN_IRRADIANCE = "Rad1h" # Unit: W/m2
    FOG_PROBABILITY = "wwM" # Unit: % (0..100)
    HUMIDITY = "humidity"  # Unit: %

class Weather:

    get_station_name(optional bool shouldUpdate) # Return Station name

    get_forecast_data(weatherDataType: see WeatherDataType, datetime, optional bool shouldUpdate) # Returns the requested weather data

    get_get_reported_weather(weatherDataType: see WeatherDataType, optional bool shouldUpdate) # Returns the latest actual reported value if available for this station

    get_daily_max(weatherDataType: see WeatherDataType, datetime, optional bool shouldUpdate) # Returns the maximum daily value

    get_timeframe_max(weatherDataType: see WeatherDataType, datetime, timeframe: hours after datetime as int, optional bool shouldUpdate) # Returns the maximum of that value within the time frame

    get_daily_min(weatherDataType: see WeatherDataType, datetime, optional bool shouldUpdate) # Returns the minimum daily value

    get_timeframe_min(weatherDataType: see WeatherDataType, datetime, timeframe: hours after datetime as int, optional bool shouldUpdate) # Returns the minimum of that value within the time frame

    get_daily_sum(weatherDataType: see WeatherDataType, datetime, optional bool shouldUpdate) # Returns the daily sum of that value

    get_timeframe_sum(weatherDataType: see WeatherDataType, datetime, timeframe: hours after datetime as int, optional bool shouldUpdate) # Returns the sum of that value within the time frame

    get_timeframe_avg(weatherDataType: see WeatherDataType, datetime, timeframe: hours after datetime as int, optional bool shouldUpdate) # Returns the average of that value within the time frame

    get_forecast_condition(datetime, optional bool shouldUpdate) # Result is condition as text

    get_daily_condition(datetime, optional bool shouldUpdate) # Result is an approximate "feeled" condition at this day

    get_timeframe_condition(datetime, timeframe: hours after datetime as int, optional bool shouldUpdate) # Result is an approximate "feeled" condition at this time frame

    get_weather_report(optional bool shouldUpdate) # Returns the weather report for the geographical region of the station as HTML

    get_uv_index(int day_from_today (values: 0-2)) # Returns the UV index for the nearest station available for today, tomorrow or the day after tomorrow

    update(self, optional bool force_hourly (default: False), optional bool with_forecast (default: True), optional bool with_measurements (default: False), optional bool with_report (default: False), optional bool with_uv (default: True)) # Updates the weather data

Advanced Usage

If you want to access the forecast data for the next 10 days directly for further processing, you can do so. All data is stored in dictonary and can be accessed like this:

from simple_dwd_weatherforecast import dwdforecast

dwd_weather​ ​=​ ​dwdforecast​.​Weather​(​"10385"​) # Station-ID For BERLIN-SCHOENEFELD​
dwd_weather.update() # This has to be done manually to fetch the data from the DWD server
access_forecast_dict = dwd_weather.forecast_data # dwd_weather.forecast_data contains the forecast as a dict

Keep in mind that the weather condition is stored as the original digit value as provided by DWD. So if you want to use them, you have to convert these yourself. You can use my simplified conversion from the source code in the variable weather_codes or the original conversion available here.

Weather maps

You can download weather maps from the DWD GeoServer with this package. There are different options for the displayed foreground and background data. See below for further information.

example picture of a map produced with this package

Usage example

from simple_dwd_weatherforecast import dwdmap

image = dwdmap.get_from_location(51.272, 8.84, radius_km=100, map_type=dwdmap.WeatherMapType.NIEDERSCHLAGSRADAR, background_type=dwdmap.WeatherBackgroundMapType.BUNDESLAENDER)

image.save("niederschlag.png")

image = dwdmap.get_germany(map_type=dwdmap.WeatherMapType.UVINDEX, image_width=520, image_height=580)

image.save("uvindex.png")

Available methods

class WeatherMapType(Enum):
    NIEDERSCHLAGSRADAR = "dwd:Niederschlagsradar"
    MAXTEMP = "dwd:GefuehlteTempMax"
    UVINDEX = "dwd:UVI_CS"
    POLLENFLUG = "dwd:Pollenflug"
    SATELLITE_RGB = "dwd:Satellite_meteosat_1km_euat_rgb_day_hrv_and_night_ir108_3h"
    SATELLITE_IR = "dwd:Satellite_worldmosaic_3km_world_ir108_3h"
    WARNUNGEN_GEMEINDEN = "dwd:Warnungen_Gemeinden"
    WARNUNGEN_KREISE = "dwd:Warnungen_Landkreise"

class WeatherBackgroundMapType(Enum):
    LAENDER = "dwd:Laender"
    BUNDESLAENDER = "dwd:Warngebiete_Bundeslaender"
    KREISE = "dwd:Warngebiete_Kreise"
    GEMEINDEN = "dwd:Warngebiete_Gemeinden"
    SATELLIT = "dwd:bluemarble"

def get_from_location(longitude, latitude, radius_km, map_type: WeatherMapType, background_type: WeatherBackgroundMapType, optional integer image_width, optional integer image_height) #Returns map as pillow image with given radius from coordinates

get_germany(map_type: WeatherMapType, optional WeatherBackgroundMapType background_type, optional integer image_width, optional integer image_height, optional string save_to_filename) #Returns map as pillow image of whole germany

get_map(minx,miny,maxx,maxy, map_type: WeatherMapType, background_type: WeatherBackgroundMapType, optional integer image_width, optional integer image_height, optional string save_to_filename) #Returns map as pillow image

Help and Contribution

Feel free to open an issue if you find one and I will do my best to help you. If you want to contribute, your help is appreciated! If you want to add a new feature, add a pull request first so we can chat about the details.

Licenses

This package uses public data from DWD OpenData. The Copyright can be viewed here.

simple_dwd_weatherforecast's People

Contributors

fl0x53 avatar fl550 avatar jonastaedcke avatar nikipore avatar spigandromeda avatar

Stargazers

 avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar  avatar

Watchers

 avatar  avatar  avatar  avatar

simple_dwd_weatherforecast's Issues

Wrong geographic coordinates in stations.json

Due to an incorrect representation of the geographic coordinates in the source file for the MOSMIX stations #1 their position is incorrect.
On #1 the position is not in decimal format, but in degree and minutes separated by a dot.

Example

ID    ICAO NAME                 LAT    LON     ELEV
10763 EDDN NUERNBERG             49.30   11.03   318

whereas #2:

Stations-Kennziffer | Stations_ID | ICAO-Kennung | Stationsname | Stationshöhe in Metern | geogr. Breite | geogr. Länge
10763 | 3668 | EDDN | Nürnberg | 314 | 49° 30' | 11° 03'

And #3 states the exact decimal coordinates as 49.503, 11.0549

So for the time being, I would suggest parsing the incorrect values. Dividing the places after the decimal point by 60 to convert from arcminute to decimal. With the values from the example resulting in 49.5, 11,05. This should be sufficient to find the next station.

I only became aware of the error because the Home Assistan Integration suggested stations that are really not that close by! 😉

Latitude and Longitude from DWD are not decimal values

I was wondering why the home assistant integration was returning a station which is far away from the lat/long from my home.
I have then checked the values that DWD provides in their station list.
If they write 50.25 they don’t mean the decimal value 50.25, but 50 degrees, 25 minutes.
I‘m on my phone right now and can’t type the degree symbol, but you can check this in google maps when you type 50*25'N 7*13'E in the search bar (replace * with the degree symbol).

Inquiry Regarding Sun Irradiance Values Returned by API

I hope this message finds you well. I am reaching out to seek further clarification regarding the sun irradiance values I have received from your API. I have been analyzing the data for different German cities, and the irradiance values I retrieved are considerably higher than I initially anticipated.

While I am not an expert in meteorology, I had understood that the maximum possible amount of solar irradiance reaching the Earth's atmosphere is approximately 1365 W/m². Particularly in Germany, I assumed this value to be somewhat lower. However, upon examining the hourly dataframes for cities like Germany and Potsdam, I noticed that the values exceed 2000 W/m².

I must admit that I am a bit perplexed by these readings and would greatly appreciate your guidance in understanding this matter better. I would like to comprehend the specifics of the "sun_irradiance" parameter better, as it appears that my current understanding might be incomplete or incorrect. Your insights into the nature of these irradiance values and the factors that could contribute to such measurements would be invaluable.

I want to express my gratitude for the functionality of your API, which has been performing remarkably well for my purposes. I eagerly await your response and am thankful for any assistance you can provide in shedding light on this situation.

Thank you for your time and support.

Best regards
Max

Deprecated forecast attribute

Right after installing i get the following message:

The dwd_weather integration is using deprecated forecast
This stops working in version 2024.3.0. Please address before upgrading.
The integration dwd_weather is using the deprecated forecast attribute.

Please report it to the custom integration author..

I am currently on Home Assistant version 2023.9.2

KML parsing error

Unfortuneatly, I found another problem and I am not sure how to fix it (I am actually a PHP developer).

The parsing of the KML file fails with an an "list index out of range" error.

File "/usr/local/lib/python3.9/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 469, in update
    self.parse_kml(kml)
File "/usr/local/lib/python3.9/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 496, in parse_kml
    result = tree.xpath("//dwd:IssueTime", namespaces=self.namespaces)[0].text
IndexError: list index out of range

The used station was 10677. I checked the file manually and it looks exactly like the others:
MOSMIX_L_2022050815_10677.kml (renamed the file to .txt to allow upload on GitHub)

I guess the error is about this XML line:

<dwd:IssueTime>2022-05-08T15:00:00.000Z</dwd:IssueTime>

I also checked lxml and there were no recent updates that could explain this error.

These are my HA infos:

Version | core-2022.5.2
Installation Type | Home Assistant OS
Development | false
Supervisor | true
Docker | true
User | root
Virtual Environment | false
Python Version | 3.9.9
Operating System Family | Linux
Operating System Version | 5.10.103-v8
CPU Architecture | aarch64
Timezone | Europe/Berlin

Home Assistant Community Store:

GitHub API | ok
GitHub Content | ok
GitHub Web | ok
GitHub API Calls Remaining | 4855
Installed Version | 1.24.5
Stage | running
Available Repositories | 1115
Downloaded Repositories | 17

best regards,
spigandromeda

P.S. the error occurred only after I installed the HTTP patch (download results with HTTP and HTTPS are the same).

Data are downloaded on every get request

I get the Impression, that the kml file ist fatched on every get query (I am fetching one hour after another in a outer loop and it takes about 2s per hour). From reading the code (I am Python beginner) I understood, that new data shall be fatched only if the kml issued time is older than 6 hours. Sometimes running the same code goes very fast. So I assume that there ist an issue with the issue time supplied by the dwd. Is there any solution to it?

If trying to get for a station multiple times crashes

If I call this
temperature = dwd_weather.get_forecast_data(dwdforecast.WeatherDataType.TEMPERATURE, time)
multiple times for the same dwd_weather (the same station) it crashes (want to use different times):
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 239, in get_forecast_data
if self.is_in_timerange(timestamp):
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/Library/Frameworks/Python.framework/Versions/3.11/lib/python3.11/site-packages/simple_dwd_weatherforecast/dwdforecast.py", line 219, in is_in_timerange
list(self.forecast_data.keys())[0]
^^^^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'keys'

Time shift at day change

Hello, I noticed a reproducable one day shift of th e data at 0:00 o'clock. The shift exists also at 0:30 but disapears at 1:00 o'clock. I am using the dwdforecast.Weather method. Basically at 0:00 o'clock the data contains only 9 days.

Regards

Meik

ICON-D2 for more precise 48h forecasts

I was wondering if it would be possible to add support for ICON-D2.

It seems to provide significantly more accurate data (thanks to higher resolutions it seems) than the "normal" ICON model.

Since it's in grib2 format, it would probably require a different method of parsing.

DWD Documentation of ICON-D2 (Section 6.3. Near surface products)
DWD OpenData Folder for ICON-D2 (the subfolders are different generation times if I understand correctly)

Also this data would need to be merged with the normal ICON (or ICON-EU, see below) model for >45h forecast.

I would be happy to help, but I thought getting an overview first (and your opinion on this) would probably help.

If you want to see it in action, windy has this model included.

EDIT: After some further reading, the normal station data seems to depend on the ICON 13km resolution. Seeing this, it would probably also help to switch to ICON-EU for the 45h+ forecast.

Resolutions
ICON: 13km
ICON-EU: 6.5km
ICON-D2: 2.2km

Motivation: ICON data differs significantly where I live (in the 5° range temperature wise, precipitation is even worse), so getting more accurate forecast (into hass) is always helpful.

Issue with predefined station list (stations.json)

Hi,
I just came around to an issue with the station.json file when updating the HCAS dwd weather add-on to v2.0.

First, I just noticed that the somehow migrated station was displaying "wrong" data. I re-added, in my case the station of Nürnberg Airport, and noticed that the station IDs differ.

Second, in home assistant when you configure the add-on you will get a list of weather stations close by. There I noticed a vast of stations that are historic: The stations were once operated by the DWD, but have since been decommissioned (e.g. Station Tennenlohe ID 11659 out of service since 24.04.1978 1).

Now comes the fun part: DWD somehow re-uses those IDs 🤦‍♂️. So the ID from my example (11659) now maps to Přibyslav in Czech Republic 2. Hence, you select a station nearby and get data from else where 😄

So I guess the station file needs a little rework.

The order of the requested layers does matter

The order of the map layers is currently wrong

url = f"https://maps.dwd.de/geoserver/dwd/wms?service=WMS&version=1.1.0&request=GetMap&layers={background_type.value},{map_type.value}&bbox={minx},{miny},{maxx},{maxy}&width={image_width}&height={image_height}&srs=EPSG:4326&styles=&format=image/png"

With the url as per the line 43, above:
https://maps.dwd.de/geoserver/dwd/wms?service=WMS&version=1.1.0&request=GetMap&layers=dwd:Laender,dwd:Satellite_meteosat_1km_euat_rgb_day_hrv_and_night_ir108_3h&bbox=4.4,46.4,16.1,55.6&width=720&height=803&srs=EPSG:4326&styles=&format=image/png

we get this image
image

Reversing the two layers to have it this way
https://maps.dwd.de/geoserver/dwd/wms?service=WMS&version=1.1.0&request=GetMap&layers=dwd:Satellite_meteosat_1km_euat_rgb_day_hrv_and_night_ir108_3h,dwd:Laender&bbox=4.4,46.4,16.1,55.6&width=720&height=803&srs=EPSG:4326&styles=&format=image/png

we get this image
image

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.