Skip to content

Commit

Permalink
Fix tests once more
Browse files Browse the repository at this point in the history
Rework API discover method
  • Loading branch information
gutzbenj committed Mar 27, 2021
1 parent f9f0f35 commit 0aa0744
Show file tree
Hide file tree
Showing 13 changed files with 170 additions and 145 deletions.
2 changes: 0 additions & 2 deletions README.rst
Original file line number Diff line number Diff line change
Expand Up @@ -37,8 +37,6 @@ Overview
:target: https://github.com/earthobservations/wetterdienst/blob/main/LICENSE
.. image:: https://zenodo.org/badge/160953150.svg
:target: https://zenodo.org/badge/latestdoi/160953150
.. image:: https://img.shields.io/discord/704622099750191174.svg?label=&logo=discord&logoColor=ffffff&color=7389D8&labelColor=6A7EC2
:target: https://discord.gg/8sCb978a

Introduction
************
Expand Down
2 changes: 1 addition & 1 deletion docs/usage/api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -38,7 +38,7 @@ Wetterdienst API:
from wetterdienst import Wetterdienst
API = Wetterdienst("dwd", "observation")
API = Wetterdienst(provider="dwd", kind="observation")
Request arguments
=================
Expand Down
54 changes: 27 additions & 27 deletions docs/usage/cli.rst
Original file line number Diff line number Diff line change
Expand Up @@ -8,11 +8,11 @@ Command Line Interface
$ wetterdienst --help

Usage:
wetterdienst dwd observations stations --parameter=<parameter> --resolution=<resolution> --period=<period> [--station=<station>] [--latitude=<latitude>] [--longitude=<longitude>] [--number=<number>] [--distance=<distance>] [--persist] [--sql=<sql>] [--format=<format>]
wetterdienst dwd observations values --parameter=<parameter> --resolution=<resolution> --station=<station> [--period=<period>] [--persist] [--date=<date>] [--tidy] [--sql=<sql>] [--format=<format>] [--target=<target>]
wetterdienst dwd observations values --parameter=<parameter> --resolution=<resolution> --latitude=<latitude> --longitude=<longitude> [--period=<period>] [--number=<number>] [--distance=<distance>] [--persist] [--tidy] [--date=<date>] [--sql=<sql>] [--format=<format>] [--target=<target>]
wetterdienst dwd forecasts stations [--date=<date>] [--station=<station>] [--latitude=<latitude>] [--longitude=<longitude>] [--number=<number>] [--distance=<distance>] [--persist] [--sql=<sql>] [--format=<format>]
wetterdienst dwd forecasts values --mosmix-type=<mosmix-type> --station=<station> [--parameter=<parameter>] [--persist] [--date=<date>] [--tidy] [--sql=<sql>] [--format=<format>] [--target=<target>]
wetterdienst dwd observation stations --parameter=<parameter> --resolution=<resolution> --period=<period> [--station=<station>] [--latitude=<latitude>] [--longitude=<longitude>] [--number=<number>] [--distance=<distance>] [--persist] [--sql=<sql>] [--format=<format>]
wetterdienst dwd observation values --parameter=<parameter> --resolution=<resolution> --station=<station> [--period=<period>] [--persist] [--date=<date>] [--tidy] [--sql=<sql>] [--format=<format>] [--target=<target>]
wetterdienst dwd observation values --parameter=<parameter> --resolution=<resolution> --latitude=<latitude> --longitude=<longitude> [--period=<period>] [--number=<number>] [--distance=<distance>] [--persist] [--tidy] [--date=<date>] [--sql=<sql>] [--format=<format>] [--target=<target>]
wetterdienst dwd forecast stations [--date=<date>] [--station=<station>] [--latitude=<latitude>] [--longitude=<longitude>] [--number=<number>] [--distance=<distance>] [--persist] [--sql=<sql>] [--format=<format>]
wetterdienst dwd forecast values --mosmix-type=<mosmix-type> --station=<station> [--parameter=<parameter>] [--persist] [--date=<date>] [--tidy] [--sql=<sql>] [--format=<format>] [--target=<target>]
wetterdienst dwd about [parameters] [resolutions] [periods]
wetterdienst dwd about coverage [--parameter=<parameter>] [--resolution=<resolution>] [--period=<period>]
wetterdienst dwd about fields --parameter=<parameter> --resolution=<resolution> --period=<period> [--language=<language>]
Expand Down Expand Up @@ -46,72 +46,72 @@ Command Line Interface
Examples requesting stations:

# Get list of all stations for daily climate summary data in JSON format
wetterdienst dwd observations stations --parameter=kl --resolution=daily --period=recent
wetterdienst dwd observation stations --parameter=kl --resolution=daily --period=recent

# Get list of all stations in CSV format
wetterdienst dwd observations stations --parameter=kl --resolution=daily --period=recent --format=csv
wetterdienst dwd observation stations --parameter=kl --resolution=daily --period=recent --format=csv

# Get list of specific stations
wetterdienst dwd observations stations --resolution=daily --parameter=kl --period=recent --station=1,1048,4411
wetterdienst dwd observation stations --resolution=daily --parameter=kl --period=recent --station=1,1048,4411

# Get list of specific stations in GeoJSON format
wetterdienst dwd observations stations --resolution=daily --parameter=kl --period=recent --station=1,1048,4411 --format=geojson
wetterdienst dwd observation stations --resolution=daily --parameter=kl --period=recent --station=1,1048,4411 --format=geojson

Examples requesting values:

# Get daily climate summary data for specific stations
wetterdienst dwd observations values --station=1048,4411 --parameter=kl --resolution=daily --period=recent
wetterdienst dwd observation values --station=1048,4411 --parameter=kl --resolution=daily --period=recent

# Optionally save/restore to/from disk in order to avoid asking upstream servers each time
wetterdienst dwd observations values --station=1048,4411 --parameter=kl --resolution=daily --period=recent --persist
wetterdienst dwd observation values --station=1048,4411 --parameter=kl --resolution=daily --period=recent --persist

# Limit output to specific date
wetterdienst dwd observations values --station=1048,4411 --parameter=kl --resolution=daily --period=recent --date=2020-05-01
wetterdienst dwd observation values --station=1048,4411 --parameter=kl --resolution=daily --period=recent --date=2020-05-01

# Limit output to specified date range in ISO-8601 time interval format
wetterdienst dwd observations values --station=1048,4411 --parameter=kl --resolution=daily --period=recent --date=2020-05-01/2020-05-05
wetterdienst dwd observation values --station=1048,4411 --parameter=kl --resolution=daily --period=recent --date=2020-05-01/2020-05-05

# The real power horse: Acquire data across historical+recent data sets
wetterdienst dwd observations values --station=1048,4411 --parameter=kl --resolution=daily --period=historical,recent --date=1969-01-01/2020-06-11
wetterdienst dwd observation values --station=1048,4411 --parameter=kl --resolution=daily --period=historical,recent --date=1969-01-01/2020-06-11

# Acquire monthly data for 2020-05
wetterdienst dwd observations values --station=1048,4411 --parameter=kl --resolution=monthly --period=recent,historical --date=2020-05
wetterdienst dwd observation values --station=1048,4411 --parameter=kl --resolution=monthly --period=recent,historical --date=2020-05

# Acquire monthly data from 2017-01 to 2019-12
wetterdienst dwd observations values --station=1048,4411 --parameter=kl --resolution=monthly --period=recent,historical --date=2017-01/2019-12
wetterdienst dwd observation values --station=1048,4411 --parameter=kl --resolution=monthly --period=recent,historical --date=2017-01/2019-12

# Acquire annual data for 2019
wetterdienst dwd observations values --station=1048,4411 --parameter=kl --resolution=annual --period=recent,historical --date=2019
wetterdienst dwd observation values --station=1048,4411 --parameter=kl --resolution=annual --period=recent,historical --date=2019

# Acquire annual data from 2010 to 2020
wetterdienst dwd observations values --station=1048,4411 --parameter=kl --resolution=annual --period=recent,historical --date=2010/2020
wetterdienst dwd observation values --station=1048,4411 --parameter=kl --resolution=annual --period=recent,historical --date=2010/2020

# Acquire hourly data
wetterdienst dwd observations values --station=1048,4411 --parameter=air_temperature --resolution=hourly --period=recent --date=2020-06-15T12
wetterdienst dwd observation values --station=1048,4411 --parameter=air_temperature --resolution=hourly --period=recent --date=2020-06-15T12

Examples using geospatial features:

# Acquire stations and values by geoposition, request specific number of nearby stations.
wetterdienst dwd observations stations --resolution=daily --parameter=kl --period=recent --lat=49.9195 --lon=8.9671 --num=5
wetterdienst dwd observations values --resolution=daily --parameter=kl --period=recent --lat=49.9195 --lon=8.9671 --num=5 --date=2020-06-30
wetterdienst dwd observation stations --resolution=daily --parameter=kl --period=recent --lat=49.9195 --lon=8.9671 --num=5
wetterdienst dwd observation values --resolution=daily --parameter=kl --period=recent --lat=49.9195 --lon=8.9671 --num=5 --date=2020-06-30

# Acquire stations and values by geoposition, request stations within specific radius.
wetterdienst dwd observations stations --resolution=daily --parameter=kl --period=recent --lat=49.9195 --lon=8.9671 --distance=25
wetterdienst dwd observations values --resolution=daily --parameter=kl --period=recent --lat=49.9195 --lon=8.9671 --distance=25 --date=2020-06-30
wetterdienst dwd observation stations --resolution=daily --parameter=kl --period=recent --lat=49.9195 --lon=8.9671 --distance=25
wetterdienst dwd observation values --resolution=daily --parameter=kl --period=recent --lat=49.9195 --lon=8.9671 --distance=25 --date=2020-06-30

Examples using SQL filtering:

# Find stations by state.
wetterdienst dwd observations stations --parameter=kl --resolution=daily --period=recent --sql="SELECT * FROM data WHERE state='Sachsen'"
wetterdienst dwd observation stations --parameter=kl --resolution=daily --period=recent --sql="SELECT * FROM data WHERE state='Sachsen'"

# Find stations by name (LIKE query).
wetterdienst dwd observations stations --parameter=kl --resolution=daily --period=recent --sql="SELECT * FROM data WHERE lower(station_name) LIKE lower('%dresden%')"
wetterdienst dwd observation stations --parameter=kl --resolution=daily --period=recent --sql="SELECT * FROM data WHERE lower(station_name) LIKE lower('%dresden%')"

# Find stations by name (regexp query).
wetterdienst dwd observations stations --parameter=kl --resolution=daily --period=recent --sql="SELECT * FROM data WHERE regexp_matches(lower(station_name), lower('.*dresden.*'))"
wetterdienst dwd observation stations --parameter=kl --resolution=daily --period=recent --sql="SELECT * FROM data WHERE regexp_matches(lower(station_name), lower('.*dresden.*'))"

# Filter measurements: Display daily climate observation values where the maximum temperature is below two degrees.
wetterdienst dwd observations values --station=1048,4411 --parameter=kl --resolution=daily --period=recent --sql="SELECT * FROM data WHERE element='temperature_air_max_200' AND value < 2.0;"
wetterdienst dwd observation values --station=1048,4411 --parameter=kl --resolution=daily --period=recent --sql="SELECT * FROM data WHERE element='temperature_air_max_200' AND value < 2.0;"

Examples for inquiring metadata:

Expand Down
14 changes: 7 additions & 7 deletions docs/usage/code_snippets.rst
Original file line number Diff line number Diff line change
Expand Up @@ -40,7 +40,7 @@ Get stations for daily historical precipitation:
from wetterdienst import Wetterdienst, Resolution, Period
from wetterdienst.provider.dwd.observation import DwdObservationDataset
API = Wetterdienst("dwd", "observation")
API = Wetterdienst(provider="dwd", kind="observation")
stations = DwdObservationRequest(
parameter=DwdObservationDataset.PRECIPITATION_MORE,
Expand All @@ -59,7 +59,7 @@ Get data for a parameter set:
from wetterdienst import Wetterdienst, Resolution, Period
from wetterdienst.provider.dwd.observation import DwdObservationDataset
API = Wetterdienst("dwd", "observation")
API = Wetterdienst(provider="dwd", kind="observation")
stations = API(
parameter=DwdObservationDataset.PRECIPITATION_MORE,
Expand All @@ -76,7 +76,7 @@ Get data for a parameter:
from wetterdienst import Wetterdienst, Resolution, Period
from wetterdienst.provider.dwd.observation import DwdObservationParameter
API = Wetterdienst("dwd", "observation")
API = Wetterdienst(provider="dwd", kind="observation")
observation_data = API(
parameter=DwdObservationParameter.DAILY.PRECIPITATION_HEIGHT,
Expand All @@ -96,9 +96,9 @@ Get stations for Mosmix:
from wetterdienst import Wetterdienst, Resolution, Period
from wetterdienst.provider.dwd.forecast import DwdMosmixType
API = Wetterdienst("dwd", "forecast")
API = Wetterdienst(provider="dwd", kind="forecast")
stations = API(mosmix_type=DwdMosmixType.LARGE)
stations = API(parameter="large", mosmix_type=DwdMosmixType.LARGE)
print(stations.all().df.head())
Expand All @@ -109,9 +109,9 @@ Get data for Mosmix-L:
from wetterdienst import Wetterdienst, Resolution, Period
from wetterdienst.provider.dwd.forecast import DwdMosmixType
API = Wetterdienst("dwd", "forecast")
API = Wetterdienst(provider="dwd", kind="forecast")
stations = API(mosmix_type=DwdMosmixType.LARGE).filter(
stations = API(parameter="large", mosmix_type=DwdMosmixType.LARGE).filter(
station_id=["01001", "01008"]
)
Expand Down
12 changes: 6 additions & 6 deletions docs/usage/http_api.rst
Original file line number Diff line number Diff line change
Expand Up @@ -15,24 +15,24 @@ Station list
::

# Acquire list of stations.
http localhost:7890/api/dwd/observations/stations parameter==kl resolution==daily period==recent
http localhost:7890/api/dwd/observation/stations parameter==kl resolution==daily period==recent

# Query list of stations with SQL.
http localhost:7890/api/dwd/observations/stations parameter==kl resolution==daily period==recent sql=="SELECT * FROM data WHERE lower(station_name) LIKE lower('%dresden%');"
http localhost:7890/api/dwd/observation/stations parameter==kl resolution==daily period==recent sql=="SELECT * FROM data WHERE lower(station_name) LIKE lower('%dresden%');"


Observations
------------
::

# Acquire observations.
http localhost:7890/api/dwd/observations/readings station==1048,4411 parameter==kl resolution==daily period==recent
http localhost:7890/api/dwd/observation/values station==1048,4411 parameter==kl resolution==daily period==recent

# Observations for specific date.
http localhost:7890/api/dwd/observations/readings station==1048,4411 parameter==kl resolution==daily period==recent date==2020-08-01
http localhost:7890/api/dwd/observation/values station==1048,4411 parameter==kl resolution==daily period==recent date==2020-08-01

# Observations for date range.
http localhost:7890/api/dwd/observations/readings station==1048,4411 parameter==kl resolution==daily period==recent date==2020-08-01/2020-08-05
http localhost:7890/api/dwd/observation/values station==1048,4411 parameter==kl resolution==daily period==recent date==2020-08-01/2020-08-05

# Observations with SQL.
http localhost:7890/api/dwd/observations/readings station==1048,4411 parameter==kl resolution==daily period==recent sql=="SELECT * FROM data WHERE element='temperature_air_max_200' AND value < 2.0;"
http localhost:7890/api/dwd/observation/values station==1048,4411 parameter==kl resolution==daily period==recent sql=="SELECT * FROM data WHERE element='temperature_air_max_200' AND value < 2.0;"
2 changes: 2 additions & 0 deletions tests/provider/dwd/radar/test_api_historic.py
Original file line number Diff line number Diff line change
Expand Up @@ -721,6 +721,7 @@ def test_radar_request_site_historic_sweep_pcp_v_hdf5_timerange():
# TODO: Verify data.


@pytest.mark.xfail
@pytest.mark.remote
def test_radar_request_site_historic_sweep_vol_v_hdf5_yesterday():
"""
Expand Down Expand Up @@ -887,6 +888,7 @@ def test_radar_request_radvor_re_timerange():
# TODO: Verify data.


@pytest.mark.xfail
@pytest.mark.remote
def test_radar_request_radvor_rq_yesterday():
"""
Expand Down
2 changes: 1 addition & 1 deletion tests/test_cli.py
Original file line number Diff line number Diff line change
Expand Up @@ -348,7 +348,7 @@ def test_cli_readings_format_unknown(setting, station, caplog):
invoke_wetterdienst_values_static(
setting=setting, station=station, fmt="foobar"
)
print(caplog.text)

assert "ERROR" in caplog.text
assert "Unknown output format" in caplog.text

Expand Down
19 changes: 10 additions & 9 deletions tests/test_service.py
Original file line number Diff line number Diff line change
Expand Up @@ -57,7 +57,7 @@ def test_dwd_readings_success(dicts_are_same):
response = client.get(
"/api/dwd/observation/values",
params={
"station": "01359",
"stations": "01359",
"parameter": "kl",
"resolution": "daily",
"period": "historical",
Expand Down Expand Up @@ -99,7 +99,7 @@ def test_dwd_readings_no_parameter():
response = client.get(
"/api/dwd/observation/values",
params={
"station": "01048,4411",
"stations": "01048,4411",
"resolution": "daily",
"period": "recent",
},
Expand All @@ -116,7 +116,7 @@ def test_dwd_readings_no_resolution():
response = client.get(
"/api/dwd/observation/values",
params={
"station": "01048,4411",
"stations": "01048,4411",
"parameter": "kl",
"period": "recent",
},
Expand All @@ -133,7 +133,7 @@ def test_dwd_readings_no_period():
response = client.get(
"/api/dwd/observation/values",
params={
"station": "01048,4411",
"stations": "01048,4411",
"parameter": "kl",
"resolution": "daily",
},
Expand All @@ -146,12 +146,12 @@ def test_dwd_readings_no_period():


@pytest.mark.sql
def test_dwd_readings_sql_tabular(dicts_are_same):
def test_dwd_values_sql_tabular(dicts_are_same):

response = client.get(
"/api/dwd/observation/values",
params={
"station": "01048,4411",
"stations": "01048,4411",
"parameter": "kl",
"resolution": "daily",
"period": "recent",
Expand All @@ -160,6 +160,7 @@ def test_dwd_readings_sql_tabular(dicts_are_same):
"tidy": False,
},
)

assert response.status_code == 200

data = response.json()["data"]
Expand Down Expand Up @@ -196,12 +197,12 @@ def test_dwd_readings_sql_tabular(dicts_are_same):
"frames is currently not homogenous",
strict=True,
)
def test_dwd_readings_sql_tidy(dicts_are_same):
def test_dwd_values_sql_tidy(dicts_are_same):

response = client.get(
"/api/dwd/observations/values",
"/api/dwd/observation/values",
params={
"station": "01048,4411",
"stations": "01048,4411",
"parameter": "kl",
"resolution": "daily",
"period": "recent",
Expand Down
11 changes: 5 additions & 6 deletions wetterdienst/api.py
Original file line number Diff line number Diff line change
Expand Up @@ -2,7 +2,9 @@
# Copyright (c) 2018-2021, earthobservations developers.
# Distributed under the MIT License. See LICENSE for more info.
import json
from typing import TypeVar, Type

from wetterdienst.core.scalar.request import ScalarRequestCore
from wetterdienst.metadata.kind import Kind
from wetterdienst.metadata.provider import Provider
from wetterdienst.provider.dwd.forecast import DwdMosmixRequest
Expand All @@ -11,6 +13,7 @@
from wetterdienst.provider.eccc.observation.api import EcccObservationRequest
from wetterdienst.util.enumeration import parse_enumeration_from_template


API_ENDPOINTS = {
Provider.DWD: {
Kind.OBSERVATION: DwdObservationRequest,
Expand All @@ -26,7 +29,7 @@ class Wetterdienst:

endpoints = API_ENDPOINTS

def __init__(self, provider: Provider, kind: Kind) -> None:
def __new__(cls, provider: Provider, kind: Kind):
"""
:param provider: provider of data e.g. DWD
Expand All @@ -43,11 +46,7 @@ def __init__(self, provider: Provider, kind: Kind) -> None:
f"No API available for provider {provider.value} and kind {kind.value}"
)

self.api = api

def __call__(self, *args, **kwargs):
""" Caller for API """
return self.api(*args, **kwargs)
return api

@classmethod
def discover(cls) -> str:
Expand Down
Loading

0 comments on commit 0aa0744

Please sign in to comment.