hydrotools._restclient._restclient module

class hydrotools._restclient._restclient.RestClient(*, base_url: str | Url | None = None, headers: dict = {}, enable_cache: bool = True, cache_filename: str = 'cache', cache_expire_after: int = 43200, retry: bool = True, n_retries: int = 3, loop: AbstractEventLoop | None = None)

Bases: object

Class that simplifies writing RESTful client libraries and retrieval scripts. Behind the scenes requests are made asynchronously, however the API is exposed using serial methods to simplify usage.

Features

  • Base url

  • SQLite request cache

  • Retry exponential backoff

  • Serial wrapper methods that make requests asynchronously (see RestClient.mget)

Parameters:

  • base_url (Union[str, Url, None], default None) – Request base url

  • headers (dict, default {}) – Headers included in every request

  • enable_cache (bool, default True) – Enable or disable caching

  • cache_filename (str, default "cache") – Cache filename with .sqlite filetype suffix

  • cache_expire_after (int, default 43200) – Cached request life in seconds

  • retry (bool, default True) – Enable exponential backoff

  • n_retries (int, default 3) – Attempt retries n times before failing

  • loop (asyncio.AbstractEventLoop, default None) – Async event loop

Examples

Simple Request >>> from hydrotools._restclient import RestClient >>> >>> client = RestClient() >>> resp = client.get(“weather.gov”) >>> print(resp.text())

USGS NLDI Requests >>> from hydrotools._restclient import RestClient >>> >>> base_url = “https://labs.waterdata.usgs.gov/api/nldi/linked-data/nwissite/” >>> headers = {“Accept”: “/”, “Accept-Encoding”: “gzip, compress”} >>> client = RestClient(base_url=base_url, headers=headers) >>> >>> sites = [“0423360405”, “05106000”,”05078520”,”05078470”,”05125039”,”05124982”] >>> site_urls = [f”/USGS-{site}/navigation/UT/flowlines” for site in sites] >>> requests = client.mget(site_urls) >>> requests = [resp.json() for resp in requests]

async _get(url: str = None, *, allow_redirects: bool = True, parameters: Dict[str, str | List[str]] = {}, headers: Dict[str, str] = {}, **kwargs: Any) → ClientResponse
async _mget(urls: str | URL = None, *, allow_redirects: bool = True, parameters: List[Dict[str, str | List[str]]] = {}, headers: List[Dict[str, str]] = {}, **kwargs: Any) → List[ClientResponse]
_patch_get(client_response: ClientResponse) → ClientResponse

Wrap aiohttp.ClientResponse text and json coros in run_until_complete. Monkeypatch text and json with wrappers.

property base_url: str

Base url

build_url(url: str | None = None, parameters: Dict[str, PRIMITIVE | List[PRIMITIVE]] = {})
close() → None

Release aiohttp.ClientSession

get(url: str | URL = None, *, allow_redirects: bool = True, parameters: Dict[str, str | List[str]] = {}, headers: Dict[str, str] = {}, **kwargs: Any) → ClientResponse

Make GET request. If base url is set, url is appended to the base url. Passed headers are given precedent over instance headers(if present), meaning passed headers replace instance headers with matching keys.

Parameters:

  • url (str, Url) – Request url

  • parameters (Dict[str, Union[str, List[str, int, float]]]) – Query parameters

  • headers (Dict[str, str]) – Request headers, if RestClient headers set provided headers are appended

Return type:

aiohttp.ClientResponse

property headers: dict

GET request headers

mget(urls: str | URL = None, *, allow_redirects: bool = True, parameters: List[Dict[str, str | List[str]]] = {}, headers: List[Dict[str, str]] = {}, **kwargs: Any) → List[ClientResponse]

Make multiple asynchronous GET requests. If base url is set, each url is appended to the base url. Passed headers are given precedent over instance headers(if present), meaning passed headers replace instance headers with matching keys.

Parameters:

  • urls (List[Union[str, Url]]) – Request urls

  • parameters (Dict[str, Union[str, List[str, int, float]]]) – Query parameters

  • headers (Dict[str, str]) – Request headers, if RestClient headers set provided headers are appended

Return type:

List[aiohttp.ClientResponse]