feat(waterdata): add get_waterdata for generalized CQL2 queries

[thodson-usgs]

Summary

Adds get_waterdata(service, cql, ...) — Python analogue of R dataRetrieval::read_waterdata. The typed wrappers (get_daily, get_continuous, get_peaks, …) only support exact-equality predicates on whitelisted parameters. Some users need more expressive queries:

• top-level or instead of just and
• like with % wildcards (e.g. HUC prefix match)
• comparison operators (<, >, between)
• nested boolean trees
• geometry predicates beyond a bbox

This function gives them a single entry point that accepts a raw CQL2 query (either a Python dict or a pre-serialized JSON string), POSTs it against any recognized OGC collection, walks pages, and runs the same post-processing pipeline (_deal_with_empty → _type_cols → _arrange_cols → _sort_rows) the typed wrappers use.

CQL2 grammar reference: https://api.waterdata.usgs.gov/docs/ogcapi/complex-queries/

Examples

from dataretrieval import waterdata

# 1. Daily values for two parameter codes at two sites — compound AND-of-INs.
cql = {
    "op": "and",
    "args": [
        {"op": "in", "args": [
            {"property": "parameter_code"},
            ["00060", "00065"],
        ]},
        {"op": "in", "args": [
            {"property": "monitoring_location_id"},
            ["USGS-07367300", "USGS-03277200"],
        ]},
    ],
}
df, md = waterdata.get_waterdata(service="daily", cql=cql)

# 2. Monitoring locations whose HUC starts with "02070010" — LIKE with %.
df, md = waterdata.get_waterdata(
    service="monitoring-locations",
    cql='{"op": "like", "args": ['
        '{"property": "hydrologic_unit_code"}, "02070010%"]}',
)

Both examples mirror the R reference.

API

def get_waterdata(
    service: str,
    cql: str | dict,
    *,
    properties: str | Iterable[str] | None = None,
    bbox: list[float] | None = None,
    limit: int | None = None,
    skip_geometry: bool | None = None,
    convert_type: bool = True,
    client: requests.Session | None = None,
) -> tuple[pd.DataFrame, BaseMetadata]:

• service validated against the recognized OGC collections (daily, continuous, latest-*, peaks, field-measurements*, channel-measurements, monitoring-locations, time-series-metadata, combined-metadata).
• cql accepts dict (JSON-serialized internally) or str (passed through verbatim).
• properties honors the same "id" → output_id rewrite the typed wrappers do.
• client lets callers reuse an HTTP session.

What's reused vs. new

Reused unchanged: _walk_pages, _deal_with_empty, _arrange_cols, _type_cols, _sort_rows, _switch_properties_id, _default_headers, BaseMetadata. The whole post-processing pipeline drops in.

New:

• _OUTPUT_ID_BY_SERVICE (utils.py): single mapping from service to the renamed-id column. Hoisted from the typed wrappers so the generalized entry point picks the right one.
• _construct_cql_request (utils.py): focused POST/CQL2 request builder. Kept separate from _construct_api_requests because that function derives the CQL body from typed kwargs; here the body comes in verbatim.
• get_waterdata (api.py): the public entry point.

Smoke test

>>> df, md = get_waterdata(service="daily",
...     cql='{"op":"in","args":[{"property":"monitoring_location_id"},["USGS-02238500"]]}',
...     limit=5)
>>> md.url
'https://api.waterdata.usgs.gov/ogcapi/v0/collections/daily/items?skipGeometry=False&limit=5'
>>> df.shape
(140, 12)

The POST goes out, the OGC server filters by the CQL body, pagination handles the multi-page response, and post-processing renames id → daily_id, types columns, and sorts rows. (140 rows came back from a single site with a small limit=5 page size — pagination cycled through ~28 pages.)

Out of scope

• Unit tests for the new function (would mirror existing tests/waterdata_test.py patterns; can follow up).
• The hash-ID drop default behavior in feat(waterdata): drop hash-valued ID columns by default #281: that PR's include_hash_ids parameter would apply uniformly to get_waterdata once it lands (no extra plumbing needed).

Test plan

• Module import + signature
• _construct_cql_request builds the right URL, headers, and body offline
• Live smoke test against the USGS OGC API with a simple CQL in predicate
• Live test with the wildcard like example (rate-limited at time of submission)
• Unit tests via requests_mock

🤖 Generated with Claude Code

[thodson-usgs]

do we need to shield this against string comparisons as we do in filters.py?