v0.3.1

[svdC1]

Added

• 10 new supported plugins — google/youtube, chatgpt/chat, shein, trip/search, trip/detail, google/play-store, google/play-store/product, google/play-store/reviews, google/shopping/product, google/shopping/product/stores. Each ships a typed *Parameters model and a matching *AsyncPlugin adapter. Sync params for YouTube, the Play Store family, and the Shopping Product family live under scrape_do.plugins.google; ChatGPT gets its own scrape_do.plugins.chatgpt sub-package. Shein and Trip.com are async-only (no per-endpoint sync docs page exists) so their params models live alongside the adapters in scrape_do.async_api.models.plugins.additional. All new *AsyncPlugin adapters participate in the AsyncPlugin discriminated union.

• Async adapters for promoted endpoints — GoogleTrendingAsyncPlugin and GoogleHotelsDetailAsyncPlugin join the discriminated union now that Scrape.do exposes these endpoints through the Async API.

• GoogleSearchAiOverviewAsyncParameters + GoogleSearchAiOverviewAsyncPlugin — async-side q-driver shape for the google/search/ai-overview plugin. The plugin handles both fetch hops internally over the Async API. The existing session-key-based GoogleSearchAiOverviewParameters continues to be the sync follow-up form for the SERP state: "deferred" flow, and both shapes now coexist. The new adapter participates in the AsyncPlugin discriminated union.

• New public literal type aliases under scrape_do.plugins.google — GoogleTrendsRegionType (regional-interest resolution for Trends), and GoogleTrendingHoursType / GoogleTrendingSortType / GoogleTrendingStatusType for the now-typed Trending model. GoogleTrendsDataType gains "GEO_MAP" alongside the existing "GEO_MAP_0". Plus new Play Store / Shopping Product literal enums: GooglePlayStoreChartType, GooglePlayStoreDeviceType, GooglePlayStoreAgeType, GoogleShoppingProductSortByType, GoogleShoppingProductDeviceType. TripCabinClassType is published under the async-only additional module.

Changed

• GoogleSearchAiModeParameters is now a strict subset of SERP. The model no longer carries start, cr, lr, time_period, filter, nfpr, or num; AI Mode is documented as a standalone endpoint whose engine rejects those fields with 400. Breaking change for callers that constructed the model with any of them.

• GoogleSearchParameters.num removed — the current per-endpoint SERP docs no longer list it. Breaking change for callers that relied on it being a typed attribute.

• GoogleTrendsParameters gains tz (timezone offset minutes from UTC, default 420 server-side) and region (geographic resolution for GEO_MAP / GEO_MAP_0 widgets).

• GoogleTrendingParameters rewritten with a typed schema — geo is now required and the previously-permissive extra="allow" shell is replaced with explicit hl / hours / cat / sort / status fields backed by literal enums. The "sync-only" warning is dropped — the endpoint is now part of the Async API plugin table.

• GoogleHotelsDetailParameters sync-only marker dropped — Scrape.do promoted the endpoint to the Async API. Field shape is unchanged.

• WalmartStoreParameters / LowesStoreParameters now enforce the documented schema from Scrape.do's async-api/plugins page instead of accepting arbitrary extras via extra="allow". Walmart requires url (walmart.com domain) and treats zipcode + storeid as a conditional pair (both or neither). Lowes requires url (lowes.com domain) plus digit-only zipcode and storeid. Both pick up the gateway-side disableretry / transparentresponse / timeout knobs. Breaking change for callers passing undocumented extras through the previous schema-free passthrough.

Internal

• Integration suite standardized around three test categories — content-dependent tests retry on transient Scrape.do gateway failures, shape-dependent tests assert only that the request wasn't rejected (HTTP 400), and error-routing tests are unchanged.

• Re-introduced google/trends and lowes/store into the live plugin sweep now that the pass criterion tolerates upstream / engine-side transient failures.

• Plugin integration tests extended with the new endpoints — google/youtube, chatgpt/chat, shein, trip/search, google/play-store, google/shopping/product, google/trending, plus the promoted google/hotels/detail adapter all participate in the parametrized sweep. The shared case list was moved to tests/integration/async_api/conftest.py::_plugin_cases so both test_client.py and test_async_client.py consume the same definitions instead of duplicating them.

• New unit-test coverage for every new model — happy-path construction + cross-field rule tests for GoogleYouTubeParameters, ChatGPTChatParameters, SheinParameters, TripSearchParameters, TripDetailParameters, the Play Store family, and the Shopping Product family. The shared AsyncPlugin discriminated-union test parametrizes all 13 new adapter keys, and test_google.py / test_chatgpt.py cover each new *AsyncPlugin adapter's default key literal + min_length=1 enforcement.

• The GoogleSearchAiOverviewAsyncPlugin integration test is omitted for now because the gateway hasn't updated to reflect the documentation changes about the google/search/ai-overview async endpoint, so requests still require the sync-only session_key parameter.

v0.3.0...v0.3.1

---

This discussion was created from the release v0.3.1.