SNOW-3243341: Python e2e tests for pyarrow and pandas support

[sfc-gh-asolarski]

TL;DR

Refactored Python CI workflow to support separate testing of pandas and non-pandas functionality, added pandas/pyarrow optional dependencies, and created dedicated pandas test modules.

What changed?

• CI Workflow Changes: Replaced YAML anchors with explicit matrix configurations that include extras dimension (no-extras, pandas) to run tests with and without pandas dependencies
• Dependency Management: Added pandas and pyarrow optional dependencies to pyproject.toml with version constraints based on Python version
• Test Environment Setup: Created separate Hatch environments (test and test-pandas) with different dependency sets and test scopes
• Test Organization: Added new tests/e2e/pandas/ directory with pandas-specific tests for Arrow and pandas fetch methods
• Cache Strategy: Updated UV cache keys to include the extras dimension for proper dependency isolation
• Reference Testing: Extended reference testing to support both pandas and non-pandas configurations

How to test?

1. Run the standard test suite: hatch run test.py3.12:cov
2. Run pandas-specific tests: hatch run test-pandas.py3.12:cov
3. Verify CI matrix runs both configurations by checking GitHub Actions workflow
4. Test optional dependency installation: pip install -e .[pandas] or pip install -e .[pyarrow]

Why make this change?

This change enables proper testing of pandas/pyarrow functionality while maintaining compatibility for users who don't need these heavy dependencies. It separates concerns by isolating pandas-specific tests and ensures the CI pipeline validates both usage patterns, preventing regressions in either scenario.

[sfc-gh-asolarski]

• SNOW-3243341: python wrapper - support pandas and pyarrow fetching #653
• SNOW-3243341: optional dependencies in cursor (pyarrow, pandas) #639
• SNOW-3243341: e2e tests for pyarrow & pandas type conversion #629
• SNOW-3243341: Python e2e tests for pyarrow and pandas support #625 👈 (View in Graphite)
• main

---

How to use the Graphite Merge Queue

Add either label to this PR to merge it via the merge queue:

• merge_queue - adds this PR to the back of the merge queue
• priority_merge_queue - for urgent changes, fast-track this PR to the front of the merge queue

You must have a Graphite account in order to use the merge queue. Sign up using this link.

An organization admin has enabled the Graphite Merge Queue in this repository.

Please do not merge from GitHub as this will restart CI on PRs being processed by the merge queue.

This stack of pull requests is managed by Graphite. Learn more about stacking.

[sfc-gh-pczajka]

Do we need as many jobs? I'd keep "pandas/no-extras" for 3.9 and 3.13, then juto do alternating "pandas/no-extras" for the rest

[sfc-gh-asolarski]

The thing is "test-pandas" is a very small suit just for testing integration with pandas (other e2e test are not executed when using this env, and not unit tests are executed).

So, we run whole test suite on different envs and additionally for 3 envs (py3.9 - smallest, py3.13 - reference, py3.14 - highest) we verify just pandas integration (rest of the tests is not re-executed, so those 3 jobs are rather tiny).

[sfc-gh-pczajka]

What is the value of no-pandas reference test? How about just doing the pandas one?

[sfc-gh-asolarski]

As explained above, pandas test is just about this single feature, the rest goes through the usual flow.

So, here we also need two flows for separate coverage reports.