Skip to content

Add disk chunk fallback for datasets containing non-cftime object variables#780

Open
charles-turner-1 wants to merge 2 commits intomainfrom
779-fallback-disk-chunking
Open

Add disk chunk fallback for datasets containing non-cftime object variables#780
charles-turner-1 wants to merge 2 commits intomainfrom
779-fallback-disk-chunking

Conversation

@charles-turner-1
Copy link
Copy Markdown
Collaborator

@charles-turner-1 charles-turner-1 commented Mar 17, 2026 •
edited
Loading

Change Summary

  • If we get a 'Can not use auto rechunking with object dtype.' error, fall back to using disk chunks (strategy pre. Change chunking default to auto for compatible xarray versions #737).
  • Remove drop variables in test_open_dataset_kerchunk to explicitly test this.
  • Pin pandas < 3.0 (unrelated but necessary as the COW stuff breaks search, etc.)

Related issue number

#779

Checklist

  • Unit tests for the changes exist
  • Tests pass on CI
  • Documentation reflects the changes where applicable

@charles-turner-1
- Add fallback to {} chunking strategy if dataset contains non-cftime…
ab8f91c 
… object variables

- Update test to ensure we fall back to working chunking strategy in
  case above
- Change pandas requirements - pin to less than 3.0.0
@charles-turner-1 charles-turner-1 linked an issue Mar 17, 2026 that may be closed by this pull request
Suggestion the default of chunks kwarg from 'auto' to {} or None #779
Open
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@mgrover1 mgrover1 Awaiting requested review from mgrover1 mgrover1 is a code owner

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

Suggestion the default of chunks kwarg from 'auto' to {} or None

1 participant

@charles-turner-1