fix(catalog): refresh schema after table replacement - #242
Conversation
Preserve Lance's zero-row overwrite commit semantics for schema-only CREATE OR REPLACE TABLE operations. After a successful overwrite, invalidate only the replaced directory dataset cache entry and evict its stale attached-catalog entry so lazy discovery reloads the committed schema. Leave cached handles and catalog state unchanged when the writer fails, and preserve cross-query reuse for untouched datasets. Add regression coverage for schema refresh, failed replacements, precise cache invalidation, and untouched cache reuse.
![lance-gatekeeper[bot]](https://avatars.githubusercontent.com/in/4433141?s=60&v=4){kind=small}
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
lance-gatekeeper
Bot
added
the
K-changes
Use the physical identifier returned by case-insensitive namespace discovery when replacing a table. This keeps writes, cache invalidation, and catalog refresh scoped to the existing dataset instead of creating a case-variant sibling.
lance-gatekeeper
Bot
removed
the
K-changes
This comment was marked as outdated.
This comment was marked as outdated.
Sorry, something went wrong.
lance-gatekeeper
Bot
added
the
K-changes
Resolve directory namespace lookups through the listed table name so case-variant replacements retain the physical dataset identifier. Use that identifier for DROP and add replacement and drop regressions.
lance-gatekeeper
Bot
removed
the
K-changes
There was a problem hiding this comment.
The case-variant replacement and DROP paths now preserve the physical table identifier, and the prior schema-only and CTAS rediscovery failure is fixed.
The author intentionally accepts that a second DuckDB connection may continue reading a stale cached Lance dataset after another connection commits a replacement. No further change is requested for that behavior in this pull request.
lance-gatekeeper
Bot
added
K-approved
|
@Xuanwo Could you review fix/dir-namespace-replace-v6? It keeps directory CREATE OR REPLACE TABLE on the existing dataset (including case-insensitive names), refreshes only that table after a successful overwrite, and adds replace/DROP regressions. Thanks. |
|
@lance-community Lance Gatekeeper pipeline could not complete on the last run, could you please rerun it? |
lance-gatekeeper
Bot
added
K-approved
1 participant
Preserve Lance's zero-row overwrite commit semantics for schema-only CREATE OR REPLACE TABLE operations.
After a successful overwrite, invalidate only the replaced directory dataset cache entry and evict its stale attached-catalog entry so lazy discovery reloads the committed schema. Leave cached handles and catalog state unchanged when the writer fails, and preserve cross-query reuse for untouched datasets.
Add regression coverage for schema refresh, failed replacements, precise cache invalidation, and untouched cache reuse.