Saving dtypes as metadata for roundtrip consistency#262
Open
yfukai wants to merge 13 commits intoroyerlab:mainfrom
Open
Saving dtypes as metadata for roundtrip consistency#262yfukai wants to merge 13 commits intoroyerlab:mainfrom
yfukai wants to merge 13 commits intoroyerlab:mainfrom
Conversation
Contributor
Author
|
Notes:
|
Contributor
Author
|
Both tasks are done. I'll review the code for SQLgraph and mark this open for review. |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## main #262 +/- ##
==========================================
- Coverage 87.92% 87.76% -0.16%
==========================================
Files 56 56
Lines 4471 4610 +139
Branches 789 810 +21
==========================================
+ Hits 3931 4046 +115
- Misses 344 360 +16
- Partials 196 204 +8 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
This was referenced Feb 26, 2026
Draft
Member
JoOkuma
reviewed
Feb 27, 2026
Comment on lines
+594
to
+597
| ordered_keys = [key for key in preferred_order if key in schemas] | ||
| ordered_keys.extend(key for key in table_class.__table__.columns.keys() if key not in ordered_keys) | ||
| ordered_keys.extend(key for key in schemas if key not in ordered_keys) | ||
| return {key: schemas[key] for key in ordered_keys} |
Member
There was a problem hiding this comment.
What do you think of this implementation? This way, we can avoid checking if it's in ordered_keys, which is a list, and it takes a bit longer to check if the column is there.
Suggested change
| ordered_keys = [key for key in preferred_order if key in schemas] | |
| ordered_keys.extend(key for key in table_class.__table__.columns.keys() if key not in ordered_keys) | |
| ordered_keys.extend(key for key in schemas if key not in ordered_keys) | |
| return {key: schemas[key] for key in ordered_keys} | |
| result = {} | |
| # return dictionary in preferred order | |
| for source in ( | |
| preferred_order, | |
| table_class.__table__.columns.keys(), | |
| schemas, | |
| ): | |
| for key in source: | |
| if key in schemas: | |
| result.setdefault(key, schemas[key]) | |
| return result |
Comment on lines
+1370
to
+1371
| else: | ||
| nodes_df = nodes_df.select([pl.col(c) for c in self._node_attr_schemas() if c in nodes_df.columns]) |
Member
There was a problem hiding this comment.
@yfukai , it's not clear to me why this is necessary.
Is it because it might have private attributes?
Could you include a comment on this and the equivalent edge_attr_code?
Comment on lines
+1735
to
+1751
| edge_schemas = self.__edge_attr_schemas | ||
| # Process arguments and create validated schema | ||
| schema = process_attr_key_args(key_or_schema, dtype, default_value, self.__edge_attr_schemas) | ||
|
|
||
| # Store schema | ||
| self.__edge_attr_schemas[schema.key] = schema | ||
| schema = process_attr_key_args(key_or_schema, dtype, default_value, edge_schemas) | ||
|
|
||
| # Add column to database | ||
| self._add_new_column(self.Edge, schema) | ||
| edge_schemas[schema.key] = schema | ||
| self.__edge_attr_schemas = edge_schemas | ||
|
|
||
| def remove_edge_attr_key(self, key: str) -> None: | ||
| if key not in self.edge_attr_keys(): | ||
| raise ValueError(f"Edge attribute key {key} does not exist") | ||
|
|
||
| edge_schemas = self.__edge_attr_schemas | ||
| self._drop_column(self.Edge, key) | ||
| self.__edge_attr_schemas.pop(key, None) | ||
| edge_schemas.pop(key, None) | ||
| self.__edge_attr_schemas = edge_schemas |
Member
There was a problem hiding this comment.
I was a bit lost at first here, but this seems required because self.__edge_attr_schemas has a setter and getter.
Do you think it would be worth being more explicit than my original implementation with @property and .setter? We could address this in another PR.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This is a follow-up on the dtype, assuming #260 has been merged.
In this PR, the graphs store the dtypes of the fields as metadata, enabling robust recovery of the original dtypes during round-trip
from_otherconversions.