Fix update() raising RevisionIdWasChanged for non-revision DuplicateKeyError#1270
Fix update() raising RevisionIdWasChanged for non-revision DuplicateKeyError#1270veeceey wants to merge 7 commits intoBeanieODM:mainfrom
Conversation
roman-right
left a comment
There was a problem hiding this comment.
Good approach. Checking the error string for id or revision_id before converting to RevisionIdWasChanged makes sense - other unique constraint violations should propagate as DuplicateKeyError. This is a better solution than #1267 which just removes the try/except entirely.
|
The CI is failing because of the outdated |
446d786 to
0d63237
Compare
staticxterm
left a comment
There was a problem hiding this comment.
Great to have this fixed, thanks!
Same comment as for the other PRs that some regression test case would be nice to have...
beanie/odm/documents.py
Outdated
| # A DuplicateKeyError mentioning revision_id is also a | ||
| # revision conflict. Any other DuplicateKeyError (e.g. on a | ||
| # user-defined unique index) should propagate as-is. | ||
| err_str = str(e) |
There was a problem hiding this comment.
| err_str = str(e) |
92bcf4d
beanie/odm/documents.py
Outdated
| # user-defined unique index) should propagate as-is. | ||
| if use_revision_id and not ignore_revision: | ||
| err_str = str(e) | ||
| if "_id_" in err_str or "revision_id" in err_str: |
There was a problem hiding this comment.
Hmm. Not sure how this would work if the user had serialized revision_id field into the DB in camelCase though...
For example, in an app I previously worked on, where we still used Pydantic v1 though, we added these overrides so it would always dump the fields in camelCase (but I guess the same could've been achieved with alias on the revision_id field with Pydantic v2):
def dict(self, *, by_alias = True, **kwargs):
output = super().dict(by_alias=by_alias, **kwargs)
return output
def json(self, *, by_alias = True, **kwargs):
return super().json(by_alias=by_alias, **kwargs)Anyway, since some fix is better than no fix at all, we could merge this in, and later see if field alias detection / (de)serialization of fields from DB could be improved.
There was a problem hiding this comment.
Yeah that's a fair point about camelCase aliases. The current fix checks for revision_id in the keyPattern which should work for the default field name, but if someone aliases it to camelCase the key in MongoDB's error details would reflect the actual index key name, not the Python field name. I think that's an edge case we can tackle separately though — happy to open a follow-up issue for it if you'd like.
|
@staticxterm Great point about the camelCase alias scenario! You're right that if someone has |
roman-right
left a comment
There was a problem hiding this comment.
Good fix -- the old code swallowed all DuplicateKeyErrors unconditionally. One suggestion on making the match more robust.
beanie/odm/documents.py
Outdated
| # revision conflict. Any other DuplicateKeyError (e.g. on a | ||
| # user-defined unique index) should propagate as-is. | ||
| if use_revision_id and not ignore_revision: | ||
| err_str = str(e) |
There was a problem hiding this comment.
Could you check if matching on str(e) is reliable across MongoDB versions? PyMongo's DuplicateKeyError inherits details from WriteError, which contains structured data like keyPattern. Using e.details.get("keyPattern", {}) would avoid depending on error message formatting.
There was a problem hiding this comment.
Great suggestion! I've switched to using e.details.get("keyPattern", {}) instead of string matching — that's definitely more robust across MongoDB versions. Updated in the latest push.
|
Thanks for the suggestion on making the match more robust, @roman-right! I'll update the string check accordingly. Also rebasing on main now to pick up the mongodb-github-action fix. |
…eyError When a Document has a unique indexed attribute, violating the uniqueness constraint was incorrectly raising RevisionIdWasChanged instead of the proper DuplicateKeyError. Updated the exception handler to check if the DuplicateKeyError is actually related to revision_id before converting it to RevisionIdWasChanged. If the error is for a different unique constraint, the original DuplicateKeyError is now re-raised. Fixes BeanieODM#1057
…tor imports conditional ConfigDict only exists in pydantic v2, and root_validator is only in pydantic v1. Move these imports to conditional blocks based on IS_PYDANTIC_V2 to fix import errors when running tests with pydantic 1.10.18.
for more information, see https://pre-commit.ci
When save() uses upsert=True with revision_id filtering, a stale revision causes the filter to miss the existing document. MongoDB then tries to insert a new doc with the same _id, raising a DuplicateKeyError on _id_ (not revision_id). This commit checks for both _id_ and revision_id in the error message when revision checking is active.
Address review feedback: only compute str(e) when we actually need it for the revision check, avoiding unnecessary work for non-revision DuplicateKeyError cases.
…tring matching Instead of matching on str(e) which can vary across MongoDB versions, use e.details["keyPattern"] to check which key caused the duplicate error. This is more robust as it uses PyMongo's structured error data. Co-Authored-By: Claude Opus 4.6 <noreply@anthropic.com>
e51b9fb
92bcf4d to
e51b9fb
Compare
|
Rebased on main and made the error string matching more robust — now using |
for more information, see https://pre-commit.ci
|
Quick update — I've rebased on main (picking up the mongodb-github-action fix) and switched from @staticxterm — regarding the camelCase alias edge case you flagged: the All 24 CI checks are passing now. Would appreciate a re-review! |
|
Hey @roman-right, I've switched to using keyPattern from the error details as you suggested. Could you take another look? All CI checks are green. Thanks! |
|
The fix is correct and well-reasoned. The final implementation using That said, I wouldn't approve it yet for two reasons. First, there are no regression tests. Both staticxterm and CAPITAINMARVEL flagged this, and the author never followed through. For exception handling logic especially, a test isn't optional — it's the only way to guarantee the fix doesn't silently regress in a future refactor. Second, @roman-right, who suggested the most impactful change in this PR (switching to The camelCase alias edge case is a known limitation and I'm fine with it being a follow-up, as long as it gets a tracking issue. |
Summary
DuplicateKeyErroron unique constraints was being converted toRevisionIdWasChangedRevisionIdWasChangedwhen the duplicate key error is actually related to revision_idChanges
Document.save()to check if theDuplicateKeyErrormentions "revision_id" before converting itDuplicateKeyErrorif it's for a different unique constraintTest plan
DuplicateKeyErroris raised (notRevisionIdWasChanged)Fixes #1057