Allow update() signals to change (add/modify/delete) kwargs

[Clement-O]

Hi!

I was looking to auto update a field (updated_at) in my document on pre_save and pre_update signals.
While it works flawlessly with save() & create(), it does not with update().

Looking at the code, save() & create() trigger their pre_save very early and use self directly so I can easily modify the instance in the signal before being 'updated' in the collection by the initial save or create function. For update() I need to modify the kwargs and not the instance, which looks currently impossible..

My code as example:

class Service(mongoz.Document):
    # Meta class
    name: str = mongoz.String()
    updated_at: datetime = mongoz.DateTime(null=True, default=None)

service = await Service.objects.get(name='a')
await service.update(a='b')

@pre_save(Service)
@pre_update(Service) # useless for my use case
async def before_update(sender: Service, instance: Service, **kwargs) -> None:
        instance.updated_at = now()

[harshalizode]

@Clement-O for running the our customized signal then you need to connect it as:
Service.meta.signals.pre_update.connect(before_update)
refer the https://mongoz.tarsild.io/signals/#custom-signals

[Clement-O]

Yeah I read the custom signals doc, but unless I missunderstand, it would not allow me to change the way the update is done and trigger said signals?

# mongoz.core.db.documents.document > Document > update() > l.75
await self.signals.pre_update.send(sender=self.__class__, instance=self)
await collection.update_one({"_id": self.id}, {"$set": data})  # type: ignore
await self.signals.post_update.send(sender=self.__class__, instance=self)

There it's already too late? data used for the update is already processed and set. I would like the possibilities to add to this data before the collection.update_one is sent.
Of course, as it is right now, I could catch the post_update signal, add the data I want and await doc.save(), but it would trigger another round of signals..

I haven't given much thought to it, but something like that would be fine / perfect for me:

await self.signals.pre_update.send(sender=self.__class__, instance=self, **kwargs)
data = kwargs_processed
await collection.update_one({"_id": self.id}, {"$set": data})  # type: ignore
await self.signals.post_update.send(sender=self.__class__, instance=self)

# in my pre_update signal
@pre_update(Service)
async def before_update(sender: Service, instance: Service, **kwargs) -> None:
        kwargs['updated_at'] = now()

Thanks for your time and answer!

[Clement-O]

Well, to anyone wanting to achieve something similar, I ended up rewriting the update() function in an Abstract BaseDocument, it uses save() instead, so it trigger the pre_save / post_save on update & save... (I needed a few other feature, such as nesting update)

[tarsil]

@Clement-O we are more than happy to have a contribution from you with a suggestion of improvement 🙂

[Clement-O]

Hi @tarsil
I'm really sorry but I do not have the energy nor time, right now, to go through the process of issues/pull request and all..

I'm quickly gonna explain my objective and some paste from my code here, if you deem it interesting for an improvement, feel free to take it!

Considering this setup:

class Nested(mongoz.EmbeddedDocument):
    string: str = mongoz.String()
    lorem: str = mongoz.String()
    
class Example(mongoz.Document):
    name: str = mongoz.String()
    nested: Nested = mongoz.Embed(Nested, default=Nested())
    updated_at: datetime = mongoz.Datetime(null=True, default=None)

Multiple things:

1. I want to update updated_at automatically on any other field update, either by using update() or save() (different use case)
2. I want to be able to update only one field on Nested and not the other (ex: lorem and not string) using update(). This might be against Mongo "rules", my memory is a bit blurry on this..

Problems:

1. update() pre_update signal trigger with data already validated / set in a dictionary with no way to edit it. save() pre_save signal has the possibility to edit the instance before the asynciomotor does the proper update.
2. Pydantic model_dump do not give a 'dotted' key: value, so mongo $set simply replace the full embedded document with given dict.
   Note: Point '2' could have the downside of making resetting to default value a tad bit more annoying.. (haven't tested, not really need it in my use case, or I have very easy model / needs to get it done easily)

I fixed both problem in one, but I'm sure they can be fixed separately, I have an idea and will share it at the end..
My fix:

# in a `Abstract` BaseDocument to be inherited.. 
async def update(self, **kwargs) -> "mongoz.Document":
        """TODO: ..."""

        def rsetattr(obj, attr, val):
            pre, _, post = attr.rpartition(".")
            return setattr(rgetattr(obj, pre) if pre else obj, post, val)

        def rgetattr(obj, attr, *args):
            def _getattr(obj, attr):
                return getattr(obj, attr, *args)

            return reduce(_getattr, [obj] + attr.split("."))

        def flatten_dict_gen(_dict: dict, parent_key: str = ""):
            for key, value in _dict.items():
                new_key = f"{parent_key}.{key}" if parent_key else key
                if isinstance(value, dict) and value:
                    yield from flatten_dict_gen(value, new_key)
                else:
                    yield new_key, value

        for key, val in flatten_dict_gen(kwargs):
            rsetattr(self, key, val)

        return await super().save()
        
# Assuming a pre_signal such as:
@pre_save(Example)
async def before_save(sender: Document, instance: Document, **kwargs) -> None:
        instance.updated_at = None if instance.id is None else utc_datetime()

# with this, I can simply do:
example = Example(name='name', nested={string:'init value', lorem: 'ipsum'})
await example.create()
assert updated_at is None

await example.update({nested: {string: 'new value'}}  # Assuming I have a `pre_save` signal to update `updated_at`..
assert example.nested.string == 'new_value'
assert example.nested.lorem == 'ipsum'
assert isinstance(updated_at, datetime)

Now, to implement it better in mongoz, I think problem one could modify Document.update() l.71

# From (l.71)
data = self.model_dump(exclude={"id": "_id"})
data.update(values)   # values being the dictionary of validated incoming changes..

# To (l.71)
data = self.model_dump(exclude={"id": "_id"})
await self.signals.pre_update.send(sender=self.__class__, instance=self, values)
# here you could also add my above code for nested update...  
# (though if you do, I think you should remove the `return await super().save()
# and instead use the same lines (372-379) (to avoid the double signal (pre_update > pre_save > post_save > post_update)).
await collection.update_one({"_id": self.id}, {"$set": values})  # type: ignore
await self.signals.post_update.send(sender=self.__class__, instance=self)
# rest of the code...

I think that's all, probably a bit messy, sorry!