order.balance task can silently drop balance.order event, missing revenue metrics

[stilla]

Bug

When the order.balance background task fails on its first attempt and retries, the retry can exit successfully even if the balance.order system event was never actually persisted. This causes the order to be missing from Tinybird revenue metrics.

Root cause

_create_balance_order_event() in server/polar/order/service.py wraps the entire event creation in a broad try/except that swallows exceptions silently — it only logs the error and returns:

async def _create_balance_order_event(...):
    try:
        assert payment_transaction.presentment_amount is not None
        assert payment_transaction.presentment_currency is not None
        # ... builds SystemEvent.balance_order
        await event_service.create_event(session, balance_order_event)
    except Exception as e:
        log.error("Could not save balance.order event", error=str(e))

So if event_service.create_event() throws (e.g. due to a DB session error), the task still completes without raising — no further retry is triggered, and the metric event is lost.

Revenue metrics in Tinybird are derived from balance.order events. If the event is never saved, the order never shows up in revenue metrics.

Impact

• Orders can be fully paid and processed (email, benefits, etc.) but not counted in revenue metrics
• No observable failure — the order.balance task reports success
• The _emit_balance_credit_order_event() method likely has the same issue

Fix

The exception should be re-raised (or at minimum not silently swallowed) so the Dramatiq task retries until the event is successfully persisted.

_{Sent by @pieterbeulque from Revenue metrics bug.}