Skip to content

Bitfinex import retry is broken: jh.random_uniform() does not exist (AttributeError kills the import worker) #601

Description

@ernestodeoliveira

Describe the bug

The retry logic in the Bitfinex Spot candle-import driver is dead: it calls
jh.random_uniform(), which does not exist in jesse.helpers. Any transient network error
during an import therefore raises AttributeError from inside the exception handler, and the
import worker dies silently instead of retrying.

jesse/modes/import_candles_mode/drivers/Bitfinex/BitfinexSpot.py:36

except (ConnectionError, RequestException) as e:
    if attempt == self.max_retries - 1:  # Last attempt
        raise e

    # Exponential backoff with jitter
    delay = (self.base_delay * 2 ** attempt) + (jh.random_uniform(0, 1))   # <-- AttributeError
    time.sleep(delay)

jesse.helpers exposes random and random_str, but there is no random_uniform:

>>> import jesse.helpers as jh
>>> hasattr(jh, 'random_uniform')
False
>>> [n for n in dir(jh) if 'rand' in n.lower()]
['random', 'random_str']

Actual traceback:

requests.exceptions.ConnectionError: HTTPSConnectionPool(host='api-pub.bitfinex.com', port=443):
Max retries exceeded with url: /v2/candles/trade:1m:tBTCUSD/hist?start=...&end=...&limit=1440&sort=1
(Caused by NewConnectionError("... Failed to establish a new connection: [Errno 111] Connection refused"))

During handling of the above exception, another exception occurred:

AttributeError: module 'jesse.helpers' has no attribute 'random_uniform'
==> Removed finished worker <import_id> from active workers

This is easy to miss because the failure is silent from the caller's perspective: the import
API/MCP call has already returned status: started, and nothing in the response indicates the
worker died. Only the container log shows it.

api-pub.bitfinex.com refuses connections fairly often during long imports (hours of consecutive
1m-candle requests), so in practice the retry path is hit regularly — and always fails.

Grep shows this is the only occurrence in the codebase; the Gate driver implements backoff without
jitter and is unaffected.

To Reproduce

  1. Import a long range of Bitfinex Spot candles, e.g. BTC-USD starting from 2017-01-01
    (several years of 1m candles).
  2. Let it run long enough for the exchange to refuse a connection (happened ~3 times over several
    hours in my case).
  3. The worker dies. The last log lines are the ConnectionError followed immediately by
    AttributeError: module 'jesse.helpers' has no attribute 'random_uniform'.

A minimal reproduction of the root cause, no network needed:

import jesse.helpers as jh
jh.random_uniform(0, 1)   # AttributeError

Expected behavior

A transient ConnectionError should trigger the exponential backoff and retry (up to
max_retries), not kill the import worker.

Proposed fix

Use the stdlib directly — random is not currently imported in this module:

 import requests
 import time
+import random
 from requests.exceptions import ConnectionError, RequestException
-                delay = (self.base_delay * 2 ** attempt) + (jh.random_uniform(0, 1))
+                delay = (self.base_delay * 2 ** attempt) + random.uniform(0, 1)

Alternatively, add random_uniform to jesse.helpers if it is meant to exist as part of the
helpers API (there may be other call sites planned).

Happy to send a PR for whichever approach you prefer.

Environment

  • OS: Docker (salehmir/jesse:latest), Linux container on Windows host
  • Version: jesse 2.5.0
  • Exchange: Bitfinex Spot, BTC-USD, 1m candles

Additional context

Two other ways a long import can stop, for context (not bugs, but they look similar in the logs and
made this one harder to isolate):

  • a full day missing on the exchange raises CandleNotFoundInExchange (e.g. Bitfinex 2016-04-02) —
    correct behaviour, but requires restarting past the gap;
  • restarting the jesse container kills any in-flight import worker.

Because of those, I initially misdiagnosed the three stops as unrelated network flakiness, until I
read the full traceback and saw the AttributeError below the ConnectionError.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions