Skip to content

[Messaging] SubscribeAsync silently fails with unobserved task exceptions when Dapr sidecar is unavailable #1663

@Euynac

Description

@Euynac

Expected Behavior

When the Dapr sidecar is unavailable (not started, crashed, or disconnected), SubscribeAsync() should throw a catchable exception (like DaprException used in other SDK methods) so applications can handle failures gracefully.

Actual Behavior

Exceptions are hidden and become unobserved task exceptions that are nearly impossible to track:

  1. If sidecar is not started: Calling SubscribeAsync() appears to succeed, but silently fails. No exception is thrown.
  2. If sidecar crashes after subscription: The failure triggers TaskScheduler.UnobservedTaskException only when the task is garbage collected (may never happen or happen much later).
  3. No error callback, event, or observable way to detect the failure.

Root Cause

In PublishSubscribeReceiver.cs:113-134, SubscribeAsync() starts background tasks with fire-and-forget:

internal async Task SubscribeAsync(CancellationToken cancellationToken = default)
{
    // ...
    var stream = await GetStreamAsync(cancellationToken);  // Line 121

    // Fire-and-forget - exceptions become unobserved
    _ = FetchDataFromSidecarAsync(stream, ...)
        .ContinueWith(HandleTaskCompletion, ..., TaskContinuationOptions.OnlyOnFaulted, ...);

    _ = ProcessAcknowledgementChannelMessagesAsync(...)
        .ContinueWith(HandleTaskCompletion, ..., TaskContinuationOptions.OnlyOnFaulted, ...);

    // Method returns immediately - appears successful even if sidecar is down
}

When GetStreamAsync() or background tasks fail, exceptions go to HandleTaskCompletion which re-throws into the void:

internal static void HandleTaskCompletion(Task task, object? state)
{
    if (task.Exception != null)
    {
        throw task.Exception;  // ⚠️ Becomes UnobservedTaskException
    }
}

Steps to Reproduce the Problem

// Don't start daprd at all
var subscription = await pubSubClient.SubscribeAsync("pubsub", "topic", ...);
// Returns successfully, no exception thrown
// Silently fails - very hard to debug

// Or: start daprd, subscribe, then kill daprd
// Exception only appears in TaskScheduler.UnobservedTaskException (if subscribed)
// Otherwise completely silent

Proposed Solution

1. Make SubscribeAsync() actually await the connection establishment and throw on failure
2. Add observable error callback: event EventHandler<Exception>? OnError
3. Store and expose background tasks for monitoring
4. Wrap RpcException in DaprException for consistency with other SDK methods

Release Note

RELEASE NOTE: FIX SubscribeAsync now properly throws exceptions when Dapr sidecar is unavailable instead of silently failing.

Metadata

Metadata

Assignees

No one assigned

    Type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions