Skip to content

Internal: Fix reader reordering in FlowSynchronizationGroup::waitForDataAt - #711

Merged
vt-tv merged 2 commits into
dmf-mxl:mainfrom
rochonma:bugfix/sync-group-waitForDataAt
Sep 9, 2026
Merged

vt-tv merged 2 commits into
dmf-mxl:mainfrom
rochonma:bugfix/sync-group-waitForDataAt

Conversation

@rochonma

@rochonma rochonma commented Sep 8, 2026

Copy link
Copy Markdown
Contributor

What's broken

A FlowSynchronizationGroup with two or more readers silently stops synchronizing after its first internal reordering. From that point on, mxlFlowSynchronizationGroupWaitForDataAt returns MXL_STATUS_OK in microseconds regardless of whether the data is actually there.

It fails by looking like success: no error code, no log line, no crash. A consumer keeps calling the API and keeps being told the data is ready, and simply reads whatever happens to be in the ring.

Why

waitForDataAt opportunistically promotes the flow with the largest observed source delay to the front of _readers, so that later calls block on the slowest source first. The promotion was written as:

_reader.splice_after(_readers.before_begin(), _readers, current);

std::forward_list::splice_after(pos, other, it) moves the element following it, not *it - a singly-linked list can only unlink an element through its predecessor. So passing the iterator of the entry to be promoted moves that entry's successor instead.

Two failure shapes:

  • Entry in the middle of the list: the wrong entry is promoted. A bad heuristic decision, but harmless.
  • Entry at then end of the list: std::next(current) is the end iterator and the call is undefined behavior. With libstdc++ the list's head pointer is nulled - the list becomes empty and its nodes are leaked. waitForDataAt then iterates over nothing and returns MXL_STATUS_OK immediately, forever.

A standalone reproducer of just the list operation, under ASan/UBSan:

before                       [ video audio ]
after splice_after(current)  [ ]
SUMMARY: AddressSanitizer: 80 byte(s) leaked in 2 allocation(s)

When it triggers

It fires the first time a non-head member's observed source delay exceeds the head's record - in practice almost immediately whenever one source is consistently later than another, which is the normal use case for sync-group.

The fix

Track the predecessor of the entry being examined and pass that to splice_after. After the splice the predecessor already precedes the loop cursor, so it must not be advanced - hence the continue.

The promotion condition can only hold for an entry that is not the head, so the tracked predecessor is always a real element and never before_begin().

Testing

Adds lib/tests/test_flow_sync_groups.cpp -
Synchronization group : Repeated waits. Two discrete flows sharing a grain rate are added to one group; the second member's grain is committed deliberately late sot that the reordering is triggered on the first wait. A second wait on the same group then asks for a grain nobody will ever write and must time out. The reordering logic is independent of reader type - variant only selects which wait function is called - so the test uses two discrete flows for determinism.

Before the fix that second wait returns MXL_STATUS_OK instantly instead of blocking for its 200 ms timeout. After the fix it times out correctly.

Built and tested locally on Linux x86_64 with both the Linux-GCC-Release and Linux-Clang-Release presets: all builds warning-free and ctest reports 60/60 passing on each. clang-format --dry-run --Werror over lib and tools is clean.

…ataAt

std::forward_list::splice_after(pos, other, it) moves the element *following*
`it`, not `*it`. waitForDataAt passed the iterator of the entry it wanted to
promote, so it moved that entry's successor instead, and when the entry to
promote was the last one in the list, `std::next(it)` was the end iterator and
the call was undefined behaviour. In practice the list ends up empty, and every
subsequent call iterates over nothing and returns MXL_STATUS_OK immediately:
the group silently stops synchronizing after its first reordering, which is a
failure that looks like success.

A single-member group never reaches that branch, since the guard requires an
entry whose observed source delay exceeds the head's, so only groups with two
or more readers are affected.

Pass the predecessor of the entry to be promoted, and keep track of it while
walking the list.

Signed-off-by: rochonma <mathieu.rochon@radio-canada.ca>
Signed-off-by: rochonma <mathieu.rochon@radio-canada.ca>

@KimonHoffmann KimonHoffmann left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thank you for catching this and fixing the issue!
The changes look good to me.

@KimonHoffmann KimonHoffmann added this to the v1.1 milestone Sep 9, 2026
@KimonHoffmann KimonHoffmann added the backport/v1.1 This PR should be back ported to the release branch of version 1.1. label Sep 9, 2026

@jonasohland jonasohland left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@vt-tv
vt-tv merged commit 26ceaf0 into dmf-mxl:main Sep 9, 2026
11 checks passed
@backport-mxl-pull-requst

Copy link
Copy Markdown

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

backport/v1.1 This PR should be back ported to the release branch of version 1.1.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants