Consider to improve the current GC for tracking

[vlrpl]

We might want to consider revisiting the GC logic (worst case can be problematic) as an LRU map may potentially replace it.

Some more investigations are needed (performance wise as well).

[github-actions]

This issue has been automatically marked as stale due to 365 day(s) of inactivity.
If this remains inactive for an additional 30 day(s), it will be automatically closed.
In case this was incorrectly marked as stale, please, remove the stale label (or simply add a comment), and add it
to a milestone, if applicable.
If you do not have sufficient permissions to complete any of the previous actions, feel free to reach out to the maintainers.