[Feature Request] Add per-list scan limit for `parallel_mode=1/2`

[Tickdack]

Problem

max_codes only works with parallel_mode=0/3. For single-query scenarios, parallel_mode=1 is preferred, but there's no way to limit scan depth per inverted list.

Use Case

In real-time search with dynamic data, hot events can cause some inverted lists to grow to millions of vectors (vs. hundreds expected). This causes severe latency spikes when queries hit these lists.

Proposed Solution

Add max_codes_per_list parameter to limit scan depth per individual list, working with all parallel modes.

// In SearchParametersIVF
size_t max_codes_per_list = 0;  // 0 = no limit

Why not existing solutions?

• max_codes + parallel_mode=0: Loses intra-query parallelism
• Reduce nprobe: Too coarse, hurts recall
• Retrain index: Doesn't handle dynamic hot spots

Implementation

Minimal change (~10 lines). Happy to contribute a PR if this approach is acceptable.

[mdouze]

I don't understand why supporting max_codes in the parallel_mode=1 case is not sufficient ?

faiss/faiss/IndexIVF.cpp

Line 631 in 1cb3d46

} else if (pmode == 1) {

The comment in the code says it's not possible to support max_codes, but it is possible, we'd just have to compute the list sizes in advance and limit the nprobe and size of the last list to max_codes.

[Tickdack]

Thanks for the suggestion!

The pre-computation approach works for total scan budget. However, my use case is different:

Scenario: Some lists grow to millions (hot events), most remain small.

With max_codes: If first list has 2M vectors, we scan only that list and skip others.

With max_codes_per_list: Each list capped, but we still visit all nprobe lists → better recall by sampling from multiple clusters.

Would you prefer I implement max_codes for parallel_mode=1, or do you see value in per-list limit as a separate parameter?

I don't understand why supporting max_codes in the parallel_mode=1 case is not sufficient ?

faiss/faiss/IndexIVF.cpp

Line 631 in 1cb3d46

} else if (pmode == 1) {
The comment in the code says it's not possible to support max_codes, but it is possible, we'd just have to compute the list sizes in advance and limit the nprobe and size of the last list to max_codes.

[mdouze]

I think this is a too specific problem to change the searching code.
Note that there is an inverted lists implementation that limits the size of inverted lists here:

https://github.com/facebookresearch/faiss/blob/main/faiss/invlists/InvertedLists.h#L449

It would make sense to make a similar implementation that caps (rather than cancelling completely) the list sizes while still allowing writing.