StopFilter: pack ASCII stop words into longs for faster lookup

[costin]

Pack short ASCII stop words (up to 8 chars, which covers all standard English stop word lists) into a LongHashSet for O(1) lookup, avoiding a pointer indirection and the Character.toLowerCase() cost.
Non-packable sets (non-ASCII or words >8 chars) fall back to the existing CharArraySet path unchanged.

Benchmark

AMD EPYC 7R32 (c5a.2xlarge)
2000 tokens per iteration, english = 50% stop words, technical = 20%:

Text type	charArraySet (ops/µs)	packedLongHashSet (ops/µs)	Speedup
english	15.328 ± 0.235	40.127 ± 0.892	2.6x
technical	12.864 ± 0.198	35.741 ± 0.547	2.8x

[uschindler]

I am not so happy with that change as it makes everything more complex. It may be 2.6 times faster for each lookup, but the overall speedup during indexing is likely much lower.

In addition to me the whole thing may be better organized if the optimization would be inside CharArrayMap and CharArraySet - so instead of hashing it could cast the prefix characters inside CharArraySet and use that one as lookup key.

So: Wouldn't it be better to implement CharArrayMap using LongHashMap when all keys are short and ASCII only?

[dweiss]

I agree with Uwe - such changes - while showing microbenchmark benefits - are probably dwarfed by other things in the indexing pipeline. I don't think they're worth the increased complexity. And I know - LLMs don't care - but there is value and beauty in simplicity.

[costin]

Thanks for the feedback!

I've moved the optimization inside CharArrayMap (and Set) so all consumers benefit from it, including StopFilter which is untouched. The cache threshold defaults to 8 characters and is configurable via constructor.

The main motivation behind the PR is improving QPS, especially as LLM-driven agents are generating more search traffic than ever.

Updated the micro-benchmark to test CharArrayMap with/without the feature (same machine type and sample size):

Text type	baseline (ops/µs)	packed (ops/µs)	Speedup
english	0.024 ± 0.002	0.068 ± 0.001	2.8x
technical	0.024 ± 0.003	0.065 ± 0.002	2.7x

Let me know what you think.

[uschindler]

The new code still has the problem of more complexity, but in general I like it more, as it can be used for more use-cases.

I have to look closer into CharArrayMap. What happens if you add new elements later (the maps are modifiable)? Do you need to explicitely call the private method to make the cache? I did not found an automatic way in the code. It only builds it for CharArraySet when using the bulk constructor.

Maybe a good thing would be to add a public "freeze()" method to CharArrayMap (and set), which improves its performance but makes it read-only at same time. Of course add() would need to throw UOE then.

[uschindler]

Another observation: It only uses a cache of "known" keys in the map variant. The LongSet is only used to figure out if the key exists (on get()), if the key does not exist it returns null, otherwise it does the lookup using the normal hash table. The idea behind this is that for StopFilter and similar use-cases have more often lookups of non-existent keys, so we only improve the non existing key (most tokens in a text are NOT stop-words, so most lookups return nothing). The downside of that approach is that looking up a key that exists may take longer.

We have two possibilities:

• When building the cache use a LongMap and store the value, too. Of course this have additional memory overhead.
• Live with the limitation, but this would require users to know of the possible slowdown, so they should not call "freeze" (see above) after populating the map.
• Maybe move the optimization only to CharArraySet only.

In general I tend to disagree with the added complexity, but as we now have more possible uses of the code I have a slight preference to include it, although we have higher memory requirements (and complexity). Of course the API should be well documented and it should be an opt-in feature of CharArrayMap/Set. Freezing the maps is a good idea anyways to make sure they can't be modified after construction of the set.

I don't know yet if I'd like it.... sorry need to think about it more!