[Feature Request] Add the capability to automatically adjust the skipFactor parameter for improved query cache utilization

[kkewwei]

Is your feature request related to a problem? Please describe

QueryCache is proven to accelerate query performance, with up to a 50% improvement achievable through proper configuration. In #18351, we added capability to adjuste the querycache skipFactor, but this expert-level setting requires manual tuning to take effect.

Cluster administrator unfamiliar with skipFactor often fail to unlock the full potential of querycache. In production, we have seen many clusters with underutilized query caches—where manual skipFactor adjustments directly boosted query performance.

Another common scenario is that users retain the default skipFactor, only to find the querycache perpetually saturated—resulting in cache entry creation and eviction. This not only causes unnecessary CPU overhead but also degrades overall query performance. In such cases, reducing the skipFactor value would effectively alleviate cache overloading.

Describe the solution you'd like

Therefore recommend adding a scheduled task to auto-adjust skipFactor based on real-time query cache usage, enabling the system to maximize query caching value without manual intervention.

Related component

Search:Performance

Describe alternatives you've considered

No response

Additional context

No response

[kkewwei]

@sgup432 Would you mind take a look?

[sgup432]

@kkewwei This does makes sense but I wonder if we should look at adding this capability for overall caches and not just limited to query cache.
For example, we introduced many different knobs in request cache as well, like this and this

We have seen in a lot of cases where request cache is underutilized as well, as by default it only caches size=0 queries. But with above setting, it can be tweaked. Plus request cache can be also be extended with tiered cache.

Are you thinking to implement this scheduler as an opt-in functionality where it can be turned on/off as per user's need? If yes, should then this be a part of the core or module/plugin?

[sgup432]

In addition, I also wonder if this "skipFactor" also needs another thought. It was added to avoid caching "costly" queries, determined via its own cost(matching documents) and lead iterator's cost. We probably can replace this with a combination of took_time and size based policy for better accuracy?

[kkewwei]

Are you thinking to implement this scheduler as an opt-in functionality where it can be turned on/off as per user's need? If yes, should then this be a part of the core or module/plugin?

@sgup432 I’m unsure if the cache-ehcache plugin can support this functionality—ideally, if it can, this would be the optimal approach, as the plugin provides extensive control capabilities that extend well beyond just the query cache.

We probably can replace this with a combination of took_time and size based policy for better accuracy?

This logic is implemented in the LRUQueryCache of Lucene. Caching based on metrics such as took_time and size would render the LRUQueryCache inapplicable, which constitutes a substantial change and is therefore recommended for the second phase of development. Maybe the primary focus of the first phase should be on implementing the capability to dynamically adjust cache parameters.

[sgup432]

@sgup432 I’m unsure if the cache-ehcache plugin can support this functionality—ideally, if it can, this would be the optimal approach, as the plugin provides extensive control capabilities that extend well beyond just the query cache.

Oh I didn't mean to use cache-ehcache plugin for this, but a separate module for this functionality if needed.

This logic is implemented in the LRUQueryCache of Lucene. Caching based on metrics such as took_time and size would render the LRUQueryCache inapplicable, which constitutes a substantial change and is therefore recommended for the second phase of development. Maybe the primary focus of the first phase should be on implementing the capability to dynamically adjust cache parameters.

I actually meant to think about this later, not right now. I was just thinking out loud. So for now, we can just focus on using the skip factor as it is.