[BUG] null_pointer_exception on knn query with rescore and explain (engine=lucene) #20484
Description
Describe the bug
I get a null_pointer_exception on knn query on knn_vector with rescore and explain when Lucene engine is used.
The NPE doesn't happen when (either one):
- the query doesn't ask for an explanation (no
explain=true) - no rescore is used
- engine other that Lucene (e.g. Faiss) is used for index construction
Related component
Search
To Reproduce
- Create a test index:
PUT test
{
"settings": {
"index": {
"knn": true
}
},
"mappings": {
"properties": {
"embedding": {
"type": "knn_vector",
"dimension": 2,
"method": {
"name": "hnsw",
"engine": "lucene"
}
}
}
}
}
- Index a document
PUT test/_doc/1?refresh=true
{
"embedding": [1, 2]
}
- Try to perform knn search with rescore and explain
GET test/_search?explain=true
{
"query": {
"knn": {
"embedding": {
"vector": [1,2],
"k": 1,
"rescore": {
"oversample_factor": 3
}
}
}
}
}
- Observe the error:
{
"error": {
"root_cause": [
{
"type": "null_pointer_exception",
"reason": "Cannot invoke \"org.opensearch.knn.index.query.KNNWeight.explain(org.apache.lucene.index.LeafReaderContext, int, float)\" because \"this.this$0.knnWeight\" is null"
}
],
"type": "search_phase_execution_exception",
"reason": "all shards failed",
"phase": "query",
"grouped": true,
"failed_shards": [
{
"shard": 0,
"index": "test",
"node": "0-TLQAxdS6Sbon3RR-Rkyg",
"reason": {
"type": "null_pointer_exception",
"reason": "Cannot invoke \"org.opensearch.knn.index.query.KNNWeight.explain(org.apache.lucene.index.LeafReaderContext, int, float)\" because \"this.this$0.knnWeight\" is null"
}
}
],
"caused_by": {
"type": "null_pointer_exception",
"reason": "Cannot invoke \"org.opensearch.knn.index.query.KNNWeight.explain(org.apache.lucene.index.LeafReaderContext, int, float)\" because \"this.this$0.knnWeight\" is null",
"caused_by": {
"type": "null_pointer_exception",
"reason": "Cannot invoke \"org.opensearch.knn.index.query.KNNWeight.explain(org.apache.lucene.index.LeafReaderContext, int, float)\" because \"this.this$0.knnWeight\" is null"
}
}
},
"status": 500
}
Expected behavior
No NPE
Additional Details
OpenSearch v3.4.0
Screenshots
