Sticky Sessions does not appear to be working

[lbatalha]

Testing with olla 0.0.26, I can't seem to trigger sticky sessions.

I've enabled them in the config (but also tried setting OLLA_PROXY_STICKY_SESSIONS_ENABLED=true):

proxy:
  engine: "sherpa"
  profile: "auto"
  load_balancer: "least-connections"
  stream_buffer_size: 8192
  sticky_sessions:
    enabled: true
    idle_ttl_seconds: 0
    max_sessions: 1000
    key_sources:
      - "session_header"
      - "prefix_hash"
      - "auth_header" 

I've also tested changing the engine to olla, and adding/changing/removing some of the sticky_sessions parameters.

I'm testing sending a request with an example header X-Olla-Session-ID: conv-abc123
But I do not see any sticky-related response headers:

< content-length: 2073
< access-control-allow-origin: 
< via: 1.1 olla/v0.0.26
< x-olla-backend-type: llamaswap
< x-olla-endpoint: home3080
< x-olla-model: qwen3.6-35b-a3b
< x-olla-request-id: vicuna-humming-5d1c
< x-olla-response-time: 8018ms
< x-olla-routing-decision: routed
< x-olla-routing-reason: model_found
< x-olla-routing-strategy: strict
< x-served-by: Olla/v0.0.26

The internal/stats/sticky endpoint appears to be working, but all values show as 0 after sending some requests:

{
  "enabled": true,
  "active_sessions": 0,
  "insertions": 0,
  "hits": 0,
  "misses": 0,
  "evictions": 0,
  "max_sessions": 1000,
  "idle_ttl_seconds": 0
}

Did I miss some particularities that would make this not work with my setup?

[thushan]

Thanks, noted this just after we did the hotfix but the fix wasn't ready to hit .26.

Will have this and another fix for the load balancer component ready soon.

[thushan]

I've merged the fixes into main if you're able to test?

[lbatalha]

I've built the image locally (using goreleaser this time :>).

Sticky sessions appear to be working correctly! I'll go ahead and close this for now. I'll keep testing and will open an issue if I find any further problems.

Thanks for work 🙏

[thushan]

thanks for raising and testing!