Support for API key batch and automatic key rotation

[FluoxetinT]

Please consider adding support for an API key pool with automatic rotation. When a key becomes temporarily unavailable due to rate limiting, the system should seamlessly switch to another key in the pool.
Additionally, it would be helpful to allow sending requests in parallel across multiple keys to maximize throughput and avoid slowdowns caused by throttled requests.

[hydropix]

Thanks for the suggestion. I had experimented with parallel requests on paid APIs, and chapter-level parallelization is definitely interesting for raw speed. Before going further, can you confirm what your main use case is? Are you mostly looking to chain several free-tier keys to get around daily quotas, or is it more about pure throughput on paid tiers? The implementation differs quite a bit between the two (rotation on 429 vs. true parallel dispatch), so it helps to know which one would actually unblock you.

[hydropix]

Hey, we just shipped the rotation part as a first step. You can now set GEMINI_API_KEY (or any other *_API_KEY env var) to a comma-separated list, and the system will switch to the next key automatically when one gets rate-limited. Behavior with a single key is unchanged. Full guide with examples and limitations is at docs/API_KEY_ROTATION.md.

Honest heads-up: we validated this with unit tests but havent run it against real APIs yet. If you can give it a try with your actual keys and let us know how it behaves in the wild, that would really help confirm everything works as expected.

Parallel dispatch (the second half of your request, true concurrent requests across keys) is a separate, more invasive change so we deferred it for now.

[virdb]

About key pool, I wondering if you also thought about the opportunity to use also a pool of provider. I.E. TBL start consuming the Gemini key pool, then switch to ChatGPT key pool, etc.
This can be useful in a scenario where the translations starts with an edge model and rely on a local one (ollama) to complete the translation if the credit on the paid provider is exhausted. Can this make sense?