[BUG] Clustering playlist naming with OpenRouter failing

[Th3K1ngP1n]

Describe the bug
I get empty responses when using openrouter as ai playlist naming.

Here are the relevant logs:

INFO]-[29-04-2026 22-38-21]-FINAL FISHER-YATES SHUFFLE applied to 'Pop_Indie_Rock_Medium_Sad_Relaxed_automatic': 27 songs
[INFO]-[29-04-2026 22-38-21]-FINAL ORDER: First song = 'Turn It Off (acoustic version)', Last song = 'Hate That I Love You'
[INFO]-[29-04-2026 22-38-21]-Truncated song list from 78 to 25 songs for AI prompt to avoid token limits
[INFO]-[29-04-2026 22-38-21]-Sending prompt to AI (OPENAI):
You are an expert music collector and MUST give a title to this playlist.
The title MUST represent the mood and the activity of when you are listening to the playlist.
The title MUST use ONLY standard ASCII (a-z, A-Z, 0-9, spaces, and - & ' ! . , ? ( ) [ ]).
The title MUST be within the range of 5 to 40 characters long.
No special fonts or emojis.

• BAD EXAMPLES: 'Ambient Electronic Space - Electric Soundscapes - Emotional Waves' (Too long/descriptive)
• BAD EXAMPLES: 'Blues Rock Fast Tracks' (Too direct/literal, not evocative enough)
• BAD EXAMPLES: '𝑯𝒘𝒆 𝒂𝒓𝒐𝒏𝒊 𝒅𝒆𝒕𝒔' (Non-standard characters)
  CRITICAL: Your response MUST be ONLY the single playlist name. No explanations, no 'Playlist Name:', no numbering, no extra text or formatting whatsoever.
  This is the playlist:

• Diamond Days (Redux) by Lonely the Brave
• Mullaghmore by Christy Moore
• Between the Bars by Elliott Smith
• He Got Away by Noah Gundersen & The Forest Rangers
• Feel It Still by The Kooks
• That’s the Way by Led Zeppelin
• It’s a Beautiful Day by Liverpool Express
• Carving Circles (Redux) by Interlaker
• Society by Eddie Vedder
• Africa by SubAudible Hum
• Rolling in the Deep by Vance Joy
• I Was a Prayer (acoustic) by Alkaline Trio
• Love Love Love (The Cabin Sessions) by Of Monsters and Men
• This Boy's in Love by Sparkadia
• Rift (acoustic) by Northlane
• Dancing Queen by Whitley
• The Foxy Devil by Christy Moore
• Dark Globe by Placebo
• Everlong (acoustic version) by Foo Fighters
• Only You by Mat McHugh and The Black Bird
• Stumbleine by The Smashing Pumpkins
• True Love Waits by Radiohead
• Senseless Tragedy by Seether
• If I Could Change Your Mind by Jake Bugg
• Time to Waste (acoustic) by Alkaline Trio

And

[WARNING]-[29-04-2026 22-38-21]-OpenAI/OpenRouter returned empty content. Full raw response:
[WARNING]-[29-04-2026 22-38-21]-AI naming failed for 'Pop_Indie_Rock_Medium_Sad_Relaxed': Error: AI returned empty content after retries.. Using original name.
[INFO]-[29-04-2026 22-38-21]-FINAL FISHER-YATES SHUFFLE applied to 'Pop_Indie_Rock_Medium_Sad_Relaxed_automatic': 27 songs
[INFO]-[29-04-2026 22-38-21]-FINAL ORDER: First song = 'Turn It Off (acoustic version)', Last song = 'Hate That I Love You'
[INFO]-[29-04-2026 22-38-21]-Truncated song list from 78 to 25 songs for AI prompt to avoid token limits
[INFO]-[29-04-2026 22-38-21]-Sending prompt to AI (OPENAI):
You are an expert music collector and MUST give a title to this playlist.
The title MUST represent the mood and the activity of when you are listening to the playlist.
The title MUST use ONLY standard ASCII (a-z, A-Z, 0-9, spaces, and - & ' ! . , ? ( ) [ ]).
The title MUST be within the range of 5 to 40 characters long.
No special fonts or emojis.

• BAD EXAMPLES: 'Ambient Electronic Space - Electric Soundscapes - Emotional Waves' (Too long/descriptive)
• BAD EXAMPLES: 'Blues Rock Fast Tracks' (Too direct/literal, not evocative enough)
• BAD EXAMPLES: '𝑯𝒘𝒆 𝒂𝒓𝒐𝒏𝒊 𝒅𝒆𝒕𝒔' (Non-standard characters)
  CRITICAL: Your response MUST be ONLY the single playlist name. No explanations, no 'Playlist Name:', no numbering, no extra text or formatting whatsoever.
  This is the playlist:

• Diamond Days (Redux) by Lonely the Brave
• Mullaghmore by Christy Moore
• Between the Bars by Elliott Smith
• He Got Away by Noah Gundersen & The Forest Rangers
• Feel It Still by The Kooks
• That’s the Way by Led Zeppelin
• It’s a Beautiful Day by Liverpool Express
• Carving Circles (Redux) by Interlaker
• Society by Eddie Vedder
• Africa by SubAudible Hum
• Rolling in the Deep by Vance Joy
• I Was a Prayer (acoustic) by Alkaline Trio
• Love Love Love (The Cabin Sessions) by Of Monsters and Men
• This Boy's in Love by Sparkadia
• Rift (acoustic) by Northlane
• Dancing Queen by Whitley
• The Foxy Devil by Christy Moore
• Dark Globe by Placebo
• Everlong (acoustic version) by Foo Fighters
• Only You by Mat McHugh and The Black Bird
• Stumbleine by The Smashing Pumpkins
• True Love Waits by Radiohead
• Senseless Tragedy by Seether
• If I Could Change Your Mind by Jake Bugg
• Time to Waste (acoustic) by Alkaline Trio
  [WARNING]-[29-04-2026 22-38-23]-OpenAI/OpenRouter returned empty content. Full raw response:
  [INFO]-[29-04-2026 22-38-23]-Retrying in 5 seconds due to empty content...
  [WARNING]-[29-04-2026 22-38-30]-OpenAI/OpenRouter returned empty content. Full raw response:
  [INFO]-[29-04-2026 22-38-30]-Retrying in 10 seconds due to empty content...
  [WARNING]-[29-04-2026 22-38-43]-OpenAI/OpenRouter returned empty content. Full raw response:
  [INFO]-[29-04-2026 22-38-43]-Retrying in 20 seconds due to empty content

I can see from my activity in OpenRouter that tokens have been used to make the requests.

Have you any ideas? Thanks.

[NeptuneHub]

Have you used any of the approved model in the documentation?
If not, can you use any of the approved model in the documentation and retest?

Thanks!

[rendyhd]

I've had this with some models on OpenRouter too while testing, especially when they're routed via slower providers using large thinking models.
OpenRouter published this last year: https://openrouter.ai/announcements/never-pay-for-empty-ai-responses-again
I figured it's not us, but something that can happen.

[Th3K1ngP1n]

@NeptuneHub Sorry I forgot to add, this was with Claude Sonet 4.6 , I tried it because it says its the best in the docs 😊

@rendyhd Ill have a look into that. Ill also run clustering again at different time.

And thank you both for rapid response.

[rendyhd]

Yeah please try again, Sonet has generally been solid for me. Claude did had a red status, might just be bad timing https://status.claude.com/

[Th3K1ngP1n]

Ok thanks for info, it does sound like maybe wrong timing.

Currently there is no way to resend prompt from previous clustering run right? and, I need to run clustering again?

[rendyhd]

Yes that's correct. But I'll take that feedback, I'm planning to look at the clustering naming after I'll look at the sonic fingerprint playlist. Now that the migration is done I should be getting to it.

[Th3K1ngP1n]

Thanks @rendyhd I really appreciate it.

[Th3K1ngP1n]

I just ran the clustering again but Im left with the same error :(

[INFO]-[30-04-2026 07-58-21]-FINAL FISHER-YATES SHUFFLE applied to 'Rock_Alternative_Indie_Medium_Aggressive_Danceable_6_automatic': 37 songs
[INFO]-[30-04-2026 07-58-21]-FINAL ORDER: First song = 'The Poison (demo)', Last song = 'Summer'
[INFO]-[30-04-2026 07-58-21]-Sending prompt to AI (OPENAI):
You are an expert music collector and MUST give a title to this playlist.
The title MUST represent the mood and the activity of when you are listening to the playlist.
The title MUST use ONLY standard ASCII (a-z, A-Z, 0-9, spaces, and - & ' ! . , ? ( ) [ ]).
The title MUST be within the range of 5 to 40 characters long.
No special fonts or emojis.
* BAD EXAMPLES: 'Ambient Electronic Space - Electric Soundscapes - Emotional Waves' (Too long/descriptive)
* BAD EXAMPLES: 'Blues Rock Fast Tracks' (Too direct/literal, not evocative enough)
* BAD EXAMPLES: '𝑯𝒘𝒆 𝒂𝒓𝒐𝒏𝒊 𝒅𝒆𝒕𝒔' (Non-standard characters)
CRITICAL: Your response MUST be ONLY the single playlist name. No explanations, no 'Playlist Name:', no numbering, no extra text or formatting whatsoever.
This is the playlist:
- Only You (acoustic) by Sarah Close
- Wrecking Ball by Miley Cyrus
- Deep End by Lily Allen
- De madrugá by ROSALÍA
- Your Song (acoustic) by Rita Ora
- Issues (acoustic) by Julia Michaels
- Male Fantasy by Billie Eilish
- hate u love u by Olivia O’Brien
- La rumba del perdón by ROSALÍA
- Starving (acoustic) by Hailee Steinfeld • Grey & Zedd
- Told Me (acoustic piano version) by Dusky Grey
- Believe by Madilyn Bailey
- Radioactive (single version) by Madilyn Bailey
- Smile by Lily Allen
- Hell Would Have to Freeze Over by Catherine McGrath
- Please Don’t Say You Love Me by Gabrielle Aplin
- not the same by ROSÉ
- Young Dumb & Broke by Thelma Plum
- Chandelier by Madilyn Bailey
- Midnight (solo) by Lianne La Havas
- Sauvignon blanc by ROSALÍA
- Turn It Off (acoustic version) by Paramore
[WARNING]-[30-04-2026 07-58-23]-OpenAI/OpenRouter returned empty content. Full raw response: 
[INFO]-[30-04-2026 07-58-23]-Retrying in 5 seconds due to empty content...
[WARNING]-[30-04-2026 07-58-29]-OpenAI/OpenRouter returned empty content. Full raw response: 
[INFO]-[30-04-2026 07-58-29]-Retrying in 10 seconds due to empty content...
[WARNING]-[30-04-2026 07-58-41]-OpenAI/OpenRouter returned empty content. Full raw response: 
[INFO]-[30-04-2026 07-58-41]-Retrying in 20 seconds due to empty content...
[WARNING]-[30-04-2026 07-59-15]-OpenAI/OpenRouter returned empty content. Full raw response: 
[WARNING]-[30-04-2026 07-59-15]-AI naming failed for 'Pop_Indie_Rock_Medium_Sad_Relaxed': Error: AI returned empty content after retries.. Using original name.

[Th3K1ngP1n]

I've been digging into this a bit more and AI tells me it has something to do with streaming being enabled.

Playlist naming with Mistral is working for me so it seems there is something fundamentally wrong with the current openrouter ai naming config. Please let me know if you require and more info from me.

Does any of this help (sorry for AI paste crap)

Your model isn’t “failing” — it did return a completion, but your pipeline is throwing it away.
Key clue here:


"tokens_completion": 6


"finish_reason": "stop"


no content surfaced in your logs


That means:
👉 The model generated something short (likely a valid title), but you’re not reading it correctly from the streamed response.

🔍 What’s actually happening
You’re using:
"api_type": "completions""streamed": true
With OpenRouter + OpenAI models, this combo means:
1. The text is delivered in stream chunks, not in one content field
So if you do something like:
response["choices"][0]["message"]["content"]
…it will often be empty, because streaming responses don’t populate it that way.

✅ Why you see “empty content”
Your logs:
OpenAI/OpenRouter returned empty content
But the raw response shows:
tokens_completion: 6finish_reason: stop
👉 So content exists, but:


you didn’t collect streamed tokens


or you’re reading the wrong field (message.content vs text vs delta)



🧠 Likely root causes
❌ 1. Wrong field for completions API
Since "api_type": "completions":
You should read:
response["choices"][0]["text"]
NOT:
response["choices"][0]["message"]["content"]

❌ 2. Streaming not assembled
If stream=True, you must accumulate:
full_text = ""for chunk in stream:    full_text += chunk["choices"][0]["text"]
Otherwise → you end up with empty output.

❌ 3. Mixing chat + completions formats
You’re sending a chat-style prompt, but using:
"api_type": "completions"
That mismatch often leads to weird behavior.

💥 The real bug (most likely)
You’re doing one of these:


Using chat parsing on a completions response


OR using non-stream parsing on a streamed response



✅ Fix options (pick one)
✅ Option A — Switch to chat API (recommended)
Use:
"api_type": "chat"
Then read:
response["choices"][0]["message"]["content"]

✅ Option B — Keep completions, fix parsing
Read:
response["choices"][0]["text"]
And if streaming:
for chunk in stream:    print(chunk["choices"][0]["text"], end="")

✅ Option C — Disable streaming (quick debug)
Set:
"stream": false
Then check if content appears → confirms parsing bug.

🧪 Bonus: sanity check
Your prompt is fine. The model would likely output something like:

"Quiet Night Under Heavy Skies"

(That’s ~5 tokens → matches your tokens_completion: 6)

⚠️ One more subtle issue
Your constraint:
ONLY ASCII characters
But your playlist includes:


I’ll Miss the System Here (curly apostrophe)


non-ASCII titles


Sometimes models get cautious and output weirdly minimal text.
👉 Not the main issue, but worth normalizing input text.

🔧 TL;DR
Your system isn’t failing — you’re just not reading the response correctly.
Fix this first:


Match API type (chat vs completions)


Handle streaming properly


Read the correct field (text vs message.content)



If you want, paste your request + parsing code and I’ll pinpoint the exact broken line.

[NeptuneHub]

I personally use llama3.1:8b deployed locally and do his job. As model on API the last that I tested was Gemini 2.5 pro.

Here the problem is that using something on API, maybe jumping on the latest and greatest model, ends up that something change and stop working. A lot of this model for example introduce the thinking / reasoning that broke the fact that we want only one title.

Anyway @rendyhd is the master and commander of the AI model, so I hope he can give a look helping in this.

[rendyhd]

Thanks for the research @Th3K1ngP1n !
I'll have a look at it. I wonder what has changed, because it definitely used to work.

[Th3K1ngP1n]

@rendyhd Happy to help. My knowledge in this area is limited, but, do please let me know if you need any more Infos or tests done from my end.

[Th3K1ngP1n]

I personally use llama3.1:8b deployed locally and do his job. As model on API the last that I tested was Gemini 2.5 pro.

Here the problem is that using something on API, maybe jumping on the latest and greatest model, ends up that something change and stop working. A lot of this model for example introduce the thinking / reasoning that broke the fact that we want only one title.

Anyway @rendyhd is the master and commander of the AI model, so I hope he can give a look helping in this.

Im also on this thinking pattern that smaller model makes more sense. The prompt to expected result doesnt need massive reasoning or thinking. I will stick with Mistral Mini for the time being but might try the self hosted variant you mentioned.

Openrouter of course has the obvious benefit of jumping around quickly to other models without much commitment.