fix: engine cannot give response to the second user request with stre…

[lihaiyin88]

see title

[CharlieFRuan]

thanks a lot for the contribution. Would it be possible for you to provide a script for reproducing the issue / elaborate on the issue? Thank you!

[Copilot]

Pull Request Overview

This PR restructures the streaming generation method in MLCEngine to ensure the model lock is always released once, fixing an issue where the engine could not handle a second streaming request.

• Flattened multiple nested try/catch blocks into a single try + finally around the main logic.
• Converted inner helper functions (_countTrailingReplacementChar, _getChunk) to arrow function expressions.
• Removed redundant lock.release() calls spread across catches; now released once in finally.

Comments suppressed due to low confidence (2)

src/engine.ts:706

• This TODO is still open—either implement the usage support for non-chat completions or file a tracking issue to ensure it isn't overlooked.

          // TODO(Charlie): support usage for completion

src/engine.ts:483

• Add a test case that performs two consecutive streaming requests (with and without errors) to verify the lock is always released and reacquired correctly.

    genConfig: GenerationConfig,

[Copilot]

Consider adding a brief comment above this try/finally to clarify its scope is for lock acquisition and release, improving future readability.

Suggested change

	const lock = this.loadedModelIdToLock.get(model)!;
	const lock = this.loadedModelIdToLock.get(model)!;
	// Acquire the lock and ensure its release in the `finally` block.

[lihaiyin88]

thanks a lot for the contribution. Would it be possible for you to provide a script for reproducing the issue / elaborate on the issue? Thank you!

Hey there! So sorry for the super late reply! 😊
I've just pushed all the updates to the forked project right here: https://github.com/lihaiyin88/web-llm/tree/demo_for_issue
If you have any questions at all, don't hesitate to let me know! I'm all ears! 😎