Windows CPU+Vulkan build error IMPORTANT

[dhadialhda]

When compiling the Qwen2-5B-Instruct model using mlc_llm compile, the TVM compiler throws a type system error: IntImm supports only int or uint type, but bool was supplied.

[dhadialhda]

Environment Information
Operating System: Windows

Python Version: 3.12

Python Environment: Miniconda (env name: mlc-cpu)

MLC Packages:

mlc-ai-nightly-cpu: 0.20.dev562

mlc-llm-nightly-cpu: 0.20.dev94

Detected Devices:

vulkan:0, vulkan:1 (currently used)

cpu:0

Not found: cuda:0, rocm:0, metal:0, opencl:0

Steps to Reproduce
bash
python -m mlc_llm compile Qwen2___5-0___5B-Instruct-q4f16_1-MLC --output Qwen2___5-0___5B-Instruct-q4f16_1-MLC/lib
Complete Error Log
text
[2025-11-26 16:03:31] INFO auto_config.py:154: Found model type: qwen2. Use --model-type to override.
Compiling with arguments:
--config QWen2Config(hidden_act='silu', hidden_size=896, intermediate_size=4864, num_attention_heads=14, num_hidden_layers=24, num_key_value_heads=2, rms_norm_eps=1e-06, rope_theta=1000000.0, vocab_size=151936, tie_word_embeddings=True, context_window_size=32768, prefill_chunk_size=2048, tensor_parallel_shards=1, head_dim=64, dtype='float32', max_batch_size=80, kwargs={})
--quantization GroupQuantize(name='q4f16_1', kind='group-quant', group_size=32, quantize_dtype='int4', storage_dtype='uint32', model_dtype='float16', linear_weight_layout='NK', quantize_embedding=True, quantize_final_fc=True, num_elem_per_storage=8, num_storage_per_group=4, max_int_value=7, tensor_parallel_shards=0)
--model-type qwen2
--target {'kind': 'vulkan', 'tag': '', 'keys': ['vulkan', 'gpu'], 'host': {'kind': 'llvm', 'tag': '', 'keys': ['cpu'], 'mtriple': 'x86_64-pc-windows-msvc', 'mcpu': 'znver4'}, 'supports_int8': True, 'supports_int64': True, 'supports_float32': True, 'max_threads_per_block': 1024, 'max_shared_memory_per_block': 49152, 'thread_warp_size': 1, 'supports_8bit_buffer': True, 'supports_float16': True, 'supports_int16': True, 'supports_16bit_buffer': True, 'supports_storage_buffer_storage_class': True, 'supports_int32': True, 'max_num_threads': 256}
--opt flashinfer=0;cublas_gemm=0;faster_transformer=0;cudagraph=0;cutlass=0;ipc_allreduce_strategy=NONE
--system-lib-prefix ""
--output Qwen2___5-0___5B-Instruct-q4f16_1-MLC\lib
--overrides context_window_size=None;sliding_window_size=None;prefill_chunk_size=None;attention_sink_size=None;max_batch_size=None;tensor_parallel_shards=None;pipeline_parallel_stages=None;disaggregation=None

error: Check failed: (dtype.is_int() || dtype.is_uint()) is false: ValueError: IntImm supports only int or uint type, but bool was supplied.
--> C:\Users\yangl\miniconda3\envs\mlc-cpu\lib\site-packages\mlc_llm\op\batch_spec_verify.py:114:25
|
114 | done[0] = False
| ^^^^^^^^^^^^^^^
note: run with TVM_BACKTRACE=1 environment variable to display a backtrace.
[16:03:35] D:\a\package\package\tvm\src\relax\ir\block_builder.cc:64: Warning: BlockBuilder destroyed with remaining blocks!
Error Location
The error occurs in mlc_llm/op/batch_spec_verify.py at line 114:

python
done[0] = False # Attempting to assign a boolean value to a variable expecting integer type

[dhadialhda]

I have tried starting the service directly and switching models, but encountered the same error.

python -m mlc_llm serve Qwen2___5-0___5B-Instruct-q4f16_1-MLC
[2025-11-26 16:18:32] INFO auto_device.py:93: Not found device: cuda:0
[2025-11-26 16:18:36] INFO auto_device.py:93: Not found device: rocm:0
[2025-11-26 16:18:40] INFO auto_device.py:93: Not found device: metal:0
[2025-11-26 16:18:51] INFO auto_device.py:82: Found device: vulkan:0
[2025-11-26 16:18:51] INFO auto_device.py:82: Found device: vulkan:1
[2025-11-26 16:18:56] INFO auto_device.py:93: Not found device: opencl:0
[2025-11-26 16:19:01] INFO auto_device.py:82: Found device: cpu:0
[2025-11-26 16:19:01] INFO auto_device.py:36: Using device: vulkan:0
[2025-11-26 16:19:02] INFO jit.py:43: MLC_JIT_POLICY = ON. Can be one of: ON, OFF, REDO, READONLY
[2025-11-26 16:19:02] INFO jit.py:118: Compiling using commands below:
[2025-11-26 16:19:02] INFO jit.py:119: 'C:\Users\yangl\miniconda3\envs\mlc-cpu\python.exe' -m mlc_llm compile Qwen2___5-0___5B-Instruct-q4f16_1-MLC --opt 'flashinfer=1;cublas_gemm=1;faster_transformer=0;cudagraph=1;cutlass=1;ipc_allreduce_strategy=NONE' --overrides '' --device vulkan:0 --output 'C:\Users\yangl\AppData\Local\Temp\tmpgylwz4rh\lib.dll'
[2025-11-26 16:19:08] INFO auto_config.py:70: Found model configuration: Qwen2___5-0___5B-Instruct-q4f16_1-MLC\mlc-chat-config.json
[2025-11-26 16:19:08] INFO auto_target.py:91: Detecting target device: vulkan:0
[2025-11-26 16:19:08] INFO auto_target.py:93: Found target: {'kind': 'vulkan', 'tag': '', 'keys': ['vulkan', 'gpu'], 'supports_int8': True, 'supports_int64': True, 'supports_float32': True, 'max_threads_per_block': 1024, 'max_shared_memory_per_block': 49152, 'thread_warp_size': 1, 'supports_8bit_buffer': True, 'supports_float16': True, 'supports_int16': True, 'supports_16bit_buffer': True, 'supports_storage_buffer_storage_class': True, 'supports_int32': True, 'max_num_threads': 256}
[2025-11-26 16:19:08] INFO auto_target.py:110: Found host LLVM triple: x86_64-pc-windows-msvc
[2025-11-26 16:19:08] INFO auto_target.py:111: Found host LLVM CPU: znver4
[2025-11-26 16:19:08] INFO auto_config.py:154: Found model type: qwen2. Use --model-type to override.
Compiling with arguments:
--config QWen2Config(hidden_act='silu', hidden_size=896, intermediate_size=4864, num_attention_heads=14, num_hidden_layers=24, num_key_value_heads=2, rms_norm_eps=1e-06, rope_theta=1000000.0, vocab_size=151936, tie_word_embeddings=True, context_window_size=32768, prefill_chunk_size=2048, tensor_parallel_shards=1, head_dim=64, dtype='float32', max_batch_size=80, kwargs={})
--quantization GroupQuantize(name='q4f16_1', kind='group-quant', group_size=32, quantize_dtype='int4', storage_dtype='uint32', model_dtype='float16', linear_weight_layout='NK', quantize_embedding=True, quantize_final_fc=True, num_elem_per_storage=8, num_storage_per_group=4, max_int_value=7, tensor_parallel_shards=0)
--model-type qwen2
--target {'kind': 'vulkan', 'tag': '', 'keys': ['vulkan', 'gpu'], 'host': {'kind': 'llvm', 'tag': '', 'keys': ['cpu'], 'mtriple': 'x86_64-pc-windows-msvc', 'mcpu': 'znver4'}, 'supports_int8': True, 'supports_int64': True, 'supports_float32': True, 'max_threads_per_block': 1024, 'max_shared_memory_per_block': 49152, 'thread_warp_size': 1, 'supports_8bit_buffer': True, 'supports_float16': True, 'supports_int16': True, 'supports_16bit_buffer': True, 'supports_storage_buffer_storage_class': True, 'supports_int32': True, 'max_num_threads': 256}
--opt flashinfer=0;cublas_gemm=0;faster_transformer=0;cudagraph=0;cutlass=0;ipc_allreduce_strategy=NONE
--system-lib-prefix ""
--output C:\Users\yangl\AppData\Local\Temp\tmpgylwz4rh\lib.dll
--overrides context_window_size=None;sliding_window_size=None;prefill_chunk_size=None;attention_sink_size=None;max_batch_size=None;tensor_parallel_shards=None;pipeline_parallel_stages=None;disaggregation=None
[2025-11-26 16:19:08] INFO compile.py:140: Creating model from: QWen2Config(hidden_act='silu', hidden_size=896, intermediate_size=4864, num_attention_heads=14, num_hidden_layers=24, num_key_value_heads=2, rms_norm_eps=1e-06, rope_theta=1000000.0, vocab_size=151936, tie_word_embeddings=True, context_window_size=32768, prefill_chunk_size=2048, tensor_parallel_shards=1, head_dim=64, dtype='float32', max_batch_size=80, kwargs={})
[2025-11-26 16:19:09] INFO compile.py:158: Exporting the model to TVM compiler
[2025-11-26 16:19:09] INFO compile.py:164: Running optimizations using TVM
[2025-11-26 16:19:09] INFO compile.py:186: Registering metadata: {'model_type': 'qwen2', 'quantization': 'q4f16_1', 'context_window_size': 32768, 'sliding_window_size': -1, 'attention_sink_size': -1, 'prefill_chunk_size': 2048, 'tensor_parallel_shards': 1, 'pipeline_parallel_stages': 1, 'disaggregation': False, 'kv_state_kind': 'kv_cache', 'max_batch_size': 80}
error: Check failed: (dtype.is_int() || dtype.is_uint()) is false: ValueError: IntImm supports only int or uint type, but bool was supplied.
--> C:\Users\yangl\miniconda3\envs\mlc-cpu\lib\site-packages\mlc_llm\op\batch_spec_verify.py:114:25
|
114 | done[0] = False
| ^^^^^^^^^^^^^^^
note: run with TVM_BACKTRACE=1 environment variable to display a backtrace.
[16:19:13] D:\a\package\package\tvm\src\relax\ir\block_builder.cc:64: Warning: BlockBuilder destroyed with remaining blocks!
Traceback (most recent call last):
File "C:\Users\yangl\miniconda3\envs\mlc-cpu\lib\runpy.py", line 196, in _run_module_as_main
return run_code(code, main_globals, None,
File "C:\Users\yangl\miniconda3\envs\mlc-cpu\lib\runpy.py", line 86, in run_code
exec(code, run_globals)
File "C:\Users\yangl\miniconda3\envs\mlc-cpu\lib\site-packages\mlc_llm_main.py", line 69, in
main()
File "C:\Users\yangl\miniconda3\envs\mlc-cpu\lib\site-packages\mlc_llm_main.py", line 50, in main
cli.main(sys.argv[2:])
File "C:\Users\yangl\miniconda3\envs\mlc-cpu\lib\site-packages\mlc_llm\cli\serve.py", line 208, in main
serve(
File "C:\Users\yangl\miniconda3\envs\mlc-cpu\lib\site-packages\mlc_llm\interface\serve.py", line 57, in serve
async_engine = engine.AsyncMLCEngine(
File "C:\Users\yangl\miniconda3\envs\mlc-cpu\lib\site-packages\mlc_llm\serve\engine.py", line 895, in init
super().init(
File "C:\Users\yangl\miniconda3\envs\mlc-cpu\lib\site-packages\mlc_llm\serve\engine_base.py", line 590, in init
) = _process_model_args(models, device, engine_config)
File "C:\Users\yangl\miniconda3\envs\mlc-cpu\lib\site-packages\mlc_llm\serve\engine_base.py", line 171, in _process_model_args
model_args: List[Tuple[str, str]] = [_convert_model_info(model) for model in models]
File "C:\Users\yangl\miniconda3\envs\mlc-cpu\lib\site-packages\mlc_llm\serve\engine_base.py", line 171, in
model_args: List[Tuple[str, str]] = [_convert_model_info(model) for model in models]
File "C:\Users\yangl\miniconda3\envs\mlc-cpu\lib\site-packages\mlc_llm\serve\engine_base.py", line 164, in _convert_model_info
model_lib = jit.jit(
File "C:\Users\yangl\miniconda3\envs\mlc-cpu\lib\site-packages\mlc_llm\interface\jit.py", line 164, in jit
_run_jit(
File "C:\Users\yangl\miniconda3\envs\mlc-cpu\lib\site-packages\mlc_llm\interface\jit.py", line 124, in _run_jit
raise RuntimeError("Cannot find compilation output, compilation failed")
RuntimeError: Cannot find compilation output, compilation failed