Add Python runtime end-to-end tests for XNNPACK on Linux#18703
Add Python runtime end-to-end tests for XNNPACK on Linux#18703Lidang-Jiang wants to merge 3 commits intopytorch:mainfrom
Conversation
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18703
Note: Links to docs will display an error until the docs builds have been completed. ✅ You can merge normally! (2 Unrelated Failures)As of commit 7e51933 with merge base ac68932 ( BROKEN TRUNK - The following jobs failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
|
@pytorchbot label "release notes: xnnpack" |
Export small models (add, linear, conv2d+relu, MLP, depthwise separable conv, classifier head) with XNNPACK delegation and verify correctness through the Python runtime (executorch.runtime.Runtime). This is the Linux XNNPACK portion of the Python runtime test gap described in the issue. Each test exports a model to .pte, loads it via Runtime, and asserts that the output matches PyTorch eager mode. Part of pytorch#11225 Signed-off-by: Lidang-Jiang <lidangjiang@gmail.com>
d27e33b to
931add3
Compare
|
@Lidang-Jiang Can you resolve the lint - https://github.com/pytorch/executorch/actions/runs/24114688479/job/71164744422?pr=18703? I can go ahead and merge once CI is green. I ended up closing the original issue, as it's old, but adding more test coverage here doesn't hurt. Thanks! |
- resolve the standard unittest collection failure - fix the UFMT import formatting in the XNNPACK runtime test Signed-off-by: Lidang-Jiang <lidangjiang@gmail.com>
|
@GregoryComer Fixed in e1b05d5. I moved the Linux-only XNNPACK Python runtime tests into |
Summary
Add Python runtime end-to-end tests for the XNNPACK backend on Linux — the first batch for the testing gap described in #11225.
This update keeps the Linux-only coverage under
runtime/test/test_runtime_xnnpack.pyso the standard unittest jobs collect it cleanly. Each test exports a small model with XNNPACK delegation, saves it as.pte, loads it viaexecutorch.runtime.Runtime, and compares the output against PyTorch eager mode.Models tested:
test_add— element-wise additiontest_linear— singleLinearlayer (fp32)test_conv2d_relu—Conv2d + ReLUtest_mlp— multi-layer perceptron (Linear -> ReLU -> Linear)test_depthwise_separable_conv— depthwise + pointwise conv (MobileNet building block)test_classifier_head—AdaptiveAvgPool2d + Flatten + Lineartest_conv_bn—Conv2d + BatchNorm2d + ReLU(skipped:FuseBatchNormPasscrashes; TODO to re-enable)Part of #11225 (Linux XNNPACK)
Before
After
Test plan
runtime/test/test_runtime_xnnpack.pyruntime/test/TARGETScoverage for the new Python runtime testlintrunner --skip MYPY runtime/test/test_runtime_xnnpack.pypython -m py_compile runtime/test/test_runtime_xnnpack.pytest_conv_bnremains skipped with TODO —FuseBatchNormPassstill has a known bug onSequential(Conv2d, BN)exportpython -m pytest runtime/test/test_runtime_xnnpack.py -vLocal blocker: this workstation does not currently have a fully provisioned ExecuTorch runtime test environment; upstream CI remains the source of truth for this suite.
cc @GregoryComer @digantdesai @cbilgin @jathu @larryliu0820