[ET-VK][runtime] Add prepack cache to avoid duplicate weight prepacking by SS-JIA · Pull Request #18361 · pytorch/executorch

SS-JIA · 2026-03-20T06:14:19Z

Stack from ghstack (oldest at bottom):

-> [ET-VK][runtime] Add prepack cache to avoid duplicate weight prepacking #18361
[ET-VK][embedding] Enable embedding weight dedup with tied linear weights #18360

When embedding and linear ops share tied weights and both use the same
prepacking function (prepack_quantized_linear_weight), the weight gets
prepacked twice, wasting GPU memory. Add a cache to ComputeGraph keyed
by (input ValueRef, kernel name) that returns the already-prepacked
tensor on cache hit, avoiding the duplicate allocation.

Differential Revision: D97430801

When embedding and linear ops share tied weights and both use the same prepacking function (prepack_quantized_linear_weight), the weight gets prepacked twice, wasting GPU memory. Add a cache to ComputeGraph keyed by (input ValueRef, kernel name) that returns the already-prepacked tensor on cache hit, avoiding the duplicate allocation. Differential Revision: [D97430801](https://our.internmc.facebook.com/intern/diff/D97430801/) [ghstack-poisoned]

pytorch-bot · 2026-03-20T06:14:23Z

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18361

📄 Preview Python docs built from this PR

Note: Links to docs will display an error until the docs builds have been completed.

❌ 1 New Failure, 2 Unrelated Failures

As of commit fc166c7 with merge base 38b40bc ():

NEW FAILURE - The following job has failed:

pull / android / run-emulator (gh)
The process '/usr/bin/sh' failed with exit code 1

BROKEN TRUNK - The following jobs failed but were present on the merge base:

👉 Rebase onto the `viable/strict` branch to avoid these failures

pull / unittest / windows / windows-job (gh) (trunk failure)
##[error]The operation was canceled.
pull / unittest-editable / windows / windows-job (gh) (trunk failure)
##[error]The operation was canceled.

This comment was automatically generated by Dr. CI and updates every 15 minutes.

When embedding and linear ops share tied weights and both use the same prepacking function (prepack_quantized_linear_weight), the weight gets prepacked twice, wasting GPU memory. Add a cache to ComputeGraph keyed by (input ValueRef, kernel name) that returns the already-prepacked tensor on cache hit, avoiding the duplicate allocation. Differential Revision: [D97430801](https://our.internmc.facebook.com/intern/diff/D97430801/) ghstack-source-id: 355089157 Pull Request resolved: #18361

github-actions · 2026-03-20T06:15:28Z

This PR needs a `release notes:` label

If your change should be included in the release notes (i.e. would users of this library care about this change?), please use a label starting with release notes:. This helps us keep track and include your important work in the next release notes.

To add a label, you can comment to pytorchbot, for example
@pytorchbot label "release notes: none"

For more information, see
https://github.com/pytorch/pytorch/wiki/PyTorch-AutoLabel-Bot#why-categorize-for-release-notes-and-how-does-it-work.

…ht prepacking" When embedding and linear ops share tied weights and both use the same prepacking function (prepack_quantized_linear_weight), the weight gets prepacked twice, wasting GPU memory. Add a cache to ComputeGraph keyed by (input ValueRef, kernel name) that returns the already-prepacked tensor on cache hit, avoiding the duplicate allocation. Differential Revision: [D97430801](https://our.internmc.facebook.com/intern/diff/D97430801/) [ghstack-poisoned]

Pull Request resolved: #18361 When embedding and linear ops share tied weights and both use the same prepacking function (prepack_quantized_linear_weight), the weight gets prepacked twice, wasting GPU memory. Add a cache to ComputeGraph keyed by (input ValueRef, kernel name) that returns the already-prepacked tensor on cache hit, avoiding the duplicate allocation. ghstack-source-id: 355234968 @exported-using-ghexport Differential Revision: [D97430801](https://our.internmc.facebook.com/intern/diff/D97430801/)

…ht prepacking" When embedding and linear ops share tied weights and both use the same prepacking function (prepack_quantized_linear_weight), the weight gets prepacked twice, wasting GPU memory. Add a cache to ComputeGraph keyed by (input ValueRef, kernel name) that returns the already-prepacked tensor on cache hit, avoiding the duplicate allocation. Differential Revision: [D97430801](https://our.internmc.facebook.com/intern/diff/D97430801/) [ghstack-poisoned]

Pull Request resolved: #18361 When embedding and linear ops share tied weights and both use the same prepacking function (prepack_quantized_linear_weight), the weight gets prepacked twice, wasting GPU memory. Add a cache to ComputeGraph keyed by (input ValueRef, kernel name) that returns the already-prepacked tensor on cache hit, avoiding the duplicate allocation. ghstack-source-id: 355269010 @exported-using-ghexport Differential Revision: [D97430801](https://our.internmc.facebook.com/intern/diff/D97430801/)

…ht prepacking" When embedding and linear ops share tied weights and both use the same prepacking function (prepack_quantized_linear_weight), the weight gets prepacked twice, wasting GPU memory. Add a cache to ComputeGraph keyed by (input ValueRef, kernel name) that returns the already-prepacked tensor on cache hit, avoiding the duplicate allocation. Differential Revision: [D97430801](https://our.internmc.facebook.com/intern/diff/D97430801/) [ghstack-poisoned]

Pull Request resolved: #18361 When embedding and linear ops share tied weights and both use the same prepacking function (prepack_quantized_linear_weight), the weight gets prepacked twice, wasting GPU memory. Add a cache to ComputeGraph keyed by (input ValueRef, kernel name) that returns the already-prepacked tensor on cache hit, avoiding the duplicate allocation. ghstack-source-id: 355353466 @exported-using-ghexport Differential Revision: [D97430801](https://our.internmc.facebook.com/intern/diff/D97430801/)

…ht prepacking" When embedding and linear ops share tied weights and both use the same prepacking function (prepack_quantized_linear_weight), the weight gets prepacked twice, wasting GPU memory. Add a cache to ComputeGraph keyed by (input ValueRef, kernel name) that returns the already-prepacked tensor on cache hit, avoiding the duplicate allocation. Differential Revision: [D97430801](https://our.internmc.facebook.com/intern/diff/D97430801/) [ghstack-poisoned]

Pull Request resolved: #18361 When embedding and linear ops share tied weights and both use the same prepacking function (prepack_quantized_linear_weight), the weight gets prepacked twice, wasting GPU memory. Add a cache to ComputeGraph keyed by (input ValueRef, kernel name) that returns the already-prepacked tensor on cache hit, avoiding the duplicate allocation. ghstack-source-id: 355397958 @exported-using-ghexport Differential Revision: [D97430801](https://our.internmc.facebook.com/intern/diff/D97430801/)

…ng (#18390) Pull Request resolved: #18361 When embedding and linear ops share tied weights and both use the same prepacking function (prepack_quantized_linear_weight), the weight gets prepacked twice, wasting GPU memory. Add a cache to ComputeGraph keyed by (input ValueRef, kernel name) that returns the already-prepacked tensor on cache hit, avoiding the duplicate allocation. ghstack-source-id: 355397958 @exported-using-ghexport Differential Revision: [D97430801](https://our.internmc.facebook.com/intern/diff/D97430801/) Co-authored-by: ssjia <ssjia@devvm26340.ftw0.facebook.com>

SS-JIA mentioned this pull request Mar 20, 2026

[ET-VK][embedding] Enable embedding weight dedup with tied linear weights #18360

Merged

meta-cla bot added the CLA Signed This label is managed by the Facebook bot. Authors need to sign the CLA before a PR can be reviewed. label Mar 20, 2026

meta-codesync bot added fb-exported meta-exported labels Mar 20, 2026

manuelcandales approved these changes Mar 20, 2026

View reviewed changes

SS-JIA merged commit caf373f into gh/SS-JIA/499/base Mar 20, 2026
136 of 140 checks passed

SS-JIA deleted the gh/SS-JIA/499/head branch March 20, 2026 23:31

SS-JIA temporarily deployed to cherry-pick-bot March 20, 2026 23:31 — with GitHub Actions Inactive

SS-JIA mentioned this pull request Mar 20, 2026

[ET-VK][runtime] Add prepack cache to avoid duplicate weight prepacking #18390

Merged

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ET-VK][runtime] Add prepack cache to avoid duplicate weight prepacking#18361

[ET-VK][runtime] Add prepack cache to avoid duplicate weight prepacking#18361
SS-JIA merged 5 commits intogh/SS-JIA/499/basefrom
gh/SS-JIA/499/head

SS-JIA commented Mar 20, 2026 •

edited

Loading

Uh oh!

pytorch-bot bot commented Mar 20, 2026 •

edited

Loading

Uh oh!

github-actions bot commented Mar 20, 2026

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

Conversation

SS-JIA commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

pytorch-bot bot commented Mar 20, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

🔗 Helpful Links

🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/18361

❌ 1 New Failure, 2 Unrelated Failures

Uh oh!

github-actions bot commented Mar 20, 2026

This PR needs a release notes: label

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

SS-JIA commented Mar 20, 2026 •

edited

Loading

pytorch-bot bot commented Mar 20, 2026 •

edited

Loading

This PR needs a `release notes:` label