fix(quantization): add missing constexpr and static_assert in INT8Quantizer::ComputeImpl by LHT129 · Pull Request #1719 · antgroup/vsag

LHT129 · 2026-03-19T06:12:05Z

Summary

This PR fixes undefined behavior in INT8Quantizer::ComputeImpl method caused by a missing return statement in the L2SQR branch.

Changes

Added static_assert to ensure only supported metric types (IP, COSINE, L2SQR) are used
Changed else if (metric == MetricType::METRIC_TYPE_L2SQR) to else if constexpr (metric == MetricType::METRIC_TYPE_L2SQR)
Ensured all branches use if constexpr for compile-time branch elimination
Eliminated undefined behavior when control flow reaches end of non-void function

Technical Details

Problem:
The ComputeImpl method had a code path that could lead to undefined behavior:

IP and COSINE branches used if constexpr (compile-time branch elimination)
L2SQR branch used regular if (no constexpr), so compiler couldn't eliminate fall-through
Missing return statement at end of function → undefined behavior for non-void function

Solution:

Added static_assert to validate metric type at compile time
Added constexpr to L2SQR branch for consistency with other branches

Testing

All INT8Quantizer tests passed (30708 assertions in 3 test cases)
All INT8 SIMD tests passed (12800 assertions in 3 test cases)
Release build successful
Code formatted with clang-format-15

Files Changed

src/quantization/int8_quantizer.cpp: Added static_assert and constexpr to ComputeImpl method

Related Issues

Fixes [Bug] Fix undefined behavior in INT8Quantizer::ComputeImpl due to missing return statement #1718

Checklist

Code follows VSAG coding style
All tests pass
No undefined behavior in the function
Compile-time branch elimination works correctly for all metric types

gemini-code-assist · 2026-03-19T06:12:18Z

Summary of Changes

Hello, I'm Gemini Code Assist¹! I'm currently reviewing this pull request and will post my feedback shortly. In the meantime, here's a summary to help you and other reviewers quickly get up to speed!

This pull request addresses a critical issue in the INT8Quantizer::ComputeImpl method where a specific code path could lead to undefined behavior due to a missing return statement. The changes ensure that all branches are correctly handled with if constexpr for compile-time optimization and introduce a static_assert to validate metric types, significantly enhancing the robustness and correctness of the quantization logic.

Highlights

Undefined Behavior Fix: Eliminated undefined behavior in INT8Quantizer::ComputeImpl by ensuring all code paths return a value, specifically addressing a missing return in the L2SQR branch.
Compile-time Branch Elimination: Converted the else if (metric == MetricType::METRIC_TYPE_L2SQR) condition to else if constexpr to ensure all metric type branches are evaluated at compile time, improving consistency and preventing runtime issues.
Metric Type Validation: Added a static_assert to INT8Quantizer::ComputeImpl to validate that only supported metric types (IP, COSINE, L2SQR) are used at compile time.

🧠 New Feature in Public Preview: You can now enable Memory to help Gemini Code Assist learn from your team's feedback. This makes future code reviews more consistent and personalized to your project's style. Click here to enable Memory in your admin console.

Using Gemini Code Assist

The full guide for Gemini Code Assist can be found on our documentation page, here are some quick tips.

Invoking Gemini

You can request assistance from Gemini at any point by creating a comment using either /gemini <command> or @gemini-code-assist <command>. Below is a summary of the supported commands on the current page.

Feature	Command	Description
Code Review	`/gemini review`	Performs a code review for the current pull request in its current state.
Pull Request Summary	`/gemini summary`	Provides a summary of the current pull request in its current state.
Comment	@gemini-code-assist	Responds in comments when explicitly tagged, both in pull request comments and review comments.
Help	`/gemini help`	Displays a list of available commands.

Customization

To customize Gemini Code Assist for GitHub experience, repository maintainers can create a configuration file and/or provide a custom code review style guide (such as PEP-8 for Python) by creating and adding files to a .gemini/ folder in the base of the repository. Detailed instructions can be found here.

Limitations & Feedback

Gemini Code Assist may make mistakes. Please leave feedback on any instances where its feedback is incorrect or counter productive. You can react with 👍 and 👎 on @gemini-code-assist comments. If you're interested in giving your feedback about your experience with Gemini Code Assist for GitHub and other Google products, sign up here.

Review the Privacy Notices, Generative AI Prohibited Use Policy, Terms of Service, and learn how to configure Gemini Code Assist in GitHub here. Gemini can make mistakes, so double check it and use code with caution. ↩

gemini-code-assist

Code Review

The pull request effectively resolves undefined behavior in the INT8Quantizer::ComputeImpl method by ensuring all code paths return a value. The addition of a static_assert provides compile-time validation for supported metric types, and the consistent use of if constexpr aids in compile-time branch elimination, improving code robustness and efficiency.

gemini-code-assist · 2026-03-19T06:14:12Z

src/quantization/int8_quantizer.cpp

        similarity /= mold1[0] * mold2[0];
        return 1.0F - std::max(-1.0F, std::min(1.0F, similarity));
-    } else if (metric == MetricType::METRIC_TYPE_L2SQR) {
+    } else if constexpr (metric == MetricType::METRIC_TYPE_L2SQR) {


Given the static_assert at the beginning of the ComputeImpl function (lines 109-112) that guarantees metric will be one of METRIC_TYPE_IP, METRIC_TYPE_COSINE, or METRIC_TYPE_L2SQR, this else if constexpr can be simplified to a plain else statement. This would logically indicate that this branch handles the only remaining valid metric type after the previous if constexpr conditions have been evaluated.

Suggested change

} else if constexpr (metric == MetricType::METRIC_TYPE_L2SQR) {

} else {

Fixed in commit c7fe425: simplified else if constexpr to else as suggested. Thanks!

Copilot

Pull request overview

This PR hardens INT8Quantizer::ComputeImpl against unsupported template instantiations and ensures compile-time branch selection for all supported metrics, eliminating the possibility of reaching the end of a non-void function without returning.

Changes:

Added a static_assert to restrict INT8Quantizer<metric> to supported metrics (IP, COSINE, L2SQR).
Converted the L2SQR branch to else if constexpr so all metric branches are compile-time selected.
Ensured ComputeImpl always returns for the explicitly instantiated metric specializations.

…ntizer::ComputeImpl - Add static_assert to ensure only supported metric types are used - Change else if to else for the final L2SQR branch - This ensures compile-time checking and removes unreachable code Signed-off-by: LHT129 <tianlan.lht@antgroup.com>

Copilot

Pull request overview

Fixes a potential undefined-behavior path in INT8Quantizer::ComputeImpl by enforcing compile-time metric constraints and ensuring all control-flow paths return a value.

Changes:

Added a static_assert to restrict INT8Quantizer instantiations to supported metrics (IP/COSINE/L2SQR).
Adjusted branch structure so the L2SQR path is handled by the final else branch, ensuring a return on all paths.

Copilot · 2026-03-20T06:05:28Z

src/quantization/int8_quantizer.cpp

+    static_assert(metric == MetricType::METRIC_TYPE_IP ||
+                      metric == MetricType::METRIC_TYPE_COSINE ||
+                      metric == MetricType::METRIC_TYPE_L2SQR,
+                  "Unsupported metric type for INT8Quantizer");
    if constexpr (metric == MetricType::METRIC_TYPE_IP) {


The PR description says the code was changed to else if constexpr (metric == MetricType::METRIC_TYPE_L2SQR), but the diff shows the L2SQR path is now handled via a plain else. Either update the PR description to match the implementation, or change the code to use else if constexpr as described.

Copilot · 2026-03-20T06:05:28Z

src/quantization/int8_quantizer.cpp

        similarity /= mold1[0] * mold2[0];
        return 1.0F - std::max(-1.0F, std::min(1.0F, similarity));
-    } else if (metric == MetricType::METRIC_TYPE_L2SQR) {
+    } else {


Using a bare else makes it less explicit that this branch is specifically for METRIC_TYPE_L2SQR (the reader has to infer it from the static_assert plus earlier branches). Consider making this branch an explicit else if constexpr (metric == MetricType::METRIC_TYPE_L2SQR) for self-documentation, while keeping the static_assert for compile-time enforcement.

Suggested change

} else {

} else if constexpr (metric == MetricType::METRIC_TYPE_L2SQR) {

Copilot AI review requested due to automatic review settings March 19, 2026 06:12

LHT129 requested review from inabao, jiaweizone and wxyucs as code owners March 19, 2026 06:12

LHT129 self-assigned this Mar 19, 2026

pull-request-size bot added the size/XS label Mar 19, 2026

LHT129 added kind/bug Something isn't working version/1.0 labels Mar 19, 2026

gemini-code-assist bot reviewed Mar 19, 2026

View reviewed changes

Copilot AI reviewed Mar 19, 2026

View reviewed changes

Copilot AI review requested due to automatic review settings March 20, 2026 05:59

LHT129 force-pushed the 2026-03-18-修复-int8-quantizer-computeimpl-缺少-return branch from c7fe425 to dd74a5b Compare March 20, 2026 05:59

Copilot AI reviewed Mar 20, 2026

View reviewed changes

Copilot started reviewing on behalf of LHT129 March 20, 2026 06:38 View session

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

fix(quantization): add missing constexpr and static_assert in INT8Quantizer::ComputeImpl#1719

Hidden character warning

fix(quantization): add missing constexpr and static_assert in INT8Quantizer::ComputeImpl#1719
LHT129 wants to merge 1 commit intoantgroup:mainfrom
LHT129:2026-03-18-修复-int8-quantizer-computeimpl-缺少-return

LHT129 commented Mar 19, 2026

Uh oh!

gemini-code-assist bot commented Mar 19, 2026

Uh oh!

gemini-code-assist bot left a comment

Uh oh!

gemini-code-assist bot Mar 19, 2026

Uh oh!

LHT129 Mar 19, 2026

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI left a comment

Uh oh!

Copilot AI Mar 20, 2026

Uh oh!

Copilot AI Mar 20, 2026

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants

	} else if constexpr (metric == MetricType::METRIC_TYPE_L2SQR) {
	} else {

Conversation

LHT129 commented Mar 19, 2026

Summary

Changes

Technical Details

Testing

Files Changed

Related Issues

Checklist

Uh oh!

gemini-code-assist bot commented Mar 19, 2026

Summary of Changes

Highlights

Footnotes

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

gemini-code-assist bot Mar 19, 2026

Choose a reason for hiding this comment

Uh oh!

LHT129 Mar 19, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Copilot AI Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

Copilot AI Mar 20, 2026

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

2 participants