Skip to content

Counter metrics that have no new data in the current time window remain visible in 'state/v1/metrics' with 'count=0', instead of being dropped from the output #35612

@yuvalshimron

Description

@yuvalshimron

Expected Behavior

Per the Javadoc for MetricSet.partialClone():

"Aggregate metrics are currently not carried over and will not be present in the returned set."

Counter metrics (aggregate metrics) should not appear when there's no data in the current time window.

Actual Behavior

Counter metrics are padded back from DimensionCache and appear with count=0.

Root Cause Analysis

DimensionCache.padMetric() pads all cached metrics regardless of type:

https://github.com/vespa-engine/vespa/blob/master/container-core/src/main/java/com/yahoo/metrics/simple/DimensionCache.java#L89-L92

if ( ! toPresent.hasIdentifier(id)) {
    toPresent.put(id, leastOld.getValue().metric.pruneData());  // Pads all types
    --toAdd;
}

This happens upstream of MetricSet.partialClone(), so counters are included before the filtering logic can exclude them.

Suggested Resolution

Either:

  1. Fix the code - Skip counters in DimensionCache.padMetric():
if ( ! toPresent.hasIdentifier(id)) {
    if (!leastOld.getValue().metric.isCounter()) {
        toPresent.put(id, leastOld.getValue().metric.pruneData());
        --toAdd;
    }
}
  1. Fix the Javadoc - If this behavior is intentional, update MetricSet.partialClone() documentation to clarify that counters may appear with count=0 when padded from the dimension cache.

Vespa version

8.526.15

Thank you! :)

Metadata

Metadata

Assignees

Labels

No labels
No labels

Type

No type

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions