Skip to content

chore: remove span-metrics leftovers and lazy-init generator clients#6618

Open
javiermolinar wants to merge 10 commits intografana:mainfrom
javiermolinar:chore/model-driven-generator-routing
Open

chore: remove span-metrics leftovers and lazy-init generator clients#6618
javiermolinar wants to merge 10 commits intografana:mainfrom
javiermolinar:chore/model-driven-generator-routing

Conversation

@javiermolinar
Copy link
Contributor

@javiermolinar javiermolinar commented Mar 5, 2026

What this PR does:

Now that we have decoupled the livestores from the MetricsGenerator service we can now complete the cleanup. This pr does:

  • Metrics generator gprc path is removed
  • Simplifies the initialization, SingleBinary mode use a simple callback for pushing traces to the Metrics generator
  • Deprecate unnecesary configs
  • Remove all query-path from generator and proto code
  • Lazy-init generator clients in the distributor. Only for single binary mode.

How this has been tested:

  • Compiled locally and ran single-binary and distributed docker compose modes. As expected the single binary do no longer connect to Kafka for the distributor

Which issue(s) this PR fixes:
Fixes #

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

t.cfg.Distributor.KafkaWritePathEnabled = t.cfg.Ingest.Enabled // TODO: Don't mix config params
t.cfg.Distributor.KafkaWritePathEnabled = false
t.cfg.Distributor.PushSpansToKafka = true
t.cfg.Distributor.PushSpansToGenerator = singleBinary
Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is necessary until we implement the LiveStore grpc path

@javiermolinar
Copy link
Contributor Author

@codex review

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 552bd48166

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

t.cfg.Distributor.KafkaConfig = t.cfg.Ingest.Kafka
t.cfg.Distributor.IngesterWritePathEnabled = false
t.cfg.Distributor.KafkaWritePathEnabled = t.cfg.Ingest.Enabled // TODO: Don't mix config params
t.cfg.Distributor.PushSpansToKafka = true

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Gate Kafka write-path enablement on actual ingest configuration

initDistributor now forces PushSpansToKafka = true for every target, which makes distributor.New() always validate Kafka settings and fail when ingest.kafka.topic is unset. With the current defaults (ingest.enabled: false, empty topic), this turns the default all target into a startup error path (ErrMissingKafkaTopic) even for configs that previously booted without Kafka; the test change to BackendScheduler masks this regression rather than fixing it. Please only enable Kafka routing when the deployment/config is actually Kafka-backed (or provide a non-empty default topic).

Useful? React with 👍 / 👎.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Kafka is the only deployment model now unless we are in singlebinary mode. So this is totally right

return nil, err
}

// In single-binary mode we also push directly to metrics-generator over gRPC,
Copy link
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

If this functionality is only for monolithic, let's avoid gRPC and push directly via a func

@javiermolinar
Copy link
Contributor Author

@codex review

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: f34eb0b7f2

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment on lines 247 to +249
t.cfg.Distributor.KafkaConfig = t.cfg.Ingest.Kafka
t.cfg.Distributor.IngesterWritePathEnabled = false
t.cfg.Distributor.KafkaWritePathEnabled = t.cfg.Ingest.Enabled // TODO: Don't mix config params
t.cfg.Distributor.PushSpansToKafka = true

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Disable deprecated ingester write path toggle

initDistributor now enables Kafka writes but no longer clears distributor.ingester_write_path_enabled, so legacy configs that still set this deprecated flag to true will re-enable sendToIngestersViaBytes in Distributor.PushTraces while Kafka remains active. In current app wiring this means each push can be sent down two write paths (or fail if the live-store endpoint doesn’t serve PushBytesV2), causing duplicate ingestion or request failures for existing deployments carrying old config.

Useful? React with 👍 / 👎.

Copy link
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

There are no ingesters anymore

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants