MINOR: Cherry-pick KAFKA-19851: Delete dynamic configs that were removed by Kafka to 4.0 - #22160
Merged
Conversation
…e#21053) When upgrading from Kafka 3.x to 4.0, the metadata log may contain dynamic configurations that were removed in 4.0 (e.g., message.format.version per KIP-724). These removed configs cause InvalidConfigurationException when users attempt to modify any configuration, because validation checks all existing configs including the removed ones. Adds filtering to prevent unsupported or invalid configurations from being applied during metadata replay. The filtering is implemented using a SupportedConfigChecker interface that is injected via dependency injection through Builder patterns. When a ConfigRecord is replayed, the checker validates whether the configuration name is supported for the given resource type. Unsupported configurations are silently ignored during replay, ensuring that only valid configurations enter the in-memory state. The SupportedConfigChecker interface provides a default TRUE implementation that accepts all configurations. The actual filtering logic is implemented by DefaultSupportedConfigChecker, which maintains a whitelist of valid configuration names per resource type (TOPIC, CLIENT_METRICS, GROUP) based on the actual config definitions. The filtering occurs in both ConfigurationDelta#replay and ConfigurationControlManager#replay methods. Added unit tests to ensure: - Removed configurations are filtered during the replay operations - Only supported configurations appear in the resulting metadata images - The filtering works correctly for all resource types (TOPIC, BROKER, CLIENT_METRICS, GROUP) - DefaultSupportedConfigChecker correctly identifies supported vs unsupported configurations for each resource type Reviewers: José Armando García Sancio <jsancio@apache.org>, Jun Rao <junrao@apache.org>, Alyssa Huang <ahuang@confluent.io>, Kevin Wu <kevin.wu2412@gmail.com>, Andrew Grant <agrant@confluent.io> (cherry picked from commit a35d649) (cherry picked from commit 3be19e4) (cherry picked from commit 90ad891)
(cherry picked from commit 3289edc)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Cherry-pick #21053: Delete dynamic configs that were removed by Kafka to 4.0
In CoordinatoRuntimeTest: added
@SuppressWarnings({"checkstyle:ClassDataAbstractionCoupling", "checkstyle:ClassFanOutComplexity"})Because
MetadataDelta.Builderincreases class fan-out beyond the checkstyle threshold.Run tests in all related modules:
./gradlew :coordinator-common:test :core:test :group-coordinator:test :metadata:test :server:test :shell:testMost local tests passed, except a few flaky ones:
rerun the failed tests. Only 2 of them failed, the errors are:
java.util.concurrent.ExecutionException: java.net.SocketException: Connection resetjava.lang.AssertionError: receiveRequest timed out at kafka.network.SocketServerTest.receiveRequest(SocketServerTest.scala:152)unrelated to our changes: