v2.0.0 release with support for Kafka 3.1.0

Breaking changes

• Change introduced in the format of metrics exported to AWS CloudWatch.
• Moved team name (ownership) to 'Buffalo' in Docker / Pierone. Images are now published to registry-write.opensource.zalan.do/buffalo instead of registry-write.opensource.zalan.do/machina.

Other Changes

• Added support for Kafka 3.1.0.
• Upgraded Docker base image to use Java 17.
• Upgraded all dependencies caused by migration from SBT (v0.13 to v1.6), and Scala (v2.11 to v2.13).

v1.3.0 release

What's Changed

• Allow to override root log level by @dtrsan in #81
• Allow configuration of AdminClient in KafkaSettings by @soceanainn in #83
• Add cloudwatch metric name whitelisting feature [Fixes #71] by @jaelee-branch in #85
• Refactor CloudWatch reporting by adding dedicated reporter class by @soceanainn in #92

Full Changelog: v1.2.0...v1.3.0

1.2.0 release

• #78 - Kafka client 2.2.1

HealthCheck

There will be a breaking change for the payload returned on the healthcheck. Please see the README

Kafka 2X

v1.0.7

readme update

Upgrades and Bug Fixes

Issues

• 45 - HTTP Gereral Counters
• 49 - Assembly errors due to dependency issues
• Upgrade to use kafka 1.1.1

DataDogIntegration

See readme for datadog integration.

Kafka1.0.0

Works with Kafka 1.0.0

Kafka0.10.2.1_1

The API need to change, the kafka clients are not backwards compatible and so it was best to just increase and fix up the code.

Show Active Consumers

http://localhost:9000/consumers

which gives

["consumer-1", "consumer-2", "consumer-3"]

Show specific consumer info

curl http://localhost:9000/consumers/<ConsumerGroupId>

State can be "Empty", "Dead", "Stable", "PreparingRebalance", "AwaitingSync"

{  
   "state":"Empty",
   "partition_assignment":[  
      {  
         "group":"console-consumer-20891",
         "coordinator":{  
            "id":0,
            "id_string":"0",
            "host":"foo.company.com",
            "port":9092
         },
         "topic":"products-in",
         "partition":1,
         "offset":3,
         "lag":0,
         "consumer_id":"-",
         "host":"-",
         "client_id":"-",
         "log_end_offset":3
      },
      {  
         "group":"console-consumer-20891",
         "coordinator":{  
            "id":0,
            "id_string":"0",
            "host":"foo.company.com",
            "port":9092
         },
         "topic":"products-in",
         "partition":0,
         "offset":3,
         "lag":0,
         "consumer_id":"consumer-1-7baba9b9-0ec3-4241-9433-f36255dd4708",
         "host":"/10.170.2.103",
         "client_id":"consumer-1",
         "log_end_offset":3
      }
   ]
}

Kafka0.10.0.1

basic application showing consumer offsets stored in kafka. Tested with kafka 0.10.0.1

Show Active Consumers

http://localhost:9000/consumers

which gives

[
    {
      "protocolType": "consumer",
      "groupId": "consumer-1"
    }
    {
      "protocolType": "consumer",
      "groupId": "consumer-2"
    }
    {
      "protocolType": "consumer",
      "groupId": "consumer-3"
    }
]

Show specific consumer info

curl http://localhost:9000/consumers/<ConsumerGroupId>

[
    {
      "owner": "consumer-2_/132.34.134.12",
      "lag": 155758,
      "log_end_offset": 2580124,
      "offset": 2424366,
      "partition": 1,
      "topic": "foobar",
      "group": "consumer-1"
    }
    {
      "owner": "consumer-2_/132.34.134.12",
      "lag": 155758,
      "log_end_offset": 2580124,
      "offset": 2424366,
      "partition": 2,
      "topic": "foobar",
      "group": "consumer-1"
    }
    {
      "owner": "consumer-2_/132.34.134.12",
      "lag": 155758,
      "log_end_offset": 2580124,
      "offset": 2424366,
      "partition": 3,
      "topic": "foobar",
      "group": "consumer-1"
    }
]