Schema registry configuration per documentation results in error.

[MartinJohannessen]

Error message

026-03-26T16:21:57,672Z INFO  io.axual.ksml.user.UserFunction      Registered function 'processor.anonymiser_melding(key:?, value:?) ==> confluent_avro:Struct'
2026-03-26T16:21:57,723Z ERROR io.axual.ksml.runner.KSMLRunner      KSML Stopping because of unhandled exception
2026-03-26T16:21:57,725Z ERROR io.axual.ksml.execution.FatalError   
FatalError
===========
Description: Missing required configuration "schema.registry.url" which has no default value.
Stack trace: 
  org.apache.kafka.common.config.ConfigDef::parseValue @ ConfigDef.java:534
  org.apache.kafka.common.config.ConfigDef::parse @ ConfigDef.java:524
  org.apache.kafka.common.config.AbstractConfig::<init> @ AbstractConfig.java:115
  org.apache.kafka.common.config.AbstractConfig::<init> @ AbstractConfig.java:133
  io.confluent.kafka.serializers.AbstractKafkaSchemaSerDeConfig::<init> @ AbstractKafkaSchemaSerDeConfig.java:452
  io.confluent.kafka.serializers.KafkaAvroSerializerConfig::<init> @ KafkaAvroSerializerConfig.java:56
  io.confluent.kafka.serializers.KafkaAvroSerializer::configure @ KafkaAvroSerializer.java:52
  io.axual.ksml.data.serde.DataObjectSerde::configure @ DataObjectSerde.java:101
  io.axual.ksml.data.notation.vendor.VendorNotation::serde @ VendorNotation.java:61
  io.axual.ksml.generator.StreamDataType::serde @ StreamDataType.java:50
  io.axual.ksml.generator.TopologyBuildContext::buildWrapper @ TopologyBuildContext.java:211
  io.axual.ksml.generator.TopologyBuildContext::getStreamWrapper @ TopologyBuildContext.java:186
  io.axual.ksml.generator.TopologyBuildContext::getStreamWrapper @ TopologyBuildContext.java:144
  io.axual.ksml.generator.TopologyBuildContext::getStreamWrapper @ TopologyBuildContext.java:114
  io.axual.ksml.TopologyGenerator::lambda$generate$6 @ TopologyGenerator.java:194
  com.google.common.collect.SingletonImmutableBiMap::forEach @ SingletonImmutableBiMap.java:69
  io.axual.ksml.TopologyGenerator::generate @ TopologyGenerator.java:190
  io.axual.ksml.TopologyGenerator::lambda$create$0 @ TopologyGenerator.java:83
  java.util.HashMap::forEach @ HashMap.java:1430
  io.axual.ksml.TopologyGenerator::create @ TopologyGenerator.java:75
  io.axual.ksml.runner.backend.KafkaStreamsRunner::<init> @ KafkaStreamsRunner.java:145
  io.axual.ksml.runner.backend.KafkaStreamsRunner::<init> @ KafkaStreamsRunner.java:123
  io.axual.ksml.runner.KSMLRunner::main @ KSMLRunner.java:296
===========

Above error caused: KSML topology error: Error in topology "anonymisering_pipeline": Missing required configuration "schema.registry.url" which has no default value.
Stack trace: 
  io.axual.ksml.TopologyGenerator

Steps to reproduce:

ksml-runner.yaml

ksml:
  definitions:
    processor: processor.yaml
  schemaRegistries:
    confluent:
      config:
        schema.registry.url: http://schema-registry:8080/apis/ccompat/v7
  notations:
    avro:
      type: confluent_avro
      schemaRegistry: confluent
      config:
        auto.register.schemas: true

processor.yaml

streams:
  # input topics
  melding:
    topic: Melding
    keyType: string
    valueType: avro
  # output topics
  anonymisert_melding:
    topic: Melding
    keyType: string
    valueType: avro

functions:  
  anonymiser_melding:
    type: valueTransformer
    code: |
      new_value = value
      # anonymisert = {
      #     "meldingId": value.get("meldingId"),
      # }
    expression: new_value
    resultType: avro

pipelines:
  anonymisering_pipeline:
    from: melding
    via:
      - type: transformValue
        mapper: anonymiser_melding
    to: anonymisert_melding

Images

registry.axual.io/opensource/images/axual/ksml:latest
registry.axual.io/opensource/images/axual/ksml:snapshot
registry.axual.io/opensource/images/axual/ksml:1.1.1

[jeroenvandisseldorp]

Thanks for reporting. Can you validate if this is still the case with v1.2.0? I think this was fixed recently.

[MartinJohannessen]

Yes, of course. Thanks for your quick response

2026-03-27T11:03:55,380Z INFO  io.axual.ksml.runner.KSMLRunner      Starting KSML Runner 1.2.0 (2026-03-25T10:39:44Z)
2026-03-27T11:03:55,658Z INFO  io.axual.ksml.runner.KSMLRunner      Using directories: config: /ksml, schema: /ksml, storage: /ksml
2026-03-27T11:03:55,659Z INFO  i.a.ksml.runner.config.KSMLConfig    Reading KSML definition from source file: /ksml/processor.yaml
2026-03-27T11:03:56,165Z INFO  i.a.k.r.backend.KafkaStreamsRunner   Constructing Kafka Backend
2026-03-27T11:03:56,227Z INFO  io.axual.ksml.TopologyGenerator      Starting processor definition: name=<anonymous topology>, version=<no version>, namespace=processor
2026-03-27T11:04:01,008Z INFO  io.axual.ksml.user.UserFunction      Registered function 'processor.anonymiser_melding(key:?, value:?) ==> confluent_avro:Struct'
2026-03-27T11:04:01,083Z ERROR io.axual.ksml.runner.KSMLRunner      KSML Stopping because of unhandled exception
2026-03-27T11:04:01,085Z ERROR io.axual.ksml.execution.FatalError   
FatalError
===========
Description: Missing required configuration "schema.registry.url" which has no default value.
Stack trace: 
  org.apac

[jeroenvandisseldorp]

Thanks for testing. I found the issue and will merge soon. In the meantime, the workaround is to add the schema registry url as notation config under auto.register.schemas: true. Please check if that works for you for now.

[MartinJohannessen]

It made no difference unfortunately.

What I ran:

ksml:
  definitions:
    processor: processor.yaml
  schemaRegistries:
    confluent:
      config:
        schema.registry.url: http://schema-registry:8080/apis/ccompat/v7
  notations:
    avro:
      type: confluent_avro
      schemaRegistry: confluent
      config:
        auto.register.schemas: true
        schema.registry.url: http://schema-registry:8080/apis/ccompat/v7

[jeroenvandisseldorp]

Oh and one more remark: when using AVRO, you need to specify an AVRO schema that your pipeline uses. So the stream definitions should be typed as avro:MySchema. KSML will load the schema from the configured schemaDirectory (in ksml-runner.yaml) and use its fields to (de)serialize Kafka messages.

Not referring to an AVRO schema is apparently a case that we do not test well, because otherwise we would have caught this, but is also is the direct cause of the bug you're hitting.

[MartinJohannessen]

In other words schema-registry isn't supported yet, and I must create the schemas manually(either directly or by creating a dto to generate from) and save the schemas locally? Schema-first approach drives me mad heh, my MO is to auto-register schemas generated from db tables. I prefer to never create schemas manually, to avoid small mistakes that can crash my connectors, so this isn't optimal.

[jeroenvandisseldorp]

You mean fetching schema from schema registry instead of loading from a local file? I understand that that would be preferable for your db use case, where a connector generates a schema dynamically. Should not be that hard to implement, but we need to consider the potential side effects of that. Currently a pipeline is supposed to know the data type in every single step. If we load the schema from sr dynamically, you can end up with different schema versions (or even completely different schema) in every pipeline step. Not sure if that would break anything. Let us think about this for a few days and come back to you on this.

[jeroenvandisseldorp]

As you can see in PR #493 we are working towards loading from SR. In the draft implementation, this is done once, upon process start, and before the first pipeline executes. Would that work for your use case?

[MartinJohannessen]

Thanks Jeroen, that would work for all of my foreseeable future use cases indeed! Sorry for slow response, been on vacation. Before I went however, I had to implement this specific use case of mine as a confluent dotnet app (for expedience, as I just needed to extend an existing program). So I only get to use KSML (with schema registry) next time I need a new transformation app, probably later this year.

[MartinJohannessen]

I was a little quick with my reply yesterday, sorry. What I will need for ksml to take over this part of my pipeline is to auto register schemas. Loading from schema registry is also a nice convenience, but secondary.

[jeroenvandisseldorp]

Can you test the new snapshot? This should make the AVRO schema be fetched from SR upon startup.

[jeroenvandisseldorp]

Hi Martin, were you able to confirm the functionality works as expected?