java.util.NoSuchElementException: key not found: schema.name with TOPIC_NAME strategy about abris

[cerveada]

When the schema is generated automatically, these values are used for schema name and namespace.

I guess Abris could provide some default value, so you doesn't have to provide those.

For now you can just provide those values in the configuration.

[tonyclyde]

That makes sense, thanks!

Then Im probably just not doing things right. Im trying to use the existing <topic>-key and <topic>-value schemas that are already in the registry for the given topic but cant seem to find the right configuration to do so. I tried setting SchemaManager.PARAM_SCHEMA_NAME_FOR_RECORD_STRATEGY to key and value respectively (which i think is the confluent default) but that gives me these errors:

java.security.InvalidParameterException: Schema could not be registered for topic '<topic>'. Make sure that the Schema Registry is available, the parameters are correct and the schemas ar compatible

Any guidance appreciated. Thanks.

Guess for now I probably just need to make sure that the schema Im publishing with really matches the one thats there....

[cerveada]

The idea was that when you write avro you also write the schema to registry and when you read avro you read the schema from there.

So to_confluent_avro will generate the schema and try to register it, the generated and the schema already in register are not compatible and that's the reason why it fails.

[cerveada]

I would try to load the schema first AvroSchemaUtils.loadForValue(schemaRegistryConf) and then call to_confluent_avro with the provided schema.

[tonyclyde]

Thanks for your help, this seems to mostly work but then Im back to being blocked by #58

to_confluent_avro(
  col("key"),
  AvroSchemaUtils.loadForKey(keyRegistryConfig).toString,
  keyRegistryConfig
)

For now Im just falling back on serializing and sending my messages manually via the KafkaProducer but will come back to this once thats resolved.

Thanks! (feel free to close this one)