KafkaAvroDeserializer should not require setting the version id. about registry

[michaelandrepearce]

Also it seems that if i write with a SpecificRecord, i must read with one, and like wise if i write with a GenericRecord, i must read with one.

This will cause issues, where an app produces with specific and another consumes without the specific (aka some generic integration) and vice-versa.

Should not serial/deserialize to Generic/Specific based on how an array is written, but based on if the app has a SpecificRecord on the CP or not. (similar to confluents)

    int recordType = payloadInputStream.read();
                LOG.debug("Received record type: [{}]", Integer.valueOf(recordType));
                Object datumReader = null;
                Schema readerSchema = readerSchemaVersion != null?(Schema)this.getSchema(new SchemaVersionKey(schemaName, readerSchemaVersion)):null;
                if(recordType == 0) {
                    datumReader = readerSchema != null?new GenericDatumReader(e, readerSchema):new GenericDatumReader(e);
                } else {
                    datumReader = readerSchema != null?new SpecificDatumReader(e, readerSchema):new SpecificDatumReader(e);
                }

[michaelandrepearce]

I have raised a PR to hopefully address this.

#115

it would be good if this could make next release (due soon i believe reading google groups), as this is blocking us currently.

[satishd]

@michaelandrepearce Which version id should be set for KafkaAvroDeserializer?

+1 to be flexible for consuming either specific record or generic record on deserializer side. We can target addressing this issue in upcoming release.

[michaelandrepearce]

For what you serialize into for specific it's redundant, as specefic it should take the schema of what's on concrete class on the class path.

For generic if not provided / present you simply don't evolve and you serialize into generic record using the inbound version.

For generic if provided then you can evolve as you serialize into generic record

This is a key point here.

[michaelandrepearce]

See the pr, This is all the same logic that's in the confluent serdes also.

[michaelandrepearce]

Logic to deal with if no readerSchema (aka no reader version id)

+    private DatumReader getDatumReader(Schema writerSchema, Schema readerSchema) {
 +        if(this.useSpecificAvroReader) {
 +            if(readerSchema == null) {
 +                readerSchema = this.getReaderSchema(writerSchema);
 +            }
 +
 +            return new SpecificDatumReader(writerSchema, readerSchema);
 +        } else {
 +            return readerSchema == null?new GenericDatumReader(writerSchema):new GenericDatumReader(writerSchema, readerSchema);
 +        }
 +    }