Make type encoders and registry Serializable about mango

[cjnolet]

What was the expected implementation / transport for this? The Accumulo side is easy with their VFS /HDFS classloader, but what about Storm?

[eawagner]

Sorry, I don't know how, but I accidentally left off the Serialization in the title.

I mean the simplest implementation is to assume the classes are available on the classpath and use standard java serialization. Basically just some way to make sure that a customized TypeRegistry can be shared with an iterator or a bolt. I don't really want to solve the problem of distribution in mango, just a means to make it possible easily.

[cjnolet]

Ah, gotcha. That makes more sense. I didn't understand the purpose.

On Sat, May 17, 2014 at 9:38 PM, eawagner [email protected] wrote:

Sorry, I don't know how, but I accidentally left off the Serialization in
the title.

I mean the simplest implementation is to assume the classes are available
on the classpath and use standard java serialization. Basically just some
way to make sure that a customized TypeRegistry can be shared with an
iterator or a bolt. I don't really want to solve the problem of
distribution in mango, just a means to make it possible easily.

—
Reply to this email directly or view it on GitHubhttps://github.com//issues/58#issuecomment-43429004
.

[cjnolet]

+1 I realized I could probably use this in the input formats & pig load/store funcs

[cjnolet]

I think it would also do us justice to go as far as coding the type encoders to a serializable. That way nobody can make the mistake of having an unserializable value they are trying to encode. I made this mistake today with the EntityRelationship class as I was integrating the Calrissian stack into my day project

[eawagner]

I don't know if that is a good idea to require the encode type to be serializable.

1. Currently you can use the TypeEncoder interface to create any type of object. Right now, we use it mostly for serializing and normalizing data to Strings. This does not mean that you need to use these for serialization.
2. Secondly, this interface allows beans that you don't control to actually be serialized if you want to use it for that. This way if you can't make a class serializable, you can still create an encoder for it.

[cjnolet]

Fair enough. Application layer devs could always force that.