Comments (9)
I am for breaking up the way types are handled. For accumulo normalization, I think this is a great idea.
For things such as converting to strings for Json representation, we need another way. Currently the two are linked in the Types normalizers.
Unless these get broken up, I am not too comfortable in having an extra third party dependency simply to use the Types apis.
from mango.
Seems like one is for normalization and one is for lexicoding. I don't mind proposing a design change and breaking these up.
from mango.
Ok looking at type normalization, we have 5 main functions on a TypeNormalizer.
resolves -> the class it handles
getAlias -> short name for the class
asString -> readable representation (used only for json)
fromString -> inverse of asString
normalize -> lexigraphically sortable representation (used only for accumulo)
denomalize -> inverse of normalize
It seems to me that if we come up with a more generic name for asString/normalize and fromString/denormalize, we can have a common types interface that can back normalizers, encoders, serializers.
Maybe something along the lines of
resolves -> the class it handles
getAlias -> short name for the class
encode -> some string representation
decode -> inverse of encode.
Then have implementations for each type specified for serialization and another set for accumulo using Lexicoders or current normalization (or both).
from mango.
+1 Per IM conversation with Edward Wagner:
We should probably use bytes for the lexicographically sortable encoders and use strings for the pretty-print encoders. Probably could use generics to type the implementation of the encoder to string or byte array.
from mango.
Opened issue Issue #27, to discuss and track the changes to the types API.
from mango.
One issue that I have found using the lexicoders is more of a recipes problem. We will need to be more careful with how we construct rows.
Currently we will do something like longval + "\u0000" + longval. Then to parse it we simply do a string split. This logic doesn't work with the lexicoders as an encoded long will have "\u0000" in the encoded value if any byte did not have any set bits.
This simply means that we need to be more careful parsing rows. For example extract 8 bytes for the first long then another 8 for the next.
Just something to keep in mind, but I still want to include this functionality at some point.
from mango.
Yeah, but at the same time, it would also be more efficient to pull split the strings using known indexes when possible rather than having an algorithm (like StringUtils) need to do a possible O(n) search through the string.
from mango.
Now that these are actually included in accumulo, does this make sense to do anymore?
Especially in mango?
from mango.
Can we close this? I'm in agreement that I don't think we need it.
from mango.
Related Issues (20)
- IP classes should be caching the byteArray used for comparisons instead of the InetAddress HOT 3
- CloseableIterables needs a toList() or toCollection method HOT 2
- Host the javadocs for release versions
- Release mango 2.0 HOT 1
- Removed deprecated jackson serialization and deserialization classes
- Most of the criteria objects don't evaluate correctly
- Entity object needs to specify a serialVersionUuid HOT 1
- "Short Alias" for type encoders HOT 7
- AbstractAttributeStore should use the ArrayListMultiMap HOT 2
- Remove ComparableComparator
- Add a close/shutdown method to the Batcher interface that returns all of the unprocessed objects in the queue.
- RangeLeaf should not just extend AbstractKeyValueLeaf HOT 1
- Cleanup Criteria domain objects
- BigDecimalLexiEncoder has a bug with lexi encoding 0
- IP Type should be Public
- Invalid error message parameters HOT 1
- Replace Query and Criteria APIs with library support HOT 1
- Fix use of methods that have been deprecated in new guava versions. HOT 6
- Update to use Java 8
- Add encoders for Java 8's Instant time class
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from mango.