Comments (3)
I'm actually more concern about being able to read the bytes data as direct ByteBuffer. In Java, it's getting to a low latency era that there are more serialization/encoding frameworks (Google FlatBuffers and Simple Binary Encoding) that keep data in the original ByteBuffer format.
from aerospike-client-java.
SocketChannel.write(ByteBuffer) is currrently only used for asynchronous commands (AsyncClient). The byte manipulation, however, is still performed using byte[]. This byte[] is copied to a direct ByteBuffer just before socket channel write(s). The byte[] is also copied from the direct ByteBuffer after socket channel read(s). Performance is actually faster than using direct ByteBuffer throughout. See this link which was verified by our own internal testing:
http://www.evanjones.ca/software/java-bytebuffers.html
For synchronous commands (AerospikeClient), direct ByteBuffers are not used at all. We could have employed the same byte[]/ByteBuffer combination, but it has not been a priority because the old socket paradigm performs reasonably well and is less likely to encounter memory fragmentation issues than direct ByteBuffer implementations.
from aerospike-client-java.
Notice that I'm not suggesting to use direct ByteBuffer because it is always faster, nor I am suggesting to replace the byte[] stuff with ByteBuffer. For normal cases that user read/write data from/to AeroSpike in the JVM only, the current implementation is fine. I'm only saying it should have a ByteBuffer interface (and for sure internally it won't turn it back to byte[]), for the cases the input data is a direct ByteBuffer already, there shouldn't be unnecessary copying, nor the data should get into the JVM heap at all.
Your link suggests "direct ByteBuffers should be the most efficient form of I/O, but it depends on how your data gets into the buffers.". If one write bytes to direct ByteBuffer like one of the tests it made, ByteBuffer won't be meaningful benefit as it shown. My scenario is I get data from the network (with Netty) in direct ByteBuffer already. And I don't really need to decode the ByteBuffer to Java primitive types or byte array for any processing, only need to store those bytes in buffer into AeroSpike. Even if I use the AeroSpike async API, first it requires me to turn my direct ByteBuffer to byte[] Bin, and then internally it seems copy the bytes around (refer to AsyncWrite, Command) before it get to the direct buffer in the AsyncCluster.
allocating byte[], copying stuff around maybe fine for many case. But in fields that require low latency, people nowadays design Java application to use more off heap storage and do stuff minimize GC. e.g. Huge Collections to store stuff off-heap, Disruptor when one use byte[], it allows a fixed number of byte[] in a Ring Buffer to avoid GC; Simple Binary Encoding as mentioned above basically provide a view over ByteBuffer rather than moving the bytes to a Java POJO to remove those serialization/deserialization cost. AeroSpike as a scalable and low latency storage system fit the need of such solutions, but the current Java client implement is fast but not the most optimized for low latency application.
I'll look at the AsyncClient and see if I can add an interface that take a ByteBuffer and have it pass to the AsyncCluster without turning to byte[].
from aerospike-client-java.
Related Issues (20)
- Streaming support for batch operations HOT 2
- rename Record to AerospikeRecord HOT 1
- View screenshot link is broken HOT 1
- Type Erasure in 5.1.11 HOT 7
- Fully support surrogates symbol in Strings HOT 5
- Wrong condition for aeropsike client connectivity HOT 6
- Add RetryListener HOT 7
- Security contact? HOT 2
- Aerospike Cluster Peer Invalid Node Docker HOT 3
- Node name is null HOT 1
- Error 20: Partition map empty ( Issue for java client ) HOT 8
- Add support for findBySinceUpdate using since_update in metadata HOT 2
- What is the difference between `BatchRecordArrayListener` and `RecordArrayListener`? HOT 2
- How to make async queries faster? HOT 3
- java.lang.NullPointerException in AerospikeClient.put() HOT 4
- Setters for Spring HOT 2
- Enable releases in github HOT 4
- Problem reading Date type with client version 7.X HOT 2
- When compiling lua, the client discards the exception which has the lua compilation error details. HOT 2
- Add listener for node addition / deletion HOT 4
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from aerospike-client-java.