Comments (6)
Thanks for your suggestion. Actually we plan to make the implementation of buffer
pluggable, as adding a few bytes of meta in front of user data is disruptive (for those who want to avoid memcpy
).
from nuraft.
Thanks for your suggestion. Actually we plan to make the implementation of
buffer
pluggable, as adding a few bytes of meta in front of user data is disruptive (for those who want to avoidmemcpy
).
May I know when would this feature plan to be added as original one really hurts performance? Or are there any suggestions to minimize memcpy issue?
from nuraft.
@sheepgrass
Currently, we don't have ETA as it is supposed to be a significant overhaul of the code.
memcpy
issue exists only when your system uses its own buffer structure and tries to put the data in the buffer to Raft using append_entries(...)
API. In such a case, you should allocate a separate ptr<buffer>
, do memcpy
from your buffer to ptr<buffer>
, and then call Raft API with ptr<buffer>
, which is quite redundant.
One possible workaround is using ptr<buffer>
for your system as well, then you can directly put data to Raft without memcpy
call.
from nuraft.
@greensky00
Thanks for your reply. Actually, I came up with the workaround yesterday just similar to what you suggested. What I want to mention is that I want to minimize dynamic memory allocations as many as possible but using the provided buffer
class would force us to use alloc()
. To workaround this, I need to 'hack' my own buffer to have structure same as the provided buffer
. I think it's a common scenario having data in a different structure other than NuRaft buffer
.
And on a side note, if I got it right, calling append_entries()
is a must when using NuRaft and I found that there are many dynamic memory allocations which I want to avoid.
from nuraft.
'hack' my own buffer to have structure same as the provided
buffer
.
Yes, this is the same way as what we are doing internally here.
Maybe we can provide a new overloaded append_entries()
function, which accepts the general memory pointer and its size as parameters, and reduces the number of internal memory allocations. It will be a much smaller work than supporting generic/pluggable buffers.
from nuraft.
Maybe we can provide a new overloaded
append_entries()
function, which accepts the general memory pointer and its size as parameters
Yes, it can help if such function can be provided. Beware that not to memcpy
inside that append_entries()
otherwise there is no point to have such function. (I think such approach is moving the 'hack' from outside of append_entries()
to inside)
Also, be aware of the handling of log_entry
inside append_entries()
which may cause dynamic memory allocations and cause difficult to implement the new approach.
from nuraft.
Related Issues (20)
- nuraft 2.0.0 cannot work on centos7 + centos8 HOT 2
- Preconditions of apply_pack HOT 1
- handle_append_entries_resp() declined append HOT 1
- How to use NuRaft with CMake's FetchContent? HOT 12
- CMake targets should be namespaced using ALIAS targets
- CMake targets are missing usage requirements
- Leadership yielding is not synchronized with replicated log HOT 1
- Should the type be uint8_t instead of size_t for serialize_v1p(...) in srv_state.hxx line 133 HOT 1
- This is a question, not an issue. About `state_machine::pre_commit` HOT 2
- Does nuraft support linearizable read if generating no raft log entry of read requests? HOT 2
- Out of order call to state_machine::create_snapshot() when manually triggering a snapshot HOT 4
- Found a MSan error in Asio HOT 4
- Data race in peer HOT 3
- auto_fwd_resp_handler should forward resp->get_result_code
- src/tracer.hxx:52 stack-buffer-overflow when using vsnprintf's return value
- cmd_result::get_result_str missing `SERVER_IS_LEAVING`
- The semantics of the nuraft::snapshot HOT 3
- Merge two clusters into one HOT 1
- Misleading append_entries result code HOT 3
- rpc_listener HOT 1
Recommend Projects
-
React
A declarative, efficient, and flexible JavaScript library for building user interfaces.
-
Vue.js
🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.
-
Typescript
TypeScript is a superset of JavaScript that compiles to clean JavaScript output.
-
TensorFlow
An Open Source Machine Learning Framework for Everyone
-
Django
The Web framework for perfectionists with deadlines.
-
Laravel
A PHP framework for web artisans
-
D3
Bring data to life with SVG, Canvas and HTML. 📊📈🎉
-
Recommend Topics
-
javascript
JavaScript (JS) is a lightweight interpreted programming language with first-class functions.
-
web
Some thing interesting about web. New door for the world.
-
server
A server is a program made to process requests and deliver data to clients.
-
Machine learning
Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.
-
Visualization
Some thing interesting about visualization, use data art
-
Game
Some thing interesting about game, make everyone happy.
Recommend Org
-
Facebook
We are working to build community through open source technology. NB: members must have two-factor auth.
-
Microsoft
Open source projects and samples from Microsoft.
-
Google
Google ❤️ Open Source for everyone.
-
Alibaba
Alibaba Open Source for everyone
-
D3
Data-Driven Documents codes.
-
Tencent
China tencent open source team.
from nuraft.