GithubHelp home page GithubHelp logo

Comments (6)

raychu86 avatar raychu86 commented on August 16, 2024 1

On the MAX_TRANSACTIONS front, there are 2 limiters:

  1. The protocol limit allows up to 2^20 transactions according to our merkle tree standard.
  2. The BFT worst case scenario formula is MAX_NUM_VALIDATORS * NUM_TRANSACTIONS_IN_PROPOSAL * MAX_GC_ROUNDS as defined by Transactions::<N>::MAX_ABORTED_TRANSACTIONS. In testnet, this value is 100 * 50 * 100 = 500,000. Which is already large enough that the problem still persists

I would say that this problem needs a more fundamental fix/avoidance than just lowering the size check.

from snarkos.

elderhammer avatar elderhammer commented on August 16, 2024

I think there are two problems to solve:

  1. Refer to the changes in #3304 and put the deserialization work into the rayon thread
  2. Appropriately reduce MAX_TRANSACTIONS (I remember that each certificate can only have a maximum of 50 transactions)

from snarkos.

vicsn avatar vicsn commented on August 16, 2024

@joske can you confirm if this is an issue (and the related PR resolves it?)

from snarkos.

joske avatar joske commented on August 16, 2024

I think #3304 should avoid the node getting stuck on big blocks. That said, there's still a deadlock in sync that we haven't found yet.

I don't know about the MAX_TRANSACTIONS

from snarkos.

elderhammer avatar elderhammer commented on August 16, 2024

I think #3304 should avoid the node getting stuck on big blocks. That said, there's still a deadlock in sync that we haven't found yet.

I don't know about the MAX_TRANSACTIONS

I've looked at change 3304, which is a change to node/router/src/inbound.rs, but the problem described there is within the BFT module:

Event::BlockResponse(block_response) => {
// If a sync sender was provided, then process the block response.
if let Some(sync_sender) = self.sync_sender.get() {
// Retrieve the block response.
let BlockResponse { request, blocks } = block_response;
// Perform the deferred non-blocking deserialization of the blocks.
let blocks = blocks.deserialize().await.map_err(|error| anyhow!("[BlockResponse] {error}"))?;
// Ensure the block response is well-formed.
blocks.ensure_response_is_well_formed(peer_ip, request.start_height, request.end_height)?;
// Send the blocks to the sync module.
if let Err(e) = sync_sender.advance_with_sync_blocks(peer_ip, blocks.0).await {
warn!("Unable to process block response from '{peer_ip}' - {e}");
}
}
Ok(())
}

The value of MAX_TRANSACTIONS is hardcoded to 1048575, which means that a fake BlockResponse may be filled with 1048575 transactions, which will make deserialization extremely time-consuming.

from snarkos.

joske avatar joske commented on August 16, 2024

it's probably a good idea to defer this deserialization to rayon in Gateway too, yes.

from snarkos.

Related Issues (20)

Recommend Projects

  • React photo React

    A declarative, efficient, and flexible JavaScript library for building user interfaces.

  • Vue.js photo Vue.js

    🖖 Vue.js is a progressive, incrementally-adoptable JavaScript framework for building UI on the web.

  • Typescript photo Typescript

    TypeScript is a superset of JavaScript that compiles to clean JavaScript output.

  • TensorFlow photo TensorFlow

    An Open Source Machine Learning Framework for Everyone

  • Django photo Django

    The Web framework for perfectionists with deadlines.

  • D3 photo D3

    Bring data to life with SVG, Canvas and HTML. 📊📈🎉

Recommend Topics

  • javascript

    JavaScript (JS) is a lightweight interpreted programming language with first-class functions.

  • web

    Some thing interesting about web. New door for the world.

  • server

    A server is a program made to process requests and deliver data to clients.

  • Machine learning

    Machine learning is a way of modeling and interpreting data that allows a piece of software to respond intelligently.

  • Game

    Some thing interesting about game, make everyone happy.

Recommend Org

  • Facebook photo Facebook

    We are working to build community through open source technology. NB: members must have two-factor auth.

  • Microsoft photo Microsoft

    Open source projects and samples from Microsoft.

  • Google photo Google

    Google ❤️ Open Source for everyone.

  • D3 photo D3

    Data-Driven Documents codes.