Discussion about "Fast-track for WindowPoSt" about fips

[anorth]

I understand the problem to be that a WindowPoSt message has some level of urgency, and it critical to broad network functionality. We need (valid) WindowPoSt messages to be selected and included by a block producers, even when there is message congestion, and not to be priced out by the auction for block gas limit.

The implicit BaseFee and message GasPremium are price per gas unit, and bounded by the message's GasFeeCap (also price per unit). Miners will prefer messages with a high GasPremium. In times of congestion, the BaseFee will increase, possibly past a message's GasFeeCap, at which point the message is ineligible for inclusion.

In order to be preferred for inclusion in blocks even during congestion, Window PoSt messages must carry a very high GasFeeCap and GasPremium. This is expensive in real terms for the sender, as this high price is multiplied by the GasLimit (in units) required for a WindowPoSt.

Proposal

Set the gas usage and limit of WindowPoSt messages to be very small (even 1 unit), so that miners can specify a very high price in GasFeeCap and GasPremium but still enjoy very low cost, because the price is multiplied by ~zero. Then profit-maximising block producers will always include these message. As BaseFee rises, WindowPoSt messages will be increasingly preferred for their high relative profit per gas unit; the system becomes more effective at prioritising PoSt when congested.

We can achieve this by setting the VerifyPoSt() gas fee to be zero or even negative (subsidising the state read/write cost) when the PoSt succeeds. Or with an explicit ChargeGas(negative) in the miner actor. Thus, PoSts which are necessary to the system are free. If failed PoSts continue to consume a larger gas limit, this will not be prone to spam. Note that a duplicate PoSt for a partition is rejected, so only the exactly necessary Window PoSts can enjoy this benefit.

Setting the WindowPoSt gas usage to a small number, not representative of its actual cost, would effectively increase the amount of work done for each tipset. On the assumption of a uniform distribution of WindowPoSt messages across epochs, we could reduce the block gas limit by an exactly compensating amount. As more partitions are added to the network, the system will essentially increase the amount of work permitted per block by exactly that amount needed to process the new WindowPoSts. Given the cost of committing new sectors, this can't change very fast (and miners already have strong incentives to seal as fast as possible). It could possibly lead to us needing to reduce the block gas limit again in the future.

Compared with the fast-track proposal, this achieves the same reservation of capacity (by reducing block gas limit) but is much simpler: there is only one gas limit, one base fee. Everything is exactly the same as now, except WindowPoSt is cheap for senders, profitable for block producers, and we adjust the block gas limit.

The one thing that the fast-track achieves that this gas limit manipulation doesn't is a clear bound on the number of Window PoSt messages that can be included in a block. On average this number is bounded by the number of valid Window PoSts required to prove all partitions, but this is an average over 40-60 epochs. A collection of miners could collude to submit many WindowPoSt messages in the same epoch, causing a slow tipset.

[Kubuxu]

There is one issue with the negative gas used for successful WindowPoSt Verification: GasLimit needs to be greater than max(GasUsed) at any point in the execution. This means that even if you refund most of the gas used before WindowPoSt is verified the GasLimit still needs to be high, which in turn will trigger OverEstimation gas burn.

A possible solution is charging a large amount of negative gas (equivalent to the amount of refund) at the start of the execution of the message, and then if WindowPoSt fails for any reason, charging that refund back.
This doesn't solve all problems as I can submit messages with a low gas limit that will trigger expensive WindowPoSt verification, which will fail but as my GasLimit is very low I'm not paying almost anything for it. I can repeat that as many times as I want during a window (unless we handle it in some special way).

[anorth]

If we're going to artificially decrease the gas cost for some message then it makes sense we might need some compensating mechanism to avoid it being attractive as a spam vector. We could add a failed Window PoSt counter to each deadline or partition and enforce a maximum number of attempts?

[zixuanzh]

It seems that what was suggested is to reduce the GasUsage of SubmitWindowPoSt. However, we need to reduce the block gas limit somewhat to account for potential increase in processing time and spam vector. If we are going down that path, we can apply the discount to Base Fee instead. It will probably make more sense as a stopgap since it doesn't interfere with gas accounting, chain capacity, or overestimation burn. For example, the network can have X% off for SubmitWindowPoSt and PublishStorageDeals and maybe Y% extra for Pre or ProveCommit given the network BaseFee).

[jbenet]

• in networking, separating the control plane and data plane was key to solving similar kinds of problems. Many protocols need control messages to bypass network congestion
• I think having a Control Gas Plane is probably the robust solution.
  • (there may be more periodic control messages in the future)
• The percentage of control plane vs data plane bandwidth could hike up over time.

---

Another option is to instead separate out pre/prove commit messages and bound them. Will help deals and other kinds of messages through

[irenegia]

Set the gas usage and limit of WindowPoSt messages to be very small

What's the cost of doing this? Can we do it?
I understood that gasUsed is basically out of our control since it depends on how many operations are required to verify a PoSt. Is this wrong? Clearly, gasUsed decreases if we make verification more efficient, but can we artificially it small?

If we keep gasUsed as it is and make gasLimit small, then someone else has to pay for the gas?

We can achieve this by setting the VerifyPoSt() gas fee to be zero or even negative (subsidising the state read/write cost) when the PoSt succeeds.

I don't understand this, gas fee (gasFeeCap) is decided by the miner (the sender), do you mean setting gasLimit to zero?
Subsidising by what/who?

[anorth]

gasUsed is basically out of our control ... can we artificially it small

Yes we can set it artificially, it's very much in our control. The cost in gas units of the VerifyPoSt() syscall is just a constant in the code (determined empirically), and we can change it.

Alternatively, and even more powerful, we can invoke ChargeGas(-X) in the miner actor (link). (Note that we may eventually want to remove this method from the Runtime API for user contracts).

The price per unit is determined by the sender, but the number of units consumed is determined by the operation, and we have direct ways to manipulate this

[irenegia]

the number of units consumed is determined by the operation, and we have direct ways to manipulate this

Yep, but my point is the operation (verify post in this case) has some intrinsic lower bound for the number of units consumed, no?

[anorth]

There is a lower bound on the amount of CPU or other resources actually used by the operation, but there is no lower bound on the number of gas units that we decide it should consume.

[anorth]

I have had my say here and will withdraw from pressing it. I think @Kubuxu understands my proposal, and probably understands the implications even better than I do. I won't object to the fast track proposal if @Kubuxu agrees it is appropriate.

[nikkolasg]

@zixuanzh From what I understand currently price of window post is a problem for small miners even when network isn't congested - reducing base fee won't solve that won't it ? whereas setting a small gas usage might solve it, am i getting this right ? @anorth

[zixuanzh]

From the perspective of gas cost incurred, applying a discount on GasUsage is equivalent to applying a discount to network BaseFee for a particular message without having to deal with the complexity of gas accounting and chain capacity. I can spell them out if useful.

Now, there is a separate problem of small miners tend not to have full partitions and may have to incur more WindowPoSts per unit of storage (which can argued as some form of production inefficiency). Lotus should consider implementing what was discussed in #32 sooner.

[steven004]

The const BlockGasLimit = 10_000_000_000 setting is a global network computing and storage handling upper-limit estimation. Both proposals, fast track and small verify-post cost setting, would possibly break the principle, and make it more complicated.

Could we simply build in auto-incentive mechanism for type-A messages into the protocol, that is, we do not change anything on the message selection, and cost calculation (which is critical and should be fair for each kind of messages), but we send partial to-be-burnt base gas fee to the miner, so that the miner would select the type-A message first; and we could reimburse partial to-be-burnt base gas fee back to sender, so that the sender would set higher feeCap for these critical positive messages.

The only impact, I would think about now, is the we burn less. This should be acceptable.

[jbenet]

@irenegia may want to write up the FIP document

[jbenet]

See also: #47 -- somewhat of an inverse to @anorth's proposal above

[kaitlin-beegle]

Closing due to inactivity.