quicwg / reliable-stream-reset Goto Github PK

QUIC section 2.4 describes operations on streams that includes:

• write data, understanding when stream flow control credit (Section 4.1) has successfully been reserved to send the written data;

• end the stream (clean termination), resulting in a STREAM frame (Section 19.8) with the FIN bit set; and

• reset the stream (abrupt termination), resulting in a RESET_STREAM frame (Section 19.4) if the stream was not already in a terminal state.

I'm going to say that implicitly this spec counts as an abrupt termination, if so it might help to state that explicitly and point to the RFC section. The reason I care is because HTTP/3 has different rules about handling stream data (frames) depending on whether things are clean or abrupt; see https://www.rfc-editor.org/rfc/rfc9114.html#section-7.1-6

As discussed at IETF 117, we can either:

• keep CLOSE_STREAM,
• go back to RELIABLE_RESET_STREAM, or
• something else.

Hi!

I was implementing RFC: https://datatracker.ietf.org/doc/draft-ietf-quic-reliable-stream-reset/
and had a few thoughts regarding support for the SENDER sending multiple CLOSE_STREAM frames to reduce ReliableSize.

Perhaps the RFC could specify details related to ACK state tracking to help implementors?

For instance, if a sender sends 100 CLOSE_STREAM frames with monotonically decreasing ReliableSize, how do we ensure that the receiver is synchronized with the minimum value of ReliableSize before the sender can close its stream?

There were 2 approaches I was considering:

1. Wait until all the CLOSE_STREAM frames get ACK'd, and then we can be sure the Receiver will have the minimum ReliableSize.

2. Sender tracks all the CLOSE_STREAM frames sent via metadata, and when processing a CLOSE_STREAM ACK, the sender can check whether the metadata is the minimum ( == current ReliableSize state).

I feel approach 2) is the most optimal as we can get lucky and the receiver can ACK the minimum CLOSE_STREAM frame first.
Perhaps the RFC could go into more detail here?

Currently, the draft allows for a sender to change the reliable offset for RESET_AT any number of times. I would argue:

1. There is no scenario that needs this.
2. This unnecessarily complicates the logic (more on this below).

Therefore, we should remove this "feature".

Details

(2) above requires the following extra logic:

Sender

• Additional API logic to allow for an application to change the value, but only to a smaller value (documentation as well).
• Update the stream send-side state machine accordingly.
• Have per-packet RESET_AT acknowledgement tracking to know exactly which reliable offset has been acknowledged, instead of per-stream "ReliableResetAtAcked" state. This is required because you must be certain the peer is in sync, in the scenario where a offset reduction was no acknowledged, but the original offset was. In this case, if the sender does not track state appropriately (and instead just has a per-stream state), it could incorrectly think the peer has everything it needs to clean up the stream and then stop sending/retansmitting the necessary data; resulting in a deadlock.

Receiver

• Allow the change (checking for current state), instead of just erroring out.
• Update the stream recv-side state machine accordingly.
• (Possibly) additional API indications/events.

Resets are always reliable in QUIC itself, so the phrasing is in general confusing. We already renamed the frame to RESET_STREAM_AT, so the rest of the draft should probably be updated too.

In Section 5.1, should this say something like:

While multiple RESET_STREAM_AT frames can reduce Reliable Size, some applications might need to ensure that a minimum amount of data is always delivered on a stream. Application protocols can establish rules for streams that ensure that Reliable Size is not reduced below a certain threshold if that is necessary to ensure correct operation of the protocol.

To explicitly signal this WebTransport, as it were.

Section 5

When resetting a stream without the intent to deliver any data to the receiver, the sender uses a RESET_STREAM frame (Section 3.2 of [RFC9000]). The sender MAY also use a RESET_STREAM_AT frame with a Reliable Size of zero in place of a RESET_STREAM frame. These two have the same effect and the behavior of RESET_STREAM frame is unchanged from the behavior described in [RFC9000].

This seems technically fine but I wonder if we can tweak the wording to make it read a little better

When receiving a RESET_STREAM_AT frame with a lower Reliable Size, the receiver only needs to deliver data up the lower Reliable Size to the application. It MUST NOT expect the delivery of any data beyond that byte offset.

There's something about the wording here that requires me to do some mental gymnastics that I think can be avoided by some rewording of the text.

We correctly remind that QUIC stacks might over-deliver, stating that: As described in Section 3.2 of [RFC9000], it MAY deliver data beyond that offset to the application.

But we do not explicitly state that QUIC stacks may over-deliver Reliable Size; i.e., the value might be greater than what is provided by the application, though I think that has always been our intent. I think it would be better to state that so that people using QUIC do not make the incorrect assumption.

Similarly, we should state that the error code might not get delivered to the peer application.

As far as I remember, "reliable reset" originally referred to the fact that this feature would make WebTransport stream resets reliable. However, outside of that context, that name does not make that much sense, since QUIC resets are already reliable.

While implementing this SPEC for MsQuic, I've had to develop some non-trivial book-keeping mechanisms so the sender / receiver can abort their send/recv paths at the appropriate times.

In the context of the core use case, an implementation could be made much simpler if the sender just re-uses the RESET frame abortive path once it has sent / ACKed enough data. Receiver doesn't need to know or do anything.

That way, the sender doesn't necessarily even need to communicate a RELIABLE RESET to the receiver, it'll just be a more appropriately timed RESET, and perhaps the ReliableSize could be something communicated / negotiated during the handshake, or as an added field in STREAM_STARTED frame or something?

Might just be me, but I read

Conceptually, the CLOSE_STREAM frame is a RESET_STREAM frame with an added Reliable Size field.

And was lead down the false path to think it is a direct substitute for RESET_STREAM. However,

When resetting a stream, the node has the choice between using a RESET_STREAM frame and a CLOSE_STREAM frame of type 0x21.

the and in this sentence threw me off.

I'm not exactly sure what is expected. This seems a problem when a peer sends STOP_SENDING and I might be able to reply with either CLOSE_STREAM or RESET_STREAM because these things behave differently. The the peer really needs one type but the sender decides arbitrarily, bad stuff could happen.

Depending if it is an && or || relationship, we might really need to pull in something like the ENOUGH frame to make this stuff really separate and unambiguous.

The draft currently uses frame type 0x20. That number requires Standards Action and can't be used in drafts as it risks conflicting with another draft. Please use a multi-byte encoding for now, we can switch to a lower number for both the transport parameter and the frame after WGLC.

The intro states:

Some applications running on top of QUIC send an identifier at the beginning of
the stream in order to associate that stream with a specific subcomponent of the
application. For example, WebTransport ({{!WEBTRANSPORT}}) uses a
variable-length encoded integer to associate a stream with a particular
WebTransport session. It is desirable that the receiver is able to associate
incoming streams with their respective subcomponent of the application, even if
the QUIC stream is reset before the identifier at the beginning of the stream
was read by the application.

Actual use cases for protocol extensions are good. I think the WebTransport normative reference is just an accident and we can make it informative (even if that ends up as a downref, it's adopted work we expect to be published someday).

I think we can tweak this text a little bit to make the extension sound as generic as it is, and then speak to the concrete examples. I'll make a PR.