IStreamSubstitution Unable to Mock Request Body about fluxzy.core

[haga-rak]

I didn't still merge it yet. I will come back later with a more detailed answer

[haga-rak]

I don't really know how you managed to get a 528 HTTP code.

However, I've tried to run the provided steps and found that the mock is not set when no initial payload was sent to the client. This is what I've addressed with the fixes.

Coming back to your example, there are a few gotchas:

• You should use DrainAsync instead of FlushAsync on the original stream. If the client has a request body, it has to be consumed; otherwise, subsequent requests using the connection will either linger indefinitely or just fail.
• You should add or update the Content-Length header as the proxy is not able to predict the length of the body.

Finally, you can check this unit tests to see an example of a full request body mock using ISubstitution.

[MichaelVanHouHei]

@haga-rak let me take look , thanks for the efficiency fix.
what i wanted to do is mortify the request body before send , so the step will be

1. Drain sync the original Stream
2. read the original Stream and json deserialize
3. mortify the object than serialize it
4. new memory stream ( mortified json byte array)

[MichaelVanHouHei]

As the below code haven't updated the content-length yet
` internal class TestReqMockAction :Action
{
public override ValueTask InternalAlter(ExchangeContext context, Exchange? exchange, Connection? connection, FilterScope scope,
BreakPointManager breakPointManager)
{
if (exchange == null)
return default;
context.RegisterRequestBodySubstitution(new TestBodySubsitition());
return default;
}

  public override FilterScope ActionScope =>FilterScope.RequestHeaderReceivedFromClient;
  public override string      DefaultDescription => "Test";

}than internal class TestBodySubsitition : IStreamSubstitution
{

public async ValueTask<Stream> Substitute(Stream originalStream)
{
    await originalStream.DrainAsync();

    var result = await System.Text.Json.JsonSerializer.DeserializeAsync<TestReq>(originalStream);` 
   // so mortification such that result.XXX = YYYY 
    return new MemoryStream( JsonSerializer.SerializeToUtf8Bytes(result));
}

}
Than i got 528 in response

[haga-rak]

Drain sync the original Stream
read the original Stream and json deserialize

You don't need to Drain the original stream if you read it.

Note: The fix is provided by 1.19.21 which is currently being validated by nuget teams

[haga-rak]

await originalStream.DrainAsync();

var result = await System.Text.Json.JsonSerializer.DeserializeAsync<TestReq>(originalStream);` 
// so mortification such that result.XXX = YYYY 
return new MemoryStream( JsonSerializer.SerializeToUtf8Bytes(result));

Just drop await originalStream.DrainAsync(); because you already consuming the stream here var result = await System.Text.Json.JsonSerializer.DeserializeAsync<TestReq>(originalStream);

You should also pay attention to the state where var result = await System.Text.Json.JsonSerializer.DeserializeAsync<TestReq>(originalStream); is not successfull (typically no JSON payload) and can lead to 528.

[MichaelVanHouHei]

(typically no JSON payload)

as tested , the whole substation function can be run without any exception thrown (as line by line breakpoint) , json can be desterilize correctly

[MichaelVanHouHei]

since my use case is a bit different from the unit test. The unit test example is pre-defined mock data.
For my use case , the program read the request body and mortify its content , so maybe its "difficult" for me to modify the content-length (since read stream and mortified stream is at ValueTask Substitute)

[haga-rak]

the program read the request body and mortify its content , so maybe its "difficult" for me to modify the content-length (since read stream and mortified stream is at ValueTask Substitute)

You can omit setting content length, in that case the proxy will just switch to Transfer-Encoding: Chunked which is perfectly fine for most web servers.

I can't figure out what you've missed. The sample below alter an existing JSON and it's working perfectly fine.

using System;
using System.IO;
using System.Net.Http.Json;
using System.Text.Json;
using System.Text.Json.Serialization;
using System.Threading.Tasks;
using Fluxzy.Core;
using Fluxzy.Core.Breakpoints;
using Fluxzy.Rules;
using Xunit;
using Action = Fluxzy.Rules.Action;

namespace Fluxzy.Tests.UnitTests.Substitutions
{
    public class ModifyJsonSubstitutionTests
    {
        [Fact]
        public async Task SubstituteJsonBody()
        {
            var setting = FluxzySetting.CreateLocalRandomPort();

            setting.ConfigureRule()
                   .WhenAny()
                   .Do(new RequestBodyJsonMockingAction());

            await using var proxy = new Proxy(setting);
            var endPoints = proxy.Run();

            using var httpClient = HttpClientUtility.CreateHttpClient(endPoints, setting);

            var payload = new DummyPayload {
                DoYouAgree = "YES"
            };

            var response = await httpClient.PostAsJsonAsync("https://httpbin.org/anything", payload);

            var httpBinResult = await response.Content.ReadFromJsonAsync<HttpBinResponse>();
            var modifiedPayload = JsonSerializer.Deserialize<DummyPayload>(httpBinResult!.Data!);

            Assert.True(response.IsSuccessStatusCode);
            Assert.NotNull(modifiedPayload!);
            Assert.Equal(modifiedPayload.DoYouAgree, "NO", StringComparer.OrdinalIgnoreCase);
        }
    }

    internal class HttpBinResponse
    {
        [JsonPropertyName("data")]
        public string? Data { get; set; }
    }

    internal class RequestBodyJsonMockingAction : Action
    {
        public override FilterScope ActionScope => FilterScope.RequestHeaderReceivedFromClient;

        public override string DefaultDescription => "Test mock with JSON";

        public override ValueTask InternalAlter(
            ExchangeContext context, Exchange? exchange, Connection? connection, FilterScope scope,
            BreakPointManager breakPointManager)
        {
            context.RegisterRequestBodySubstitution(new ModifyJsonSubstitution());

            return default;
        }
    }

    internal class ModifyJsonSubstitution : IStreamSubstitution
    {
        public async ValueTask<Stream> Substitute(Stream originalStream)
        {
            var result = (await JsonSerializer.DeserializeAsync<DummyPayload>(originalStream))!;
            result.DoYouAgree = "NO";

            return new MemoryStream(JsonSerializer.SerializeToUtf8Bytes(result));
        }
    }

    internal class DummyPayload
    {
        public string DoYouAgree { get; set; } = "";
    }
}

[MichaelVanHouHei]

Yup,the code you provide is quite similar to what I have alrrady
this issue can be closed.
Seems the remote server close the connection.
possibly there are some validations when the server receives my modified request.

[haga-rak]

Seems the remote server close the connection.

Hmmm. It's possible that the remote is confused with the presence of Content-length and Transfer-Encoding headers at the same time, thus closing the connection.

You should explicitely remove the previous Content-Length header from the original request. Like this

internal class RequestBodyJsonMockingAction : Action
{
    public override FilterScope ActionScope => FilterScope.RequestHeaderReceivedFromClient;

    public override string DefaultDescription => "Test mock with JSON";

    public override ValueTask InternalAlter(
        ExchangeContext context, Exchange? exchange, Connection? connection, FilterScope scope,
        BreakPointManager breakPointManager)
    {
       // Add the following line
        context.RequestHeaderAlterations.Add(new HeaderAlterationDelete("Content-length"));

        context.RegisterRequestBodySubstitution(new ModifyJsonSubstitution());

        return default;
    }
}