Issues running Silo in Docker Compose: Unexpected direct silo connection on proxy endpoint about orleans

[SeppPenner]

Some more logging output (Shows that the options are set properly:

silo-1  | 05.03.24 15:42:26.609    [INF]   Configuration Orleans.Configuration.EndpointOptions:
silo-1  | AdvertisedIPAddress: 192.168.201.36
silo-1  | SiloPort: 11111
silo-1  | GatewayPort: 30000
silo-1  | SiloListeningEndpoint: 0.0.0.0:11111
silo-1  | GatewayListeningEndpoint: 0.0.0.0:30000

[SeppPenner]

The server has the following configuration options (Should be ok as well, I guess)

{
    "SiloIpAddress": "192.168.201.36",
    "GatewayPort": 30000
}

[SeppPenner]

Something more to mention: We use a background service that implements BackgroundService from Microsoft.Extensions.Hosting. In there, we create a Orleans client. The client is needed to perform some checks on Silo startup. The client runs "locally" in the Silo itself and seems to be causing the issue...

private async Task<IClusterClient> CreateLocalOrleansClient()
{
    var siloIpAddress = this.advertisedSiloIp;

    if (Program.InCompose)
    {
        siloIpAddress = IPAddress.Any;
    }

    this.logger.Information("Initializing OrleansRpc client for silo ip {IpAddress}...", siloIpAddress);

    var orleansClient = new ClientBuilder().Configure<ClusterOptions>(
            options =>
            {
                options.ClusterId = this.configuration.ClusterId;
                options.ServiceId = this.configuration.ServiceId;
            })
        .UseStaticClustering(new IPEndPoint(siloIpAddress, this.configuration.GatewayPort))
        .ConfigureApplicationParts(
            parts =>
            {
                parts.AddApplicationPart(typeof(ISomethingGrain).Assembly).WithReferences();
            })
        .ConfigureLogging(l => l.AddSerilog())
        .AddSimpleMessageStreamProvider(StreamGlobals.SmsProvider)
        .Build();
    await orleansClient.Connect();
    return orleansClient;
}

[SeppPenner]

Something more to mention: We use a background service that implements BackgroundService from Microsoft.Extensions.Hosting. In there, we create a Orleans client. The client is needed to perform some checks on Silo startup. The client runs "locally" in the Silo itself and seems to be causing the issue...

private async Task<IClusterClient> CreateLocalOrleansClient()
{
    var siloIpAddress = this.advertisedSiloIp;

    if (Program.InCompose)
    {
        siloIpAddress = IPAddress.Any;
    }

    this.logger.Information("Initializing OrleansRpc client for silo ip {IpAddress}...", siloIpAddress);

    var orleansClient = new ClientBuilder().Configure<ClusterOptions>(
            options =>
            {
                options.ClusterId = this.configuration.ClusterId;
                options.ServiceId = this.configuration.ServiceId;
            })
        .UseStaticClustering(new IPEndPoint(siloIpAddress, this.configuration.GatewayPort))
        .ConfigureApplicationParts(
            parts =>
            {
                parts.AddApplicationPart(typeof(ISomethingGrain).Assembly).WithReferences();
            })
        .ConfigureLogging(l => l.AddSerilog())
        .AddSimpleMessageStreamProvider(StreamGlobals.SmsProvider)
        .Build();
    await orleansClient.Connect();
    return orleansClient;
}

I have reworked this now following https://learn.microsoft.com/en-us/dotnet/orleans/host/client?pivots=orleans-3-x.

The IClusterClient is now injected per DI, the issue persists, however...

[SeppPenner]

Again an update: The issue persists, but the server project now causes the issue, the Silo itself can be started properly...
The client in the server project is initialized like this:

var gateways = this.configuration.OrleansSiloEndPoints; // This is just 192.168.201.36, port 30000 from the configuration.
// It's loaded properly, I check that. --> It's irrelevant as well as the gatewys are overwritten below with the environment variable.

if (Environment.GetEnvironmentVariable("DOTNET_RUNNING_IN_COMPOSE")?.ToLowerInvariant() == "true")
{
    var addresses = Dns.GetHostAddresses("server");
    gateways = addresses.Select(a => new IPEndPoint(a, 30000)).ToArray();
}

var client = new ClientBuilder()
    .Configure<ClusterOptions>(
        options =>
        {
            options.ClusterId = this.configuration.ClusterId;
            options.ServiceId = this.configuration.ServiceId;
        })
    .Configure<ClientMessagingOptions>(opts =>
    {
        opts.ResponseTimeout = TimeSpan.FromSeconds(90);
    })
    .Configure<MessagingOptions>(opts =>
    {
        opts.ResponseTimeout = TimeSpan.FromSeconds(90);
    })
    .UseStaticClustering(gateways)
    .ConfigureApplicationParts(parts =>
        {
            parts.AddApplicationPart(typeof(ISomethingGrain).Assembly).WithReferences();
        })
    .ConfigureLogging(logging => logging.AddSerilog())
    .Configure<ClientMessagingOptions>(options => options.MaxMessageBodySize = 134217728) //128 MB
        .AddSimpleMessageStreamProvider("SmsProvider")
        .Build();